I want to run a multiGPU program in my system with 2 Quadro M4000 (I’m using Ubuntu 16.04 64 bits and CUDA 8.0).
I ran the simpleP2P test and I realized that they cannot have access to each other.
./simpleP2P
[./simpleP2P] - Starting...
Checking for multiple GPUs...
CUDA-capable device count: 2
> GPU0 = " Quadro M4000" IS capable of Peer-to-Peer (P2P)
> GPU1 = " Quadro M4000" IS capable of Peer-to-Peer (P2P)
Checking GPU(s) for support of peer to peer memory access...
> Peer access from Quadro M4000 (GPU0) -> Quadro M4000 (GPU1) : No
> Peer access from Quadro M4000 (GPU1) -> Quadro M4000 (GPU0) : No
Two or more GPUs with SM 2.0 or higher capability are required for ./simpleP2P.
Peer to Peer access is not available amongst GPUs in the system, waiving test.
I would like to attach the nvidia-smi -a results:
==============NVSMI LOG==============
Timestamp : Thu Nov 3 14:48:34 2016
Driver Version : 367.48
Attached GPUs : 2
GPU 0000:01:00.0
Product Name : Quadro M4000
Product Brand : Quadro
Display Mode : Enabled
Display Active : Enabled
Persistence Mode : Disabled
Accounting Mode : Disabled
Accounting Mode Buffer Size : 1920
Driver Model
Current : N/A
Pending : N/A
Serial Number : 0323416045901
GPU UUID : GPU-0ef125ab-4e4c-cb60-20f7-d94d8be6375d
Minor Number : 0
VBIOS Version : 84.04.88.00.06
MultiGPU Board : No
Board ID : 0x100
GPU Part Number : N/A
Inforom Version
Image Version : G400.0501.01.03
OEM Object : 1.1
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization mode : None
PCI
Bus : 0x01
Device : 0x00
Domain : 0x0000
Device Id : 0x13F110DE
Bus Id : 0000:01:00.0
Sub System Id : 0x115310DE
GPU Link Info
PCIe Generation
Max : 2
Current : 1
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays since reset : 0
Tx Throughput : 0 KB/s
Rx Throughput : 5000 KB/s
Fan Speed : 46 %
Performance State : P8
Clocks Throttle Reasons
Idle : Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
Sync Boost : Not Active
Unknown : Not Active
FB Memory Usage
Total : 8120 MiB
Used : 148 MiB
Free : 7972 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 4 MiB
Free : 252 MiB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 2 %
Encoder : 0 %
Decoder : 0 %
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending : N/A
Temperature
GPU Current Temp : 36 C
GPU Shutdown Temp : 104 C
GPU Slowdown Temp : 99 C
Power Readings
Power Management : Supported
Power Draw : 16.51 W
Power Limit : 120.00 W
Default Power Limit : 120.00 W
Enforced Power Limit : 120.00 W
Min Power Limit : 10.00 W
Max Power Limit : 120.00 W
Clocks
Graphics : 135 MHz
SM : 135 MHz
Memory : 324 MHz
Video : 405 MHz
Applications Clocks
Graphics : 772 MHz
Memory : 3005 MHz
Default Applications Clocks
Graphics : 772 MHz
Memory : 3005 MHz
Max Clocks
Graphics : 772 MHz
SM : 772 MHz
Memory : 3005 MHz
Video : 710 MHz
Clock Policy
Auto Boost : On
Auto Boost Default : On
Processes
Process ID : 932
Type : G
Name : /usr/lib/xorg/Xorg
Used GPU Memory : 101 MiB
Process ID : 1612
Type : G
Name : compiz
Used GPU Memory : 45 MiB
GPU 0000:02:00.0
Product Name : Quadro M4000
Product Brand : Quadro
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : Disabled
Accounting Mode : Disabled
Accounting Mode Buffer Size : 1920
Driver Model
Current : N/A
Pending : N/A
Serial Number : 0320916049028
GPU UUID : GPU-e8d10210-2bff-57fa-0ae7-555d49adb1fb
Minor Number : 1
VBIOS Version : 84.04.88.00.06
MultiGPU Board : No
Board ID : 0x200
GPU Part Number : N/A
Inforom Version
Image Version : G400.0501.01.03
OEM Object : 1.1
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization mode : None
PCI
Bus : 0x02
Device : 0x00
Domain : 0x0000
Device Id : 0x13F110DE
Bus Id : 0000:02:00.0
Sub System Id : 0x115310DE
GPU Link Info
PCIe Generation
Max : 1
Current : 1
Link Width
Max : 16x
Current : 4x
Bridge Chip
Type : N/A
Firmware : N/A
Replays since reset : 0
Tx Throughput : 0 KB/s
Rx Throughput : 0 KB/s
Fan Speed : 46 %
Performance State : P8
Clocks Throttle Reasons
Idle : Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
Sync Boost : Not Active
Unknown : Not Active
FB Memory Usage
Total : 8120 MiB
Used : 1 MiB
Free : 8119 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 4 MiB
Free : 252 MiB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Encoder : 0 %
Decoder : 0 %
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending : N/A
Temperature
GPU Current Temp : 31 C
GPU Shutdown Temp : 104 C
GPU Slowdown Temp : 99 C
Power Readings
Power Management : Supported
Power Draw : 10.60 W
Power Limit : 120.00 W
Default Power Limit : 120.00 W
Enforced Power Limit : 120.00 W
Min Power Limit : 10.00 W
Max Power Limit : 120.00 W
Clocks
Graphics : 135 MHz
SM : 135 MHz
Memory : 324 MHz
Video : 405 MHz
Applications Clocks
Graphics : 772 MHz
Memory : 3005 MHz
Default Applications Clocks
Graphics : 772 MHz
Memory : 3005 MHz
Max Clocks
Graphics : 772 MHz
SM : 772 MHz
Memory : 3005 MHz
Video : 710 MHz
Clock Policy
Auto Boost : On
Auto Boost Default : On
Processes : None
Also, the results of nvidia-smi topo -m
GPU0 GPU1 CPU Affinity
GPU0 X PHB 0-3
GPU1 PHB X 0-3
Am I missing something?
I think that with this config I should have access from one to each other. So what is the problem?