Multiple GPUs, Peer-to-Peer Question

Hi all,

when I run simpleP2P, I got following output. I want to ask why GPU0 cannot peer access GPU2? Hardware limit or wrong software configuration?

following is the lspci output.

thank you.

[./simpleP2P] - Starting…
Checking for multiple GPUs…
CUDA-capable device count: 4

GPU0 = “GeForce GTX 1080” IS capable of Peer-to-Peer (P2P)
GPU1 = “GeForce GTX 1080” IS capable of Peer-to-Peer (P2P)
GPU2 = “GeForce GTX 1080” IS capable of Peer-to-Peer (P2P)
GPU3 = “GeForce GTX 1080” IS capable of Peer-to-Peer (P2P)

Checking GPU(s) for support of peer to peer memory access…

Peer access from GeForce GTX 1080 (GPU0) → GeForce GTX 1080 (GPU1) : Yes
Peer access from GeForce GTX 1080 (GPU0) → GeForce GTX 1080 (GPU2) : No
Peer access from GeForce GTX 1080 (GPU0) → GeForce GTX 1080 (GPU3) : No
Peer access from GeForce GTX 1080 (GPU1) → GeForce GTX 1080 (GPU0) : Yes
Peer access from GeForce GTX 1080 (GPU1) → GeForce GTX 1080 (GPU2) : No
Peer access from GeForce GTX 1080 (GPU1) → GeForce GTX 1080 (GPU3) : No
Peer access from GeForce GTX 1080 (GPU2) → GeForce GTX 1080 (GPU0) : No
Peer access from GeForce GTX 1080 (GPU2) → GeForce GTX 1080 (GPU1) : No
Peer access from GeForce GTX 1080 (GPU2) → GeForce GTX 1080 (GPU3) : Yes
Peer access from GeForce GTX 1080 (GPU3) → GeForce GTX 1080 (GPU0) : No
Peer access from GeForce GTX 1080 (GPU3) → GeForce GTX 1080 (GPU1) : No
Peer access from GeForce GTX 1080 (GPU3) → GeForce GTX 1080 (GPU2) : Yes

[root@gpu1 simpleP2P]# lspci | grep NVIDIA
02:00.0 VGA compatible controller: NVIDIA Corporation Device 1b80 (rev a1)
02:00.1 Audio device: NVIDIA Corporation Device 10f0 (rev a1)
03:00.0 VGA compatible controller: NVIDIA Corporation Device 1b80 (rev a1)
03:00.1 Audio device: NVIDIA Corporation Device 10f0 (rev a1)
82:00.0 VGA compatible controller: NVIDIA Corporation Device 1b80 (rev a1)
82:00.1 Audio device: NVIDIA Corporation Device 10f0 (rev a1)
83:00.0 VGA compatible controller: NVIDIA Corporation Device 1b80 (rev a1)
83:00.1 Audio device: NVIDIA Corporation Device 10f0 (rev a1)

[root@gpu1 simpleP2P]# lspci -t
-±[0000:ff]-±08.0
| ±08.2
| ±08.3
| ±09.0
| ±09.2
| ±09.3
| ±0b.0
| ±0b.1
| ±0b.2
| ±0c.0
| ±0c.1
| ±0c.2
| ±0c.3
| ±0c.4
| ±0c.5
| ±0c.6
| ±0c.7
| ±0d.0
| ±0d.1
| ±0d.2
| ±0d.3
| ±0d.4
| ±0d.5
| ±0f.0
| ±0f.1
| ±0f.2
| ±0f.3
| ±0f.4
| ±0f.5
| ±0f.6
| ±10.0
| ±10.1
| ±10.5
| ±10.6
| ±10.7
| ±12.0
| ±12.1
| ±12.4
| ±12.5
| ±13.0
| ±13.1
| ±13.2
| ±13.3
| ±13.6
| ±13.7
| ±14.0
| ±14.1
| ±14.2
| ±14.3
| ±14.4
| ±14.5
| ±14.6
| ±14.7
| ±16.0
| ±16.1
| ±16.2
| ±16.3
| ±16.6
| ±16.7
| ±17.0
| ±17.1
| ±17.2
| ±17.3
| ±17.4
| ±17.5
| ±17.6
| ±17.7
| ±1e.0
| ±1e.1
| ±1e.2
| ±1e.3
| ±1e.4
| ±1f.0
| -1f.2
±[0000:80]-±00.0-[81]–±00.0
| | -00.1
| ±02.0-[82]–±00.0
| | -00.1
| ±03.0-[83]–±00.0
| | -00.1
| ±04.0
| ±04.1
| ±04.2
| ±04.3
| ±04.4
| ±04.5
| ±04.6
| ±04.7
| ±05.0
| ±05.1
| ±05.2
| -05.4
±[0000:7f]-±08.0
| ±08.2
| ±08.3
| ±09.0
| ±09.2
| ±09.3
| ±0b.0
| ±0b.1
| ±0b.2
| ±0c.0
| ±0c.1
| ±0c.2
| ±0c.3
| ±0c.4
| ±0c.5
| ±0c.6
| ±0c.7
| ±0d.0
| ±0d.1
| ±0d.2
| ±0d.3
| ±0d.4
| ±0d.5
| ±0f.0
| ±0f.1
| ±0f.2
| ±0f.3
| ±0f.4
| ±0f.5
| ±0f.6
| ±10.0
| ±10.1
| ±10.5
| ±10.6
| ±10.7
| ±12.0
| ±12.1
| ±12.4
| ±12.5
| ±13.0
| ±13.1
| ±13.2
| ±13.3
| ±13.6
| ±13.7
| ±14.0
| ±14.1
| ±14.2
| ±14.3
| ±14.4
| ±14.5
| ±14.6
| ±14.7
| ±16.0
| ±16.1
| ±16.2
| ±16.3
| ±16.6
| ±16.7
| ±17.0
| ±17.1
| ±17.2
| ±17.3
| ±17.4
| ±17.5
| ±17.6
| ±17.7
| ±1e.0
| ±1e.1
| ±1e.2
| ±1e.3
| ±1e.4
| ±1f.0
| -1f.2
-[0000:00]-±00.0
±01.0-[01]–
±02.0-[02]–±00.0
| -00.1
±03.0-[03]–±00.0
| -00.1
±04.0
±04.1
±04.2
±04.3
±04.4
±04.5
±04.6
±04.7
±05.0
±05.1
±05.2
±05.4
±11.0
±11.4
±14.0
±16.0
±16.1
±1a.0
±1b.0
±1c.0-[04]–
±1c.3-[05-06]----00.0-[06]----00.0
±1c.4-[07-3f]–
±1d.0
±1f.0
±1f.2
-1f.3

this topic is covered extensively in other questions on this forum if you want to search for them