Does the numbering or GPU ID returned by device query match the ordering of the physical location (pcie slot) of the cards? For example, if the motherboard has 4 pcie slots, there are 4 GPU’s, then the following is true? (1st pcie slot is the closest to the CPU)
device ID 0 → 1st pcie slot
device ID 1 → second pcie slot
device ID 2 → third pcie slot
device ID 3 → fourth pcie slot
If not, is there a way to match the ID’s to the physical pcie slot?
The ordering definitely does not follow PCI-E slot ordering. I have 4 GPUs in a system, and the CUDA device ordering goes 1, 0, 2, 3. I’m not even sure if the numbering is stable if you swap out just one of the cards.
You can use cudaGetDeviceProperties(&deviceProp, cudaID) to retrieve the cudaDeviceProp struct for each CUDA device and look at the deviceProp.pciBusID and deviceProp.pciDeviceID fields. On my 4-GPU system, pciDeviceID is always zero, but the pciBusID value seems to go in ascending (but not consecutive) order away from the CPU.
It is possible that pciDeviceID fields can be non-zero if you have a dual GPU card, but I don’t have any plugged in to check at the moment.
Thanks Seibert, I will give it a try :)
Yes, It worked exactly as you described. I have three GPU’s and the Device ID is 0 for all three. Here’s the result after removing all the unrelated properties:
Detected 3 CUDA Capable device(s)
Device 0: “GeForce GTX TITAN”
Device PCI Bus ID / PCI location ID: 4 / 0
Device 1: “GeForce GTX TITAN”
Device PCI Bus ID / PCI location ID: 8 / 0
Device 2: “GeForce GTX TITAN”
Device PCI Bus ID / PCI location ID: 3 / 0
Thanks again for your help.