Can no longer connect to GMSL camera

Hi there,

Hopefully someone can help me, we have been using a DPX2 now for over a month to begin with it was fine we could connect to our ar0231 camera perfectly with no problems. A few weeks ago however, The connection could no longer be established. It is worth mentioning that the setup had not changed in anyway (the cameras were not unplug/swapped etc) we just came in one morning powered up the PX2 and could no longer connect to the camera. Initially we thought it was a problem with the camera so we returned it to leopard imaging who ran a number of tests on it all of which passed (both in a windows environment, using their GMSL to USB adapter and using their DPX2). They informed me that they had experienced similar issues in the past where all of a sudden the connection would stop working, they said that they fixed it by updating to the latest firmware (4.1.8.0). So I flashed our DPX2 to 4.1.8.0 and aurix 2.02.04 using Drive install however the connection problems still remain.

So far we have tried:

  • Flashing the unit with FW 4.1.4.0, 4.1.6.1 & 4.1.8.0
  • Connected the camera to all 12 of the GMSL connectors
  • Tried all of the example programs which support the ar0231 sensors
  • Tried our own in house software which uses the NVmedia IPP interface
  • Ordered an additional camera to check (this showed the same issue)
  • Used different cables

None of which solved the connection issue. Has anyone experienced similar problems? are they anymore troubleshooting methods that we can try?

Thanks!

Edit: Forgot to mention it’s the autoChauffeur unit

Log created using “sample_GMSL_camera”:

nvidia@nvidia:/usr/local/driveworks-0.3/bin$ ./sample_camera_gmsl  --camera-type=ar0231
Program Arguments:
--camera-type=ar0231
--csi-port=ab
--fifo-size=3
--offscreen=0
--serialize-bitrate=8000000
--serialize-framerate=30
--serialize-type=h264
--slave=0
--write-file=

Initialize DriveWorks SDK v0.3.422
Release build with GNU 4.9.2 from v0.3.1-rc2-0-g0475f26 against PDK v4.1.6.3
SDK: Resources mounted from /usr/local/driveworks-0.3/data/resources
SDK: Create NvMediaDevice
SDK: use EGL display as provided
SDK: iGPU has been detected as device number 0
SDK: Default GPU Device set to device number 0
SAL: identified board as DrivePX2-TegraA
SensorFactory::createSensor() -> camera.gmsl, camera-type=ar0231,csi-port=ab,fifo-size=3,offscreen=0,serialize-bitrate=8000000,serialize-framerate=30,serialize-type=h264,slave=0,write-file=,output-format=yuv
CameraGMSL: required FPS = 30, resolution = 1920x1208
receive_from: Timeout Error.
Please check Ethernet ip configuration or
check Aurix firmware version
iscRootDevOpen: Failed to power on camera(s)
NvMediaISCRootDeviceCreate: Unable to open root device
nvmedia: ERROR: Init: Failed to create NvMedia ISC root device
Driveworks exception thrown: DW_SAL_CANNOT_CREATE_SENSOR: CameraGMSL: cannot connect to camera
Cannot create driver: camera.gmsl with params: camera-type=ar0231,csi-port=ab,fifo-size=3,offscreen=0,serialize-bitrate=8000000,serialize-framerate=30,serialize-type=h264,slave=0,write-file=,output-format=yuv
Error: DW_SAL_CANNOT_CREATE_SENSOR

Dear Andy_P,

Sorry for late reply.
Could you please help to check the following and provide logs? Thanks.

  1. On Aurix Console
    a. Check output of “version” command and “status” command , below should be the default IP address set in Aurix
    i. IP-address (Tegra-A): 10.42.0.28
    ii. IP-address (Tegra-B): 10.42.0.29
    iii. IP-address (AURIX): 10.42.0.146
  2. On Target Console (Tegra A) check the following
    b. nvidia@nvidia:/usr/local/driveworks-0.3/bin$ cat /etc/nvidia/version-nv-pdk.txt
    4.1.8.0-9387945
    c. nvidia@nvidia:/usr/local/driveworks-0.3/bin$ cat /etc/tacp/tacp.cfg
    AURIX_IP_ADDRESS=10.42.0.146
    TEGRA_A_IP_ADDRESS=10.42.0.28
    TEGRA_B_IP_ADDRESS=10.42.0.29
    TEGRA_IP_ADDRESS=10.42.0.28
    AURIX_SERVER_PORT=5000
    drive-px2-a=eth0.200
    drive-px2-b=eth0.200
    drive-px2-autocruise=lo
    RECEIVE_TIMEOUT_S=5
    RECEIVE_TIMEOUT_US=0
    SPI_RECEIVE_TIMEOUT_US=1000000

Here IP’s should match for Aurix and Tegra
3) In case IP’s are correct try pinging Aurix from Tegra Console and check for packet loss
• Ping 10.42.0.146

Hi Steve thanks for getting back to me here are the logs you requested

Version:

version
Info: Executing cmd: version, argc: 0, args: 
SW Version: DPX2-P2379-EB-Aurix-2.02.04
Compilation date: Jul 18 2017, 18:48:06
Command Executed

Status (seems odd how there are no mac address?):

shell> status
Info: Executing cmd: status, argc: 0, args: 
Alive       : 00:12:32
CPU load     Core 0: 7%
CPU load max Core 0: 8%
CPU load     Core 1: 0%
CPU load max Core 1: 0%
CPU load     Core 2: 0%
CPU load max Core 2: 0%
                                                                                
Hardware information:                                                           
PMU-chip: B-Step                                                                
SCU_CHIPID: 0x4B459051                                                          
SystemUpInit-Time[ms]: 45                                                       
FPD-switch = 1 (Tegra-B)                                                        
                                                                                
IP-address (Tegra-A): 10.42.0.29                                                
IP-address (Tegra-B): 10.42.0.29                                                
IP-address   (AURIX): 10.42.0.146                                               
                                                                                
MAC-address (Tegra-A): 0x0000000000000000                                       
MAC-address (Tegra-B): 0x0000000000000000                                       
MAC-address   (AURIX): 0x0000000000000000                                       
MAC-address    (FPGA): 0x0000000000000000                                       
                                                                                
ADC0_HRNS2: 0x0AA8                                                              
ADC1_HRNS2: 0x0AA8                                                              
Command Executed

I get back the same nv-pdk version and tacp.cfg as you do.

When doing a ping I get host unreachable

nvidia@nvidia:~$ ping 10.42.0.146
PING 10.42.0.146 (10.42.0.146) 56(84) bytes of data.
From 10.42.0.78 icmp_seq=1 Destination Host Unreachable
From 10.42.0.78 icmp_seq=2 Destination Host Unreachable
From 10.42.0.78 icmp_seq=3 Destination Host Unreachable
From 10.42.0.78 icmp_seq=4 Destination Host Unreachable
From 10.42.0.78 icmp_seq=5 Destination Host Unreachable
From 10.42.0.78 icmp_seq=6 Destination Host Unreachable
From 10.42.0.78 icmp_seq=7 Destination Host Unreachable
From 10.42.0.78 icmp_seq=8 Destination Host Unreachable

For some reason I cannot edit my last post but one thing to note is that the ping test is trying to get a response for 10.42.0.78

Hello, it seems I have met with the same problem.

I can’t access to the px2 from my pc.

The same is " Destination Host Unreachable"

Have you found out the solution yet?

Dear Andy_P,

Clearly there is mismatch of Aurix version Below see ORANGE highlighted area
, please change aurix IP to default as per below GREEN text
.

a. Check output of “version” command and “status” command , below should be the default IP address set in Aurix
i. IP-address (Tegra-A): 10.42.0.28

ii. IP-address (Tegra-B): 10.42.0.29
iii. IP-address (AURIX): 10.42.0.146
2) On Target Console (Tegra A) check the following
b. nvidia@nvidia:/usr/local/driveworks-0.3/bin$ cat /etc/nvidia/version-nv-pdk.txt
4.1.8.0-9387945
c. nvidia@nvidia:/usr/local/driveworks-0.3/bin$ cat /etc/tacp/tacp.cfg

AURIX_IP_ADDRESS=10.42.0.146
TEGRA_A_IP_ADDRESS=10.42.0.28

TEGRA_B_IP_ADDRESS=10.42.0.29
TEGRA_IP_ADDRESS=10.42.0.28

Status (seems odd how there are no mac address?):

  1. shell> status

  2. Info: Executing cmd: status, argc: 0, args:

  3. Alive : 00:12:32

  4. CPU load Core 0: 7%

  5. CPU load max Core 0: 8%

  6. CPU load Core 1: 0%

  7. CPU load max Core 1: 0%

  8. CPU load Core 2: 0%

  9. CPU load max Core 2: 0%

  10. Hardware information:

  11. PMU-chip: B-Step

  12. SCU_CHIPID: 0x4B459051

  13. SystemUpInit-Time[ms]: 45

  14. FPD-switch = 1 (Tegra-B)

  15. IP-address (Tegra-A): 10.42.0.29

  16. IP-address (Tegra-B): 10.42.0.29

  17. IP-address (AURIX): 10.42.0.146

  18. MAC-address (Tegra-A): 0x0000000000000000

  19. MAC-address (Tegra-B): 0x0000000000000000

  20. MAC-address (AURIX): 0x0000000000000000

  21. MAC-address (FPGA): 0x0000000000000000

  22. ADC0_HRNS2: 0x0AA8

  23. ADC1_HRNS2: 0x0AA8

  24. Command Executed

I get back the same nv-pdk version and tacp.cfg as you do.

When doing a ping I get host unreachable

  1. nvidia@nvidia:~$ ping 10.42.0.146

  2. PING 10.42.0.146 (10.42.0.146) 56(84) bytes of data.

  3. From 10.42.0.78 icmp_seq=1 Destination Host Unreachable

  4. From 10.42.0.78 icmp_seq=2 Destination Host Unreachable

  5. From 10.42.0.78 icmp_seq=3 Destination Host Unreachable

  6. From 10.42.0.78 icmp_seq=4 Destination Host Unreachable

  7. From 10.42.0.78 icmp_seq=5 Destination Host Unreachable

  8. From 10.42.0.78 icmp_seq=6 Destination Host Unreachable

  9. From 10.42.0.78 icmp_seq=7 Destination Host Unreachable

  10. From 10.42.0.78 icmp_seq=8 Destination Host Unreachable

Ahh of course! cant believe I didn’t spot that, thanks Steve fixed my problem :)