Data Logging

How does DriveWorks support data acquisition with DRIVE PX 2?
The NVIDIA DriveWorks command-line and GUI recorder tools save sensor data to files. For more information, see “Data Acquisition” in DriveWorks Development Guide.

How can I achieve the maximum USB 3.0 write throughput for data acquisition?
You should be able to achieve USB3.0-SSD write speeds between 400 and 500-megabytes per second, depending on the brand of the USB3.0 SSD being used. NVIDIA has tested extensively with Samsung 850 EVO and Samsung 750 EVO SSDs, where the SSDs are formatted with the EXT4 file system. The following are guidelines for ensuring optimal disk speed.

To mount and initialize your USB disk

  1. On the target platform, enter: # mount -o rw,async,noatime,nodiratime,data=writeback <path_to_drive For example: # mount -o rw,async,noatime,nodiratime,data=writeback /dev/sdX1 /mnt
  2. Enable the CFQ schedule for the USB disk: # echo cfq > /sys/block/sda/queue/scheduler The directory /sys/block/device/queue/iosched contains files that allow the administrator to retrieve and set tunable values related to the I/O scheduler.
  3. Change the memory limit for the I/O scheduler. # echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb
  4. Confirm you successfully updated the memory limit. # cat /sys/module/usbcore/parameters/usbfs_memory_mb

To optimize your platform to store recorded data

  1. On the DRIVE PX 2 console: # echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb
  2. Disable journaling in the USB disk: tune2fs -O ^has_journal /dev/sdXN This destroys the journal on the USB drive permanently. To restore it: # mkfs.ext4 /dev/sdX1
  3. Make the following changes in the /etc/sysctl.conf file in DRIVE PX 2 to maximize network and USB performance. These are persistent changes and are preserved between reboots. net.core.rmem_default = 1048576 net.core.rmem_max = 10485760 net.core.wmem_default = 1048576 net.core.wmem_max = 10485760 net.core.netdev_max_backlog = 30000 net.ipv4.ipfrag_high_thresh = 8388608 net.ipv6.conf.all.disable_ipv6 = 1 vm.dirty_background_ratio = 5 vm.dirty_ratio = 80
  4. Check the IP in /etc/tacp/tacp.cfg and give the same IP from the AURIX console using: # setip a 10.42.0.83 for aurix IP set
  5. Check setip ta and setip tb in same way.

How can I ensure the USB disk speed is adequate to record my sensors?
The required write speed of the drives depends on the number of sensors and the recording format. If you are recording RAW data, calculate the required drive speed as:
width * height * 2 * fps = bytes/second
RAW recording consumes most of the bandwidth.

To determine the write speed of the mounted jbod disk, enter:

dd if=/dev/zero of=/media/NVidia/folder#/file bs=1G count=10

JBod is mounted on:
/media/NVidia/

How do I modify my JSON config file for distributed recording?

You can simultaneously record sensors on both Tegra processors on a platform. This can be helpful when your cameras generate more data than can be recorded with a single USB storage device. In these instructions, Tegra A is the master and Tegra B is the slave.

Step 1: Set up Tegra A

  1. Follow Step 1: Verify the Sensors Are Collecting Data in the “Data Acquisition” chapter of DriveWorks Development Guide.
  2. Create a new master configuration file at: driveworks/tools/recorder-qt-config_TA1.json
  3. Populate the new file with the following content. Do not change the setting for recorder-config-tb1.json on the master at /home/Nvidia.
    "vin": "your_vin"
    "record-group": {
      "name": "TA1",
      "master": true,
    
      "peer-info": [
        {
        "name": "TB1",
        "configurations": [
          {"name": "your_prod","path":"/home/nvidia/recorder-config-tb1.json"}
          ]
        }
      ]
    },
    
    "pre-requisite-script":
          "${APP_DIR}/scripts/bash/default-pre-requisite-script.sh",
        "pre-record-script":
          "${APP_DIR}/scripts/bash/default-pre-record-script.sh",
        "post-record-script":
          "${APP_DIR}/scripts/bash/default-post-record-script.sh",
    
        "configurations": [
            {"name":"your_prod",
             "path":"${CONFIG_FILE_DIR}/recorder-config.json"}
        ]
    }
    

    record-group specifies to simultaneously start recorder on multiple devices.

    peer-info specifies to pass the JSON configuration files to the named slaves. This applies only when “master” is true.

  4. Attach USB storage.

Step 2: Set up Tegra B

  1. Follow Step 1: Verify the Sensors Are Collecting Data in the “Data Acquisition” chapter of DriveWorks Development Guide.
  2. Create a new slave configuration file at: driveworks/tools/recorder-qt-config_TB1.json
  3. Populate the new configuration file with the following content.
    "vin": "your_vin"
    "record-group": {
      "name": "TB1",
      "master": false
             
    },
    
      "pre-requisite-script": 
        "${APP_DIR}/scripts/bash/default-pre-requisite-script.sh",
        "pre-record-script": 
          "${APP_DIR}/scripts/bash/default-pre-record-script.sh",
        "post-record-script":
          "${APP_DIR}/scripts/bash/default-post-record-script.sh"
       
    }
    
  4. Ensure the file does not contain a configurations block. That block begins with:
    configurations":[  ......      ]
    
  5. Attach USB storage to Tegra B.

Step 3: Synchronize and then run

  1. Synchronize the Tegra clocks as described in Synchronizing system clocks in these release notes.
  2. On Tegra A, start the recording application, specifying the corresponding configuration file: # sudo tools/recorder-qt --config-file=./recorder-qt-config_TA1.json

    Recording is automatically started on Tegra B.