GPU usage monitoring on TX1

Dear all,

Any hints on how can I get some stats on the GPU usage (similar to nvidia-smi) on the TX1 ?
Would love to have some nice stats to benchmark some neural simulations.

Cheers,
Cristian

Try running ~/tegrastats and look for GR3D:

sudo ~/tegrastats
[sudo] password for ubuntu:
RAM 277/3854MB (lfb 700x4MB) SWAP 0/0MB (cached 0MB) cpu [0%,0%,0%,0%]@1912 EMC 0%@1600 AVP 5%@115 VDE 0 GR3D 0%@76 EDP limit 1912
RAM 277/3854MB (lfb 700x4MB) SWAP 0/0MB (cached 0MB) cpu [1%,0%,0%,1%]@102 EMC 1%@408 AVP 33%@12 VDE 0 GR3D 0%@76 EDP limit 1912

Excellent, thanks Dustin !

The last columns are suppressed if not running as super user. Briefly, what are the AVP, VDE, and GR3D fields ?

Thanks,
Cristian

Don’t bother, found it in a previous forum post :)

EMC – memory controller
AVP – audio/video processor
VDE – video decoder engine
GR3D – GPU

Cheers,
Cristian

I’m also interested in monitoring the GPU usage on the TX1.
How should I interpret the GR3D field? Its in the form xx%@yy.
I assume xx is the percentage of GPU load, but what does yy tell?

RAM 1160/3854MB (lfb 489x4MB) SWAP 0/0MB (cached 0MB) cpu [55%,26%,27%,27%]@1734 EMC 39%@1600 AVP 0%@408 VDE 0 GR3D 65%@921 EDP limit 1734
RAM 1161/3854MB (lfb 489x4MB) SWAP 0/0MB (cached 0MB) cpu [45%,24%,29%,26%]@1555 EMC 40%@1600 AVP 0%@408 VDE 0 GR3D 99%@921 EDP limit 1734
RAM 1160/3854MB (lfb 487x4MB) SWAP 0/0MB (cached 0MB) cpu [29%,12%,14%,8%]@710 EMC 3%@665 AVP 4%@12 VDE 0 GR3D 0%@76 EDP limit 1734

Thanks,
Nick

These are linked to devices frequencies. You can get the details of it with

sudo ~/jetson_clocks.sh --show

tegrastats shows GPU load percentage and total clock rate, but how do I see the GPU memory utilization, or how do I monitor the GPU memory?

On Jetson, the same memory is shared by CPUs and GPU.

Of course, my bad. I wanted info about the Drive PX2 I’m working on, which has Tegra X2s, and discrete GPUs. The s/w package for it uses the same tooling as the Jetson, including tegrastats.

Hi ,

  1. Me trying to get the performance numbers and GPU utilization for H.265 encoding using hardware accelerators on Tegra Tx2 Board.

    1.a) I am getting MSENC number how to related % utilization of hardware accelerators ? below is the snap of output.
    1.b) Command for encoding {./video_encode COSTARICA4k.yuv 3840 2160 H265 out1.h265 -rc vbr -br 10000000 -fps 30 1}
    1.c) Command for checking resource utilization.

root@tegra-ubuntu:~# ./tegrastats
RAM 3548/7851MB (lfb 1x2MB) cpu [0%@1575,off,off,0%@1574,0%@1574,0%@1572] EMC 20%@1600 APE 150 MSENC 1113 GR3D 0%@114
RAM 3518/7851MB (lfb 1x2MB) cpu [16%@653,off,off,33%@652,58%@652,13%@654] EMC 27%@1062 APE 150 MSENC 1113 GR3D 0%@318
RAM 3488/7851MB (lfb 1x2MB) cpu [32%@806,off,off,22%@806,43%@806,34%@805] EMC 27%@1062 APE 150 MSENC 1113 GR3D 0%@114
RAM 3462/7851MB (lfb 1x1MB) cpu [47%@1114,off,off,25%@1113,32%@1113,54%@1113] EMC 22%@1331 APE 150 MSENC 1113 GR3D 30%@114
RAM 3445/7851MB (lfb 4x256kB) cpu [27%@806,off,off,49%@806,15%@806,31%@806] EMC 24%@1331 APE 150 MSENC 1113 GR3D 0%@420
RAM 3440/7851MB (lfb 4x256kB) cpu [19%@1153,off,off,54%@1153,4%@1156,19%@1157] EMC 23%@1600 APE 150 MSENC 1113 GR3D 0%@114
RAM 3441/7851MB (lfb 4x256kB) cpu [25%@1581,off,off,21%@1573,7%@1574,37%@1575] EMC 25%@1600 APE 150 MSENC 1113 GR3D 20%@114
RAM 3411/7851MB (lfb 5x256kB) cpu [36%@806,off,off,35%@806,32%@806,44%@806] EMC 24%@1331 APE 150 MSENC 1113 GR3D 24%@114
RAM 3402/7851MB (lfb 3x256kB) cpu [35%@652,off,off,34%@653,35%@652,31%@652] EMC 24%@1331 APE 150 MSENC 1113 GR3D 0%@420
RAM 3395/7851MB (lfb 3x256kB) cpu [35%@652,off,off,28%@805,46%@806,37%@806] EMC 24%@1331 APE 150 MSENC 1113 GR3D 24%@114

  1. In the sample example code , Their is two encoders one is "01_video_encode" and other is 03_video_cuda_enc?
    2.a) what is the difference ?
    2.b) which one uses cuda core and hardware accelerators (NVENC)?
    {both code looks similar just 01_video_encode has more option for configuration }

  2. MSENC for h.264 is showing MSENC only ? which means hardware accelerators for H.264 is MSENC (dedicated hardware chip)

Discussing at https://devtalk.nvidia.com/default/topic/1024771/tx2-with-jetpack-3-1-gpu-utilization-issue-for-video-encoding-msenc-nvenc/