[MMAPI R28.2/R28.1] deinitPlane() of NvVideoEncoder -- Memory Leak ?

I’m developing a video streaming system with MMAPI both R28.2 and R28.1.

(1) Server: CAMERA → V4L2 → NvVideoConverter → NvVideoEncoder → RTP Packetise / Socket → Network
(2) Client: Network → Socket / RTP-NAL reassemble → NvVideoDecoder → NvVideoConverter → EglRenderer

Sometimes the H.264 encoder (streaming server) crashes with these messages.

===== MSENC blits (mode: 1) into tiled surfaces =====
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 497, SyncPointValue = 0)
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 496, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 497, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 496, SyncPointValue = 0)

Duaring this analysis, I found that the available memory of JetsonTX2 decrease.
Then I make a test code that de-allocate and re-allocate buffers every a few seconds.

Log of tegrastats:

nvidia@tegra-ubuntu:~$ sudo ./tegrastats 
RAM 1895/7851MB (lfb 963x4MB) cpu [0%@2022,0%@2035,0%@2033,0%@2022,0%@2027,0%@2027]
RAM 1895/7851MB (lfb 963x4MB) cpu [0%@2003,0%@2035,0%@2036,0%@2019,0%@2019,0%@2020]
RAM 1895/7851MB (lfb 963x4MB) cpu [0%@2035,0%@2035,0%@2035,1%@2037,0%@2037,0%@2034]
RAM 1996/7851MB (lfb 964x4MB) cpu [0%@2031,17%@2035,0%@2034,1%@2041,0%@2038,0%@2034]
RAM 2008/7851MB (lfb 964x4MB) cpu [0%@2019,2%@2035,0%@2035,0%@2035,0%@2032,0%@2034]
RAM 2011/7851MB (lfb 964x4MB) cpu [1%@2034,39%@2034,1%@2035,0%@2035,4%@2034,35%@2036]
RAM 2011/7851MB (lfb 964x4MB) cpu [3%@2035,17%@2035,0%@2035,1%@2035,1%@2035,56%@2033]
RAM 2011/7851MB (lfb 964x4MB) cpu [2%@1995,17%@2034,1%@2036,2%@1995,2%@1997,25%@1998]
RAM 2013/7851MB (lfb 964x4MB) cpu [3%@2008,32%@2035,6%@2034,3%@2025,4%@2025,72%@2029]
RAM 2014/7851MB (lfb 964x4MB) cpu [2%@2010,20%@2035,8%@2034,1%@2019,1%@2027,52%@2013]
RAM 2018/7851MB (lfb 964x4MB) cpu [1%@2013,36%@2035,3%@2035,1%@2025,2%@2016,33%@2025]
RAM 2019/7851MB (lfb 964x4MB) cpu [1%@2035,19%@2035,6%@2035,0%@2034,1%@2036,38%@2034]
RAM 2019/7851MB (lfb 964x4MB) cpu [1%@2009,26%@2035,8%@2034,5%@2016,1%@2029,25%@2030]
RAM 2019/7851MB (lfb 964x4MB) cpu [0%@2019,16%@2035,0%@2035,2%@2027,1%@2028,11%@2028]
RAM 2019/7851MB (lfb 964x4MB) cpu [1%@2009,18%@2035,0%@2035,1%@2026,0%@2025,38%@2029]
RAM 2019/7851MB (lfb 964x4MB) cpu [0%@2010,28%@2034,0%@2034,1%@2023,1%@2031,26%@2029]
RAM 2019/7851MB (lfb 964x4MB) cpu [1%@2035,12%@2035,0%@2035,2%@2034,0%@2035,10%@2033]
RAM 2019/7851MB (lfb 964x4MB) cpu [0%@2019,14%@2034,2%@2036,0%@2028,0%@2027,11%@2024]
RAM 2019/7851MB (lfb 964x4MB) cpu [0%@1993,13%@2035,0%@2035,0%@1996,0%@1996,33%@1996]
RAM 2020/7851MB (lfb 964x4MB) cpu [1%@1992,28%@2035,1%@2035,1%@1996,1%@1996,26%@1994]
RAM 2020/7851MB (lfb 964x4MB) cpu [0%@2008,18%@2035,2%@2034,1%@2020,0%@2023,20%@2030]
RAM 2020/7851MB (lfb 964x4MB) cpu [1%@2035,18%@2036,0%@2036,0%@2034,1%@2035,46%@2035]
RAM 2020/7851MB (lfb 964x4MB) cpu [2%@2020,33%@2035,3%@2034,2%@2025,1%@2024,15%@2027]
RAM 2020/7851MB (lfb 964x4MB) cpu [1%@2033,20%@2034,3%@2035,0%@2024,2%@2023,11%@2030]
RAM 2022/7851MB (lfb 964x4MB) cpu [2%@2017,30%@2035,3%@2036,2%@2027,1%@2021,33%@2016]
RAM 2022/7851MB (lfb 964x4MB) cpu [0%@2017,29%@2036,7%@2034,2%@2023,2%@2022,21%@2025]
RAM 4604/7851MB (lfb 609x4MB) cpu [0%@2012,16%@2036,0%@2037,0%@2063,0%@2034,20%@2035]
RAM 4604/7851MB (lfb 609x4MB) cpu [1%@2020,36%@2035,2%@2035,1%@2020,0%@2024,15%@2025]
RAM 4605/7851MB (lfb 609x4MB) cpu [0%@2034,14%@2034,0%@2034,2%@2034,1%@2035,27%@2035]
RAM 4605/7851MB (lfb 609x4MB) cpu [0%@2019,12%@2035,2%@2033,1%@2024,1%@2020,0%@2021]
RAM 4605/7851MB (lfb 609x4MB) cpu [0%@2018,35%@2034,2%@2036,2%@2024,2%@2019,15%@2020]
RAM 4604/7851MB (lfb 609x4MB) cpu [1%@2008,17%@2035,1%@2034,1%@2021,1%@2023,20%@2026]
RAM 4606/7851MB (lfb 609x4MB) cpu [0%@2034,17%@2035,3%@2035,1%@2034,1%@2034,20%@2035]
RAM 4733/7851MB (lfb 577x4MB) cpu [1%@1992,42%@2034,6%@2035,1%@1996,3%@1996,15%@1996]
RAM 4733/7851MB (lfb 577x4MB) cpu [0%@2020,18%@2034,4%@2035,2%@2032,0%@2020,20%@2026]
RAM 4734/7851MB (lfb 576x4MB) cpu [3%@2014,35%@2034,2%@2035,1%@2025,2%@2030,21%@2027]
RAM 4734/7851MB (lfb 576x4MB) cpu [1%@2035,16%@2035,0%@2034,1%@2035,0%@2034,0%@2034]
RAM 4734/7851MB (lfb 576x4MB) cpu [2%@2002,35%@2035,0%@2035,2%@1996,2%@1997,11%@1997]
RAM 4734/7851MB (lfb 576x4MB) cpu [1%@2022,13%@2035,1%@2036,2%@2024,0%@2027,11%@2033]
RAM 4734/7851MB (lfb 576x4MB) cpu [0%@2035,13%@2035,0%@2036,0%@2035,0%@2035,50%@2035]
RAM 4734/7851MB (lfb 576x4MB) cpu [1%@2034,16%@2036,0%@2034,1%@2035,0%@2034,27%@2034]
RAM 4735/7851MB (lfb 576x4MB) cpu [0%@2006,38%@2035,0%@2034,2%@2016,1%@2027,20%@2024]
RAM 4735/7851MB (lfb 576x4MB) cpu [0%@2006,12%@2034,2%@2035,1%@2026,2%@2023,27%@2023]
RAM 4735/7851MB (lfb 576x4MB) cpu [1%@2035,12%@2033,1%@2035,1%@2035,1%@2034,0%@2034]
RAM 4735/7851MB (lfb 576x4MB) cpu [1%@2036,36%@2034,2%@2034,1%@2035,0%@2035,21%@2035]
RAM 4735/7851MB (lfb 576x4MB) cpu [1%@2007,15%@2036,2%@2035,2%@2023,1%@2022,52%@2032]
RAM 4735/7851MB (lfb 576x4MB) cpu [1%@2017,17%@2035,3%@2036,0%@2029,2%@2020,33%@2033]
RAM 4863/7851MB (lfb 545x4MB) cpu [2%@2018,43%@2034,4%@2036,3%@2019,1%@2024,29%@2026]
RAM 4863/7851MB (lfb 545x4MB) cpu [2%@2010,22%@2035,7%@2036,1%@2020,4%@2029,41%@2009]
RAM 4863/7851MB (lfb 544x4MB) cpu [1%@2015,39%@2034,3%@2035,1%@2034,3%@2027,28%@2029]
RAM 4863/7851MB (lfb 544x4MB) cpu [2%@2034,20%@2035,0%@2034,0%@2035,0%@2035,18%@2034]
RAM 4864/7851MB (lfb 544x4MB) cpu [0%@2035,25%@2034,0%@2035,1%@2036,1%@2034,21%@2036]
RAM 4864/7851MB (lfb 544x4MB) cpu [1%@2008,20%@2035,0%@2036,0%@2027,2%@2025,20%@2024]
RAM 4864/7851MB (lfb 544x4MB) cpu [1%@2035,20%@2034,1%@2033,3%@2034,1%@2035,20%@2035]
RAM 4864/7851MB (lfb 544x4MB) cpu [1%@2034,21%@2034,0%@2036,0%@2035,0%@2034,20%@2035]
RAM 4864/7851MB (lfb 544x4MB) cpu [1%@1989,36%@2034,0%@2036,1%@1990,1%@1996,20%@1993] 
RAM 4864/7851MB (lfb 544x4MB) cpu [1%@1990,19%@2036,1%@2035,4%@1995,2%@1996,33%@1997]
RAM 4866/7851MB (lfb 544x4MB) cpu [1%@2035,23%@2033,2%@2035,0%@2035,2%@2035,16%@2035]
RAM 7329/7851MB (lfb 15x4MB) cpu [3%@1988,22%@2033,1%@2035,1%@1991,1%@1995,33%@1995]
RAM 7329/7851MB (lfb 15x4MB) cpu [1%@2003,45%@2036,4%@2035,2%@2020,0%@2017,17%@2019]
RAM 7329/7851MB (lfb 15x4MB) cpu [0%@2037,20%@2034,5%@2035,1%@2035,0%@2035,50%@2034]
RAM 7329/7851MB (lfb 15x4MB) cpu [0%@2005,20%@2033,3%@2036,1%@2041,2%@2013,27%@2028]
RAM 7454/7851MB (lfb 15x4MB) cpu [6%@2034,50%@2035,5%@2034,5%@2034,1%@2035,12%@2036]
RAM 7453/7851MB (lfb 15x4MB) cpu [0%@1990,18%@2034,8%@2036,1%@1992,3%@1995,53%@1995]
RAM 7453/7851MB (lfb 15x4MB) cpu [1%@1989,40%@2034,2%@2034,3%@1994,3%@1996,17%@1995]
RAM 7454/7851MB (lfb 15x4MB) cpu [0%@1988,17%@2036,0%@2035,1%@1993,2%@1995,33%@1994]
RAM 7454/7851MB (lfb 15x4MB) cpu [1%@2003,41%@2036,0%@2035,0%@2015,2%@2021,0%@2020]
RAM 7454/7851MB (lfb 15x4MB) cpu [1%@2013,12%@2034,1%@2035,1%@2027,0%@2024,27%@2022]
RAM 7453/7851MB (lfb 15x4MB) cpu [0%@1995,12%@2035,1%@2035,1%@1991,1%@1989,27%@1990]
RAM 7453/7851MB (lfb 15x4MB) cpu [0%@2020,12%@2035,1%@2034,0%@2021,1%@2023,38%@2019]
RAM 7457/7851MB (lfb 15x4MB) cpu [1%@2035,36%@2034,1%@2036,1%@2034,0%@2035,32%@2036]
RAM 7454/7851MB (lfb 15x4MB) cpu [0%@2021,19%@2036,2%@2034,1%@2017,1%@2018,23%@2023]
RAM 7454/7851MB (lfb 15x4MB) cpu [1%@2019,17%@2035,0%@2035,0%@2017,0%@2022,20%@2022]
RAM 7454/7851MB (lfb 15x4MB) cpu [1%@2006,43%@2035,1%@2035,0%@2010,0%@2015,5%@2014]
RAM 7454/7851MB (lfb 15x4MB) cpu [0%@2035,18%@2035,4%@2034,1%@2034,0%@2035,20%@2035]
RAM 7454/7851MB (lfb 15x4MB) cpu [1%@2006,23%@2036,3%@2035,3%@2022,1%@2020,33%@2017]
RAM 7582/7851MB (lfb 15x4MB) cpu [1%@2033,67%@2035,8%@2035,24%@2034,18%@2035,54%@2035]
RAM 6114/7851MB (lfb 67x4MB) cpu [4%@2018,26%@2035,37%@2033,10%@2020,5%@2015,5%@2028]
RAM 1878/7851MB (lfb 762x4MB) cpu [0%@1994,24%@2035,7%@2035,3%@1995,0%@1997,0%@1995]
RAM 1877/7851MB (lfb 762x4MB) cpu [0%@1995,0%@2035,0%@2035,0%@1995,0%@1997,0%@1996]
RAM 1877/7851MB (lfb 762x4MB) cpu [0%@2011,1%@2037,0%@2034,0%@2024,0%@2019,0%@2023]

And I added system calls at some points of the streaming server.

  • send EOS to NvVideoCovnerter
  • waitForDQThread() of NvVideoConverter
  • deinitPlane() of NvVideoConverter capture buffer
  • deinitPlane() of NvVideoEncoder output and capture planes
  • setPlaneFormat() and setupPlane() of each planes
  • setStreamStatus() and startDQThread() of each planes
system("grep MemAvailable /proc/meminfo");

After setupPlane() that allocates buffers,
the available memory decreases.
It is nomaly operation, I think.

But after deinitPlane() that de-allocate buffers,
the available memory does not increase enough to thought.
I suspect there is something wrong.
Have I got the wrong idea about it ?

Log of my application:

1524809650.417699: [V4L2][ChgRes] process_image(): Change Resolution 18075371 bps, [1920]x[1080]->[3840]x[2160], Duration=[75]
MemAvailable:    5964644 kB
1524809650.430521: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809650.468605: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809650.468616: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5964784 kB
1524809650.496326: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5964708 kB
1524809650.509185: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809650.509236: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5964876 kB
1524809650.523794: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809650.523845: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809650.524143: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5964984 kB
1524809650.537976: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809650.538073: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5964984 kB
1524809650.552744: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809650.558060: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809650.558115: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5921572 kB
1524809650.585905: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5921296 kB
1524809650.599006: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5921108 kB
1524809650.618844: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5921216 kB
1524809650.632098: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5921044 kB
1524809650.644385: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809652.584369: [V4L2][ChgRes] process_image(): Change Resolution 1000000 bps, [3840]x[2160]->[640]x[360], Duration=[61]
MemAvailable:    5864548 kB
1524809652.597334: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809652.636407: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809652.636418: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5864052 kB
1524809652.663784: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5864052 kB
1524809652.676730: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
MemAvailable:    5864144 kB
1524809652.693052: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809652.693082: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809652.693341: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5864316 kB
1524809652.709407: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809652.709513: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5864532 kB
1524809652.724176: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809652.725505: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809652.725534: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5864532 kB
1524809652.739109: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5864532 kB
1524809652.752073: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5864312 kB
1524809652.772865: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5864372 kB
1524809652.787706: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5864684 kB
1524809652.799951: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809655.017734: [V4L2][ChgRes] process_image(): Change Resolution 1801148 bps, [640]x[360]->[960]x[540], Duration=[69]
MemAvailable:    5864096 kB
1524809655.030680: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809655.068666: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809655.068678: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5863832 kB
1524809655.095820: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5864176 kB
1524809655.109007: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809655.109064: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5863956 kB
1524809655.122469: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809655.122561: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809655.122847: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5864160 kB
1524809655.136117: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809655.136235: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5864040 kB
1524809655.149760: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809655.151010: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809655.151037: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5864212 kB
1524809655.164750: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5863972 kB
1524809655.177897: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5864268 kB
1524809655.203204: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5863724 kB
1524809655.216702: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5863848 kB
1524809655.229266: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809659.151167: [V4L2][ChgRes] process_image(): Change Resolution 4828227 bps, [960]x[540]->[1280]x[720], Duration=[120]
MemAvailable:    5863492 kB
1524809659.164120: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809659.202181: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809659.202191: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5863836 kB
1524809659.230212: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5863636 kB
1524809659.243157: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809659.243213: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5863636 kB
1524809659.256870: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809659.256921: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809659.257193: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5863668 kB
1524809659.270714: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809659.270786: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5863636 kB
1524809659.284785: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809659.286330: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809659.286353: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5863652 kB
1524809659.301134: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5863652 kB
1524809659.314273: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5863568 kB
1524809659.334565: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5863572 kB
1524809659.347808: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5863572 kB
1524809659.360777: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809662.017865: [V4L2][ChgRes] process_image(): Change Resolution 9643861 bps, [1280]x[720]->[1920]x[1080], Duration=[82]
MemAvailable:    5863352 kB
1524809662.030807: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809662.068826: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809662.068837: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5863372 kB
1524809662.096544: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5863388 kB
1524809662.109926: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809662.109957: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5863212 kB
1524809662.123704: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809662.123755: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809662.124026: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5863400 kB
1524809662.137643: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809662.137737: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5863196 kB
1524809662.152051: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809662.154068: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809662.154094: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5863500 kB
1524809662.169763: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5863212 kB
1524809662.182943: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5863236 kB
1524809662.204030: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5862988 kB
1524809662.217181: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5863160 kB
1524809662.229891: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809664.651273: [V4L2][ChgRes] process_image(): Change Resolution 18075371 bps, [1920]x[1080]->[3840]x[2160], Duration=[75]
MemAvailable:    5862468 kB
1524809664.664360: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809664.702436: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809664.702448: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5862624 kB
1524809664.729534: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5862548 kB
1524809664.742686: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809664.742740: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5862720 kB
1524809664.757085: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809664.757136: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809664.757432: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5862560 kB
1524809664.771773: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809664.771872: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5862748 kB
1524809664.786939: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809664.792237: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809664.792289: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5786504 kB
1524809664.824995: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5786196 kB
1524809664.838088: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5786196 kB
1524809664.859978: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5786160 kB
1524809664.873401: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5786100 kB
1524809664.885992: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809666.817943: [V4L2][ChgRes] process_image(): Change Resolution 1000000 bps, [3840]x[2160]->[640]x[360], Duration=[60]
MemAvailable:    5730068 kB
1524809666.831200: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809666.869235: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809666.869247: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5730180 kB
1524809666.902029: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5729956 kB
1524809666.917649: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809666.917701: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5730200 kB
1524809666.934919: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809666.934973: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809666.935297: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5730236 kB
1524809666.951884: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809666.951992: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5730200 kB
1524809666.966792: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809666.968084: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809666.968110: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5730152 kB
1524809666.981650: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5730156 kB
1524809666.995128: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5730312 kB
1524809667.015636: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5730140 kB
1524809667.029292: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5730088 kB
1524809667.042056: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809669.284662: [V4L2][ChgRes] process_image(): Change Resolution 1801148 bps, [640]x[360]->[960]x[540], Duration=[70]
MemAvailable:    5729744 kB
1524809669.297852: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809669.336880: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809669.336892: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5729468 kB
1524809669.363814: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5729464 kB
1524809669.377165: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809669.377217: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5729620 kB
1524809669.391308: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809669.391346: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809669.391709: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5729588 kB
1524809669.405128: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809669.405226: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5729652 kB
1524809669.418642: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809669.419974: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809669.420001: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5729556 kB
1524809669.434001: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5729728 kB
1524809669.447400: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5729452 kB
1524809669.468310: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5729696 kB
1524809669.481856: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5729476 kB
1524809669.494872: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809673.384734: [V4L2][ChgRes] process_image(): Change Resolution 4828227 bps, [960]x[540]->[1280]x[720], Duration=[119]
MemAvailable:    5729236 kB
1524809673.397805: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809673.436875: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809673.436885: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5729244 kB
1524809673.464037: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5728988 kB
1524809673.477823: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809673.477845: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5728928 kB
1524809673.493131: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809673.493152: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809673.493437: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5729116 kB
1524809673.507639: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809673.507735: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5729180 kB
1524809673.521234: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809673.522900: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809673.522941: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5728880 kB
1524809673.537458: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5729084 kB
1524809673.550901: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5729040 kB
1524809673.572059: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5728988 kB
1524809673.585569: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5728864 kB
1524809673.598823: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809676.284791: [V4L2][ChgRes] process_image(): Change Resolution 9643861 bps, [1280]x[720]->[1920]x[1080], Duration=[83]
MemAvailable:    5728844 kB
1524809676.298293: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809676.336284: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809676.336295: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5728876 kB
1524809676.363731: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5728892 kB
1524809676.377040: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809676.377093: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5728668 kB
1524809676.391354: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809676.391396: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809676.391763: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5728856 kB
1524809676.405941: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809676.406037: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5728752 kB
1524809676.419964: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809676.422055: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809676.422102: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5728656 kB
1524809676.437723: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5728656 kB
1524809676.451066: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5728780 kB
1524809676.471997: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5728796 kB
1524809676.485546: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5728620 kB
1524809676.499110: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809678.918149: [V4L2][ChgRes] process_image(): Change Resolution 18075371 bps, [1920]x[1080]->[3840]x[2160], Duration=[75]
MemAvailable:    5728688 kB
1524809678.931557: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809678.969682: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809678.969691: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5728316 kB
1524809678.996884: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5728196 kB
1524809679.010493: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809679.010521: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5728176 kB
1524809679.026368: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809679.026396: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809679.026661: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5728456 kB
1524809679.041691: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809679.041793: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5728492 kB
1524809679.057104: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809679.062190: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809679.062240: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5652900 kB
1524809679.095171: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5652372 kB
1524809679.109318: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5652148 kB
1524809679.146781: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5652164 kB
1524809679.160079: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809681.051521: [V4L2][ChgRes] process_image(): Change Resolution 1000000 bps, [3840]x[2160]->[640]x[360], Duration=[59]
MemAvailable:    5595468 kB
1524809681.064776: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809681.102779: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809681.102790: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5595488 kB
1524809681.130996: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5595456 kB
1524809681.144603: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809681.144652: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5595364 kB
1524809681.161142: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809681.161193: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809681.161480: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5595828 kB
1524809681.178284: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809681.178387: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5596000 kB
1524809681.193589: [V4L2][ChgRes] change_resolution(): (8) Reconfigure Converter res=[0]
1524809681.194814: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809681.194843: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5595840 kB
1524809681.209020: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5596060 kB
1524809681.222715: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5595596 kB
1524809681.245372: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5595700 kB
1524809681.259324: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5595808 kB
1524809681.272482: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

-----------------------------------------------------------------------

1524809683.518233: [V4L2][ChgRes] process_image(): Change Resolution 1801148 bps, [640]x[360]->[960]x[540], Duration=[70]
MemAvailable:    5595572 kB
1524809683.531957: [V4L2][ChgRes] change_resolution(): (1) pause converter loop
1524809683.568886: [V4L2][ChgRes] change_resolution(): (2) Pause Converter res=[0]
1524809683.568898: [V4L2][ChgRes] change_resolution(): (3) pause encoder loop
MemAvailable:    5595716 kB
1524809683.597147: [V4L2][ChgRes] change_resolution(): (4) Pause Encoder res=[0]
MemAvailable:    5595732 kB
1524809683.610943: [V4L2][ChgRes] change_resolution(): (5) Clear inBufferPool res=[0]
1524809683.610977: [CNV] freeing_converter(): capture_plane.deinitPlane()
MemAvailable:    5595592 kB
1524809683.625212: [V4L2][ChgRes] change_resolution(): (6) Freeing-up Converter res=[0]
1524809683.625267: [ENC] freeing_encoder(): output_plane.deinitPlane()
1524809683.625613: [ENC] freeing_encoder(): capture_plane.deinitPlane()
MemAvailable:    5595400 kB
1524809683.639932: [V4L2][ChgRes] change_resolution(): (7) Freeing-up Encoder res=[0]
1524809683.640031: [CNV] reconfigure_converter(): capture_plane.setupPlane(MMAP, 4, false, false)
libv4l2_nvvidconv (0):(808) (INFO) : Allocating (4) CAPTURE PLANE BUFFERS Layout=0
MemAvailable:    5595560 kB
1524809683.654941: [ENC] reconfigure_encoder(): output_plane.setupPlane(DMABUF, 10, false, false)
1524809683.654968: [ENC] reconfigure_encoder(): capture_plane.setupPlane(MMAP, 10, true, false)
MemAvailable:    5595372 kB
1524809683.669318: [V4L2][ChgRes] change_resolution(): (9) Reconfigure Encoder res=[0]
MemAvailable:    5595700 kB
1524809683.682996: [V4L2][ChgRes] change_resolution(): (10) Reconfigure inBufferPool res=[0]
MemAvailable:    5595352 kB
1524809683.704216: [V4L2][ChgRes] change_resolution(): (11) Restart Encoder res=[0]
MemAvailable:    5595444 kB
1524809683.725175: [V4L2][ChgRes] change_resolution(): (12) Restart Converter res=[0]
MemAvailable:    5595292 kB
1524809683.738467: [V4L2][ChgRes] change_resolution(): (13) re-start converter/encoder loop

Best Regards

Hi Mynaemi,
===== MSENC blits (mode: 1) into tiled surfaces =====
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 497, SyncPointValue = 0)
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 496, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 497, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 496, SyncPointValue = 0)

Above error are mostly caused by you are submitting a dmabuf fd which has been destroyed.

For dmabuf memory leakage issue,
You can check following sysnode:
root@tegra-ubuntu:/sys/kernel/debug/nvmap/iovmm# cat all_allocations
BASE SIZE USERFLAGS REFS KMAPS UMAPS SHARE UID
0 8640K 10000001 2 0 0 1 ffffffc227498b00
0 4096K 10000003 3 0 1 1 ffffffc227498d00
0 4096K 10000003 3 0 1 1 ffffffc227499000

Hi waynezhu,

I appreciate your support.

I cannot read and understand these information.
Should I compare them before and after execution of my application ?

If it has been increased, is that a memory leak (non-deallocated memory) ?
Then what field do I check, USERFLAGS, UID, or … ?

Best Regards

Yes, just compare them before and after execution of my application.

Or you can upload the result before and after execution, I can take a look.

Hi waynezhu,

Thanks a lot for your quick response.

Before execution and after termination by error, the information of all_allocations were same.

all_allocations before termination

BASE        SIZE USERFLAGS   REFS  KMAPS  UMAPS  SHARE      UID
       0       9600K  10000001      3      0      0      1 ffffffc0738c1500
       0      10240K  10000001      2      0      0      1 ffffffc07a323a00
       0         64K  10000001      2      0      0      1 ffffffc07ac89000
       0         64K  10000001      2      0      0      1 ffffffc07ac89200
       0       8192K  10000001      3      0      1      1 ffffffc07ac89a00
       0       1048K  10000003      3      0      1      1 ffffffc07ad4e500
       0          4K  10000001      4      1      1      1 ffffffc07ad4e800
       0         64K  10000001      3      0      1      1 ffffffc07ad4e900
       0       2432K  10000001      2      0      0      1 ffffffc07b127100
       0        128K  10000001      2      0      0      1 ffffffc07b127d00
       0         64K  10000001      2      0      0      1 ffffffc07b22fc00
       0       4096K  10000001      2      0      0      1 ffffffc1cc3baf00
       0      10240K  10000001      2      0      0      1 ffffffc1cc3dbe00
       0        256K  10000001      2      0      0      1 ffffffc1cc3dbf00
       0         64K  10000001      2      0      0      1 ffffffc1ce282a00
       0        256K  10000001      3      0      1      1 ffffffc1d3e0d000
       0       9600K  10000001      2      0      0      1 ffffffc1d3e0d100
       0        384K  10000001      2      0      0      1 ffffffc1d3e0d600
       0          4K  10000001      2      0      0      1 ffffffc1d3e0df00
       0       1024K  10000001      2      0      0      1 ffffffc1d71d4200
       0      10240K  10000001      2      0      0      1 ffffffc1d71d4600
       0          4K  10000003      4      1      1      1 ffffffc1d73c0700
       0         84K  10000001      3      0      1      1 ffffffc1d73c0d00
       0          4K  10000003      3      1      1      1 ffffffc1e2448100
       0         64K  10000001      2      0      0      1 ffffffc1e24f2c00
       0        256K  10000003      3      1      1      1 ffffffc1e45e8700
       0         64K  10000001      3      0      1      1 ffffffc1e4696100
       0          4K  10000001      2      0      0      1 ffffffc1e4696f00
       0        128K  10000001      2      0      0      1 ffffffc1e5b14300

Then I inserted it into my code.

system("sudo cat /sys/kernel/debug/nvmap/iovmm/all_allocations");

Attached ZIP file includes logs below:

  • Program is started
  • 1st Allocate Buffer and Start Converter/Encoder
  • 1st Stop Converter/Encoder and De-allocate Buffer
    —> 20180517_deinit_1.txt
  • 2nd Allocate Buffer and Start Converter/Encoder
    —> 20180517_deinit_2.txt
  • 2nd Stop Converter/Encoder and De-allocate Buffer
    —> 20180517_deinit_3.txt
  • 3rd Allocate Buffer and Start Converter/Encoder
    —> 20180517_deinit_4.txt
  • 3rd Stop Converter/Encoder and De-allocate Buffer
  • 4th Allocate Buffer and Start Converter/Encoder
  • 5th Stop Converter/Encoder and De-allocate Buffer
    : : : : : : : : : : :
  • 232th Stop Converter/Encoder and De-allocate Buffer
  • 233th Allocate Buffer and Start Converter/Encoder
    —> 20180517_deinit_5.txt
  • 233th Stop Converter/Encoder and De-allocate Buffer
    —> 20180517_deinit_6.txt
  • 234th Allocate Buffer and Start Converter/Encoder
    —> 20180517_deinit_7.txt
  • Encoder is stopped (after dqBuffer bytesused == 23)
  • Program is terminated with error

Log of “Encoder is stopped”: Encoder output → SPS/PPS is 23 Bytes

1526527006.678085: [CNV0] inBuf_to_conv0(): qbuf output  plane[00] :1526527006.436261 q:00
 qBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1], index=[0], bytesused=[8294400]
 qBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(25)]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(0)], index=[0], bytesused=[8294400]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[10], memory_type=[1]
dqThread():  After dqBuffer(): index=[0], bytesused=[8294400], buffer=[0x7f84006a20], shared_buffer=[(nil)] --> CallBack()
1526527006.683484: [CNV0] conv0_output_dqbuf_thread_callback(): dqbuf output  plane[00] :         0.000000 q:00
1526527006.683546: [CNV0] conv0_output_dqbuf_thread_callback(): Push inBuf(Empty) [0/8]
dqThread():  After callback(): index=[0], buffer=[0x7f84006a20], shared_buffer=[(nil)], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[9], memory_type=[1], ret=[0(11)], index=[0], bytesused=[8388608]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[9], memory_type=[1]
dqThread():  After dqBuffer(): index=[0], bytesused=[8388608], buffer=[0x7f56c43ee0], shared_buffer=[0x7f8b7ac57f] --> CallBack()
1526527006.683584: [CNV0] Unlocked
1526527006.683592: [CNV0] conv0_capture_dqbuf_thread_callback(): dqbuf capture plane[00] :1526527006.436261 q:03 byteused=[8388608]
1526527006.683602: [CNV0] conv0_capture_dqbuf_thread_callback(): Get outBuf(Empty) [0/5]
1526527006.683615: [CNV0] conv0_capture_dqbuf_thread_callback(): Push outBuf(Filled) [0/5] 1526527006.436261
dqThread():  After callback(): index=[0], buffer=[0x7f56c43ee0], shared_buffer=[0x7f8b7ac57f], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[9], memory_type=[1]
1526527006.683651: [ENC] inBuf_to_enc(): Get inBuf(Filled) 1526527006.436261 [0/5]
1526527006.683695: [ENC] controlEnc(): setBitrate:16511115
1526527006.683704: [ENC] inBuf_to_enc():  qbuf output  plane[00] :1526527006.436261 q:00
 qBuffer(): Before ioctl(): fd=[32], buf_type=[10], memory_type=[4], index=[0], bytesused=[8388608]
 qBuffer(): After  ioctl(): fd=[32], buf_type=[10], memory_type=[4], ret=[0(0)]
1526527006.683772: [ENC] inBuf_to_enc(): Push inBuf(Empty) [0/5]
1526527006.683791: [ENC] ThreadFunc(): LoopTop flag=[false], q:00

1526527006.699123: [CNV0] inBuf_to_conv0(): Get inBuf(Filled) [0/9]
1526527006.699173: [CNV0] inBuf_to_conv0(): Try to Lock
1526527006.699186: [CNV0] inBuf_to_conv0(): Locked
1526527006.699194: [CNV0] inBuf_to_conv0(): qbuf output  plane[01] :1526527006.469608 q:00
 qBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1], index=[1], bytesused=[8294400]
 qBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(25)]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(0)], index=[1], bytesused=[8294400]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[10], memory_type=[1]
dqThread():  After dqBuffer(): index=[1], bytesused=[8294400], buffer=[0x7f84006b10], shared_buffer=[(nil)] --> CallBack()
1526527006.704808: [CNV0] conv0_output_dqbuf_thread_callback(): dqbuf output  plane[01] :         0.000000 q:00
1526527006.704872: [CNV0] conv0_output_dqbuf_thread_callback(): Push inBuf(Empty) [0/8]
dqThread():  After callback(): index=[1], buffer=[0x7f84006b10], shared_buffer=[(nil)], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[9], memory_type=[1], ret=[0(11)], index=[1], bytesused=[8388608]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[9], memory_type=[1]
dqThread():  After dqBuffer(): index=[1], bytesused=[8388608], buffer=[0x7f574bc630], shared_buffer=[0x7f8b7ac57f] --> CallBack()
1526527006.704928: [CNV0] Unlocked
1526527006.704936: [CNV0] conv0_capture_dqbuf_thread_callback(): dqbuf capture plane[01] :1526527006.469608 q:02 byteused=[8388608]
1526527006.704946: [CNV0] conv0_capture_dqbuf_thread_callback(): Get outBuf(Empty) [0/5]
1526527006.704951: [CNV0] conv0_capture_dqbuf_thread_callback(): Push outBuf(Filled) [0/5] 1526527006.469608
dqThread():  After callback(): index=[1], buffer=[0x7f574bc630], shared_buffer=[0x7f8b7ac57f], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[9], memory_type=[1]
1526527006.704980: [ENC] inBuf_to_enc(): Get inBuf(Filled) 1526527006.469608 [0/5]
1526527006.704993: [ENC] inBuf_to_enc():  qbuf output  plane[01] :1526527006.469608 q:01
 qBuffer(): Before ioctl(): fd=[32], buf_type=[10], memory_type=[4], index=[1], bytesused=[8388608]
 qBuffer(): After  ioctl(): fd=[32], buf_type=[10], memory_type=[4], ret=[0(0)]
1526527006.705023: [ENC] inBuf_to_enc(): Push inBuf(Empty) [0/5]
1526527006.705032: [ENC] ThreadFunc(): LoopTop flag=[false], q:00

1526527006.705646: [CNV0] inBuf_to_conv0(): Get inBuf(Filled) [0/9]
1526527006.705690: [CNV0] inBuf_to_conv0(): Try to Lock
1526527006.705705: [CNV0] inBuf_to_conv0(): Locked
1526527006.705712: [CNV0] inBuf_to_conv0(): qbuf output  plane[02] :1526527006.502948 q:00
 qBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1], index=[2], bytesused=[8294400]
 qBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(25)]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(0)], index=[2], bytesused=[8294400]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[10], memory_type=[1]
dqThread():  After dqBuffer(): index=[2], bytesused=[8294400], buffer=[0x7f84006c00], shared_buffer=[(nil)] --> CallBack()
1526527006.711221: [CNV0] conv0_output_dqbuf_thread_callback(): dqbuf output  plane[02] :         0.000000 q:00
1526527006.711280: [CNV0] conv0_output_dqbuf_thread_callback(): Push inBuf(Empty) [0/8]
dqThread():  After callback(): index=[2], buffer=[0x7f84006c00], shared_buffer=[(nil)], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[9], memory_type=[1], ret=[0(11)], index=[2], bytesused=[8388608]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[9], memory_type=[1]
dqThread():  After dqBuffer(): index=[2], bytesused=[8388608], buffer=[0x7f55c15760], shared_buffer=[0x7f8b7ac57f] --> CallBack()
1526527006.711328: [CNV0] Unlocked
1526527006.711336: [CNV0] conv0_capture_dqbuf_thread_callback(): dqbuf capture plane[02] :1526527006.502948 q:01 byteused=[8388608]
1526527006.711345: [CNV0] conv0_capture_dqbuf_thread_callback(): Get outBuf(Empty) [0/5]
1526527006.711350: [CNV0] conv0_capture_dqbuf_thread_callback(): Push outBuf(Filled) [0/5] 1526527006.502948
dqThread():  After callback(): index=[2], buffer=[0x7f55c15760], shared_buffer=[0x7f8b7ac57f], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[9], memory_type=[1]
1526527006.711378: [ENC] inBuf_to_enc(): Get inBuf(Filled) 1526527006.502948 [0/5]
1526527006.711401: [ENC] controlEnc(): setBitrate:16642939
1526527006.711407: [ENC] inBuf_to_enc():  qbuf output  plane[02] :1526527006.502948 q:02
 qBuffer(): Before ioctl(): fd=[32], buf_type=[10], memory_type=[4], index=[2], bytesused=[8388608]
 qBuffer(): After  ioctl(): fd=[32], buf_type=[10], memory_type=[4], ret=[0(0)]
1526527006.711447: [ENC] inBuf_to_enc(): Push inBuf(Empty) [0/5]
1526527006.711523: [ENC] ThreadFunc(): LoopTop flag=[false], q:00

1526527006.712661: [CNV0] inBuf_to_conv0(): Get inBuf(Filled) [0/9]
1526527006.712723: [CNV0] inBuf_to_conv0(): Try to Lock
1526527006.712738: [CNV0] inBuf_to_conv0(): Locked
1526527006.712745: [CNV0] inBuf_to_conv0(): qbuf output  plane[03] :1526527006.536297 q:00
 qBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1], index=[3], bytesused=[8294400]
 qBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(25)]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(0)], index=[3], bytesused=[8294400]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[10], memory_type=[1]
dqThread():  After dqBuffer(): index=[3], bytesused=[8294400], buffer=[0x7f84006cf0], shared_buffer=[(nil)] --> CallBack()
1526527006.717770: [CNV0] conv0_output_dqbuf_thread_callback(): dqbuf output  plane[03] :         0.000000 q:00
1526527006.717833: [CNV0] conv0_output_dqbuf_thread_callback(): Push inBuf(Empty) [0/9]
dqThread():  After callback(): index=[3], buffer=[0x7f84006cf0], shared_buffer=[(nil)], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[9], memory_type=[1], ret=[0(11)], index=[3], bytesused=[8388608]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[9], memory_type=[1]
dqThread():  After dqBuffer(): index=[3], bytesused=[8388608], buffer=[0x7f57be8260], shared_buffer=[0x7f8b7ac57f] --> CallBack()
1526527006.717862: [CNV0] Unlocked
1526527006.717869: [CNV0] conv0_capture_dqbuf_thread_callback(): dqbuf capture plane[03] :1526527006.536297 q:00 byteused=[8388608]
1526527006.717879: [CNV0] conv0_capture_dqbuf_thread_callback(): Get outBuf(Empty) [0/5]
1526527006.717884: [CNV0] conv0_capture_dqbuf_thread_callback(): Push outBuf(Filled) [0/5] 1526527006.536297
dqThread():  After callback(): index=[3], buffer=[0x7f57be8260], shared_buffer=[0x7f8b7ac57f], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[9], memory_type=[1]

1526527006.717911: [ENC] inBuf_to_enc(): Get inBuf(Filled) 1526527006.536297 [0/5]
1526527006.717925: [ENC] inBuf_to_enc():  qbuf output  plane[03] :1526527006.536297 q:03
 qBuffer(): Before ioctl(): fd=[32], buf_type=[10], memory_type=[4], index=[3], bytesused=[8388608]
 qBuffer(): After  ioctl(): fd=[32], buf_type=[10], memory_type=[4], ret=[0(0)]
1526527006.717955: [ENC] inBuf_to_enc(): Push inBuf(Empty) [0/5]
1526527006.717964: [ENC] ThreadFunc(): LoopTop flag=[false], q:00

1526527006.747395: [CNV0] inBuf_to_conv0(): Get inBuf(Filled) [0/9]
1526527006.747450: [CNV0] inBuf_to_conv0(): Try to Lock
1526527006.747463: [CNV0] inBuf_to_conv0(): Locked
1526527006.747472: [CNV0] inBuf_to_conv0(): qbuf output  plane[04] :1526527006.702965 q:00
 qBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1], index=[4], bytesused=[8294400]
 qBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(25)]

dqBuffer(): After  ioctl(): fd=[32], buf_type=[9], memory_type=[1], ret=[0(0)], index=[0], bytesused=[23]
dqBuffer(): Exit do-while(): fd=[32], buf_type=[9], memory_type=[1]
dqThread():  After dqBuffer(): index=[0], bytesused=[23], buffer=[0x7f57d96a00], shared_buffer=[(nil)] --> CallBack()
1526527006.747994: [ENC] encoder_capture_plane_dq_callback(): dqbuf capture plane[00] :1526527006.436261 q:02 fps:25.76 bytesused=[23]
1526527006.748037: [ENC] encoder_capture_plane_dq_callback(): Push outBuf(Filled) [0/29] 1526527006.436261
1526527006.748072: [ENC] encoder_capture_plane_dq_callback(): qbuf capture plane[00] :1526527006.436261 q:02
 qBuffer(): Before ioctl(): fd=[32], buf_type=[9], memory_type=[1], index=[0], bytesused=[23]
 qBuffer(): After  ioctl(): fd=[32], buf_type=[9], memory_type=[1], ret=[0(0)]
dqThread():  After callback(): index=[0], buffer=[0x7f57d96a00], shared_buffer=[(nil)], ret=[1]

dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[32], buf_type=[9], memory_type=[1]
dqBuffer(): After  ioctl(): fd=[32], buf_type=[10], memory_type=[4], ret=[0(0)], index=[0], bytesused=[0]
dqBuffer(): Exit do-while(): fd=[32], buf_type=[10], memory_type=[4]
dqThread():  After dqBuffer(): index=[0], bytesused=[0], buffer=[0x7f57c3f120], shared_buffer=[0x7f56c43ee0] --> CallBack()
1526527006.748117: [ENC] encoder_output_plane_dq_callback(): dqbuf output  plane[00] :         0.000000 q:03, bytesused=[0][0][8388608]

1526527006.748132: [CNV0] capture_plane_qBuffer(): qbuf capture plane[00] :         0.000000 q:00
 qBuffer(): Before ioctl(): fd=[35], buf_type=[9], memory_type=[1], index=[0], bytesused=[8388608]
 qBuffer(): After  ioctl(): fd=[35], buf_type=[9], memory_type=[1], ret=[0(0)]
dqThread():  After callback(): index=[0], buffer=[0x7f57c3f120], shared_buffer=[0x7f56c43ee0], ret=[1]

dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[32], buf_type=[10], memory_type=[4]
1526527006.748178: [ENC] inBuf_to_enc(): Timeout inBuf(Filled) [0/6]
1526527006.748191: [ENC] ThreadFunc(): LoopTop flag=[false], q:00
dqBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(0)], index=[4], bytesused=[8294400]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[10], memory_type=[1]
dqThread():  After dqBuffer(): index=[4], bytesused=[8294400], buffer=[0x7f84006de0], shared_buffer=[(nil)] --> CallBack()
1526527006.759672: [CNV0] conv0_output_dqbuf_thread_callback(): dqbuf output  plane[04] :         0.000000 q:00
1526527006.759749: [CNV0] conv0_output_dqbuf_thread_callback(): Push inBuf(Empty) [0/9]
dqThread():  After callback(): index=[4], buffer=[0x7f84006de0], shared_buffer=[(nil)], ret=[1]
dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1]
dqBuffer(): After  ioctl(): fd=[35], buf_type=[9], memory_type=[1], ret=[0(11)], index=[0], bytesused=[8388608]
dqBuffer(): Exit do-while(): fd=[35], buf_type=[9], memory_type=[1]
dqThread():  After dqBuffer(): index=[0], bytesused=[8388608], buffer=[0x7f56c43ee0], shared_buffer=[0x7f8b7ac57f] --> CallBack()
1526527006.759790: [CNV0] Unlocked
1526527006.759801: [CNV0] conv0_capture_dqbuf_thread_callback(): dqbuf capture plane[00] :1526527006.702965 q:00 byteused=[8388608]
1526527006.759811: [CNV0] conv0_capture_dqbuf_thread_callback(): Get outBuf(Empty) [0/5]
1526527006.759819: [CNV0] conv0_capture_dqbuf_thread_callback(): Push outBuf(Filled) [0/5] 1526527006.702965
dqThread():  After callback(): index=[0], buffer=[0x7f56c43ee0], shared_buffer=[0x7f8b7ac57f], ret=[1]

dqThread(): Before dqBuffer(): index=[0]
dqBuffer(): Before ioctl(): fd=[35], buf_type=[9], memory_type=[1]
1526527006.759848: [ENC] inBuf_to_enc(): Get inBuf(Filled) 1526527006.702965 [0/5]
1526527006.759874: [ENC] controlEnc(): setBitrate:16775816
1526527006.759882: [ENC] inBuf_to_enc():  qbuf output  plane[04] :1526527006.702965 q:03
 qBuffer(): Before ioctl(): fd=[32], buf_type=[10], memory_type=[4], index=[4], bytesused=[8388608]
 qBuffer(): After  ioctl(): fd=[32], buf_type=[10], memory_type=[4], ret=[0(0)]
1526527006.759910: [ENC] inBuf_to_enc(): Push inBuf(Empty) [0/5]
1526527006.759920: [ENC] ThreadFunc(): LoopTop flag=[false], q:00

1526527006.775220: [CNV0] inBuf_to_conv0(): Get inBuf(Filled) [0/9]
1526527006.775275: [CNV0] inBuf_to_conv0(): Try to Lock
1526527006.775288: [CNV0] inBuf_to_conv0(): Locked
1526527006.775296: [CNV0] inBuf_to_conv0(): qbuf output  plane[05] :1526527006.736266 q:00
 qBuffer(): Before ioctl(): fd=[35], buf_type=[10], memory_type=[1], index=[5], bytesused=[8294400]
 qBuffer(): After  ioctl(): fd=[35], buf_type=[10], memory_type=[1], ret=[0(25)]

** Converter bytesused: 8294400 = 3840 x 2160, 8388608 = 2^23 (4096 x 2048)

tegrastats near by termination

RAM 6983/7851MB (lfb 15x4MB) cpu [0%@1992,16%@2035,1%@2035,0%@1996,0%@1996,9%@1992] EMC 10%@1866 APE 150 MSENC 1164 GR3D 0%@1300
RAM 6983/7851MB (lfb 15x4MB) cpu [1%@2009,31%@2034,2%@2034,2%@2024,1%@2025,0%@2027] EMC 11%@1866 APE 150 MSENC 1164 GR3D 0%@1300
RAM 6984/7851MB (lfb 15x4MB) cpu [0%@2007,21%@2034,2%@2036,1%@2024,3%@2019,9%@2018] EMC 12%@1866 APE 150 MSENC 1164 GR3D 0%@1300
RAM 6984/7851MB (lfb 15x4MB) cpu [0%@2012,20%@2035,5%@2034,0%@2026,2%@2035,0%@2033] EMC 13%@1866 APE 150 MSENC 1164 GR3D 0%@1300
RAM 7114/7851MB (lfb 15x4MB) cpu [10%@2021,43%@2034,30%@2035,7%@2023,1%@2024,17%@2025] EMC 10%@1866 APE 150 GR3D 0%@1300
RAM 1416/7851MB (lfb 926x4MB) cpu [10%@2018,5%@2035,48%@2035,0%@2027,0%@2026,0%@2023] EMC 8%@1866 APE 150 GR3D 0%@1300
RAM 1416/7851MB (lfb 926x4MB) cpu [0%@2017,0%@2035,0%@2034,0%@2027,0%@2026,0%@2026] EMC 4%@1866 APE 150 GR3D 0%@1300

Error message at termination

===== MSENC blits (mode: 1) into tiled surfaces =====
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 495, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 495, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 495, SyncPointValue = 0)

all_allocations after termination

BASE        SIZE USERFLAGS   REFS  KMAPS  UMAPS  SHARE      UID
       0       9600K  10000001      3      0      0      1 ffffffc0738c1500
       0      10240K  10000001      2      0      0      1 ffffffc07a323a00
       0         64K  10000001      2      0      0      1 ffffffc07ac89000
       0         64K  10000001      2      0      0      1 ffffffc07ac89200
       0       8192K  10000001      3      0      1      1 ffffffc07ac89a00
       0       1048K  10000003      3      0      1      1 ffffffc07ad4e500
       0          4K  10000001      4      1      1      1 ffffffc07ad4e800
       0         64K  10000001      3      0      1      1 ffffffc07ad4e900
       0       2432K  10000001      2      0      0      1 ffffffc07b127100
       0        128K  10000001      2      0      0      1 ffffffc07b127d00
       0         64K  10000001      2      0      0      1 ffffffc07b22fc00
       0       4096K  10000001      2      0      0      1 ffffffc1cc3baf00
       0      10240K  10000001      2      0      0      1 ffffffc1cc3dbe00
       0        256K  10000001      2      0      0      1 ffffffc1cc3dbf00
       0         64K  10000001      2      0      0      1 ffffffc1ce282a00
       0        256K  10000001      3      0      1      1 ffffffc1d3e0d000
       0       9600K  10000001      2      0      0      1 ffffffc1d3e0d100
       0        384K  10000001      2      0      0      1 ffffffc1d3e0d600
       0          4K  10000001      2      0      0      1 ffffffc1d3e0df00
       0       1024K  10000001      2      0      0      1 ffffffc1d71d4200
       0      10240K  10000001      2      0      0      1 ffffffc1d71d4600
       0          4K  10000003      4      1      1      1 ffffffc1d73c0700
       0         84K  10000001      3      0      1      1 ffffffc1d73c0d00
       0          4K  10000003      3      1      1      1 ffffffc1e2448100
       0         64K  10000001      2      0      0      1 ffffffc1e24f2c00
       0        256K  10000003      3      1      1      1 ffffffc1e45e8700
       0         64K  10000001      3      0      1      1 ffffffc1e4696100
       0          4K  10000001      2      0      0      1 ffffffc1e4696f00
       0        128K  10000001      2      0      0      1 ffffffc1e5b14300

Best Regards

20180517_MemoryLeak.zip (88.9 KB)

Hi waynezhu,

I inserted this into NvV4l2ElementPlane::deinitPlane()

if (num_buffers == 0) return;

And added a process to delete objects (Converter/Encoder) and to re-allocate output Buffer of Converter.

Then the memory leakage become less than before.

In the comment #5, my program was running about 11minutes.
But after the modification (delete objects), it was running over 30 minutes.

(This comment is edited by mynaemi at 24th May.)

Attached file is the file “all_allocates” read from my application software while running.
(23 minutes after the start)

20180517_deinit_and_delete.txt (77 KB)

I drawed a graph of used memory.

Resolution Change without “delete NvV4l2Element(Converter/Encoder)”

  • 201805171205_memory.png

Resolution Change with “delete NvV4l2Element”

  • 201805171619_memory.png

Best Regards.


hi mynaemi,
Sorry for late,
When you cat all_allocations, did your program exit?
You need to add a wait/delay in the end of program to hold the program then cat all_allocation and compare.

BTW, can you share a small app to reproduce this issue?

Thanks
wayne zhu

Hi waynezhu,

Thank you have me remember.

I wrote it in #5.

It is in my application while running:

system("sudo cat /sys/kernel/debug/nvmap/iovmm/all_allocations");

What I think a problem is

  • decrease of the available memory while running
  • abnormal termination due to it

It is related to deinitPlane() and setupPlane() with/without delete and new NvV4l2Element objects.
Therefore it is not enough after exitting, I beleave.

I’m implementing dynamic resolution function in my application (Streaming Server).
When it changes a video resolution by congestion or bandwidth,
it stop the NvVideoConverter and the NvVideoEncoder, deinitPlane(), setupPlane(), and restart them.
I tried it with/without delete and new objects.
Unfortunately the available memory was decreased, and my application was terminated abnormaly for 10-30 minutes.

It can be considered the same thing in a different application without dynamic resolution changing.
For example,
if a streaming server has [START] and [STOP] user interface (GUI buttons),
when the user clicks a [START] button after selecting a resolution and bit rate,
it executes new objects, call setupPlane() and starts them.
when the user clicks a [STOP] button, it stops them, call deinitPlane() and delete them.
in the next time the user clicks a [START] button, it repeats them.
Hence the available memory is decreased.
When you click a [START] button, don’t you want to terminate it abnormaly ?

A program to reproduce this issue demands to repeat to start and stop the NvVideoConverter and the NvVideoEncoder.
My application is too complicated (to need a streaming client and a network).
I will try to reproduce it in the following ways:

  • Using samples/10_camera_recording
  • Modify main() function
  • Repeat to call ArgusSamples::execute()

Best Regards

Hi waynezhu,

I have tried to reproduce it using a modified sample code.

The modification part:

  • Loop of execution()
  • Read /proc/meminfo

Log of this test program:

nvidia@tegra-ubuntu:~/tegra_multimedia_api/samples/memoryLeakTest$ ./camera_recording -r 3840x2160 -t H264 -d 2 2> /dev/null | grep "Memory"
1527150501.745255: [Memory] Total = [8034572], Free = [3495856], Available = [5667888]
1527150504.711174: [Memory] Total = [8034572], Free = [3427432], Available = [5606328]
1527150507.556324: [Memory] Total = [8034572], Free = [3400768], Available = [5583084]
1527150510.396212: [Memory] Total = [8034572], Free = [3377296], Available = [5567000]
1527150513.234329: [Memory] Total = [8034572], Free = [3363904], Available = [5556912]
1527150516.037403: [Memory] Total = [8034572], Free = [3306324], Available = [5506996]
1527150518.877686: [Memory] Total = [8034572], Free = [3293580], Available = [5497544]
1527150521.705814: [Memory] Total = [8034572], Free = [3172652], Available = [5383272]
1527150524.511762: [Memory] Total = [8034572], Free = [3166084], Available = [5380148]
1527150527.353340: [Memory] Total = [8034572], Free = [3055584], Available = [5277308]
1527150530.173333: [Memory] Total = [8034572], Free = [3046820], Available = [5272224]
1527150533.025829: [Memory] Total = [8034572], Free = [2931872], Available = [5164936]
1527150535.853060: [Memory] Total = [8034572], Free = [2923848], Available = [5160252]
1527150538.695121: [Memory] Total = [8034572], Free = [2811200], Available = [5054800]
1527150541.487711: [Memory] Total = [8034572], Free = [2807212], Available = [5054220]
1527150544.315535: [Memory] Total = [8034572], Free = [2679048], Available = [4933780]
1527150547.120793: [Memory] Total = [8034572], Free = [2675044], Available = [4933312]
1527150549.938450: [Memory] Total = [8034572], Free = [2560096], Available = [4825664]
1527150552.805425: [Memory] Total = [8034572], Free = [2553960], Available = [4822272]
1527150555.643120: [Memory] Total = [8034572], Free = [2425696], Available = [4701836]
1527150558.456893: [Memory] Total = [8034572], Free = [2419096], Available = [4698608]
1527150561.211630: [Memory] Total = [8034572], Free = [2312272], Available = [4599168]
1527150564.024818: [Memory] Total = [8034572], Free = [2308360], Available = [4598996]
1527150566.848534: [Memory] Total = [8034572], Free = [2191404], Available = [4489392]
1527150569.682497: [Memory] Total = [8034572], Free = [2185556], Available = [4486760]
1527150572.448863: [Memory] Total = [8034572], Free = [2068100], Available = [4377156]
1527150575.187710: [Memory] Total = [8034572], Free = [2061072], Available = [4373492]
1527150577.973811: [Memory] Total = [8034572], Free = [1960660], Available = [4281132]
1527150580.821254: [Memory] Total = [8034572], Free = [1956132], Available = [4279616]
1527150583.662461: [Memory] Total = [8034572], Free = [1820956], Available = [4151720]
1527150586.475134: [Memory] Total = [8034572], Free = [1817804], Available = [4152300]
1527150589.314207: [Memory] Total = [8034572], Free = [1703588], Available = [4045084]
1527150592.146781: [Memory] Total = [8034572], Free = [1700440], Available = [4045760]
1527150594.969559: [Memory] Total = [8034572], Free = [1591976], Available = [3944704]
1527150597.814977: [Memory] Total = [8034572], Free = [1586032], Available = [3941616]
1527150600.636728: [Memory] Total = [8034572], Free = [1467884], Available = [3830884]
1527150603.445961: [Memory] Total = [8034572], Free = [1462332], Available = [3828804]
1527150606.275190: [Memory] Total = [8034572], Free = [1343576], Available = [3718028]
1527150609.114616: [Memory] Total = [8034572], Free = [1340844], Available = [3718772]
1527150611.877247: [Memory] Total = [8034572], Free = [1233552], Available = [3619020]
1527150614.753694: [Memory] Total = [8034572], Free = [1230032], Available = [3618384]
1527150617.586796: [Memory] Total = [8034572], Free = [1116864], Available = [3512380]
1527150620.397833: [Memory] Total = [8034572], Free = [1112324], Available = [3511616]
1527150623.178132: [Memory] Total = [8034572], Free = [989928], Available = [3396744]
1527150626.017377: [Memory] Total = [8034572], Free = [986004], Available = [3396480]
1527150628.814649: [Memory] Total = [8034572], Free = [875204], Available = [3292564]
1527150631.665418: [Memory] Total = [8034572], Free = [871464], Available = [3292464]
1527150634.448265: [Memory] Total = [8034572], Free = [763832], Available = [3192468]
1527150637.237817: [Memory] Total = [8034572], Free = [760236], Available = [3192544]
1527150640.058844: [Memory] Total = [8034572], Free = [639168], Available = [3079280]
1527150642.867559: [Memory] Total = [8034572], Free = [636316], Available = [3079624]
1527150645.697450: [Memory] Total = [8034572], Free = [527752], Available = [2978012]
1527150648.441621: [Memory] Total = [8034572], Free = [524532], Available = [2978164]
1527150651.230524: [Memory] Total = [8034572], Free = [411540], Available = [2872992]
1527150653.986519: [Memory] Total = [8034572], Free = [408284], Available = [2873256]
1527150656.823834: [Memory] Total = [8034572], Free = [297168], Available = [2769444]
1527150659.614696: [Memory] Total = [8034572], Free = [293816], Available = [2769328]
1527150662.405878: [Memory] Total = [8034572], Free = [188764], Available = [2672064]
1527150665.192759: [Memory] Total = [8034572], Free = [183236], Available = [2669992]
1527150667.976390: [Memory] Total = [8034572], Free = [116232], Available = [2558012]
1527150670.791113: [Memory] Total = [8034572], Free = [116148], Available = [2557500]
1527150673.537124: [Memory] Total = [8034572], Free = [118904], Available = [2441192]
1527150676.365866: [Memory] Total = [8034572], Free = [117988], Available = [2441956]
1527150679.175211: [Memory] Total = [8034572], Free = [115472], Available = [2352880]
1527150682.030702: [Memory] Total = [8034572], Free = [111900], Available = [2352868]
1527150684.834777: [Memory] Total = [8034572], Free = [116100], Available = [2230352]
1527150687.642010: [Memory] Total = [8034572], Free = [112032], Available = [2229112]
1527150690.467973: [Memory] Total = [8034572], Free = [115352], Available = [2124288]
1527150693.314068: [Memory] Total = [8034572], Free = [112200], Available = [2124448]
1527150696.164812: [Memory] Total = [8034572], Free = [111812], Available = [2019488]
1527150698.964350: [Memory] Total = [8034572], Free = [112912], Available = [2021588]
1527150701.728117: [Memory] Total = [8034572], Free = [114948], Available = [1913148]
1527150704.541004: [Memory] Total = [8034572], Free = [112592], Available = [1914328]
1527150707.366380: [Memory] Total = [8034572], Free = [113160], Available = [1816728]
1527150710.201556: [Memory] Total = [8034572], Free = [109520], Available = [1816900]
1527150713.009379: [Memory] Total = [8034572], Free = [112624], Available = [1723200]
1527150715.811322: [Memory] Total = [8034572], Free = [109276], Available = [1722664]
1527150718.648811: [Memory] Total = [8034572], Free = [117592], Available = [1598664]
1527150721.459330: [Memory] Total = [8034572], Free = [114880], Available = [1599116]
1527150724.290053: [Memory] Total = [8034572], Free = [119792], Available = [1497828]
1527150727.109834: [Memory] Total = [8034572], Free = [115060], Available = [1496580]
1527150729.904886: [Memory] Total = [8034572], Free = [115536], Available = [1388864]
1527150732.686555: [Memory] Total = [8034572], Free = [111476], Available = [1387712]
1527150735.479901: [Memory] Total = [8034572], Free = [117568], Available = [1290588]
1527150738.301639: [Memory] Total = [8034572], Free = [113800], Available = [1290376]
1527150741.122745: [Memory] Total = [8034572], Free = [111808], Available = [1196180]
1527150743.949381: [Memory] Total = [8034572], Free = [113044], Available = [1198912]
1527150746.725156: [Memory] Total = [8034572], Free = [117136], Available = [1081108]
1527150749.536207: [Memory] Total = [8034572], Free = [117128], Available = [1082860]
1527150752.334938: [Memory] Total = [8034572], Free = [115392], Available = [976952]

Log of tegrastats: (from the middle of the activity)

nvidia@tegra-ubuntu:~$ ./tegrastats 
RAM 3363/7846MB (lfb 266x4MB) CPU [0%@1728,0%@345,0%@345,0%@1728,0%@1728,0%@1728]
RAM 3366/7846MB (lfb 266x4MB) CPU [16%@345,1%@345,0%@345,17%@345,23%@345,21%@345]
RAM 3471/7846MB (lfb 266x4MB) CPU [31%@499,0%@345,0%@345,36%@499,33%@499,34%@499]
RAM 3471/7846MB (lfb 266x4MB) CPU [39%@2035,0%@345,0%@345,37%@2035,36%@2035,29%@2035]
RAM 3473/7846MB (lfb 266x4MB) CPU [27%@499,0%@345,1%@345,17%@499,19%@499,24%@499]
RAM 3474/7846MB (lfb 266x4MB) CPU [35%@345,0%@345,0%@345,27%@345,29%@345,29%@345]
RAM 3464/7846MB (lfb 266x4MB) CPU [31%@1113,0%@2035,0%@2035,29%@1113,21%@1113,29%@1113]
RAM 3576/7846MB (lfb 247x4MB) CPU [38%@1881,1%@345,1%@345,38%@1881,33%@1881,29%@1881]
RAM 3581/7846MB (lfb 246x4MB) CPU [45%@499,3%@345,1%@345,38%@499,42%@499,28%@499]
RAM 3571/7846MB (lfb 246x4MB) CPU [39%@345,0%@345,0%@345,26%@345,30%@345,30%@345]
RAM 3584/7846MB (lfb 245x4MB) CPU [22%@345,0%@345,0%@345,24%@345,21%@345,26%@345]
RAM 3585/7846MB (lfb 245x4MB) CPU [37%@2035,0%@345,0%@345,31%@2035,32%@2035,36%@2035]
RAM 3583/7846MB (lfb 245x4MB) CPU [43%@1420,4%@345,0%@345,38%@1420,41%@1420,37%@1420]
RAM 3676/7846MB (lfb 223x4MB) CPU [29%@345,0%@345,0%@345,32%@345,28%@345,37%@345]
RAM 3676/7846MB (lfb 223x4MB) CPU [34%@1267,0%@345,0%@345,26%@1267,27%@1267,32%@1267]
RAM 3673/7846MB (lfb 223x4MB) CPU [15%@499,2%@345,1%@345,31%@499,13%@499,14%@499]
RAM 3676/7846MB (lfb 223x4MB) CPU [30%@345,0%@345,0%@345,23%@345,22%@345,25%@345]
RAM 3672/7846MB (lfb 223x4MB) CPU [24%@806,0%@345,0%@345,26%@806,20%@806,21%@806]
RAM 3787/7846MB (lfb 196x4MB) CPU [23%@499,1%@345,0%@345,18%@499,23%@499,18%@499]
RAM 3801/7846MB (lfb 193x4MB) CPU [32%@345,0%@345,0%@345,37%@345,31%@345,31%@345]
RAM 3791/7846MB (lfb 193x4MB) CPU [34%@960,0%@345,0%@345,32%@960,28%@960,25%@960]
RAM 3800/7846MB (lfb 192x4MB) CPU [14%@345,0%@345,0%@345,26%@345,21%@345,23%@345]
RAM 3801/7846MB (lfb 192x4MB) CPU [29%@499,0%@345,0%@345,29%@499,26%@499,24%@499]
RAM 3793/7846MB (lfb 192x4MB) CPU [25%@2035,0%@345,1%@345,18%@2035,25%@2035,23%@2035]
RAM 3905/7846MB (lfb 167x4MB) CPU [24%@345,0%@345,0%@345,32%@345,23%@345,28%@345]
RAM 3906/7846MB (lfb 167x4MB) CPU [34%@499,0%@345,0%@345,30%@499,32%@499,30%@499]
RAM 3904/7846MB (lfb 167x4MB) CPU [19%@345,1%@345,0%@345,22%@345,23%@345,21%@345]
RAM 3906/7846MB (lfb 167x4MB) CPU [21%@345,0%@345,0%@345,30%@345,21%@345,23%@345]
RAM 3905/7846MB (lfb 167x4MB) CPU [25%@806,0%@345,0%@345,24%@806,25%@806,21%@806]
RAM 3904/7846MB (lfb 166x4MB) CPU [17%@1267,0%@345,0%@345,16%@1267,20%@1267,29%@1267]
RAM 4004/7846MB (lfb 142x4MB) CPU [29%@345,0%@345,0%@345,34%@345,32%@345,29%@345]
RAM 4004/7846MB (lfb 142x4MB) CPU [33%@960,0%@345,0%@345,29%@960,31%@960,28%@960]
RAM 4006/7846MB (lfb 142x4MB) CPU [23%@345,0%@345,1%@345,20%@345,18%@345,21%@345]
RAM 4007/7846MB (lfb 142x4MB) CPU [20%@345,0%@345,0%@345,34%@345,21%@345,22%@345]
RAM 3997/7846MB (lfb 142x4MB) CPU [20%@499,0%@2035,0%@2035,26%@499,25%@499,30%@499]
RAM 4108/7846MB (lfb 117x4MB) CPU [25%@652,1%@345,0%@345,22%@652,20%@652,25%@652]
RAM 4116/7846MB (lfb 116x4MB) CPU [34%@345,0%@345,0%@345,32%@345,33%@345,30%@345]
RAM 4105/7846MB (lfb 116x4MB) CPU [24%@345,0%@345,0%@345,26%@345,20%@345,27%@345]
RAM 4117/7846MB (lfb 116x4MB) CPU [18%@345,0%@345,0%@345,26%@345,28%@345,25%@345]
RAM 4117/7846MB (lfb 116x4MB) CPU [25%@345,0%@345,0%@345,27%@345,25%@345,24%@345]
RAM 4116/7846MB (lfb 115x4MB) CPU [23%@499,0%@345,1%@345,21%@499,25%@499,17%@499]
RAM 4226/7846MB (lfb 88x4MB) CPU [30%@499,0%@345,0%@345,40%@345,24%@345,30%@345]
RAM 4227/7846MB (lfb 88x4MB) CPU [30%@345,0%@345,0%@345,34%@345,34%@345,33%@345]
RAM 4224/7846MB (lfb 88x4MB) CPU [15%@960,1%@345,0%@345,29%@960,17%@960,24%@960]
RAM 4226/7846MB (lfb 88x4MB) CPU [23%@345,0%@345,0%@345,28%@345,26%@345,17%@345]
RAM 4226/7846MB (lfb 88x4MB) CPU [25%@652,0%@345,0%@345,25%@652,24%@652,21%@652]
RAM 4310/7846MB (lfb 67x4MB) CPU [26%@499,1%@345,1%@345,22%@499,20%@499,24%@499]
RAM 4324/7846MB (lfb 64x4MB) CPU [29%@499,0%@345,0%@345,39%@499,30%@499,31%@499]
RAM 4313/7846MB (lfb 64x4MB) CPU [30%@806,0%@806,0%@806,31%@806,27%@806,29%@806]
RAM 4324/7846MB (lfb 63x4MB) CPU [24%@345,1%@345,0%@345,18%@345,17%@345,23%@345]
RAM 4324/7846MB (lfb 63x4MB) CPU [22%@345,0%@345,0%@345,28%@345,23%@345,20%@345]
RAM 4313/7846MB (lfb 63x4MB) CPU [29%@345,0%@652,0%@652,21%@345,21%@345,24%@345]
RAM 4427/7846MB (lfb 38x4MB) CPU [29%@345,1%@345,0%@345,22%@345,24%@345,27%@345]
RAM 4428/7846MB (lfb 38x4MB) CPU [34%@499,0%@345,0%@345,33%@499,32%@499,31%@499]
RAM 4425/7846MB (lfb 38x4MB) CPU [29%@806,2%@345,0%@345,29%@806,25%@806,25%@806]
RAM 4428/7846MB (lfb 38x4MB) CPU [26%@345,0%@345,0%@345,27%@345,24%@345,29%@345]
RAM 4428/7846MB (lfb 38x4MB) CPU [24%@345,0%@345,0%@345,31%@345,21%@345,23%@345]
RAM 4428/7846MB (lfb 38x4MB) CPU [23%@1113,0%@345,0%@345,21%@1113,23%@1113,25%@1113]
RAM 4541/7846MB (lfb 15x4MB) CPU [31%@345,1%@345,0%@345,32%@345,34%@345,30%@345]
RAM 4540/7846MB (lfb 15x4MB) CPU [29%@1420,0%@345,0%@345,26%@1420,29%@1420,28%@1420]
RAM 4540/7846MB (lfb 15x4MB) CPU [23%@345,1%@345,0%@345,20%@345,19%@345,21%@345]
RAM 4540/7846MB (lfb 15x4MB) CPU [27%@345,0%@345,0%@345,25%@345,19%@345,29%@345]
RAM 4531/7846MB (lfb 15x4MB) CPU [24%@1574,0%@806,0%@806,31%@1574,27%@1574,25%@1574]
RAM 4642/7846MB (lfb 15x4MB) CPU [25%@345,0%@345,0%@345,12%@345,26%@345,24%@345]
RAM 4643/7846MB (lfb 15x4MB) CPU [33%@345,0%@345,0%@345,27%@345,34%@345,24%@345]
RAM 4632/7846MB (lfb 15x4MB) CPU [27%@652,0%@345,0%@345,25%@652,21%@652,27%@652]
RAM 4643/7846MB (lfb 15x4MB) CPU [25%@345,2%@345,2%@345,25%@345,22%@345,31%@345]
RAM 4642/7846MB (lfb 15x4MB) CPU [28%@345,0%@345,0%@345,22%@345,24%@345,30%@345]
RAM 4640/7846MB (lfb 15x4MB) CPU [28%@499,0%@345,0%@345,26%@499,27%@499,22%@499]
RAM 4740/7846MB (lfb 15x4MB) CPU [33%@499,0%@345,0%@345,28%@499,33%@499,27%@499]
RAM 4741/7846MB (lfb 15x4MB) CPU [36%@345,0%@345,0%@345,25%@345,31%@345,27%@345]
RAM 4739/7846MB (lfb 15x4MB) CPU [30%@960,0%@345,0%@345,20%@806,22%@806,18%@806]
RAM 4741/7846MB (lfb 15x4MB) CPU [33%@499,0%@345,0%@345,21%@499,22%@499,25%@499]
RAM 4740/7846MB (lfb 15x4MB) CPU [31%@652,0%@345,0%@345,25%@652,24%@652,21%@652]
RAM 4838/7846MB (lfb 15x4MB) CPU [21%@960,0%@345,0%@345,25%@960,22%@960,25%@960]
RAM 4852/7846MB (lfb 15x4MB) CPU [39%@499,0%@345,0%@345,29%@499,28%@499,30%@499]
RAM 4841/7846MB (lfb 15x4MB) CPU [39%@806,0%@345,0%@345,30%@806,29%@806,32%@806]
RAM 4851/7846MB (lfb 15x4MB) CPU [26%@1420,0%@345,0%@345,21%@1420,12%@1420,18%@1420]
RAM 4851/7846MB (lfb 15x4MB) CPU [28%@345,0%@345,0%@345,22%@345,24%@345,24%@345]
RAM 4848/7846MB (lfb 15x4MB) CPU [25%@2035,0%@499,0%@499,28%@2035,21%@2035,19%@2035]
RAM 4950/7846MB (lfb 15x4MB) CPU [27%@345,0%@345,0%@345,34%@345,21%@345,29%@345]
RAM 4951/7846MB (lfb 15x4MB) CPU [34%@499,0%@345,0%@345,29%@499,31%@499,34%@499]
RAM 4947/7846MB (lfb 15x4MB) CPU [27%@345,1%@345,0%@345,23%@345,24%@345,27%@345]
RAM 4949/7846MB (lfb 15x4MB) CPU [31%@499,0%@345,0%@345,22%@499,24%@499,29%@499]
RAM 4949/7846MB (lfb 15x4MB) CPU [39%@1267,0%@345,0%@345,27%@1267,34%@1267,32%@1267]
RAM 5040/7846MB (lfb 15x4MB) CPU [26%@806,0%@345,0%@345,16%@806,29%@806,23%@806]
RAM 5054/7846MB (lfb 15x4MB) CPU [29%@345,0%@345,0%@345,36%@345,36%@345,32%@345]
RAM 5043/7846MB (lfb 15x4MB) CPU [28%@2035,0%@2035,0%@2035,31%@2035,33%@2035,39%@2035]
RAM 5053/7846MB (lfb 15x4MB) CPU [18%@345,0%@345,0%@345,25%@345,21%@345,22%@345]
RAM 5053/7846MB (lfb 15x4MB) CPU [34%@345,0%@345,0%@345,25%@345,29%@345,31%@345]
RAM 5043/7846MB (lfb 15x4MB) CPU [33%@345,0%@345,0%@345,27%@345,25%@345,28%@345]
RAM 5155/7846MB (lfb 15x4MB) CPU [34%@345,0%@345,0%@345,30%@345,27%@345,26%@345]
RAM 5155/7846MB (lfb 15x4MB) CPU [38%@499,0%@345,0%@345,31%@499,35%@499,34%@499]
RAM 5151/7846MB (lfb 15x4MB) CPU [24%@652,0%@345,0%@345,20%@652,28%@652,27%@652]
RAM 5154/7846MB (lfb 15x4MB) CPU [31%@345,0%@345,0%@345,24%@345,24%@345,28%@345]
RAM 5154/7846MB (lfb 15x4MB) CPU [31%@345,0%@345,0%@345,23%@345,23%@345,25%@345]
RAM 5154/7846MB (lfb 15x4MB) CPU [28%@806,0%@345,1%@345,21%@806,26%@806,21%@806]
RAM 5251/7846MB (lfb 15x4MB) CPU [36%@345,0%@345,0%@345,35%@345,30%@345,29%@345]
RAM 5251/7846MB (lfb 15x4MB) CPU [29%@806,0%@345,0%@345,30%@806,32%@806,29%@806]
RAM 5251/7846MB (lfb 15x4MB) CPU [26%@345,0%@345,0%@345,16%@345,23%@345,16%@345]
RAM 5252/7846MB (lfb 15x4MB) CPU [36%@345,0%@345,0%@345,24%@345,21%@345,23%@345]
RAM 5242/7846MB (lfb 15x4MB) CPU [33%@499,0%@345,0%@345,23%@499,26%@499,28%@499]
RAM 5359/7846MB (lfb 15x4MB) CPU [26%@345,0%@345,2%@345,32%@345,17%@345,25%@345]
RAM 5365/7846MB (lfb 15x4MB) CPU [34%@345,0%@345,0%@345,26%@345,38%@345,33%@345]
RAM 5360/7846MB (lfb 15x4MB) CPU [29%@1267,1%@345,0%@345,33%@1267,24%@1267,20%@1267]
RAM 5364/7846MB (lfb 15x4MB) CPU [26%@345,0%@345,0%@345,19%@345,26%@345,18%@345]
RAM 5363/7846MB (lfb 15x4MB) CPU [35%@345,0%@345,0%@345,20%@345,28%@345,26%@345]
RAM 5360/7846MB (lfb 15x4MB) CPU [28%@806,0%@345,0%@345,24%@806,27%@806,16%@806]
RAM 5478/7846MB (lfb 15x4MB) CPU [31%@345,2%@345,0%@345,36%@345,39%@345,27%@345]
RAM 5477/7846MB (lfb 15x4MB) CPU [29%@2035,0%@345,0%@345,28%@2035,35%@2035,29%@2035]
RAM 5474/7846MB (lfb 15x4MB) CPU [23%@499,1%@345,1%@345,19%@499,27%@499,17%@499]
RAM 5475/7846MB (lfb 15x4MB) CPU [22%@345,0%@345,0%@345,26%@345,29%@345,23%@345]
RAM 5465/7846MB (lfb 15x4MB) CPU [34%@499,0%@960,0%@960,21%@499,27%@499,18%@499]
RAM 5563/7846MB (lfb 15x4MB) CPU [34%@345,0%@345,1%@345,21%@345,26%@345,17%@345]
RAM 5563/7846MB (lfb 15x4MB) CPU [34%@345,0%@345,0%@345,29%@345,34%@345,34%@345]
RAM 5552/7846MB (lfb 15x4MB) CPU [31%@345,0%@345,0%@345,23%@345,22%@345,25%@345]
RAM 5561/7846MB (lfb 15x4MB) CPU [18%@345,1%@345,0%@345,16%@345,25%@345,30%@345]
RAM 5562/7846MB (lfb 15x4MB) CPU [29%@345,0%@345,0%@345,22%@345,25%@345,19%@345]
RAM 5560/7846MB (lfb 15x4MB) CPU [28%@960,0%@345,0%@345,15%@960,22%@960,28%@960]
RAM 5685/7846MB (lfb 15x4MB) CPU [31%@345,0%@345,0%@345,32%@345,25%@345,25%@345]
RAM 5685/7846MB (lfb 15x4MB) CPU [33%@345,0%@345,0%@345,29%@345,31%@345,37%@345]
RAM 5680/7846MB (lfb 15x4MB) CPU [14%@1113,1%@345,0%@345,25%@1113,28%@1113,21%@1113]
RAM 5682/7846MB (lfb 15x4MB) CPU [22%@345,0%@345,0%@345,28%@345,24%@345,23%@345]
RAM 5683/7846MB (lfb 15x4MB) CPU [21%@345,0%@345,0%@345,20%@345,20%@345,31%@345]
RAM 5774/7846MB (lfb 15x4MB) CPU [23%@652,0%@345,0%@345,16%@652,30%@652,28%@652]
RAM 5788/7846MB (lfb 15x4MB) CPU [32%@345,0%@345,0%@345,33%@345,30%@345,30%@345]
RAM 5776/7846MB (lfb 15x4MB) CPU [32%@1267,0%@806,0%@806,37%@1267,34%@1267,20%@1267]
RAM 5785/7846MB (lfb 15x4MB) CPU [18%@499,1%@345,0%@345,21%@499,24%@499,15%@499]
RAM 5785/7846MB (lfb 15x4MB) CPU [30%@499,0%@345,0%@345,27%@499,19%@499,24%@499]
RAM 5776/7846MB (lfb 15x4MB) CPU [24%@345,0%@345,0%@345,19%@345,24%@345,24%@345]
RAM 5886/7846MB (lfb 15x4MB) CPU [32%@345,0%@345,0%@345,25%@345,31%@345,27%@345]
RAM 5891/7846MB (lfb 15x4MB) CPU [37%@499,0%@345,0%@345,26%@499,36%@499,26%@499]
RAM 5886/7846MB (lfb 15x4MB) CPU [27%@652,0%@345,0%@345,22%@652,28%@652,27%@652]
RAM 5889/7846MB (lfb 15x4MB) CPU [21%@345,0%@345,0%@345,20%@345,25%@345,24%@345]
RAM 5887/7846MB (lfb 15x4MB) CPU [32%@345,0%@345,0%@345,22%@345,25%@345,28%@345]
RAM 5885/7846MB (lfb 15x4MB) CPU [26%@1574,0%@345,0%@345,24%@1574,19%@1574,21%@1574]
RAM 5993/7846MB (lfb 15x4MB) CPU [32%@345,0%@345,0%@345,31%@345,38%@345,31%@345]
RAM 5993/7846MB (lfb 15x4MB) CPU [26%@2035,0%@345,0%@345,29%@2035,27%@2035,28%@2035]
RAM 5990/7846MB (lfb 15x4MB) CPU [20%@345,0%@345,0%@345,19%@345,21%@345,26%@345]
RAM 5991/7846MB (lfb 15x4MB) CPU [28%@345,0%@345,0%@345,26%@345,26%@345,19%@345]
RAM 5981/7846MB (lfb 15x4MB) CPU [26%@652,0%@960,0%@960,31%@652,26%@652,28%@652]
RAM 6088/7846MB (lfb 15x4MB) CPU [25%@652,0%@345,0%@345,27%@652,24%@652,23%@652]
RAM 6089/7846MB (lfb 15x4MB) CPU [33%@499,0%@345,0%@345,27%@499,33%@499,31%@499]
RAM 6077/7846MB (lfb 15x4MB) CPU [25%@499,0%@345,0%@345,22%@499,24%@499,31%@499]
RAM 6087/7846MB (lfb 15x4MB) CPU [26%@345,2%@345,0%@345,31%@345,21%@345,18%@345]
RAM 6087/7846MB (lfb 15x4MB) CPU [29%@345,0%@345,0%@345,30%@345,22%@345,25%@345]
RAM 6085/7846MB (lfb 15x4MB) CPU [19%@345,0%@345,0%@345,29%@345,18%@345,18%@345]
RAM 6173/7846MB (lfb 15x4MB) CPU [27%@345,0%@345,0%@345,30%@345,36%@345,33%@345]
RAM 6180/7846MB (lfb 15x4MB) CPU [31%@345,0%@345,0%@345,29%@345,27%@345,35%@345]
RAM 6176/7846MB (lfb 15x4MB) CPU [26%@806,0%@345,0%@345,19%@806,20%@806,26%@806]
RAM 6178/7846MB (lfb 15x4MB) CPU [36%@345,0%@345,0%@345,24%@345,20%@345,18%@345]
RAM 6176/7846MB (lfb 15x4MB) CPU [22%@960,0%@345,0%@345,25%@960,23%@960,31%@960]
RAM 6289/7846MB (lfb 15x4MB) CPU [20%@499,2%@345,1%@345,24%@499,20%@499,29%@499]
RAM 6303/7846MB (lfb 15x4MB) CPU [32%@345,0%@345,0%@345,29%@345,35%@345,38%@345]
RAM 6290/7846MB (lfb 15x4MB) CPU [36%@1113,0%@652,0%@652,29%@1113,30%@1113,31%@1113]
RAM 6300/7846MB (lfb 15x4MB) CPU [17%@345,1%@345,0%@345,23%@345,17%@345,27%@345]
RAM 6301/7846MB (lfb 15x4MB) CPU [23%@345,0%@345,0%@345,21%@345,27%@345,21%@345]
RAM 6290/7846MB (lfb 15x4MB) CPU [16%@499,0%@806,0%@806,34%@499,26%@499,21%@499]
RAM 6390/7846MB (lfb 15x4MB) CPU [27%@345,1%@345,1%@345,32%@345,30%@345,27%@345]
RAM 6391/7846MB (lfb 15x4MB) CPU [32%@345,0%@345,0%@345,29%@345,29%@345,34%@345]
RAM 6396/7846MB (lfb 15x4MB) CPU [28%@499,0%@345,0%@345,18%@499,31%@499,27%@499]
RAM 6400/7846MB (lfb 15x4MB) CPU [30%@345,0%@345,0%@345,20%@345,22%@345,26%@345]
RAM 6400/7846MB (lfb 15x4MB) CPU [28%@345,0%@345,0%@345,20%@345,24%@345,25%@345]
RAM 6400/7846MB (lfb 15x4MB) CPU [21%@806,1%@345,1%@345,25%@806,27%@806,22%@806]
RAM 6506/7846MB (lfb 15x4MB) CPU [38%@499,0%@345,0%@345,36%@499,34%@499,28%@499]
RAM 6504/7846MB (lfb 15x4MB) CPU [28%@652,0%@345,0%@345,38%@652,23%@652,30%@652]
RAM 6503/7846MB (lfb 15x4MB) CPU [19%@345,0%@345,0%@345,23%@345,22%@345,22%@345]
RAM 6505/7846MB (lfb 15x4MB) CPU [35%@345,0%@345,0%@345,27%@345,23%@345,29%@345]
RAM 6494/7846MB (lfb 15x4MB) CPU [31%@499,0%@345,0%@345,21%@499,22%@499,26%@499]
RAM 6600/7846MB (lfb 15x4MB) CPU [30%@345,1%@345,1%@345,23%@345,21%@345,31%@345]
RAM 6601/7846MB (lfb 15x4MB) CPU [36%@345,0%@345,0%@345,29%@345,37%@345,31%@345]
RAM 6596/7846MB (lfb 15x4MB) CPU [21%@1420,0%@345,1%@345,24%@1420,29%@1420,28%@1420]
RAM 6600/7846MB (lfb 15x4MB) CPU [26%@345,0%@345,0%@345,20%@345,23%@345,20%@345]
RAM 6600/7846MB (lfb 15x4MB) CPU [29%@345,0%@345,0%@345,26%@345,32%@345,21%@345]
RAM 6598/7846MB (lfb 15x4MB) CPU [19%@345,1%@345,1%@345,17%@345,26%@345,23%@345]
RAM 6692/7846MB (lfb 15x4MB) CPU [38%@345,1%@345,0%@345,28%@345,30%@345,36%@345]
RAM 6691/7846MB (lfb 15x4MB) CPU [34%@1881,0%@345,0%@345,29%@1881,31%@1881,33%@1881]
RAM 6690/7846MB (lfb 15x4MB) CPU [24%@345,0%@345,0%@345,19%@345,18%@345,18%@345]
RAM 6689/7846MB (lfb 15x4MB) CPU [27%@345,0%@345,0%@345,27%@345,25%@345,28%@345]
RAM 6681/7846MB (lfb 15x4MB) CPU [31%@499,0%@345,0%@345,22%@499,26%@499,22%@499]
RAM 6799/7846MB (lfb 15x4MB) CPU [26%@345,1%@345,0%@345,31%@345,17%@345,21%@345]
RAM 6804/7846MB (lfb 15x4MB) CPU [44%@345,0%@345,0%@345,26%@345,29%@345,27%@345]
RAM 6792/7846MB (lfb 15x4MB) CPU [29%@499,0%@345,0%@345,28%@499,22%@499,28%@499]
RAM 6801/7846MB (lfb 15x4MB) CPU [24%@345,0%@345,0%@345,20%@345,23%@345,28%@345]
RAM 6801/7846MB (lfb 15x4MB) CPU [28%@345,0%@345,0%@345,23%@345,24%@345,25%@345]
RAM 6799/7846MB (lfb 15x4MB) CPU [21%@652,0%@345,0%@345,22%@652,21%@652,25%@652]
RAM 6904/7846MB (lfb 15x4MB) CPU [35%@345,7%@345,2%@345,34%@345,36%@345,32%@345]
RAM 6905/7846MB (lfb 15x4MB) CPU [35%@345,0%@345,0%@345,30%@345,31%@345,33%@345]
RAM 2346/7846MB (lfb 79x4MB) CPU [18%@345,0%@345,27%@345,26%@345,16%@345,20%@345]
RAM 2346/7846MB (lfb 79x4MB) CPU [5%@345,0%@345,0%@345,0%@345,8%@345,5%@345]
RAM 2346/7846MB (lfb 79x4MB) CPU [18%@345,0%@345,0%@345,8%@345,5%@345,14%@345]
RAM 2346/7846MB (lfb 79x4MB) CPU [8%@345,0%@345,0%@345,2%@345,7%@345,7%@345]

Best Regards

20180524_MemoryLeakTest.zip (133 KB)

Hi mynaemi,
Do you still see memory leakage issue if you called deinitPlane when stop?

I will have a try on my side, will get back once have update.

Hi waynezhu,

The test code in #10 is not my program.
There was a behavior to decreace the available memory.
I think that you can see the behavior easily.

I’m not sure that it is correct to refer it as “memory leakage issue”.

Best Regards

It will be soon in 2 weeks.
Does anyone try to reproduce this behavior ?

Yes, I can reproduce this issue, in debug.

Hi waynezhu,

It’s been such a long time.
Thank you so much for debugging.

Best Regards

Hi Mynaemi,
Below is how Memory leakage happen.
Number of qbuffer and dqbuffer is not exactly same, so for some fd, NvBufferDestroy(fd) is not called.

Can you use a queue to store same number of buffers as parameter in setupPlane? and reuse them when acquiring frame?
When finished, destroy(NvBufferDestroy) all buffers in the queue.

ret = m_VideoEncoder->capture_plane.setupPlane(V4L2_MEMORY_MMAP, 10[i][/i], true, false);

// Keep acquire frames and queue into encoder
while (!m_gotError)
{
    NvBuffer *buffer;
    int fd = -1;

    struct v4l2_buffer v4l2_buf;
    struct v4l2_plane planes[MAX_PLANES];

    memset(&v4l2_buf, 0, sizeof(v4l2_buf));
    memset(planes, 0, MAX_PLANES * sizeof(struct v4l2_plane));

    v4l2_buf.m.planes = planes;

    // Check if we need dqBuffer first
    if (bufferIndex < MAX_ENCODER_FRAMES &&
        m_VideoEncoder->output_plane.getNumQueuedBuffers() <
        m_VideoEncoder->output_plane.getNumBuffers())
    {
        // The queue is not full, no need to dqBuffer
        // Prepare buffer index for the following qBuffer
        v4l2_buf.index = bufferIndex++;
    }
    else
    {
        // Output plane full or max outstanding number reached
        CHECK_ERROR(m_VideoEncoder->output_plane.dqBuffer(v4l2_buf, &buffer,
                                                          NULL, 10));
        // Release the frame.
        fd = v4l2_buf.m.planes[0].m.fd;
        printf("wayne add, destroy fd:%d\n", fd);
        [b]NvBufferDestroy(fd);[/b]
        if (VERBOSE_ENABLE)
            CONSUMER_PRINT("Released frame. %d\n", fd);
    }

    // Acquire a frame.
    UniqueObj<Frame> frame(iFrameConsumer->acquireFrame());
    IFrame *iFrame = interface_cast<IFrame>(frame);
    if (!iFrame)
    {
        // Send EOS
        v4l2_buf.m.planes[0].m.fd = fd;
        v4l2_buf.m.planes[0].bytesused = 0;
        CHECK_ERROR(m_VideoEncoder->output_plane.qBuffer(v4l2_buf, NULL));
        break;
    }

    // Get the IImageNativeBuffer extension interface and create the fd.
    NV::IImageNativeBuffer *iNativeBuffer =
        interface_cast<NV::IImageNativeBuffer>(iFrame->getImage());
    if (!iNativeBuffer)
        ORIGINATE_ERROR("IImageNativeBuffer not supported by Image.");
    [b]fd = iNativeBuffer->createNvBuffer(STREAM_SIZE,
                                       NvBufferColorFormat_YUV420,
                                       (DO_CPU_PROCESS)?NvBufferLayout_Pitch:NvBufferLayout_BlockLinear);[/b]
    if (VERBOSE_ENABLE)
        CONSUMER_PRINT("Acquired Frame. %d\n", fd);

    if (DO_CPU_PROCESS) {
        NvBufferParams par;
        NvBufferGetParams (fd, &par);
        void *ptr_y;
        uint8_t *ptr_cur;
        int i, j, a, b;
        NvBufferMemMap(fd, Y_INDEX, NvBufferMem_Write, &ptr_y);
        NvBufferMemSyncForCpu(fd, Y_INDEX, &ptr_y);
        ptr_cur = (uint8_t *)ptr_y + par.pitch[Y_INDEX]*START_POS + START_POS;

        // overwrite some pixels to put an 'N' on each Y plane
        // scan array_n to decide which pixel should be overwritten
        for (i=0; i < FONT_SIZE; i++) {
            for (j=0; j < FONT_SIZE; j++) {
                a = i>>SHIFT_BITS;
                b = j>>SHIFT_BITS;
                if (array_n[a][b])
                    (*ptr_cur) = 0xff; // white color
                ptr_cur++;
            }
            ptr_cur = (uint8_t *)ptr_y + par.pitch[Y_INDEX]*(START_POS + i)  + START_POS;
        }
        NvBufferMemSyncForDevice (fd, Y_INDEX, &ptr_y);
        NvBufferMemUnMap(fd, Y_INDEX, &ptr_y);
    }

    // Push the frame into V4L2.
    v4l2_buf.m.planes[0].m.fd = fd;
    v4l2_buf.m.planes[0].bytesused = 1; // byteused must be non-zero
    printf("wayne add, create fd:%d,  size:%d\n", fd, STREAM_SIZE);
    CHECK_ERROR(m_VideoEncoder->output_plane.qBuffer(v4l2_buf, NULL));
}

Thanks
wayne zhu

Hi waynezhu,

Thank you to support so much.

Unfortunately, sorry I cannot understand your reply.

Is this a cause ?

And,

Is this a workaround ?
I don’t understand what I have to do.

To refer the sample code,
I get new buffers using setupPlane(),
call qBuffer() for acquiring an image/frame to the encoder,
and reuse empty buffers from dqBuffer().

Do I have to use NvBufferDestroy(fd) and createNvBuffer()
instead of dqBuffer() and reuse the empty buffers of output plane ?

ret = m_VideoEncoder->capture_plane.setupPlane(V4L2_MEMORY_MMAP, 10, true, false);

// Keep acquire frames and queue into encoder
while (!m_gotError)
{
	NvBuffer *buffer;
	int fd = -1;

	struct v4l2_buffer v4l2_buf;
	struct v4l2_plane planes[MAX_PLANES];

	memset(&v4l2_buf, 0, sizeof(v4l2_buf));
	memset(planes, 0, MAX_PLANES * sizeof(struct v4l2_plane));

	v4l2_buf.m.planes = planes;

	// Check if we need dqBuffer first
	if (bufferIndex < MAX_ENCODER_FRAMES &&
		m_VideoEncoder->output_plane.getNumQueuedBuffers() <
		m_VideoEncoder->output_plane.getNumBuffers())
	{
		// The queue is not full, no need to dqBuffer
		// Prepare buffer index for the following qBuffer
		v4l2_buf.index = bufferIndex++;
	}
	else
	{
		// Output plane full or max outstanding number reached
		CHECK_ERROR(m_VideoEncoder->output_plane.dqBuffer(v4l2_buf, &buffer, NULL, 10));
		// Release the frame.
		fd = v4l2_buf.m.planes[0].m.fd;
		printf("wayne add, destroy fd:%d\n", fd);
		NvBufferDestroy(fd);
		if (VERBOSE_ENABLE) CONSUMER_PRINT("Released frame. %d\n", fd);
	}

	// Acquire a frame.
	UniqueObj<Frame> frame(iFrameConsumer->acquireFrame());
	IFrame *iFrame = interface_cast<IFrame>(frame);
	if (!iFrame)
	{
		// Send EOS
		v4l2_buf.m.planes[0].m.fd = fd;
		v4l2_buf.m.planes[0].bytesused = 0;
		CHECK_ERROR(m_VideoEncoder->output_plane.qBuffer(v4l2_buf, NULL));
		break;
	}

	// Get the IImageNativeBuffer extension interface and create the fd.
	NV::IImageNativeBuffer *iNativeBuffer =
	interface_cast<NV::IImageNativeBuffer>(iFrame->getImage());
	if (!iNativeBuffer) ORIGINATE_ERROR("IImageNativeBuffer not supported by Image.");
	fd = iNativeBuffer->createNvBuffer(STREAM_SIZE,
			NvBufferColorFormat_YUV420,
			(DO_CPU_PROCESS)?NvBufferLayout_Pitch:NvBufferLayout_BlockLinear);
	if (VERBOSE_ENABLE) CONSUMER_PRINT("Acquired Frame. %d\n", fd);

	if (DO_CPU_PROCESS) {
		NvBufferParams par;
		NvBufferGetParams (fd, &par);
		void *ptr_y;
		uint8_t *ptr_cur;
		int i, j, a, b;
		NvBufferMemMap(fd, Y_INDEX, NvBufferMem_Write, &ptr_y);
		NvBufferMemSyncForCpu(fd, Y_INDEX, &ptr_y);
		ptr_cur = (uint8_t *)ptr_y + par.pitch[Y_INDEX]*START_POS + START_POS;

		// overwrite some pixels to put an 'N' on each Y plane
		// scan array_n to decide which pixel should be overwritten
		for (i=0; i < FONT_SIZE; i++) {
			for (j=0; j < FONT_SIZE; j++) {
				a = i>>SHIFT_BITS;
				b = j>>SHIFT_BITS;
				if (array_n[a][b])
					(*ptr_cur) = 0xff; // white color
				ptr_cur++;
			}
			ptr_cur = (uint8_t *)ptr_y + par.pitch[Y_INDEX]*(START_POS + i) + START_POS;
		}
		NvBufferMemSyncForDevice (fd, Y_INDEX, &ptr_y);
		NvBufferMemUnMap(fd, Y_INDEX, &ptr_y);
	}

	// Push the frame into V4L2.
	v4l2_buf.m.planes[0].m.fd = fd;
	v4l2_buf.m.planes[0].bytesused = 1; // byteused must be non-zero
	printf("wayne add, create fd:%d, size:%d\n", fd, STREAM_SIZE);
	CHECK_ERROR(m_VideoEncoder->output_plane.qBuffer(v4l2_buf, NULL));
}

Best Regards

Mynaemi,
I will give a sample for how to implement today or tomorrow.

You can replace below function for memory leakage.
BTW, even with this, I can still see some other memory leakage, which may be from our library side, I am in syncing code.

bool ConsumerThread::threadExecute()
{
IStream *iStream = interface_cast(m_stream);
IFrameConsumer *iFrameConsumer = interface_cast(m_consumer);

// Wait until the producer has connected to the stream.
CONSUMER_PRINT("Waiting until producer is connected...\n");
if (iStream->waitUntilConnected() != STATUS_OK)
    ORIGINATE_ERROR("Stream failed to connect.");
CONSUMER_PRINT("Producer has connected; continuing.\n");

int bufferIndex;

bufferIndex = 0;

int ret;
int fd[10];
//queue<int> filled_queue;
queue<int> empty_queue;
for( int i = 0; i < 10; i++ )
{
    NvBufferCreateParams input_params = {0};
    input_params.payloadType = NvBufferPayload_SurfArray;
    input_params.width = STREAM_SIZE.width();
    input_params.height = STREAM_SIZE.height();
    input_params.layout = NvBufferLayout_Pitch;
    input_params.colorFormat = NvBufferColorFormat_YUV420;
    input_params.nvbuf_tag = NvBufferTag_VIDEO_DEC;
    ret = NvBufferCreateEx (&fd[i], &input_params);
    if (ret >= 0)
    {
        empty_queue.push(fd[i]);
    }
}

// Keep acquire frames and queue into encoder
while (!m_gotError)
{
    NvBuffer *buffer;
    int fd = -1;

    struct v4l2_buffer v4l2_buf;
    struct v4l2_plane planes[MAX_PLANES];

    memset(&v4l2_buf, 0, sizeof(v4l2_buf));
    memset(planes, 0, MAX_PLANES * sizeof(struct v4l2_plane));

    v4l2_buf.m.planes = planes;

    // Check if we need dqBuffer first
    if (bufferIndex < MAX_ENCODER_FRAMES &&
        m_VideoEncoder->output_plane.getNumQueuedBuffers() <
        m_VideoEncoder->output_plane.getNumBuffers())
    {
        // The queue is not full, no need to dqBuffer
        // Prepare buffer index for the following qBuffer
        v4l2_buf.index = bufferIndex++;
    }
    else
    {
        // Output plane full or max outstanding number reached
        CHECK_ERROR(m_VideoEncoder->output_plane.dqBuffer(v4l2_buf, &buffer,
                                                          NULL, 10));
        // Release the frame.
        /*fd = v4l2_buf.m.planes[0].m.fd;
        printf("wayne add, destroy fd:%d\n", fd);
        NvBufferDestroy(fd);
        if (VERBOSE_ENABLE)
            CONSUMER_PRINT("Released frame. %d\n", fd);*/
        fd = v4l2_buf.m.planes[0].m.fd;
        empty_queue.push(fd);
    }

    // Acquire a frame.
    UniqueObj<Frame> frame(iFrameConsumer->acquireFrame());
    IFrame *iFrame = interface_cast<IFrame>(frame);
    if (!iFrame)
    {
        // Send EOS
        v4l2_buf.m.planes[0].m.fd = fd;
        v4l2_buf.m.planes[0].bytesused = 0;
        CHECK_ERROR(m_VideoEncoder->output_plane.qBuffer(v4l2_buf, NULL));
        break;
    }

    // Get the IImageNativeBuffer extension interface and create the fd.
    /*NV::IImageNativeBuffer *iNativeBuffer =
        interface_cast<NV::IImageNativeBuffer>(iFrame->getImage());
    if (!iNativeBuffer)
        ORIGINATE_ERROR("IImageNativeBuffer not supported by Image.");
    fd = iNativeBuffer->createNvBuffer(STREAM_SIZE,
                                       NvBufferColorFormat_YUV420,
                                       (DO_CPU_PROCESS) ? NvBufferLayout_Pitch: NvBufferLayout_BlockLinear);*/
    fd = empty_queue.front();
    empty_queue.pop();

    if (VERBOSE_ENABLE)
        CONSUMER_PRINT("Acquired Frame. %d\n", fd);

    if (DO_CPU_PROCESS) {
        NvBufferParams par;
        NvBufferGetParams (fd, &par);
        void *ptr_y;
        uint8_t *ptr_cur;
        int i, j, a, b;
        NvBufferMemMap(fd, Y_INDEX, NvBufferMem_Write, &ptr_y);
        NvBufferMemSyncForCpu(fd, Y_INDEX, &ptr_y);
        ptr_cur = (uint8_t *)ptr_y + par.pitch[Y_INDEX]*START_POS + START_POS;

        // overwrite some pixels to put an 'N' on each Y plane
        // scan array_n to decide which pixel should be overwritten
        for (i=0; i < FONT_SIZE; i++) {
            for (j=0; j < FONT_SIZE; j++) {
                a = i>>SHIFT_BITS;
                b = j>>SHIFT_BITS;
                if (array_n[a][b])
                    (*ptr_cur) = 0xff; // white color
                ptr_cur++;
            }
            ptr_cur = (uint8_t *)ptr_y + par.pitch[Y_INDEX]*(START_POS + i)  + START_POS;
        }
        NvBufferMemSyncForDevice (fd, Y_INDEX, &ptr_y);
        NvBufferMemUnMap(fd, Y_INDEX, &ptr_y);
    }

    // Push the frame into V4L2.
    v4l2_buf.m.planes[0].m.fd = fd;
    v4l2_buf.m.planes[0].bytesused = 1; // byteused must be non-zero
    printf("wayne add, create fd:%d,  size:%d\n", fd, STREAM_SIZE);
    CHECK_ERROR(m_VideoEncoder->output_plane.qBuffer(v4l2_buf, NULL));
}

for( int i = 0; i < 10; i++ )
{
    NvBufferDestroy(fd[i]);
}
while (!empty_queue.empty())
    empty_queue.pop();

// Wait till capture plane DQ Thread finishes
// i.e. all the capture plane buffers are dequeued
m_VideoEncoder->capture_plane.waitForDQThread(2000);

CONSUMER_PRINT("Done.\n");

requestShutdown();

return true;

}