nvxFindHomographyNode low performance
I run video stabilizier sample from visionworks and I see that on "big" shakings performance of this node is very low, f. ex: Find Homography time : 47.0594 ms or Find Homography time : 43.8856 ms But on "small" shakings it's < 2 ms. How fix it?
I run video stabilizier sample from visionworks and I see that on "big" shakings performance of this node is very low, f. ex:
Find Homography time : 47.0594 ms or Find Homography time : 43.8856 ms
But on "small" shakings it's < 2 ms.

How fix it?

#1
Posted 11/01/2017 03:22 PM   
Hi, Please remember to maximize the CPU/GPU frequency to have a stable computing power: [code]sudo ~/jetson_clocks.sh[/code] Thanks.
Hi,

Please remember to maximize the CPU/GPU frequency to have a stable computing power:
sudo ~/jetson_clocks.sh


Thanks.

#2
Posted 11/02/2017 03:06 AM   
Yes, I use jetson_clocks.sh. I tested video stabilizier on JTX1, JTX2 and host PC. On all devices I have similar results: in some cases nvxFindHomographyNode is too slow.
Yes, I use jetson_clocks.sh. I tested video stabilizier on JTX1, JTX2 and host PC. On all devices I have similar results: in some cases nvxFindHomographyNode is too slow.

#3
Posted 11/02/2017 08:22 AM   
Hi, Do you use our video stabilizer sample? And what are your input source? Video or camera? We have checked our video stabilizer sample and didn't find the issue you mentioned. Could you help us reproduce this issue? Thanks.
Hi,

Do you use our video stabilizer sample?
And what are your input source? Video or camera?

We have checked our video stabilizer sample and didn't find the issue you mentioned.
Could you help us reproduce this issue?

Thanks.

#4
Posted 11/02/2017 09:52 AM   
Ok, I upload to gdrive short video from webcam Logitech C270: [url]https://drive.google.com/open?id=0B0L0yxSRGe5vRVoxcXhQUVJDZ0U[/url] I run video stabilizier with [code]./nvx_demo_video_stabilizer -s ~/test.avi[/code] Sometimes I see next output from console: Graph Time : 57.3297 ms <...> Find Homography time : 56.1456 ms Graph Time : 46.7193 ms <...> Find Homography time : 45.5733 ms Graph Time : 30.0361 ms <...> Find Homography time : 29.0394 ms
Ok, I upload to gdrive short video from webcam Logitech C270:
https://drive.google.com/open?id=0B0L0yxSRGe5vRVoxcXhQUVJDZ0U

I run video stabilizier with
./nvx_demo_video_stabilizer -s ~/test.avi


Sometimes I see next output from console:
Graph Time : 57.3297 ms <...> Find Homography time : 56.1456 ms
Graph Time : 46.7193 ms <...> Find Homography time : 45.5733 ms
Graph Time : 30.0361 ms <...> Find Homography time : 29.0394 ms

#5
Posted 11/02/2017 10:49 AM   
Hi, We will reproduce this issue. Could you share how long you meet a long execution time? In minutes, hours or days? Thanks.
Hi,

We will reproduce this issue.
Could you share how long you meet a long execution time? In minutes, hours or days?

Thanks.

#6
Posted 11/03/2017 02:29 AM   
I see long execution time on some frames from video. It's full stabilizier log output on video from previous post: Jetson TX2: https://pastebin.com/SKG5Jd9T Host (MacBook Pro GT750M): https://pastebin.com/SzKzT9V5
I see long execution time on some frames from video.

It's full stabilizier log output on video from previous post:
Jetson TX2: https://pastebin.com/SKG5Jd9T

Host (MacBook Pro GT750M): https://pastebin.com/SzKzT9V5

#7
Posted 11/03/2017 08:34 AM   
Hi, Thanks for your video. We can reproduce this issue now. We find it takes a long time to calculate the homography matrix. Will update more information to you later. [code] Graph Time : 51.4923 ms RGB to gray time : 0.079839 ms Copy time : 0.108319 ms Pyramid time : 0.143071 ms Optical Flow time : 0.466239 ms [b] Find Homography time : 49.8289 ms[/b] Homography Filter time : 0.02672 ms Matrices Smoothing time : 0.064863 ms Truncate Stab Transform time : 0.012544 ms Warp Perspective time: 0.221822 ms Feature Track time : 0.407677 ms Display Time : 62.78 ms [/code] Thanks.
Hi,

Thanks for your video. We can reproduce this issue now.
We find it takes a long time to calculate the homography matrix.
Will update more information to you later.

Graph Time : 51.4923 ms
RGB to gray time : 0.079839 ms
Copy time : 0.108319 ms
Pyramid time : 0.143071 ms
Optical Flow time : 0.466239 ms
Find Homography time : 49.8289 ms
Homography Filter time : 0.02672 ms
Matrices Smoothing time : 0.064863 ms
Truncate Stab Transform time : 0.012544 ms
Warp Perspective time: 0.221822 ms
Feature Track time : 0.407677 ms
Display Time : 62.78 ms

Thanks.

#8
Posted 11/03/2017 10:51 AM   
Hi, Please based on your requirement, modify the parameter of homography algorithm. Due to the textless input source, it's not easy to find a robust homography matrix. The algorithm will keep finding a suitable matrix (with RANSAC) until reach iteration maximal. You can modify these parameters to balance accuracy and performance: [code] vx_node nvxFindHomographyNode ( vx_graph graph, vx_array srcPoints, vx_array dstPoints, vx_matrix homography, vx_enum method, vx_float32 threshold, vx_int32 maxEstimateIters, vx_int32 maxRefineIters, vx_float32 confidence, vx_float32 outlierRatio, vx_array inliers ) [Graph] Computes homography matrix. Parameters [in] graph Specifies the graph. [in] srcPoints Specifies the coordinates of the points in the original plane (array of vx_keypoint_t, nvx_keypointf_t or nvx_point2f_t structs, number of the points must be >= 4). For vx_keypoint_t and nvx_keypointf_t types, the method uses points only with non-zero tracking_status. [in] dstPoints Specifies the coordinates of the points in the target plane (it must have the same number of elements and type as srcPoints). For vx_keypoint_t and nvx_keypointf_t types, the method uses points only with non-zero tracking_status. The correspondence is between points with the same indexes in arrays. [out] homography Specifies the output homography matrix (VX_TYPE_FLOAT32 3x3 matrix). [in] method Specifies the method used to compute a homography matrix (see nvx_find_homography_method_e). [in] threshold Specifies the maximum allowed reprojection error to treat a point pair as an inlier, if RANSAC method is chosen. If srcPoints and dstPoints are measured in pixels, it usually makes sense to set this parameter somewhere in the range of 1 to 10. Default value is 3. [in] maxEstimateIters Specifies the maximum allowed number of estimation iterations for robust methods (RANSAC and LMeDS). Default value is 2000. [in] maxRefineIters Specifies the maximum allowed number of refinement iterations for robust methods (RANSAC and LMeDS). Default value is 10. [in] confidence Specifies the confidence level for robust methods (RANSAC and LMeDS). It must be a value between 0 and 1. Default value is 0.995. [in] outlierRatio Specifies the outlier ratio if LMeDS method is chosen. Default value is 0.45. [out] inliers Specifies the optional inliers/outliers output mask. The inliers[i] value is set to 0 if corresponding points pair srcPoints[i] and dstPoints[i] is marked as outlier by the method. It must be a VX_TYPE_UINT8 array with capacity greater than or equal to the capacity of srcPoints. Returns vx_node [/code] For example, we fix maximal iteration to 100, and no over 10ms execution time is observed. [code] diff --git a/demos/video_stabilizer/stabilizer.cpp b/demos/video_stabilizer/stabilizer.cpp index 0b6333a..5e7d602 100644 --- a/demos/video_stabilizer/stabilizer.cpp +++ b/demos/video_stabilizer/stabilizer.cpp @@ -267,7 +267,7 @@ namespace kp_curr_list, homography, NVX_FIND_HOMOGRAPHY_METHOD_RANSAC, 3.0f, - 2000, 10, + 100, 10, 0.995f, 0.45f, mask); NVXIO_CHECK_REFERENCE(find_homography_node_); [/code] Thanks.
Answer Accepted by Original Poster
Hi,

Please based on your requirement, modify the parameter of homography algorithm.

Due to the textless input source, it's not easy to find a robust homography matrix.
The algorithm will keep finding a suitable matrix (with RANSAC) until reach iteration maximal.
You can modify these parameters to balance accuracy and performance:

vx_node nvxFindHomographyNode 	( 	vx_graph  	graph,
vx_array srcPoints,
vx_array dstPoints,
vx_matrix homography,
vx_enum method,
vx_float32 threshold,
vx_int32 maxEstimateIters,
vx_int32 maxRefineIters,
vx_float32 confidence,
vx_float32 outlierRatio,
vx_array inliers
)


[Graph] Computes homography matrix.

Parameters
[in] graph Specifies the graph.
[in] srcPoints Specifies the coordinates of the points in the original plane (array of vx_keypoint_t, nvx_keypointf_t or nvx_point2f_t structs, number of the points must be >= 4). For vx_keypoint_t and nvx_keypointf_t types, the method uses points only with non-zero tracking_status.
[in] dstPoints Specifies the coordinates of the points in the target plane (it must have the same number of elements and type as srcPoints). For vx_keypoint_t and nvx_keypointf_t types, the method uses points only with non-zero tracking_status. The correspondence is between points with the same indexes in arrays.
[out] homography Specifies the output homography matrix (VX_TYPE_FLOAT32 3x3 matrix).
[in] method Specifies the method used to compute a homography matrix (see nvx_find_homography_method_e).
[in] threshold Specifies the maximum allowed reprojection error to treat a point pair as an inlier, if RANSAC method is chosen. If srcPoints and dstPoints are measured in pixels, it usually makes sense to set this parameter somewhere in the range of 1 to 10. Default value is 3.
[in] maxEstimateIters Specifies the maximum allowed number of estimation iterations for robust methods (RANSAC and LMeDS). Default value is 2000.
[in] maxRefineIters Specifies the maximum allowed number of refinement iterations for robust methods (RANSAC and LMeDS). Default value is 10.
[in] confidence Specifies the confidence level for robust methods (RANSAC and LMeDS). It must be a value between 0 and 1. Default value is 0.995.
[in] outlierRatio Specifies the outlier ratio if LMeDS method is chosen. Default value is 0.45.
[out] inliers Specifies the optional inliers/outliers output mask. The inliers[i] value is set to 0 if corresponding points pair srcPoints[i] and dstPoints[i] is marked as outlier by the method. It must be a VX_TYPE_UINT8 array with capacity greater than or equal to the capacity of srcPoints.

Returns
vx_node


For example, we fix maximal iteration to 100, and no over 10ms execution time is observed.
diff --git a/demos/video_stabilizer/stabilizer.cpp b/demos/video_stabilizer/stabilizer.cpp
index 0b6333a..5e7d602 100644
--- a/demos/video_stabilizer/stabilizer.cpp
+++ b/demos/video_stabilizer/stabilizer.cpp
@@ -267,7 +267,7 @@ namespace
kp_curr_list,
homography,
NVX_FIND_HOMOGRAPHY_METHOD_RANSAC, 3.0f,
- 2000, 10,
+ 100, 10,
0.995f, 0.45f,
mask);
NVXIO_CHECK_REFERENCE(find_homography_node_);


Thanks.

#9
Posted 11/07/2017 08:35 AM   
Thanks
Thanks

#10
Posted 11/08/2017 03:37 PM   
Scroll To Top

Add Reply