Nvcamerasrc wbMode OFF setting not working properly

Hi,

I am testing the on-board OV5693 camera devkit on a Jetson TX1 with Jetpack 3.0. Specifically I am evaluating the wbMode property settings, but I can’t see any difference between AUTO mode and OFF mode. No matter if it is configured in AUTO or OFF mode, the white balance settings seems to always adjust and correct color temperature.

I expect that if I set the wbMode propert to OFF mode, the camera will not make any adjustment of the colors temperature, but even in OFF mode setting I can obviously note an adjustment in the color temperature. To discard variables I run the tests with the auto-exposure configured as ON and OFF mode, and got the same results. I think that wbMode OFF is not working properly.

I ran another test with the the wbMode property set to MANUAL mode, and in that case effectively, the color temperature of the picture remains always equal, no adjustment is appreciated. But when I change the wbMode setting back to OFF mode, the color temperature adjust immediately.

So,is wbMode OFF mode not working properly? or I am correct about the OFF mode purpose?

The final implementation of my use case that I am seeking is the following:
1) Run a simple capture to display pipeline using nvcamerasrc with wbMode=OFF
2) Change the wbMode setting to AUTO mode for around 5 seconds to calibrate the camera with the
scene.
3) Change the wbMode=OFF, so the camera maintains the white balances settings calibrated in the
above step.

Those this use case have any sense?

Thank you in advance for any help.

Best regards,
-Daniel

How do you check the white balance settings seems to always adjust and correct color temperature?

Hi ShaneCCC,

When I do the tests as described in the previous post, the way that I used to check the color temperature adjustment, was zoom in and out progressively towards the camera lens a solid color object (totally green or red for example). When the wbMode=AUTO, the color temperature of the solid color object adjusts automatically, the same occurs when wbMode=OFF. On the other hand, when wbMode=MANUAL, no matter who much I zoom in/out the object towards the camera lens, the color temperature of the object always remains the same, no adjustment is executed.

The behavior of the wbMode=MANUAL mode is as expected because it uses a default fixed valued for the white balances gains, so if you do not modify those gains, the color temperature should not suffer any change.

You can determine when an adjustment of the color temperature is applied, by observing that the color tone changes and becomes more gray when you bring the object closer and then the color becomes more intense when you move it away.

The final implementation of my use case that I am seeking is the following:

  1. Run a simple capture to display pipeline using nvcamerasrc with wbMode=OFF
  2. Change the wbMode setting to AUTO mode for around 5 seconds to calibrate the camera with the
    scene.
  3. Change the wbMode=OFF, so the camera maintains the white balance settings calibrated in the
    above step.

In regards with my use case, I cant’t switch to the MANUAL mode because it doesn’t use the same WB gains calculated on the AUTO mode, rather it uses some default fixed values for the gains, that then you can manually adjust. So, considering that I need the automatically adjustment of the WB gains provided by the AUTO mode, and then stops automatic adjustment but maintain the last WB gains configured, the MANUAL mode is not an option for me.

The wbMode=OFF should be exactly what I am seeking for, but apparently it isn’t working and exposes the same behavior as the AUTO mode.

Is the OFF mode not working properly or I am wrong accordingly to the OFF mode purpose?

Thanks in advance for any help!

Best regards,
-Daniel

@dgarba
Could you try the argus_camera and the AWB Lock possible match your request.

@ShaneCCC

I will give a try to it for testing purposes and post my result on this post. But, accordingly to the current project stage and the progress achieved until now, I think that is difficult and hard to change our GStreamer nvcamerasrc approach with libargus.

For more information, I also test my use case with nvgstcapture-1.0, but got the same results. The wbMode=OFF is not working properly.

I think that I must have to use nvcamerasrc. Could you provide me a fix for the nvcamerasrc wbMode=OFF mode? Maybe you could fix this bug and provided me a compiled binary fixed version of the nvcamerasrc plugin as as you have done before?

Thanks for all the provided help!

@ShaneCCC

I have tested your suggestion of using the argus_camera app with the “awb_lock” property switched between off and on state. That is exactly what I am looking for. I ran the argus_camera live preview test with wb_mode=Auto and awb_lock=OFF, in this case the white balance automatically adjust the color temperature of the scene. Once I calibrated the scene, then I changed to awb_lock=ON, in that case the color temperature do not automatically adjust and maintains the same calibrated settings just before I changed to awb_lock=ON.

I review the nvcamerasrc element with gst-inspect, looking for the “awb_lock” property, but unfortunately this property isn’t exposed on the nvcamerasrc plugin. I noticed that “ae_lock” property is both exposed in argus_camera app and nvcamerasrc plugin. Why is “awb_lock” property not exposed on nvcamerasrc plugin?

At the current stage of my project, it is not viable to change the capture implementation from nvcamerasrc to libargus. Is there any way that you could help me in exposing the “awb_lock” property in the nvcamerasrc plugin?

Maybe you could provided me a modified binary version of the nvcamerasrc plugin that has the “awb_lock” property exposed? In the past, Nvidia has provided me modified binary versions of nvcamerasrc that removes framerate and queue size limitations.

Maybe you could share the src code of the nvcamerasrc plugin so I can modify it and expose the “awb_lock” property?

I really need that property exposed on the nvcamerasrc plugin, so I can continue with my project development. Any help on that will be much appreciated.

In case that you could provide me the modified binary version of nvcamerasrc, this are my system settings:

  • TX1
  • Jetpack 3.0
  • L4T-24.2.1
  • kernel-3.10.96

Best regards,
-Daniel

Try this lib. Replace the lib at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcamera.so
libgstnvcamera.so.txt (109 KB)

Hi @ShaneCCC.

Thanks a lot for the quick response in providing a modified binary of nvcamerasrc.

I have tested the modified binary, but unfortunately it doesn’t has the “awb_lock” property exposed. When I executed “gst-inspect-1.0 nvcamerasrc” the following message appears:

(gst-inspect-1.0:1828): GLib-GObject-WARNING **: When installing property: type 'GstNvCameraSrc' already has a property named 'aeLock'

I think that this message appears because there is a duplicate name of the property “ae_lock”. I suppose that maybe, you used the code of the “ae_lock” property as base to expose the “awb_lock” property, but forgot to change the name of the property on the property installation/registration section of the code.

Also, I need and extra change that you made it on the past on the same binary. I need to modify the “queue-size” property range and default value as follows:

queue-size          : Number of buffers for the driver to enqueue
                        flags: readable, writable
                        Integer. Range: 2 - 100 Default: 4

The normal settings are:

queue-size          : Number of buffers for the driver to enqueue
                        flags: readable, writable
                        Integer. Range: 10 - 100 Default: 10

This modification to the queue-size settings, is required to reduced the glass to glass latency on the video capture and display task. Some months ago on this post (https://devtalk.nvidia.com/default/topic/1023668/jetson-tx1/nvcamerasrc-queue-size-property-limitation/post/5208881/#5208881) Nvidia provided me the nvcamerasrc binary with that queue-size modification.

Since I am working on the same project, I need both modifications (“awb_lock” property exposed and “queue-size” range and default values modification) applied to this requested modified nvcamerasrc binary.

As a reminder my system settings are:

  • TX1
  • Jetpack 3.0
  • L4T-24.2.1
  • kernel-3.10.96

Thanks a lot for all the provided help. I am very glad with the Nvidia support service.

Below you will find the whole “gst-inspect-1.0 nvcamerasrc” output log info of the current provided nvcamerasrc modified binary:

ubuntu@tegra-ubuntu:~$ gst-inspect-1.0 nvcamerasrc

(gst-inspect-1.0:1828): GLib-GObject-WARNING **: When installing property: type 'GstNvCameraSrc' already has a property named 'aeLock'
Factory Details:
  Rank                     primary (256)
  Long-name                Nvidia Camera Source
  Klass                    Video/Capture
  Description              Nvidia Camera Source
  Author                   Jitendra Kumar <jitendrak@nvidia.com>

Plugin Details:
  Name                     nvcamera
  Description              Nvidia Video Capture Component 
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcamera.so
  Version                  1.2.0
  License                  Proprietary
  Source module            nvcamera
  Binary package           NvCamera
  Origin URL               http://nvidia.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSrc
                         +----GstNvCameraSrc

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
                 format: { I420, NV12, UYVY }
              framerate: [ 0/1, 120/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_nvcamerasrc_change_state

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "nvcamerasrc0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  blocksize           : Size in bytes to read per buffer (-1 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  num-buffers         : Number of buffers to output before sending EOS (-1 = unlimited)
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  typefind            : Run typefind before negotiating
                        flags: readable, writable
                        Boolean. Default: false
  do-timestamp        : Apply current stream time to buffers
                        flags: readable, writable
                        Boolean. Default: true
  queue-size          : Number of buffers for the driver to enqueue
                        flags: readable, writable
                        Integer. Range: 10 - 100 Default: 10 
  wbmode              : White balance affects the color temperature of the photo
                        flags: readable, writable
                        Enum "GstNvCamWBMode" Default: 1, "auto"
                           (0): off              - GST_NVCAM_WB_MODE_OFF
                           (1): auto             - GST_NVCAM_WB_MODE_AUTO
                           (2): incandescent     - GST_NVCAM_WB_MODE_INCANDESCENT
                           (3): fluorescent      - GST_NVCAM_WB_MODE_FLUORESCENT
                           (4): warm-fluorescent - GST_NVCAM_WB_MODE_WARM_FLUORESCENT
                           (5): daylight         - GST_NVCAM_WB_MODE_DAYLIGHT
                           (6): cloudy-daylight  - GST_NVCAM_WB_MODE_CLOUDY_DAYLIGHT
                           (7): twilight         - GST_NVCAM_WB_MODE_TWILIGHT
                           (8): shade            - GST_NVCAM_WB_MODE_SHADE
                           (9): manual           - GST_NVCAM_WB_MODE_MANUAL
  scene-mode          : property to set scene mode
                        flags: readable, writable
                        Enum "GstNvCamSceneMode" Default: 0, "face-priority"
                           (0): face-priority    - GST_NVCAM_SCENE_MODE_FACE_PRIORITY
                           (1): action           - GST_NVCAM_SCENE_MODE_ACTION
                           (2): portrait         - GST_NVCAM_SCENE_MODE_PORTRAIT
                           (3): landscape        - GST_NVCAM_SCENE_MODE_LANDSCAPE
                           (4): night            - GST_NVCAM_SCENE_MODE_NIGHT
                           (5): night-portrait   - GST_NVCAM_SCENE_MODE_NIGHT_PORTRAIT
                           (6): theatre          - GST_NVCAM_SCENE_MODE_THEATRE
                           (7): beach            - GST_NVCAM_SCENE_MODE_BEACH
                           (8): snow             - GST_NVCAM_SCENE_MODE_SNOW
                           (9): sunset           - GST_NVCAM_SCENE_MODE_SUNSET
                           (10): steady-photo     - GST_NVCAM_SCENE_MODE_STEADYSHOT
                           (11): fireworks        - GST_NVCAM_SCENE_MODE_FIREWORKS
                           (12): sports           - GST_NVCAM_SCENE_MODE_SPORTS
                           (13): party            - GST_NVCAM_SCENE_MODE_PARTY
                           (14): candle-light     - GST_NVCAM_SCENE_MODE_CANDLE_LIGHT
                           (15): barcode          - GST_NVCAM_SCENE_MODE_BARCODE
  color-effect        : property to set color effect
                        flags: readable, writable
                        Enum "GstNvCamColorEffectMode" Default: 1, "off"
                           (1): off              - GST_NVCAM_COLOR_EFFECT_OFF
                           (2): mono             - GST_NVCAM_COLOR_EFFECT_MONO
                           (3): negative         - GST_NVCAM_COLOR_EFFECT_NEGATIVE
                           (4): solarize         - GST_NVCAM_COLOR_EFFECT_SOLARIZE
                           (5): sepia            - GST_NVCAM_COLOR_EFFECT_SEPIA
                           (6): posterize        - GST_NVCAM_COLOR_EFFECT_POSTERIZE
                           (7): aqua             - GST_NVCAM_COLOR_EFFECT_AQUA
  auto-exposure       : property to set auto exposure mode
                        flags: readable, writable
                        Enum "GstNvCamAutoExposureMode" Default: 2, "on"
                           (1): off              - GST_NVCAM_AUTO_EXPOSURE_OFF
                           (2): on               - GST_NVCAM_AUTO_EXPOSURE_ON
                           (3): OnAutoFlash      - GST_NVCAM_AUTO_EXPOSURE_AUTOFLASH
                           (4): OnAlwaysFlash    - GST_NVCAM_AUTO_EXPOSURE_ALWAYSFLASH
                           (5): OnFlashRedEye    - GST_NVCAM_AUTO_EXPOSURE_AUTO_RED_EYE
  flash               : property to set flash mode
                        flags: readable, writable
                        Enum "GstNvCamFlashMode" Default: 0, "off"
                           (0): off              - GST_NVCAM_FLASH_OFF
                           (1): on               - GST_NVCAM_FLASH_ON
                           (2): torch            - GST_NVCAM_FLASH_TORCH
                           (3): auto             - GST_NVCAM_FLASH_AUTO
  flicker             : property to select flicker detection and avoidance mode
                        flags: readable, writable
                        Enum "GstNvCamFlickerMode" Default: 3, "auto"
                           (0): off              - GST_NVCAM_FLICKER_OFF
                           (1): 50Hz             - GST_NVCAM_FLICKER_50HZ
                           (2): 60Hz             - GST_NVCAM_FLICKER_60HZ
                           (3): auto             - GST_NVCAM_FLICKER_AUTO
  contrast            : property to adjust contrast value
                        flags: readable, writable
                        Float. Range:               0 -               1 Default:               0 
  saturation          : property to adjust saturation value
                        flags: readable, writable
                        Float. Range:               0 -               2 Default:               1 
  tnr-strength        : property to adjust temporal noise reduction strength
                        flags: readable, writable
                        Float. Range:               0 -               1 Default:               0 
  tnr-mode            : property to select temporal noise reduction mode
                        flags: readable, writable
                        Enum "GstNvCamTNRMode" Default: 0, "NoiseReduction_Off"
                           (0): NoiseReduction_Off - GST_NVCAM_NR_OFF
                           (1): NoiseReduction_Fast - GST_NVCAM_NR_FAST
                           (2): NoiseReduction_HighQuality - GST_NVCAM_NR_HIGHQUALITY
  edge-enhancement    : property to adjust edge enhnacement value
                        flags: readable, writable
                        Float. Range:               0 -               1 Default:              -1 
  trigger             : property to fire pre capture trigger
                        flags: readable, writable
                        Enum "GstNvCamCaptureTrigger" Default: 0, ""
                           (1): idle             - GST_NVCAM_TRIGGER_IDLE
                           (2): start            - GST_NVCAM_TRIGGER_START
  intent              : property to select capture intent
                        flags: readable, writable
                        Enum "GstNvCamCaptureIntent" Default: 1, "preview"
                           (1): preview          - GST_NVCAM_INTENT_PREVIEW
                           (2): still            - GST_NVCAM_INTENT_STILL
                           (3): video            - GST_NVCAM_INTENT_VIDEO
                           (4): video-snapshot   - GST_NVCAM_INTENT_VIDEO_SNAPSHOT
                           (5): zsl              - GST_NVCAM_INTENT_ZSL
  sensor-id           : Set the id of camera sensor to use. Default 0.)
                        flags: readable, writable
                        Integer. Range: 0 - 255 Default: 0 
  enable-exif         : Return Exif data with captured frame
                        flags: readable, writable
                        Boolean. Default: false
  enable-meta         : Return Sensor meta data with captured frame
                        flags: readable, writable
                        Boolean. Default: false
  aeRegion            : Property to set region of interest for auto exposure.
			 Use GArray, with values of ROI coordinates (top,left,bottom,right)
			 and weight in that order, to set the property.
                        flags: readable, writable
                        Boxed pointer of type "GArray"
  wbRegion            : Property to set region of interest for white balance.
			 Use GArray, with values of ROI coordinates (top,left,bottom,right)
			 and weight in that order, to set the property.
                        flags: readable, writable
                        Boxed pointer of type "GArray"
  fpsRange            : Property to set FPS range.
			 Use string with values of FPS Range (low, high)
			 in that order, to set the property.
                        flags: readable, writable
                        String. Default: null
  exposure-time       : Property to adjust exposure time.
			 This property will have effect only when Exposure mode is Off.
                        flags: readable, writable
                        Float. Range:               0 -    3.402823e+38 Default:           0.033 
  wbManualMode        : Property to set manual mode of White balance.
			 This property will have effect only when wbMode is manual.
                        flags: readable, writable
                        Enum "GstNvCamWBManualMode" Default: 0, "off"
                           (0): off              - GST_NVCAM_WB_MANUAL_OFF
                           (1): reset            - GST_NVCAM_WB_MANUAL_RESET
                           (2): restore          - GST_NVCAM_WB_MANUAL_RESTORE
                           (3): external-gains   - GST_NVCAM_WB_MANUAL_EXTGAINS
  wbGains             : Property to set external white balance Gains.
			 Use GArray, with values of WB gains (R, GR, GB, B)
			 in that order, to set the property.
			 This will have effect only when wbMode is manual.
                        flags: readable, writable
                        Boxed pointer of type "GArray"
  aeLock              : Property to lock auto exposure.
                        flags: readable, writable
                        Boolean. Default: false
  callback            : Property to set user callback. It is required if user wants 
			 to change capture settings per frame. Provide address 
			 of GstNvCamSrcCallbacks type struct as value. 

                        flags: writable
                        Pointer. Write only
  dump-bayer          : Dump raw bayer data in still capture mode.
                        flags: readable, writable
                        Boolean. Default: false
  num-sensors         : Property to get number of sensors.
			 Returns GArray containing number of sensors {usb,csi}

                        flags: readable
                        Boxed pointer of type "GArray"
  enable-surround-cameras: Enable Surround Camera (Res: 3456x2304)
                        flags: readable, writable
                        Boolean. Default: false
  stitch-in-w         : Set the stitch in W.
                        flags: readable, writable
                        Integer. Range: 0 - 4336 Default: 4336 
  stitch-in-h         : Set the stitch in H.
                        flags: readable, writable
                        Integer. Range: 0 - 2440 Default: 2440 
  surface-layout      : Set the surface layout. Default 0. (0=Blocklinear, 1=Pitch)
                        flags: readable, writable
                        Integer. Range: 0 - 1 Default: 0 
  orientation         : Sensor orientation value. This must be accessed after PAUSED state.
                        flags: readable
                        Integer. Range: 0 - 360 Default: -1 

Element Actions:
  "start-capture" :  void user_function (GstElement* object);
  "stop-capture" :  void user_function (GstElement* object);

Have a try this lib I will add the queue-size after confirm this lib is working.
libgstnvcamera.so.txt (109 KB)

Hi @ShaneCCC.

I have tested the last provided binary. Now the “awbLock” property is correctly exposed. It is shown with the “gst-inspect-1.0 nvcamerasrc” and I could set values to it without errors on the running pipeline.

Nevertheless, the “awbLock” property of nvcamerasrc is not taking any effect on the capture settings. No matter if it is set to true or false value, the color temperature is always automatically adjusting. I run again the argus_camera app, as a control point to compare the “awb_lock” behavior with the “awbLock” nvcamerasrc recently exposed property.

On the argus_camera app the “awb_lock” property is really taking effect on the capture settings. I ran the argus_camera live preview test with wb_mode=Auto and awb_lock=OFF, in this case the white balance automatically adjust the color temperature of the scene. Once I calibrated the scene, then I changed to awb_lock=ON, in that case the color temperature do not automatically adjust and maintains the same calibrated settings just before I changed to awb_lock=ON.

Maybe something is missing on the property activation function implementation in comparison with the argus_camera app.

Please could you review, what is causing this issue. I will wait for your answer.

Thanks a lot for all the provided help!

Have a try this again.
libgstnvcamera.so.txt (109 KB)

Hi @ShaneCCC.

I have tested the last provided nvcamerasrc binary and it works perfectly. The awbLock property is working exactly the same way as it does on the argus_camera app. Just what I needed.

Thanks a lot for all the provided help!