I have been experimenting with the cudaHistogram and the cudaBayerDemosaic examples in the tegra_multimedia_api code on my Jetson Xavier. I’m trying to get the timestamp and get acces to the pixel data on the planes.
If I use FrameConsumer::acquireFrame, and then cast the Frame to an IFrame, I can use getTime() to get the timestamp. However, I haven’t found any functions on the IFrame or IFrame2D interfaces that allow you to access the data - all I see is a way to get the fd for a buffer.
In the CUDA examples, I can use cudaEGLStreamConsumerAcquireFrame to pull an image from the EGL stream. I can map the cudaResource to a cudaArray, so I can access the data. However, I can’t see how I can get the timestamp using this method.
Any suggestions? I’m looking for a way to get the frame timestamp from the cudaEGLStreamComsumerAcquireFrame call (ex: cudaHistogram or cudaDebayerMosaic examples), OR, a way to access the pixel data and planes using the FrameConsumer/Frame/IFrame method.
Hi BareMetalCoder,
If you are using CudaConsumer you can get the Metadata from the EGLStream using the MetadataContainer object. Please check the following code snippet.
This sample has been implemented in the CudaBayerDemosaicConsumer::threadExecute() function.
+ UniqueObj<EGLStream::MetadataContainer> container(MetadataContainer::create(<replace EGLDisplay>, <replace EGLStream>);
+ IArgusCaptureMetadata *iArgusMeta = interface_cast<IArgusCaptureMetadata>(container.get());
+ if (!iArgusMeta)
+ ORIGINATE_ERROR("Failed to create Metadata Container");
+ CaptureMetadata *metadata = iArgusMeta->getMetadata();
+ ICaptureMetadata *iCaptureMeta = interface_cast<ICaptureMetadata>(metadata);
+ if (!iCaptureMeta)
+ ORIGINATE_ERROR("Unable to get Capture Metadata from frame");
+ uint64_t sensorTimeStamp = iCaptureMeta->getSensorTimestamp();
// On the first frame, print the information contained in the CUDA EGL frame structure.
if (frameID == 0)
{
Ok, found it. I didn’t know I had to enable metadata while setting up my EGLStream! EGL_NO_DISPLAY causes no problems.
IEGLOutputStreamSettings * iEGLStreamSettings = interface_cast<IEGLOutputStreamSettings>(streamSettings);
if (!iEGLStreamSettings) {
ORIGINATE_ERROR("Failed to create OutputStreamSettings");
<b>iEGLStreamSettings->setMetadataEnable(true);</b>
Waiss, You saved me a lot of time, as I was preparing to reimplement our pipeline using the suggestions that preceded your comment. I am deeply thankful.
Hi BareMetalCoder,
Aplologies for the late response looks like you have already figured out the change.
Anyways I am glad that I could be of your help. :)
I tried the MetadataContainer::create method as proposed in this thread.
It works great for one stream, but if I use two streams it always returns the metadata of one stream.
e.g. if you adapt the syncSensor to use the following code, it will return the same metadata twice
UniqueObj<EGLStream::MetadataContainer> metadataContainerLeft(
EGLStream::MetadataContainer::create(m_leftStream->getEGLDisplay(), m_leftStream->getEGLStream()));
EGLStream::IArgusCaptureMetadata *iArgusCaptureMetadataLeft =
interface_cast<EGLStream::IArgusCaptureMetadata>(metadataContainerLeft);
if (iArgusCaptureMetadataLeft)
{
CaptureMetadata *metadata = iArgusCaptureMetadataLeft->getMetadata();
ICaptureMetadata *iMetadata = interface_cast<ICaptureMetadata>(metadata);
if (!iMetadata)
ORIGINATE_ERROR("Failed to get ICaptureMetadata interface.");
CONSUMER_PRINT("\tSensor Timestamp: %llu, LUX: %f\n",
static_cast<unsigned long long>(iMetadata->getSensorTimestamp()),
iMetadata->getSceneLux());
}
UniqueObj<EGLStream::MetadataContainer> metadataContainerRight(
EGLStream::MetadataContainer::create(m_rightStream->getEGLDisplay(), m_rightStream->getEGLStream()));
EGLStream::IArgusCaptureMetadata *iArgusCaptureMetadataRight =
interface_cast<EGLStream::IArgusCaptureMetadata>(metadataContainerRight);
if (iArgusCaptureMetadataRight)
{
CaptureMetadata *metadata = iArgusCaptureMetadataRight->getMetadata();
ICaptureMetadata *iMetadata = interface_cast<ICaptureMetadata>(metadata);
if (!iMetadata)
ORIGINATE_ERROR("Failed to get ICaptureMetadata interface.");
CONSUMER_PRINT("\tSensor Timestamp: %llu, LUX: %f\n",
static_cast<unsigned long long>(iMetadata->getSensorTimestamp()),
iMetadata->getSceneLux());
}