problem with extracting image data array of RGBA format from dwImageCPU

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hey there,

I’m currently developing with DPX2 base on Driveworks 5.0.5.0a using Driveworks Version6.0. I have a question about extracting image data array of RGBA format from dwImageCPU.

Here, dwImage_streamer is used for posting dwImageNVMEDIA to dwImageCPU (dwImageType of dwImageNVMEDIA has already covert to RGBA through an ImageFormatCoverter), in this way, we can extract the uint8_t * image data array from the dwImageCPU struct theoretically. But we got an data array of size 0(which means there is no handle pointing to any Rgba image data array in dwImageCPU>>(uint8_t ) data[DW_MAX_IMAGE_PLANES]). By the way, we have already confirmed that the input dwImageNVMEDIA is valid by passing through a dwImage_streamer to dw_Image_GL, and receiving display successfully. More to mention, both dwImageStreamer_postNVMEDIA and dwImageStreamer_receiveCPU resulted as DW_SUCCESS.

Is there some determinant parameter causing this error? And what is the difference between DW_CAMERA_PROCESSED_IMAGE and DW_CAMERA_RAW_IMAGE in dwCameraOutputType while using function dwSensorCamera_getImageProperties? Will it be the key point of initializing dwImage_streamer?

Looking forward for your answer
Thank you

Best Regards
Chaofan

2

Dear chaofanyin,
DW_CAMERA_RAW_IMAGE captures the image in RAW format, DW_CAMERA_PROCESSED_IMAGE captures image in processed(yuv) format. Have you checked printing data[0] array? can you double check dwImageProperties of dwImageCPU?
I believe, you have captured image DW_CAMERA_PROCESSED_IMAGE flag and used dwImageFormatConverter_copyConvertNvMedia to convert to RGBA format before sending it to CPU using imageStreamer.

3

Thank you for your quick reply. And I do process the image frame as you described. I have double checked the dwImageProperties of dwImageCPU, and the properties seem to be normal(type=0[dwImageCPU], width=1280, height=1080, pixelFormat=3000[RGBA],pxlType=64).By the way, the image size should be 12801080,however dwImageCPU.pitch[0] equals to 5120(which should be 12801080*4), and data[0] prints to be empty.
Did something wrong? Or is there any other way to extract the RGBA image and save it?

Looking forward to your apply.

4

hey there,

can anyone help me to figure out this issue. Really appreciate it!

Thanks
Chaofan

5

Dear chaofanyin,
Is it possible to share the code here to figure out the issue?

6

———————Initialize———————————
dwImageProperties cameraImageProperties;
dwSensorCamera_getImageProperties(&cameraImageProperties, DW_CAMERA_PROCESSED_IMAGE, cameraSensor[csiPort].sensor);
dwImageProperties nvm2cudaImageProperties = cameraImageProperties;
nvm2cudaImageProperties.pxlFormat = DW_IMAGE_RGBA;
nvm2cudaImageProperties.planeCount = 1;
std::cout<<"image width : "<<nvm2cudaImageProperties.width<<std::endl;
std::cout<<"image height : "<<nvm2cudaImageProperties.height<<std::endl;
result = dwImageStreamer_initialize(&cameraSensor[csiPort].nvm2cpu, &nvm2cudaImageProperties, DW_IMAGE_CPU, sdk);
if (result != DW_SUCCESS) {
std::cerr << "Cannot init image streamer nvm2cpu: " << dwGetStatusName(result) << std::endl;
g_run = false; }
———————Calling Fuction———————————
result = dwImageStreamer_postNvMedia(g_frameRGBAPtr[csiPort][cameraIdx],
cameraSensor[csiPort].nvm2cpu);
if (result != DW_SUCCESS) {
std::cerr << "Cannot post nvmedia to cpu: " << dwGetStatusName(result) << std::endl;
}else{
dwImageCPU frameCPU = nullptr;
std::string filename;
result = dwImageStreamer_receiveCPU(&frameCPU, 60000, cameraSensor[csiPort].nvm2cpu);//CPU receive image
std::cout<<"receiveCPU: "<<result<<std::endl;
if (result == DW_SUCCESS && frameCPU)
{
std::cout<<"sizeofdata: "<<strlen((char)(frameCPU->data[0]))<<std::endl;
std::cout<<“sizeofpitch: “<pitch[0]<<std::endl;
std::cout<<“ImageCPUProperties:type”<prop.type;
std::cout<<“ImageCPUProperties:width”<prop.width;
std::cout<<“ImageCPUProperties:height”<prop.height;
std::cout<<“ImageCPUProperties:dwImagePixelFormat”<prop.pxlFormat;
std::cout<<“ImageCPUProperties:dwTrivialDataType”<prop.pxlType;
(char*)memcpy(RGBAarray,(char*)(frameCPU->data[0]),128010804);
filename += std::string(“port_”) + std::to_string(csiPort) + std::string(“cam”)+std::to_string(cameraIdx) + std::string(”_Image.txt”);
FILE *fp=fopen(filename.c_str(),“w+”);
fprintf(fp, “%s”, RGBAarray);
fclose(fp);
}
dwImageStreamer_returnReceivedCPU(frameCPU, cameraSensor[csiPort].nvm2cpu);
}
// return frame which has been lately processed by the streamer
dwImageNvMedia *processed = nullptr;
result = dwImageStreamer_waitPostedNvMedia(&processed, 60000, cameraSensor[csiPort].nvm2cpu);
if (result != DW_SUCCESS) {
std::cerr << "Cannot waitpost nvmedia from cpu: " << dwGetStatusName(result) << std::endl;
g_run = false;
break; }
————————————————————————————————————————————————————————————
This code is created basing on sample_camera_multiple_gmsl. Our purpose is to capture image array in dwImageCPU and certificate its validity by saving as file with the above code. However it results in an empty txt file, and we have tried post dwImageCPU to dwImageGL with DW_BUSY_WAITING returned and display failed.
Looking forward to your reply!

7

Dear chaofanyin,
We will look into it. Can you quickly check printing data[0], data[1]… before writing it to file. Also, I do not see any dwImageFormatConverter_copyConvertNvMedia calls in the code as you mentioned.

8

Dear chaofanyin,
I have modified camera_gsml sample to get camera image as RGBA values on CPU side. I have replaced GL consumer with CPU consumer. I am sharing the code snippet below.

if (status != DW_SUCCESS) {
                    std::cout << "\n ERROR postNvMedia: " << dwGetStatusName(status) << std::endl;
                } else {
                    dwImageCPU *frameCPU = nullptr;
		    std::string filename;
                    status             = dwImageStreamer_receiveCPU(&frameCPU, 60000, nvm2gl);

                    if (status == DW_SUCCESS && frameCPU) {
                       
			std::cout << "\nsizeofdata: " << ((char*)(frameCPU->data[0])) << "\n";
			std::cout << "sizeofpitch: " << frameCPU->pitch[0] << "\n";
			std::cout<<"type "<<frameCPU->prop.type;
			std::cout<<"width "<<frameCPU->prop.width;
			std::cout<<"height "<<frameCPU->prop.height;
			std::cout<<"dwImagePixelFormat "<<frameCPU->prop.pxlFormat;
			std::cout<<"dwTrivialDataType "<<frameCPU->prop.pxlType<< "\n";
			memcpy(RGBAarray,(char*)(frameCPU->data[0]),frameCPU->prop.width * frameCPU->prop.height * sizeof(char));
			filename =   std::to_string(count)+ std::string("_Image.txt");
			FILE *fp=fopen(filename.c_str(),"w+");
			fprintf(fp, "%s", RGBAarray);
			fclose(fp);
			count++;
                        dwImageStreamer_returnReceivedCPU(frameCPU, nvm2gl);
                    }
                }

                // any image returned back, we put back into the pool
                dwImageNvMedia *retimg = nullptr;
                dwImageStreamer_waitPostedNvMedia(&retimg, 33000, nvm2gl);

Looks like you are trying to copy 128010804 values. It has to be 1280*1080(frameCPU->prop.width * frameCPU->prop.height). I hope this helps.

9

Thank you so much for replying. The problem has been solved successfully.