9 Replies Latest reply: Nov 29, 2012 7:24 AM by Francisco Alberto Carrillo Dominguez RSS

Trying to encode H.264 files.

ricardo sousa Level 1

Greetings,

 

 

My name is Ricardo, and recently I'm working on a board based on a i.MX53 module. I'm using the 11.09 BSP and IMX_MMCODECS releases. As says the title, I'm trying to encode H.264 files from a USB camera using the VPU-based gstreamer plugins. Through several researches and a lot of pipelines, we discovered some issues:

1. The USB camera provides jpeg data(image/jpeg), so we couldn't use mfw_v4lsrc, that is for data in raw format(video/x-raw-yuv).

2.  In order to encode, we need first to decode to raw video and change the PixelFormat(Y42B to NV12). This is a important thing to do, because, despite the information of gst-inspect, the mfw_vpuencoder doesn't work with Y42B pixelformat.

3. After that, we encode to H.264 using mfw_vpuencoder.

 

 

This is the working pipeline:

gst-launch v4l2src device=/dev/video0  ! image/jpeg,width=1280,height=720,framerate=15/1 ! mfw_vpudecoder codec_type=std_mjpg loopback=true ! video/x-raw-yuv,format=\(fourcc\)Y42B ! ffmpegcolorspace ! video/x-raw-yuv,format=\(fourcc\)NV12  ! mfw_vpuencoder loopback=true codec-type=std_avc ! avimux ! filesink location=teste.avi

 

 

Although it works, it's not the better way. Because we need to use the ffmpegcolorspace plugin, the CPU usage reaches 80%. We tried to use the IPU Colorspace Plugin (mfw_ipucsc), but it it doesn't accept Y42B frames as input. We tried several ways to force the VPU decoder to output frames in I420 or NV12 format, but we failed all the times. Our approaches:

1. Set the fmt parameter of mfw_vpudecoder to 1. It supposed to generate I420 formats;

2. Put a caps after the decoder;

3. Instead of use mfw_ipucsc, use mfw_deinterlacer. Doesn't work too.

 

 

Some pipelines and its debugs:

 

gst-launch -vvv v4l2src device=/dev/video0 ! image/jpeg,width=1280,height=720,framerate=15/1 ! mfw_vpudecoder codec_type=std_mjpg ! mfw_deinterlacer chrom-fmt=0 ! fakesink

 

 

Setting pipeline to PAUSED ...

[INFO]  Product Info: i.MX53

VPU Version: firmware 13.4.32; libvpu: 5.1.4

MFW_GST_VPU_DECODER_PLUGIN 2.0.3 build on Jul 13 2012 06:30:57.

/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:sink: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)Y42B, width=(int)1280, height=(int)720, width_align=(int)16, height_align=(int)16,0

/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)Y42B, width=(int)1280, height=(int)720, width_align=(int)16, height_align=(int)16,0

Deinterlacer default input format is I420

Deinterlacer default input format is I420

/GstPipeline:pipeline0/MfwGstDeinterlace:mfwgstdeinterlace0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)Y42B, width=(int)1280, height=(int)720, width_align=(int)16, height_align=0

/GstPipeline:pipeline0/MfwGstDeinterlace:mfwgstdeinterlace0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)Y42B, width=(int)1280, height=(int)720, width_align=(int)16, height_align=0

/GstPipeline:pipeline0/MfwGstDeinterlace:mfwgstdeinterlace0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)Y42B, width=(int)1280, height=(int)720, width_align=(int)16, height_align0

 

 

gst-launch -vvv v4l2src device=/dev/video0 ! image/jpeg,width=1280,height=720,framerate=15/1 ! mfw_vpudecoder codec_type=std_mjpg fmt=1 ! mfw_ipucsc ! fakesink

 

 

IPU_CSC_CORE_LIBRARY_VERSION_INFOR_01.00.

MFW_GST_IPU_CSC_PLUGIN 2.0.3 build on Jul 13 2012 06:30:48.

Setting pipeline to PAUSED ...

[INFO]  Product Info: i.MX53

VPU Version: firmware 13.4.32; libvpu: 5.1.4

MFW_GST_VPU_DECODER_PLUGIN 2.0.3 build on Jul 13 2012 06:30:57.

/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:sink: caps = image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)15/1

/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)Y42B, width=(int)1280, height=(int)720, width_align=(int)16, height_align=(int)16,0

ERROR: from element /GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0: fatal error

Additional debug info:

Allocation of the Frame Buffers Failed

Execution ended after 11049559049 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:src: caps = NULL

/GstPipeline:pipeline0/MfwGstVPU_Dec:mfwgstvpu_dec0.GstPad:sink: caps = NULL

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = NULL

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = NULL

/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = NULL

Setting pipeline to NULL ...

 

 

 

 

>>VPU_DEC: State: Ready to Null

Freeing pipeline ...

[--->FINALIZE vpu_dec

 

 

 

As you can see, I can't generate another format different of Y42B.

Somebody knows a better way to link these plugins?

 

 

Thanks in advance,

Ricardo Gurgel.