Libxcoder supports hardware filters in Quadra that make use of the hardware scaling, cropping, padding, and overlay features of the 2D engine, provide access to the multiple outputs of the decoder and to support transferring software AVFrames to the hardware so they can be used as hardware AVFrames.
The table below lists the NETINT hardware filters, all of which are implemented using features of the 2D and AI engines (except for ni_quadra_split and ni_quadra_hwupload).
Ni_quadra_split is used to provide access to the 2nd and 3rd decoder output. The FFmpeg split function can also be used on hardware AVFrames but will only split the first output.
Ni_quadra_hwupload transfers the YUV data from a software frame to the hardware and generates a hardware frame as an output.
See also APPS548 Codensity Quadra Software and Hardware Frames (YUVbypass) Application Note to learn more about hardware frames.
Quadra libxcoder filters
Filter name | Description | Resources Used |
|---|---|---|
ni_quadra_scale | NETINT Scaling Filter | 2D Engine |
ni_quadra_overlay | NETINT Overlay filter | 2D Engine |
ni_quadra_split | NETINT Split filter (gives access to 2nd and 3rd decoder outputs) | |
ni_quadra_crop | NETINT Crop filter | 2D Engine |
ni_quadra_pad | NETINT Pad filter | 2D Engine |
ni_quadra_hwupload | NETINT Hardware Upload Filter (transfers a software YUV or RGBA frame to the hardware for encode or filtering) | |
ni_quadra_roi | NETINT ROI filter | AI Engine, 2D Engine |
ni_quadra_bg | NETINT background removal filter | AI Engine, 2D Engine |
ni_quadra_xstack | NETINT Stacking filter | 2D Engine |
ni_quadra_rotate | NETINT Rotate Filter | 2D Engine |
ni_quadra_drawbox | NETINT Draw box Filter | 2D Engine |
ni_quadra_drawtext | NETINT Draw text Filter | 2D Engine |
ni_quadra_delogo | NETINT Delogo Filter | 2D Engine |
ni_quadra_merge | NETINT Merge Filter | 2D Engine |
ni_quadra_flip | NETINT Flip Filter | 2D Engine |
ni_quadra_hvsplus | NETINT HVS+ Filter | AI Engine, 2D Engine |
ni_quadra_scale2ref | NETINT Scaling Filter | 2D Engine |
The NetInt 2D Engine filters are hardware-assisted filters to provide the application the ability to scale, crop, pad, overlay, stack, rotate, and draw video inside the hardware without the need to transfer YUV data to and from the host. For example if we decode video in 16:9 aspect ratio, scale, crop, then re-encode to video with a 4:3 aspect ratio, we can do all of this in the hardware without needing to transfer any YUV using hardware frames and the Netint filters. If we were to use FFmpeg’s native scale and crop filters we would need to transfer the decoded YUV to the host to scale and crop and then transfer back to the hardware to encode.
The ni_quadra_scale, ni_quadra_overlay, ni_quadra_crop, ni_quadra_pad, ni_quadra_xstack, ni_quadra_rotate, ni_quadra_drawtext, ni_quadra_drawbox, ni_quadra_delogo, ni_quadra_merge, ni_quadra_flip and ni_quadra_scale2ref filters only work with hardware frames. To use these NETINT filters with software frames, you must first use ni_quadra_hwupload to upload the software frame to the Quadra device to create a hardware frame. Similarly, FFmpeg has a native hwdownload filter to retrieve the YUV data from the hardware, converting a hardware frame to a software frame.
The ni_quadra_split filter may be used on software frames though the behavior will default to what the FFmpeg native split filter will do. The ni_quadra_roi and ni_quadra_hvsplus filters work with hardware and software frames.
The 2D Engine supports raster YUV input and output as well as RGBA format for overlay. It requires input pictures to have even width and height for RGBA. For YUV the stride must be a multiple of 128 bytes for both the luma and chroma planes. The minimum picture height is 16 pixels and the minimum picture width is 16 pixels.
The maximum resolution for ARGB, ABGR, RGBA, BGRA is 7040x7040. The maximum resolution for other pixel formats is 8192x8192.
For ni_quadra_overlay, ni_quadra_crop, ni_quadra_pad, ni_quadra_drawtext, ni_quadra_drawbox, ni_quadra_delogo, ni_quadra_merge and ni_quadra_flip the output format is the same as input format. It depends on the input formats supported by ni_quadra_hwupload (see section 10.1.6).