1. to enumerate

There are many codecs commonly used in the industry, some purely use software such as x264, and some rely on hardware. For example, Nvidia's NVENC development kit uses its own graphics card GPU acceleration, and Intel's Quick Sync development kit uses the integrated GPU acceleration on Intel CPU.

By encapsulating the above-mentioned third-party codecs and a few self-implemented codecs, ffmpeg abstractly defines a AVCodecstructure and supports ffmpeg -decodersenumerating all supported decoders through commands. Since there is too much content, the ffmpeg -decoders | grep h264h264 decoder is displayed below. :

VFS..D h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10V....D h264_qsv             H264 video (Intel Quick Sync Video acceleration) (codec h264)V..... h264_cuvid           Nvidia CUVID H264 decoder (codec h264)

The first decoding implementation is the official development of ffmpeg. As can also be seen by the name, the second is based on Intel QSV and the third is based on Nvidia graphics cards. The meaning of the beginning of each decoder is as follows, and when grep is piped from the command line, the output is filtered:

Decoders：V..... = Video A..... = Audio S..... = Subtitle .F.... = Frame-level multithreading ..S... = Slice-level multithreading ...X.. = Codec is experimental ....B. = Supports draw_horiz_band .....D = Supports direct rendering method 1

Similarly, use ffmpeg -encoders | grep h264 to display the h264 encoder:

V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)V....D h264_amf             AMD AMF H.264 Encoder (codec h264)V....D h264_mf              H264 via MediaFoundation (codec h264)V....D h264_nvenc           NVIDIA NVENC H.264 encoder (codec h264)V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)V..... nvenc                NVIDIA NVENC H.264 encoder (codec h264)V..... nvenc_h264           NVIDIA NVENC H.264 encoder (codec h264)

The first is based on x264, Intel QSV with the word qsv, and Nvidia graphics card with the word nvenc. The meaning of the beginning part is as follows:

Encoders：V..... = Video A..... = Audio S..... = Subtitle .F.... = Frame-level multithreading ..S... = Slice-level multithreading ...X.. = Codec is experimental ....B. = Supports draw_horiz_band .....D = Supports direct rendering method 1

2. AVCodec

Each codec describes its own data by defining its own AVCodec structure variable, providing name, type, CodecId, several key callback functions, etc., so that external visitors can shield the differences between different implementations as long as they find the AVCodec corresponding to the codec :

Type

  • Audio AVMEDIA_TYPE_AUDIO

  • Video AVMEDIA_TYPE_VIDEO

  • subtitle AVMEDIA_TYPE_AUDIO

CodecId

  • AV_CODEC_ID_H264

  • AV_CODEC_ID_AC3

Callback

  • init

  • close

  • flush

  • receive_packet

  • decode

  • receive_frame

The standard for distinguishing AVCodec is very simple. Those AVCodecs that provide decode or receive_frame function pointers represent decoders, otherwise they are encoders. The following is the definition of FFMPEG's built-in h264 decoder (using CPU decoding), which is derived from libavcodec/h264dec.c and only retains key data:

AVCodec ff_h264_decoder = {    .name                  = "h264",    .long_name             = NULL_IF_CONFIG_SMALL("H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10"),    .type                  = AVMEDIA_TYPE_VIDEO,    .id                    = AV_CODEC_ID_H264,    .priv_data_size        = sizeof(H264Context),    .init                  = h264_decode_init,    .close                 = h264_decode_end,    .decode                = h264_decode_frame,    .capabilities          = /*AV_CODEC_CAP_DRAW_HORIZ_BAND |*/ AV_CODEC_CAP_DR1 |AV_CODEC_CAP_DELAY | AV_CODEC_CAP_SLICE_THREADS |AV_CODEC_CAP_FRAME_THREADS,                               HWACCEL_DXVA2(h264),    .caps_internal         = FF_CODEC_CAP_INIT_THREADSAFE | FF_CODEC_CAP_EXPORTS_CROPPING |                             FF_CODEC_CAP_ALLOCATE_PROGRESS | FF_CODEC_CAP_INIT_CLEANUP,    .flush                 = h264_decode_flush,    .update_thread_context = ONLY_IF_THREADS_ENABLED(ff_h264_update_thread_context),    .profiles              = NULL_IF_CONFIG_SMALL(ff_h264_profiles),    .priv_class            = &h264_class,};

3. Nvidia NvENC with hardware acceleration

ffmpeg supports Nvidia NvENC hardware codec h264, h265, av1, vp9 and other video formats. It defines different decoder corresponding AVCodec variables for each video format in libavcodec/cuviddec.c:

  • ff_av1_cuvid_decoder

  • ff_h264_cuvid_decoder

  • ff_hevc_cuvid_decoder

  • ff_vp9_cuvid_decoder

The name string in each variable corresponds to the decoder name listed by ffmpeg -decoders | grep cuvid:

  • av1_cuvid

  • h264_cuvid

  • hevc_cuvid

  • vp9_cuvid

4. Codec array codec_list

In libavcodec/allcodecs.c, declare the codec variables scattered in different source codes one by one through extern syntax, for example:

extern AVCodec ff_av1_cuvid_decoder;extern AVCodec ff_h264_cuvid_decoder;extern AVCodec ff_hevc_cuvid_decoder;extern AVCodec ff_vp9_cuvid_decoder;

Then when configure is executed, the variable name of the encoder with the encoder name is extracted and filled into the codec_list.c array. Similarly, the decoder continues to be extracted and filled. After that, the codec_list.c array contains all the codecs supported by ffmpeg, and is used by libavcodec/allcodecs. c provides API to access her.

Here is the corresponding code in the script:

find_things_extern(){    thing=$1    pattern=$2    file=$source_path/$3    out=${4:-$thing}    sed -n "s/^[^#]*extern.*$pattern *ff_\([^ ]*\)_$thing;/\1_$out/p" "$file"}
find_filters_extern(){    file=$source_path/$1    sed -n 's/^extern AVFilter ff_[avfsinkrc]\{2,5\}_\([[:alnum:]_]\{1,\}\);/\1_filter/p' $file}
ENCODER_LIST=$(find_things_extern encoder AVCodec libavcodec/allcodecs.c)DECODER_LIST=$(find_things_extern decoder AVCodec libavcodec/allcodecs.c)CODEC_LIST="    $ENCODER_LIST    $DECODER_LIST"