H264 Analysis Tools
Introduction
This wiki contains a list of commands based on open source software that will help you analyze an H.264 bit stream.
Dependencies
Most of the commands are based on the very great FFMPEG toolkit.
To install FFMPEG in Debian based systems run:
sudo apt install ffmpeg
To install FFMPEG in Mac OS using Brew run:
brew install ffmpeg
Plot Frame Type and Size
The following command will plot the frame size and type vs time.
plotframes -I file.h264
Sample output:
Display Frame Information
In order to display per-frame information you may use the following command.
ffprobe -v trace -show_frames file.h264
Sample output:
[FRAME] media_type=video stream_index=0 key_frame=1 pkt_pts=N/A pkt_pts_time=N/A pkt_dts=N/A pkt_dts_time=N/A best_effort_timestamp=N/A best_effort_timestamp_time=N/A pkt_duration=48000 pkt_duration_time=0.040000 pkt_pos=0 pkt_size=1606 width=320 height=240 pix_fmt=yuv420p sample_aspect_ratio=1:1 pict_type=I coded_picture_number=0 display_picture_number=0 interlaced_frame=0 top_field_first=0 repeat_pict=0 color_range=tv color_space=smpte170m color_primaries=smpte170m color_transfer=smpte170m chroma_location=center [SIDE_DATA] side_data_type=H.26[45] User Data Unregistered SEI message [/SIDE_DATA] [/FRAME] [h264 @ 0x7fd318813800] nal_unit_type: 9(AUD), nal_ref_idc: 0 [h264 @ 0x7fd318813800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 [FRAME]t message repeated 1 times media_type=video stream_index=0 key_frame=0 pkt_pts=N/A pkt_pts_time=N/A pkt_dts=N/A pkt_dts_time=N/A best_effort_timestamp=N/A best_effort_timestamp_time=N/A pkt_duration=40000 pkt_duration_time=0.033333 pkt_pos=1606 pkt_size=545 width=320 height=240 pix_fmt=yuv420p sample_aspect_ratio=1:1 pict_type=P coded_picture_number=1 display_picture_number=0 interlaced_frame=0 top_field_first=0 repeat_pict=0 color_range=tv color_space=smpte170m color_primaries=smpte170m color_transfer=smpte170m chroma_location=center [/FRAME]
Display Nal Unit / Slice Information
In order to display per-nal unit (including separate slices) you may run the following command:
ffmpeg -i file.h264 -c copy -bsf:v trace_headers -f null -
Sample output:
[trace_headers @ 0x7f81a8416d00] Access Unit Delimiter [trace_headers @ 0x7f81a8416d00] 0 forbidden_zero_bit 0 = 0 [trace_headers @ 0x7f81a8416d00] 1 nal_ref_idc 00 = 0 [trace_headers @ 0x7f81a8416d00] 3 nal_unit_type 01001 = 9 [trace_headers @ 0x7f81a8416d00] 8 primary_pic_type 001 = 1 [trace_headers @ 0x7f81a8416d00] 11 rbsp_stop_one_bit 1 = 1 [trace_headers @ 0x7f81a8416d00] 12 rbsp_alignment_zero_bit 0 = 0 [trace_headers @ 0x7f81a8416d00] 13 rbsp_alignment_zero_bit 0 = 0 [trace_headers @ 0x7f81a8416d00] 14 rbsp_alignment_zero_bit 0 = 0 [trace_headers @ 0x7f81a8416d00] 15 rbsp_alignment_zero_bit 0 = 0 [trace_headers @ 0x7f81a8416d00] Slice Header [trace_headers @ 0x7f81a8416d00] 0 forbidden_zero_bit 0 = 0 [trace_headers @ 0x7f81a8416d00] 1 nal_ref_idc 10 = 2 [trace_headers @ 0x7f81a8416d00] 3 nal_unit_type 00001 = 1 [trace_headers @ 0x7f81a8416d00] 8 first_mb_in_slice 1 = 0 [trace_headers @ 0x7f81a8416d00] 9 slice_type 00110 = 5 [trace_headers @ 0x7f81a8416d00] 14 pic_parameter_set_id 1 = 0 [trace_headers @ 0x7f81a8416d00] 15 frame_num 0100 = 4 [trace_headers @ 0x7f81a8416d00] 19 num_ref_idx_active_override_flag 0 = 0 [trace_headers @ 0x7f81a8416d00] 20 ref_pic_list_modification_flag_l0 0 = 0 [trace_headers @ 0x7f81a8416d00] 21 adaptive_ref_pic_marking_mode_flag 0 = 0 [trace_headers @ 0x7f81a8416d00] 22 slice_qp_delta 00000110101 = -26 [trace_headers @ 0x7f81a8416d00] 33 disable_deblocking_filter_idc 1 = 0 [trace_headers @ 0x7f81a8416d00] 34 slice_alpha_c0_offset_div2 1 = 0 [trace_headers @ 0x7f81a8416d00] 35 slice_beta_offset_div2 1 = 0 [trace_headers @ 0x7f81a8416d00] Slice Header [trace_headers @ 0x7f81a8416d00] 0 forbidden_zero_bit 0 = 0 [trace_headers @ 0x7f81a8416d00] 1 nal_ref_idc 10 = 2 [trace_headers @ 0x7f81a8416d00] 3 nal_unit_type 00001 = 1 [trace_headers @ 0x7f81a8416d00] 8 first_mb_in_slice 000000010100001 = 160 [trace_headers @ 0x7f81a8416d00] 23 slice_type 00110 = 5 [trace_headers @ 0x7f81a8416d00] 28 pic_parameter_set_id 1 = 0 [trace_headers @ 0x7f81a8416d00] 29 frame_num 0100 = 4 [trace_headers @ 0x7f81a8416d00] 33 num_ref_idx_active_override_flag 0 = 0 [trace_headers @ 0x7f81a8416d00] 34 ref_pic_list_modification_flag_l0 0 = 0 [trace_headers @ 0x7f81a8416d00] 35 adaptive_ref_pic_marking_mode_flag 0 = 0 [trace_headers @ 0x7f81a8416d00] 36 slice_qp_delta 00000110101 = -26 [trace_headers @ 0x7f81a8416d00] 47 disable_deblocking_filter_idc 1 = 0 [trace_headers @ 0x7f81a8416d00] 48 slice_alpha_c0_offset_div2 1 = 0 [trace_headers @ 0x7f81a8416d00] 49 slice_beta_offset_div2 1 = 0 [trace_headers @ 0x7f81a8416d00] Packet: 589 bytes, no pts, dts 207998, duration 40000.
Display Motion Vectors
The following commands will help you visualize the motion vectors on top of the original video.
To display the motion vectors on a newly opened a window:
ffplay -flags2 +export_mvs file.h264 -vf codecview=mv=pf+bf+bb
To save a copy of the video with motion vectors overlaid:
ffmpeg -flags2 +export_mvs -I file.h264 -vf codecview=mv=pf+bf+bb output.mp4
Sample output:
Display Macroblock Types
This command will print the type of each macroblock:
ffmpeg -threads 1 -debug 'mb_type' -i file.h264 -f null -
The legend for each Macroblock can be found very clearly in the source code. Sample output:
[h264 @ 0x7febe480fc00] New frame, type: B [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d < < <| d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d X| > X| > d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d X| > X- > d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d X| X| d d d d [h264 @ 0x7febe480fc00] d d d d d d d d d d d d d d d d d d d d [h264 @ 0x7febe480fc00] New frame, type: P [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S I > >| S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S > > >- >- S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S > > >+ >| S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S >- > S S S S [h264 @ 0x7febe480fc00] S S S S S S S S S S S S S S S S S S S S
Display Macroblock Quantization Parameter (QP)
The following command will display the QP for each Macroblock in a frame:
ffmpeg -threads 1 -debug 'qp' -i file.h264 -f null -
Sample output:
[h264 @ 0x7fcf9c819000] New frame, type: B [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 4 4 4 4 4 [h264 @ 0x7fcf9c819000] 4 4 4 4 4 4 4 4 4 4 4 4 4 7 0 7 7 7 7 7 [h264 @ 0x7fcf9c819000] 7 7 7 7 7 7 7 7 7 7 7 7 7 7 0 7 7 7 7 7 [h264 @ 0x7fcf9c819000] 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 4 4 4 4 4 [h264 @ 0x7fcf9c819000] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 [h264 @ 0x7fcf9c819000] New frame, type: P [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [h264 @ 0x7fcf9c819000] 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 7 3 3 3 3 [h264 @ 0x7fcf9c819000] 3 3 3 3 3 3 3 3 3 3 3 3 3 6 0 3 5 5 5 5 [h264 @ 0x7fcf9c819000] 5 5 5 5 5 5 5 5 5 5 5 5 5 5 7 7 7 7 7 7 [h264 @ 0x7fcf9c819000] 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7