How frame rate is calculated or read from h264 bitstream sending through rtsp protocol by live555
<p><strong>Problem:</strong></p>

<p>In my case, bit stream coming from h264 encoder which is feed by live camera. Bitstream is sending by live555 library using RTSP protocol. RTSP packets are received by one of rtsp client like vlc player( the most important), totem player, mplayer.
I need to know how to set lower framerate because my camera produces 9 fps but most of RTSP clients are expecting in default 25 fps and it cause that video stream is not played properly ( buffered, dropping late packets etc.).</p>

<p><strong>Question:</strong></p>

<p>I would like to know how frame rate is read from bit stream (or evaluated) by RTSP client (playback software). Where this informations are stored and how are sending to playback software?</p>

<p><strong>More infos:</strong></p>

<p>I am using testOnDemandRTSPServer example code from live555. I have wrote my implementation of DeviceSource class from live555 lib. I have set presentation time properly:</p>

<pre><code>gettimeofday(&amp;fPresentationTime, NULL);
fDurationInMicroseconds = 1100000;
</code></pre>

<p>Sequence Parameter Set NAL also contains proper timing infos:</p>

<pre><code>timing_info_present_flag : 1 
       num_units_in_tick : 1001 
       time_scale : 9000 
       fixed_frame_rate_flag : 1
</code></pre>

<p>RTSP Header contains framerate:</p>

<pre><code>a=framerate:9.0
</code></pre>

<p>Unfortunately it doesn't help. I have thought that video player will be  calmly waiting for new frame and framerate will be correlated with frequency of receiving another RTP packets (another NALs). </p>

<p><strong>How video players behave:</strong></p>

<p>VLC player - video is choppy because vlc expects next frame earlier (25 fps instead of 9 fps). In results, vlc increases buffering time to e.g. 10s, during this time video is frozen.</p>

<p>Totem player - Timing informations from SPS Nal are parsed by totem. Framerate is  showing properly but video is still choppy ( ~500ms delayed ). It seems like is waiting for more frames (25 per second).</p>

<p>Mplayer recognized wrong framerate value, shows 25 fps, but video is very smooth and good quality:</p>

<pre><code>mplayer ffmpeg://rtsp://192.168.1.82:8554/testStream
.
.
.
VIDEO:  [H264]  320x240  0bpp  25.000 fps    0.0 kbps ( 0.0 kbyte/s)
</code></pre>

<p>Live555 RTSP client test application
./live555/testProgs/testRTSPClient rtsp://192.168.1.82:8554/testStream</p>

<pre><code>Opening connection to 192.168.1.82, port 8554...
...remote connection opened
Sending request: DESCRIBE rtsp://192.168.1.82:8554/testStream RTSP/1.0
CSeq: 2
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Accept: application/sdp


Received 675 new bytes of response data.
Received a complete DESCRIBE response:
RTSP/1.0 200 OK
CSeq: 2
Date: Fri, Jan 01 1988 00:00:23 GMT
Content-Base: rtsp://192.168.1.82:8554/testStream/
Content-Type: application/sdp
Content-Length: 506

v=0
o=- 567993603275971 1 IN IP4 192.168.1.82
s=Session streamed by "testOnDemandRTSPServer"
i=testStream
t=0 0
a=tool:LIVE555 Streaming Media v2016.02.22
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "testOnDemandRTSPServer"
a=x-qt-text-inf:testStream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42801E;sprop-parameter-sets=Z0KAHpWgUHxA,aM44gA==
a=framerate:9.0
a=control:track1

[URL:"rtsp://192.168.1.82:8554/testStream/"]: Got a SDP description:
v=0
o=- 567993603275971 1 IN IP4 192.168.1.82
s=Session streamed by "testOnDemandRTSPServer"
i=testStream
t=0 0
a=tool:LIVE555 Streaming Media v2016.02.22
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "testOnDemandRTSPServer"
a=x-qt-text-inf:testStream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42801E;sprop-parameter-sets=Z0KAHpWgUHxA,aM44gA==
a=framerate:9.0
a=control:track1

[URL:"rtsp://192.168.1.82:8554/testStream/"]: Initiated the "video/H264" subsession (client ports 57660-57661)
Sending request: SETUP rtsp://192.168.1.82:8554/testStream/track1 RTSP/1.0
CSeq: 3
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Transport: RTP/AVP;unicast;client_port=57660-57661


Received 215 new bytes of response data.
Received a complete SETUP response:
RTSP/1.0 200 OK
CSeq: 3
Date: Fri, Jan 01 1988 00:00:23 GMT
Transport: RTP/AVP;unicast;destination=192.168.1.109;source=192.168.1.82;client_port=57660-57661;server_port=6970-6971
Session: 06B1DFA4;timeout=65


[URL:"rtsp://192.168.1.82:8554/testStream/"]: Set up the "video/H264" subsession (client ports 57660-57661)
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Created a data sink for the "video/H264" subsession
Sending request: PLAY rtsp://192.168.1.82:8554/testStream/ RTSP/1.0
CSeq: 4
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Session: 06B1DFA4
Range: npt=0.000-


Received 96 new bytes of response data.
Received 94 new bytes of response data.
Received a complete PLAY response:
RTSP/1.0 200 OK
CSeq: 4
Date: Fri, Jan 01 1988 00:00:23 GMT
Range: npt=0.000-
Session: 06B1DFA4
RTP-Info: url=rtsp://192.168.1.82:8554/testStream/track1;seq=52833;rtptime=3161001994


[URL:"rtsp://192.168.1.82:8554/testStream/"]: Started playing session...
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 10874 bytes.   Presentation time: 567993623.442288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12054 bytes.   Presentation time: 567993623.482288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12227 bytes.   Presentation time: 567993623.522288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12281 bytes.   Presentation time: 567993623.562288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12384 bytes.   Presentation time: 567993623.602288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12129 bytes.   Presentation time: 567993623.642288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12141 bytes.   Presentation time: 567993623.682288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12287 bytes.   Presentation time: 567993623.722288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 16415 bytes.   Presentation time: 567993623.762288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12360 bytes.   Presentation time: 567993623.802288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12167 bytes.   Presentation time: 567993623.842288
</code></pre>

<p>I parsed my NALs by tool h264_analyze and below is output ( only a few first NALs):</p>

<pre><code>!! Found NAL at offset 4 (0x0004), size 18 (0x0012) 
XX 00 00 00 01 67 42 80 1E 95 A0 50 7C 84 00 00 0F 

==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 3 
 nal_unit_type : 7 ( Sequence parameter set ) 
======= SPS =======
 profile_idc : 66 
 constraint_set0_flag : 1 
 constraint_set1_flag : 0 
 constraint_set2_flag : 0 
 constraint_set3_flag : 0 
 constraint_set4_flag : 0 
 constraint_set5_flag : 0 
 reserved_zero_2bits : 0 
 level_idc : 30 
 seq_parameter_set_id : 0 
 chroma_format_idc : 1 
 residual_colour_transform_flag : 0 
 bit_depth_luma_minus8 : 0 
 bit_depth_chroma_minus8 : 0 
 qpprime_y_zero_transform_bypass_flag : 0 
 seq_scaling_matrix_present_flag : 0 
 log2_max_frame_num_minus4 : 4 
 pic_order_cnt_type : 2 
   log2_max_pic_order_cnt_lsb_minus4 : 0 
   delta_pic_order_always_zero_flag : 0 
   offset_for_non_ref_pic : 0 
   offset_for_top_to_bottom_field : 0 
   num_ref_frames_in_pic_order_cnt_cycle : 0 
 num_ref_frames : 1 
 gaps_in_frame_num_value_allowed_flag : 0 
 pic_width_in_mbs_minus1 : 19 
 pic_height_in_map_units_minus1 : 14 
 frame_mbs_only_flag : 1 
 mb_adaptive_frame_field_flag : 0 
 direct_8x8_inference_flag : 0 
 frame_cropping_flag : 0 
   frame_crop_left_offset : 0 
   frame_crop_right_offset : 0 
   frame_crop_top_offset : 0 
   frame_crop_bottom_offset : 0 
 vui_parameters_present_flag : 1 
=== VUI ===
 aspect_ratio_info_present_flag : 0 
   aspect_ratio_idc : 0 
     sar_width : 0 
     sar_height : 0 
 overscan_info_present_flag : 0 
   overscan_appropriate_flag : 0 
 video_signal_type_present_flag : 0 
   video_format : 0 
   video_full_range_flag : 0 
   colour_description_present_flag : 0 
     colour_primaries : 0 
   transfer_characteristics : 0 
   matrix_coefficients : 0 
 chroma_loc_info_present_flag : 0 
   chroma_sample_loc_type_top_field : 0 
   chroma_sample_loc_type_bottom_field : 0 
 timing_info_present_flag : 1 
   num_units_in_tick : 1001 
   time_scale : 9000 
   fixed_frame_rate_flag : 1
 nal_hrd_parameters_present_flag : 0 
 vcl_hrd_parameters_present_flag : 0 
   low_delay_hrd_flag : 0 
 pic_struct_present_flag : 0 
 bitstream_restriction_flag : 0 
   motion_vectors_over_pic_boundaries_flag : 0 
   max_bytes_per_pic_denom : 0 
   max_bits_per_mb_denom : 0 
   log2_max_mv_length_horizontal : 0 
   log2_max_mv_length_vertical : 0 
   num_reorder_frames : 0 
   max_dec_frame_buffering : 0 
=== HRD ===
 cpb_cnt_minus1 : 0 
 bit_rate_scale : 0 
 cpb_size_scale : 0 
   bit_rate_value_minus1[0] : 0 
   cpb_size_value_minus1[0] : 0 
   cbr_flag[0] : 0 
 initial_cpb_removal_delay_length_minus1 : 0 
 cpb_removal_delay_length_minus1 : 0 
 dpb_output_delay_length_minus1 : 0 
 time_offset_length : 0 
!! Found NAL at offset 26 (0x001A), size 4 (0x0004) 
XX 00 00 00 01 68 CE 38 80 
==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 3 
 nal_unit_type : 8 ( Picture parameter set ) 
======= PPS =======
 pic_parameter_set_id : 0 
 seq_parameter_set_id : 0 
 entropy_coding_mode_flag : 0 
 pic_order_present_flag : 0 
 num_slice_groups_minus1 : 0 
 slice_group_map_type : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 weighted_pred_flag : 0 
 weighted_bipred_idc : 0 
 pic_init_qp_minus26 : 0 
 pic_init_qs_minus26 : 0 
 chroma_qp_index_offset : 0 
 deblocking_filter_control_present_flag : 0 
 constrained_intra_pred_flag : 0 
 redundant_pic_cnt_present_flag : 0 
 transform_8x8_mode_flag : 0 
 pic_scaling_matrix_present_flag : 0 
 second_chroma_qp_index_offset : 0 
!! Found NAL at offset 33 (0x0021), size 339 (0x0153) 
XX 80 00 00 01 65 B8 04 04 3F FF F8 7A 28 03 EF BE 

==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 3 
 nal_unit_type : 5 ( Coded slice of an IDR picture ) 
======= Slice Header =======
 first_mb_in_slice : 0 
 slice_type : 2 ( I slice ) 
 pic_parameter_set_id : 0 
 frame_num : 0 
 field_pic_flag : 0 
 bottom_field_flag : 0 
 idr_pic_id : 0 
 pic_order_cnt_lsb : 0 
 delta_pic_order_cnt_bottom : 0 
 redundant_pic_cnt : 0 
 direct_spatial_mv_pred_flag : 0 
 num_ref_idx_active_override_flag : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 cabac_init_idc : 0 
 slice_qp_delta : -16 
 sp_for_switch_flag : 0 
 slice_qs_delta : 0 
 disable_deblocking_filter_idc : 0 
 slice_alpha_c0_offset_div2 : 0 
 slice_beta_offset_div2 : 0 
 slice_group_change_cycle : 0 
=== Prediction Weight Table ===
 luma_log2_weight_denom : 0 
 chroma_log2_weight_denom : 0 
=== Ref Pic List Reordering ===
 ref_pic_list_reordering_flag_l0 : 0 
 ref_pic_list_reordering_flag_l1 : 0 
=== Decoded Ref Pic Marking ===
 no_output_of_prior_pics_flag : 0 
 long_term_reference_flag : 0 
 adaptive_ref_pic_marking_mode_flag : 0 
!! Found NAL at offset 376 (0x0178), size 1897 (0x0769) 
XX 00 00 00 01 41 E0 20 3F 36 5F FF 87 A2 80 03 7E 

==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 2 
 nal_unit_type : 1 ( Coded slice of a non-IDR picture ) 
======= Slice Header =======
 first_mb_in_slice : 0 
 slice_type : 0 ( P slice ) 
 pic_parameter_set_id : 0 
 frame_num : 1 
 field_pic_flag : 0 
 bottom_field_flag : 0 
 idr_pic_id : 0 
 pic_order_cnt_lsb : 0 
 delta_pic_order_cnt_bottom : 0 
 redundant_pic_cnt : 0 
 direct_spatial_mv_pred_flag : 0 
 num_ref_idx_active_override_flag : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 cabac_init_idc : 0 
 slice_qp_delta : -15 
 sp_for_switch_flag : 0 
 slice_qs_delta : 0 
 disable_deblocking_filter_idc : 0 
 slice_alpha_c0_offset_div2 : 0 
 slice_beta_offset_div2 : 0 
 slice_group_change_cycle : 0 
=== Prediction Weight Table ===
 luma_log2_weight_denom : 0 
 chroma_log2_weight_denom : 0 
=== Ref Pic List Reordering ===
 ref_pic_list_reordering_flag_l0 : 0 
 ref_pic_list_reordering_flag_l1 : 0 
=== Decoded Ref Pic Marking ===
 no_output_of_prior_pics_flag : 0 
 long_term_reference_flag : 0 
 adaptive_ref_pic_marking_mode_flag : 0 
!! Found NAL at offset 2277 (0x08E5), size 2989 (0x0BAD) 
XX 00 00 00 01 41 E0 40 3F 37 0F 0C 3E D8 A0 00 4E 

==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 2 
 nal_unit_type : 1 ( Coded slice of a non-IDR picture ) 
======= Slice Header =======
 first_mb_in_slice : 0 
 slice_type : 0 ( P slice ) 
 pic_parameter_set_id : 0 
 frame_num : 2 
 field_pic_flag : 0 
 bottom_field_flag : 0 
 idr_pic_id : 0 
 pic_order_cnt_lsb : 0 
 delta_pic_order_cnt_bottom : 0 
 redundant_pic_cnt : 0 
 direct_spatial_mv_pred_flag : 0 
 num_ref_idx_active_override_flag : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 cabac_init_idc : 0 
 slice_qp_delta : -15 
 sp_for_switch_flag : 0 
 slice_qs_delta : 0 
 disable_deblocking_filter_idc : 0 
 slice_alpha_c0_offset_div2 : 0 
 slice_beta_offset_div2 : 0 
 slice_group_change_cycle : 0 
=== Prediction Weight Table ===
 luma_log2_weight_denom : 0 
 chroma_log2_weight_denom : 0 
=== Ref Pic List Reordering ===
 ref_pic_list_reordering_flag_l0 : 0 
 ref_pic_list_reordering_flag_l1 : 0 
=== Decoded Ref Pic Marking ===
 no_output_of_prior_pics_flag : 0 
 long_term_reference_flag : 0 
 adaptive_ref_pic_marking_mode_flag : 0 
!! Found NAL at offset 5270 (0x1496), size 4074 (0x0FEA) 
XX 00 00 00 01 41 E0 60 10 CD B7 C1 BF B8 36 00 6A 
</code></pre>