{"action":"created","issue":{"url":"https://api.github.com/repos/raspberrypi/firmware/issues/456","labels_url":"https://api.github.com/repos/raspberrypi/firmware/issues/456/labels{/name}","comments_url":"https://api.github.com/repos/raspberrypi/firmware/issues/456/comments","events_url":"https://api.github.com/repos/raspberrypi/firmware/issues/456/events","html_url":"https://github.com/raspberrypi/firmware/issues/456","id":100285576,"number":456,"title":"MMAL video decoder sometimes outputs corrupted timestamps","user":{"login":"wm4","id":1387750,"avatar_url":"https://avatars.githubusercontent.com/u/1387750?v=3","gravatar_id":"","url":"https://api.github.com/users/wm4","html_url":"https://github.com/wm4","followers_url":"https://api.github.com/users/wm4/followers","following_url":"https://api.github.com/users/wm4/following{/other_user}","gists_url":"https://api.github.com/users/wm4/gists{/gist_id}","starred_url":"https://api.github.com/users/wm4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wm4/subscriptions","organizations_url":"https://api.github.com/users/wm4/orgs","repos_url":"https://api.github.com/users/wm4/repos","events_url":"https://api.github.com/users/wm4/events{/privacy}","received_events_url":"https://api.github.com/users/wm4/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":32,"created_at":"2015-08-11T10:42:35Z","updated_at":"2015-09-22T15:38:49Z","closed_at":null,"body":"I'm not really sure what's happening here. But it almost looks like MMAL is outputting corrupted timestamps every 10 seconds with a specific sample I have.\r\n\r\nHere are the first 15 seconds or so of the sample: https://0x0.st/sV_.mkv\r\n\r\nWith ``ffmpeg  -c:v h264_mmal  -i sample.mkv  -vf showinfo -f null -`` I get this at around the 10 second mark (use FFmpeg git master as of now; older versions have a bug):\r\n\r\n```\r\n[Parsed_showinfo_0 @ 0x1d5f0c0] n: 248 pts:  10344 pts_time:10.344  pos:  3503692 fmt:yuv420p sar:1/1 s:1920x800 i:P iskey:1 type:? checksum:1B950DC2 plane_checksum:[78DD99FA 0A659D1D 47F3D68D] mean:[99 125 133] stdev:[40.8 10.0 10.3]\r\n[Parsed_showinfo_0 @ 0x1d5f0c0] n: 249 pts:  10385 pts_time:10.385  pos:  3520414 fmt:yuv420p sar:1/1 s:1920x800 i:P iskey:1 type:? checksum:A53CA625 plane_checksum:[46F0DA3F 34EA651D 581366BA] mean:[99 125 133] stdev:[41.3 9.9 10.2]\r\n[Parsed_showinfo_0 @ 0x1d5f0c0] n: 250 pts:  52093 pts_time:52.093  pos:  3549459 fmt:yuv420p sar:1/1 s:1920x800 i:P iskey:1 type:? checksum:1D6286E5 plane_checksum:[E445B642 80B84E4F 534B8245] mean:[98 124 133] stdev:[41.7 9.8 10.1]\r\n[Parsed_showinfo_0 @ 0x1d5f0c0] n: 251 pts:  10469 pts_time:10.469  pos:  3565022 fmt:yuv420p sar:1/1 s:1920x800 i:P iskey:1 type:? checksum:846BF201 plane_checksum:[7FF96E8C C144BFE4 E152C382] mean:[98 124 133] stdev:[42.1 9.8 10.0]\r\n[null @ 0x1db8b30] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1249 >= 251\r\n```\r\n\r\nFrame number 250 outputs an odd timestamp. Frame number 251 is back to normal.\r\n\r\nIt happens after the next 250 frames too:\r\n```\r\n[Parsed_showinfo_0 @ 0x1b178b0] n: 499 pts:  20812 pts_time:20.812  pos:  7366248 fmt:yuv420p sar:1/1 s:1920x800 i:P iskey:1 type:? checksum:92C985CC plane_checksum:[5B9282A8 8780C19F 12244176] mean:[46 135 122] stdev:[47.5 9.2 10.2]\r\n[null @ 0x1b6b830] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1249 >= 499                                             \r\n[Parsed_showinfo_0 @ 0x1b178b0] n: 500 pts:  62520 pts_time:62.52   pos:  7383549 fmt:yuv420p sar:1/1 s:1920x800 i:P iskey:1 type:? checksum:698E2BFF plane_checksum:[7911A759 6150BC3F DEEEC849] mean:[46 135 122] stdev:[47.5 9.2 10.1]       \r\n[Parsed_showinfo_0 @ 0x1b178b0] n: 501 pts:  20896 pts_time:20.896  pos:  7386065 fmt:yuv420p sar:1/1 s:1920x800 i:P iskey:1 type:? checksum:6AE589CA plane_checksum:[ACF45616 2EB42ABA 1B5308FA] mean:[46 135 122] stdev:[47.4 9.1 10.1] \r\n```\r\n(not included in the sample, but I can upload the full one if needed)\r\n\r\nBut the packet timestamp seems to be fine:\r\n\r\n```\r\n[PACKET]\r\ncodec_type=video\r\nstream_index=1\r\npts=10427\r\npts_time=10.427000\r\ndts=10344\r\ndts_time=10.344000\r\nduration=41\r\nduration_time=0.041000\r\nconvergence_duration=N/A\r\nconvergence_duration_time=N/A\r\nsize=50353\r\npos=3270843\r\nflags=K\r\n[/PACKET]\r\n```\r\n\r\n(from ``ffprobe sample.mkv  -show_packets``)\r\n\r\nThe FFmpeg h264 software decoder has no issues here, it outputs the frame like this:\r\n\r\n```\r\n[FRAME]\r\nmedia_type=video\r\nstream_index=1\r\nkey_frame=1\r\npkt_pts=10427\r\npkt_pts_time=10.427000\r\npkt_dts=10427\r\npkt_dts_time=10.427000\r\nbest_effort_timestamp=10427\r\nbest_effort_timestamp_time=10.427000\r\npkt_duration=41\r\npkt_duration_time=0.041000\r\npkt_pos=3270843\r\npkt_size=50353\r\nwidth=1920\r\nheight=800\r\npix_fmt=yuv420p\r\nsample_aspect_ratio=1:1\r\npict_type=I\r\ncoded_picture_number=250\r\ndisplay_picture_number=0\r\ninterlaced_frame=0\r\ntop_field_first=0\r\nrepeat_pict=0\r\n[/FRAME]\r\n```\r\n\r\n(from ``ffprobe -show_frames -select_streams 1 sample.mkv``)\r\n\r\nIs it possible that MMAL is doing something wrong here?\r\n\r\n(Side note: ffprobe is a useful tool, but you can't force the video decoder yet - so it can't be used with the mmal wrapper.)"},"comment":{"url":"https://api.github.com/repos/raspberrypi/firmware/issues/comments/142327184","html_url":"https://github.com/raspberrypi/firmware/issues/456#issuecomment-142327184","issue_url":"https://api.github.com/repos/raspberrypi/firmware/issues/456","id":142327184,"user":{"login":"6by9","id":5822349,"avatar_url":"https://avatars.githubusercontent.com/u/5822349?v=3","gravatar_id":"","url":"https://api.github.com/users/6by9","html_url":"https://github.com/6by9","followers_url":"https://api.github.com/users/6by9/followers","following_url":"https://api.github.com/users/6by9/following{/other_user}","gists_url":"https://api.github.com/users/6by9/gists{/gist_id}","starred_url":"https://api.github.com/users/6by9/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/6by9/subscriptions","organizations_url":"https://api.github.com/users/6by9/orgs","repos_url":"https://api.github.com/users/6by9/repos","events_url":"https://api.github.com/users/6by9/events{/privacy}","received_events_url":"https://api.github.com/users/6by9/received_events","type":"User","site_admin":false},"created_at":"2015-09-22T15:38:49Z","updated_at":"2015-09-22T15:38:49Z","body":"> Also, what's MMAL_ES_FORMAT_T.extradata for? I never could see it having any effect, and kodi doesn't use it either.\r\n\r\nThe MMAL header documentation markup actually gives you a worked example that uses extradata.\r\nhttps://github.com/raspberrypi/userland/blob/master/interface/mmal/mmal.h#L251\r\nIt gives you a method of providing the codec header bytes out of band, rather than in buffers. When converted into the IL layer, it is converted into a setParameter(OMX_IndexParamCodecConfig).\r\n\r\nSorry, I'm not a codec expert, and a quick Google has failed me on providing detail of annex B vs MP4 format. A quick read of the code says it supports searching for start codes, or the client guaranteeing one NAL per buffer. I don't know how it handles things beyond that.\r\n(Oddly the encoder supports start codes, or FourByteInterleavedLength, which puts a 32bit length field at the start of every buffer instead of the start code).\r\n\r\n\r\n"}}