{"action":"created","issue":{"url":"https://api.github.com/repos/raspberrypi/userland/issues/230","labels_url":"https://api.github.com/repos/raspberrypi/userland/issues/230/labels{/name}","comments_url":"https://api.github.com/repos/raspberrypi/userland/issues/230/comments","events_url":"https://api.github.com/repos/raspberrypi/userland/issues/230/events","html_url":"https://github.com/raspberrypi/userland/issues/230","id":62800415,"number":230,"title":"Decoding in lockstep with MMAL","user":{"login":"wm4","id":1387750,"avatar_url":"https://avatars.githubusercontent.com/u/1387750?v=3","gravatar_id":"","url":"https://api.github.com/users/wm4","html_url":"https://github.com/wm4","followers_url":"https://api.github.com/users/wm4/followers","following_url":"https://api.github.com/users/wm4/following{/other_user}","gists_url":"https://api.github.com/users/wm4/gists{/gist_id}","starred_url":"https://api.github.com/users/wm4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wm4/subscriptions","organizations_url":"https://api.github.com/users/wm4/orgs","repos_url":"https://api.github.com/users/wm4/repos","events_url":"https://api.github.com/users/wm4/events{/privacy}","received_events_url":"https://api.github.com/users/wm4/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":18,"created_at":"2015-03-18T21:04:50Z","updated_at":"2015-03-20T16:54:31Z","closed_at":"2015-03-20T16:54:31Z","body":"I want to add a MMAL decoder wrapper to FFmpeg. FFmpeg's API requires that  output frame for each input packet is produced. The API user must pass a packet containing exactly one video frame to the decode function, and gets back a decoded frame. To support b-frames and multithreaded decoding, there can be a delay with which decoded frames are returned: the first N decode calls will not return decoded frames, but in exchange you must call the decode function N times without a packet to \"flush\" the decoder.\r\n\r\nThe MMAL API is asynchronous and quite different. You keep pushing in packets (until the input buffer is exhausted, I guess), and the decoder will return decoded frames at completely independent from this.\r\n\r\nWhen connecting these two APIs, there is one main problem: how do you know after how many packets you can expect an output frame from the decoder? This is needed to determine the minimum value for N. Basically: once the user has fed N packets to the decoder, the decoder must return an output frame for each packet. (And wait for the hw decoder if no output frame is ready yet.)\r\n\r\nYou could choose N based on when the first video frame happens to be decoded. This makes N basically random (based on the exact timings). The worst about it is that it can lead to excessive buffering, and a \"logic\" deadlock as a consequence: the decoder can suddenly return a large number of frames in a burst. After this, it won't consume more input until it has fed new video frames through the output port. But the FFmpeg wrapper can't return these frames quickly enough to the user (it can return 1 frame for 1 input packet). The input queue quickly fills up, the input pool gets exhausted, no output frames can be recycled, and everything locks up. Basically, you have a full output queue and empty input/output pools. I plan to work this around by queuing input packets in RAM using a queue outside of the MMAL API, but the point is that there's too much buffering.\r\n\r\nSo, to make it short: is there a way to determine when the MMAL component was fed enough input that we can expect output? I want to wait on the decoder, but not forever.\r\n\r\nI've also managed to freeze the GPU completely quite often. The CPU still worked, but every single process trying to access VCOS went into uninterruptible sleep. I suspect this was due to out of memory conditions on the GPU. But I can't know for sure.  If this is true, then this excessive buffering is quite dangerous and I absolutely have to avoid it.\r\n\r\n(Sorry if such API questions are inappropriate on this issue tracker.)"},"comment":{"url":"https://api.github.com/repos/raspberrypi/userland/issues/comments/84068166","html_url":"https://github.com/raspberrypi/userland/issues/230#issuecomment-84068166","issue_url":"https://api.github.com/repos/raspberrypi/userland/issues/230","id":84068166,"user":{"login":"wm4","id":1387750,"avatar_url":"https://avatars.githubusercontent.com/u/1387750?v=3","gravatar_id":"","url":"https://api.github.com/users/wm4","html_url":"https://github.com/wm4","followers_url":"https://api.github.com/users/wm4/followers","following_url":"https://api.github.com/users/wm4/following{/other_user}","gists_url":"https://api.github.com/users/wm4/gists{/gist_id}","starred_url":"https://api.github.com/users/wm4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wm4/subscriptions","organizations_url":"https://api.github.com/users/wm4/orgs","repos_url":"https://api.github.com/users/wm4/repos","events_url":"https://api.github.com/users/wm4/events{/privacy}","received_events_url":"https://api.github.com/users/wm4/received_events","type":"User","site_admin":false},"created_at":"2015-03-20T16:54:31Z","updated_at":"2015-03-20T16:54:31Z","body":">port_disable will block until the callback has been made for all buffers.\r\n\r\nI know, but port_flush is still documented as being asynchronous. I assume this asynchronous flush behavior is not even user-visible, then. I removed my own code for waiting for the buffers, and could simpliy it a lot.\r\n\r\nThe MMAL source is unfortunately not that useful, because all the interesting parts are on the videocore and closed source.\r\n\r\nI pushed my updated code. It's much cleaner and simpler. I dropped the mutex too. This is probably what I will send to FFmpeg.\r\n\r\nThanks for your help, it made my life so much easier!\r\n\r\nThe core question of this issue can probably be answered with: there's no way, but the max. buffering is 16 frames."}}