How to extract frames at 30 fps using FFMPEG APIs on Android?
<p>We are working on a project that consumes <code>FFMPEG</code> library for video frame extraction on Android platform.</p>

<p>On Windows, we have observed: </p>

<ul>
<li>Using CLI, ffmpeg is capable of extracting frames at 30 fps using command <code>ffmpeg -i input.flv -vf fps=1 out%d.png</code>.</li>
<li>Using Xuggler, we are able to extract frames at 30 fps.</li>
<li>Using FFMPEG APIs directly in code, we are getting frames at 30 fps.</li>
</ul>

<p>But when we use FFMPEG APIs directly on Android (<a href="http://www.gsmarena.com/sony_xperia_m2_dual-6162.php" rel="nofollow">See Hardware Details</a>), we are getting following results:</p>

<ul>
<li>720p video (1280 x 720) - 16 fps (approx. 60 ms/frame)</li>
<li>1080p video (1920 x 1080) - 7 fps (approx. 140 ms/frame)</li>
</ul>

<p>We haven't tested Xuggler/CLI on Android yet.</p>

<p>Ideally, we should be able to get the data in constant time (approx. 30 ms/frame).</p>

<p>How can we get 30 fps on Android?</p>

<p><strong>Code being used on Android:</strong></p>

<pre><code>if (avformat_open_input(&amp;pFormatCtx, pcVideoFile, NULL, NULL)) {
    iError = -1;  //Couldn't open file
}

if (!iError) {
    //Retrieve stream information
    if (avformat_find_stream_info(pFormatCtx, NULL) &lt; 0)
        iError = -2; //Couldn't find stream information
}

//Find the first video stream
if (!iError) {

    for (i = 0; i &lt; pFormatCtx-&gt;nb_streams; i++) {
        if (AVMEDIA_TYPE_VIDEO
                == pFormatCtx-&gt;streams[i]-&gt;codec-&gt;codec_type) {
            iFramesInVideo = pFormatCtx-&gt;streams[i]-&gt;nb_index_entries;
            duration = pFormatCtx-&gt;streams[i]-&gt;duration;
            begin = pFormatCtx-&gt;streams[i]-&gt;start_time;
            time_base = (pFormatCtx-&gt;streams[i]-&gt;time_base.num * 1.0f)
                    / pFormatCtx-&gt;streams[i]-&gt;time_base.den;

            pCodecCtx = avcodec_alloc_context3(NULL);
            if (!pCodecCtx) {
                iError = -6;
                break;
            }

            AVCodecParameters params = { 0 };
            iReturn = avcodec_parameters_from_context(&amp;params,
                    pFormatCtx-&gt;streams[i]-&gt;codec);
            if (iReturn &lt; 0) {
                iError = -7;
                break;
            }

            iReturn = avcodec_parameters_to_context(pCodecCtx, &amp;params);
            if (iReturn &lt; 0) {
                iError = -7;
                break;
            }

            //pCodecCtx = pFormatCtx-&gt;streams[i]-&gt;codec;

            iVideoStreamIndex = i;
            break;
        }
    }
}

if (!iError) {
    if (iVideoStreamIndex == -1) {
        iError = -3; // Didn't find a video stream
    }
}

if (!iError) {
    // Find the decoder for the video stream
    pCodec = avcodec_find_decoder(pCodecCtx-&gt;codec_id);
    if (pCodec == NULL) {
        iError = -4;
    }
}

if (!iError) {
    // Open codec
    if (avcodec_open2(pCodecCtx, pCodec, NULL) &lt; 0)
        iError = -5;
}

if (!iError) {
    iNumBytes = av_image_get_buffer_size(AV_PIX_FMT_RGB24, pCodecCtx-&gt;width,
            pCodecCtx-&gt;height, 1);

    // initialize SWS context for software scaling
    sws_ctx = sws_getContext(pCodecCtx-&gt;width, pCodecCtx-&gt;height,
            pCodecCtx-&gt;pix_fmt, pCodecCtx-&gt;width, pCodecCtx-&gt;height,
            AV_PIX_FMT_RGB24,
            SWS_BILINEAR,
            NULL,
            NULL,
            NULL);
    if (!sws_ctx) {
        iError = -7;
    }
}
clock_gettime(CLOCK_MONOTONIC_RAW, &amp;end);
delta_us = (end.tv_sec - start.tv_sec) * 1000000
        + (end.tv_nsec - start.tv_nsec) / 1000;
start = end;
//LOGI("Starting_Frame_Extraction: %lld", delta_us);
if (!iError) {
    while (av_read_frame(pFormatCtx, &amp;packet) == 0) {
        // Is this a packet from the video stream?
        if (packet.stream_index == iVideoStreamIndex) {
            pFrame = av_frame_alloc();
            if (NULL == pFrame) {
                iError = -8;
                break;
            }

            // Decode video frame
            avcodec_decode_video2(pCodecCtx, pFrame, &amp;iFrameFinished,
                    &amp;packet);
            if (iFrameFinished) {
                //OUR CODE
            }
            av_frame_free(&amp;pFrame);
            pFrame = NULL;
        }
        av_packet_unref(&amp;packet);
    }
}
</code></pre>