Android camera preview stretched using Grafika CameraCapture code
<p>I'm looking for help with an issue I'm facing using <a href="https://github.com/google/grafika/blob/master/src/com/android/grafika/CameraCaptureActivity.java" rel="noreferrer">Grafika's CameraCaptureActivity</a> code. I want to build an app that can record camera and display a preview, so this sample and code looked like exactly what I wanted and so far it's been great, this issue appart.</p>

<p>The issue I have is that when the camera preview size doesn't match the exact size of the <code>GLSurfaceView</code> I use to display the preview it shows a stretched preview.</p>

<p>Here's how it looks: </p>

<p><a href="https://i.stack.imgur.com/BdssX.png" rel="noreferrer"><img src="https://i.stack.imgur.com/BdssX.png" alt="Preview stretched"></a></p>

<p>As you can see it's stretched horizontally since the note sheet is a perfect square.</p>

<p>For this precise exemple on a Samsung Galaxy S4, the camera preview size is <code>1280x720</code> (extracted from available preview sizes using <code>Camera2</code> API), and the <code>GLSurfaceView</code> is <code>1920x1080</code> to cover the entire screen. </p>

<p><strong>When does it appear?</strong></p>

<p>The issue appears when the SurfaceView is upscaled compared to the Camera output, even if I always make sure the ratio remains correct (<code>1920x1080</code> = 1,5 * <code>1280x720</code>).</p>

<p><strong>What code are you using?</strong></p>

<p>I use Grafika's code, including:</p>

<ul>
<li><a href="https://github.com/google/grafika/blob/master/src/com/android/grafika/CameraCaptureActivity.java" rel="noreferrer">CameraCaptureActivity</a></li>
<li><a href="https://github.com/google/grafika/blob/master/src/com/android/grafika/AspectFrameLayout.java" rel="noreferrer">AspectFrameLayout</a></li>
<li><a href="https://github.com/google/grafika/blob/master/src/com/android/grafika/gles/FullFrameRect.java" rel="noreferrer">FullFrameRect</a></li>
<li><a href="https://github.com/google/grafika/blob/master/src/com/android/grafika/gles/Drawable2d.java" rel="noreferrer">Drawable2d</a></li>
<li><a href="https://github.com/google/grafika/blob/master/src/com/android/grafika/gles/EglCore.java" rel="noreferrer">EglCore</a></li>
<li><a href="https://github.com/google/grafika/blob/master/src/com/android/grafika/gles/EglSurfaceBase.java" rel="noreferrer">EglBaseSurface</a></li>
</ul>

<p><strong>Some log?</strong></p>

<p>Here's what SurfaceFlinger dump looks like:</p>

<pre><code>Hardware Composer state (version 01030000):
  mDebugForceFakeVSync=0
  Display[0] configurations (* current):
    * 0: 1080x1920, xdpi=442.450989, ydpi=439.351013, secure=1 refresh=16666667
  numHwLayers=3, flags=00000000
    type   |  handle  | hint | flag | tr | blnd |  format     |     source crop(l,t,r,b)       |           frame        |      dirtyRect         |  name 
------------+----------+----------+----------+----+-------+----------+-----------------------------------+---------------------------+-------------------
       HWC | b3b12ba0 | 0002 | 0000 | 00 | 0100 | RGB_888     |    0.0,    0.0, 1080.0, 1920.0 |    0,    0, 1080, 1920 | [    0,    0, 1080, 1920] | SurfaceView
       HWC | b3d12740 | 0002 | 0000 | 00 | 0105 | RGBA_8888   |    0.0,    0.0, 1065.0, 1905.0 |    0,    0, 1065, 1905 | [    0,    0, 1065, 1905] | com.mybundle.myapp/com.mybundle.myapp.activity.MyActivity
 FB TARGET | b6a55c40 | 0000 | 0000 | 00 | 0105 | RGBA_8888   |    0.0,    0.0, 1080.0, 1920.0 |    0,    0, 1080, 1920 | [    0,    0,    0,    0] | HWC_FRAMEBUFFER_TARGET

Layer 0xb4094000 (SurfaceView)
  Region transparentRegion (this=0xb4094160, count=1)
    [  0,   0,   0,   0]
  Region visibleRegion (this=0xb4094008, count=1)
    [  0,   0, 1920, 1080]
      layerStack=   0, z=    21015, pos=(0,0), size=(1920,1080), crop=(   0,   0,1920,1080), isOpaque=1, invalidate=0, alpha=0xff, flags=0x00000002, tr=[1.00, 0.00][0.00, 1.00]
      client=0xb66190c0
      format= 4, activeBuffer=[1080x1920:1536,  3], queued-frames=0, mRefreshPending=0
            mTexName=116 mCurrentTexture=0
            mCurrentCrop=[0,0,0,0] mCurrentTransform=0x7
            mAbandoned=0
            -BufferQueue mMaxAcquiredBufferCount=1, mDequeueBufferCannotBlock=0, default-size=[1920x1080], default-format=4, transform-hint=04, FIFO(0)={}
            &gt;[00:0xb1c18500] state=ACQUIRED, 0xb3b12ba0 [1080x1920:1536,  3]
</code></pre>

<p><strong>Camera preview size code?</strong></p>

<pre><code>StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

// Retrieve display size
WindowManager wm = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
Display display = wm.getDefaultDisplay();
Point displaySize = new Point();
display.getSize(displaySize);

// Choose the sizes for camera preview
mPreviewSize = chooseOptimalSize(map.getOutputSizes(SurfaceTexture.class),
                    displaySize.x, displaySize.y, captureWidth, captureHeight);

private static Size chooseOptimalSize(Size[] choices, int screenWidth, int screenHeight, int captureWidth, int captureHeight)
    {
        // Collect the supported resolutions that are at least as big as the screen &amp;&amp; wanted capture size
        List&lt;Size&gt; bigEnough = new ArrayList&lt;&gt;();
        // Collect the supported resolutions that are at least as big as the wanted capture size but &lt; to screensize
        List&lt;Size&gt; smallerButEnough = new ArrayList&lt;&gt;();

        for (Size option : choices) {
            if (option.getHeight() == option.getWidth() * captureHeight / captureWidth) {
                if( option.getWidth() &gt;= screenWidth &amp;&amp; option.getHeight() &gt;= screenHeight )
                {
                    bigEnough.add(option);
                }
                else
                {
                    smallerButEnough.add(option);
                }
            }
        }

        // Pick the smallest of those, assuming we found any
        if (bigEnough.size() &gt; 0)
        {
            return Collections.min(bigEnough, new CompareSizesByArea());
        }
        // Pick the biggest of those, assuming we found any
        else if( smallerButEnough.size() &gt; 0 )
        {
            return Collections.max(smallerButEnough, new CompareSizesByArea());
        }
        else
        {
            Log.e(TAG, "Couldn't find any suitable preview size");
            return choices[0];
        }
    }
</code></pre>

<p><strong>What have you tried?</strong></p>

<p>I first thought it was a <code>Camera2</code> issue, so I tried to use only the old Camera API, but result is the same.</p>

<p>Then I thought it was a ratio issue, but as you can see the ratio here is ok horizontally and vertically. </p>

<p>My bet is that there's an OpenGL transformation to apply when the Surface is resized but I'm a pure noob in OpenGL so I haven't been able to find anything that works. I try added those lines but it doesn't help:</p>

<pre><code>@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
    Log.d(TAG, "onSurfaceChanged " + width + "x" + height);

    gl.glViewport(0, 0, width, height);
    gl.glClear(GLES10.GL_COLOR_BUFFER_BIT | GLES10.GL_DEPTH_BUFFER_BIT);
}
</code></pre>

<p><strong>Another example, right from Grafika:</strong></p>

<p>Here's an other example of the issue, right from the Grafika app. </p>

<p>Phone: Galaxy S3 Neo (<code>GT-I9301L</code>) - Android 4.4.2</p>

<p>The phone has a <code>1280x720</code> screen, camera opened with the <code>640x480</code> resolution, displayed in a <code>573x430</code>. Here's how it looks:</p>

<p><a href="https://i.stack.imgur.com/7NArs.png" rel="noreferrer"><img src="https://i.stack.imgur.com/7NArs.png" alt="Stretching on GS3 Neo"></a></p>

<p>SurfaceFlinger logs:</p>

<pre><code>Hardware Composer state (version  1030000):
  mDebugForceFakeVSync=0
  Display[0] : 720x1280, xdpi=304.799011, ydpi=306.716003, refresh=16666667
  numHwLayers=4, flags=00000000
    type    |  handle  |   hints  |   flags  | tr | blend |  format  |          source crop            |           frame           name 
------------+----------+----------+----------+----+-------+----------+---------------------------------+--------------------------------
        HWC | b84a3d18 | 00000002 | 00000000 | 00 | 00100 | 00000002 | [    0.0,    0.0,  430.0,  573.0] | [   32,  353,  462,  926] SurfaceView
        HWC | b841b080 | 00000002 | 00000000 | 00 | 00105 | 00000001 | [    0.0,    0.0,  670.0, 1280.0] | [    0,    0,  670, 1280] com.android.grafika/com.android.grafika.CameraCaptureActivity
        HWC | b8423250 | 00000002 | 00000000 | 00 | 00105 | 00000001 | [    0.0,    0.0,   50.0, 1280.0] | [  670,    0,  720, 1280] StatusBar
  FB TARGET | b8412d50 | 00000000 | 00000000 | 00 | 00105 | 00000001 | [    0.0,    0.0,  720.0, 1280.0] | [    0,    0,  720, 1280] HWC_FRAMEBUFFER_TARGET


+ Layer 0xb8416080 (SurfaceView) id=112
  Region transparentRegion (this=0xb84169d0, count=1)
    [  0,   0,   0,   0]
  Region visibleRegion (this=0xb8416088, count=1)
    [353, 258, 926, 688]
      layerStack=   0, z=    21015, pos=(353,258), size=( 573, 430), crop=(   0,   0, 573, 430), isOpaque=1, invalidate=0, alpha=0xff, flags=0x00000000, tr=[1.00, 0.00][0.00, 1.00]
      client=0xb841d7c8
      format= 4, activeBuffer=[ 430x 573: 448,  2], queued-frames=0, mRefreshPending=0
            mTexName=95 mCurrentTexture=1
            mCurrentCrop=[0,0,0,0] mCurrentTransform=0x7
            mAbandoned=0
            -BufferQueue mMaxAcquiredBufferCount=1, mDequeueBufferCannotBlock=0, default-size=[573x430], default-format=4, transform-hint=04, FIFO(0)={}
             [00:0xb84439a8] state=FREE    , 0xb849a528 [ 430x 573: 448,  2]
            &gt;[01:0xb8421128] state=ACQUIRED, 0xb84a3d18 [ 430x 573: 448,  2]
             [02:0xb84238e8] state=FREE    , 0xb84432f8 [ 430x 573: 448,  2]
</code></pre>

<p>Grafika's app log:</p>

<pre><code>04-18 18:06:49.954 14383-14383/com.android.grafika D/Grafika: onCreate complete: com.android.grafika.CameraCaptureActivity@42adc288
04-18 18:06:49.954 14383-14383/com.android.grafika D/Grafika: onResume -- acquiring camera
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika: Camera preferred preview size for video is 1920x1080
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika-AFL: Setting aspect ratio to 1.3333333333333333 (was -1.0)
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika: onResume complete: com.android.grafika.CameraCaptureActivity@42adc288
04-18 18:06:50.064 14383-32312/com.android.grafika D/Grafika: setCameraPreviewSize
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 526]
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: new size=701x526 + padding 0x0
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 430]
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: new size=573x430 + padding 0x0
04-18 18:06:50.124 14383-32312/com.android.grafika D/Grafika: onSurfaceCreated
04-18 18:06:50.134 14383-32312/com.android.grafika D/Grafika: Created program 3 (TEXTURE_EXT)
04-18 18:06:50.144 14383-32312/com.android.grafika D/Grafika: onSurfaceChanged 573x430
04-18 18:06:50.144 14383-32312/com.android.grafika D/Grafika: Updating filter to 0
04-18 18:06:50.154 14383-14383/com.android.grafika D/Grafika: onItemSelected: 0
04-18 18:06:50.154 14383-14383/com.android.grafika D/Grafika: CameraHandler [Handler (com.android.grafika.CameraCaptureActivity$CameraHandler) {42aeec70}]: what=0
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 526]
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: new size=701x526 + padding 0x0
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 430]
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: new size=573x430 + padding 0x0
04-18 18:06:50.544 14383-14383/com.android.grafika D/AbsListView: onVisibilityChanged() is called, visibility : 4
04-18 18:06:50.544 14383-14383/com.android.grafika D/AbsListView: unregisterIRListener() is called 
</code></pre>

<p><strong>So...</strong></p>

<p>If one of you guys have an idea of how to make it work, I would love to hear from him.</p>

<p>Thanks!</p>