Proper fix for HAL_PIXEL_FORMAT_RGBX_8888

This is an appropriate fix for HAL_PIXEL_FORMAT_RGBX_8888.

The handling of RGBX_8888 is very subtle. Most of time we want it to
be treated as RGBA_8888, with the exception that alpha is always
ignored and treated as 1. The solution is to create 3 channel RGB
texture on the host while telling the guest to use 4 channel RGBA for
read/write. The host GL will handle the Alpha channel approriately.

BUG: 78602661

This cl does not impact real devices.

Test: atest CtsNativeHardwareTestCases
Change-Id: Ife299bfc55ce327d0213616b9902ba537cfb0d82

1 files changed, 11 insertions, 2 deletions

diff --git a/system/gralloc/gralloc.cpp b/system/gralloc/gralloc.cpp
index 9569d344..7c28215b 100644
--- a/system/gralloc/gralloc.cpp
+++ b/system/gralloc/gralloc.cpp
@@ -712,10 +712,19 @@ static int gralloc_alloc(alloc_device_t* dev,
 
     if (needHostCb) {
         if (hostCon && rcEnc) {
+            GLenum allocFormat = glFormat;
+            // The handling of RGBX_8888 is very subtle. Most of the time
+            // we want it to be treated as RGBA_8888, with the exception
+            // that alpha is always ignored and treated as 1. The solution
+            // is to create 3 channel RGB texture instead and host GL will
+            // handle the Alpha channel.
+            if (HAL_PIXEL_FORMAT_RGBX_8888 == format) {
+                allocFormat = GL_RGB;
+            }
             if (s_grdma) {
-                cb->hostHandle = rcEnc->rcCreateColorBufferDMA(rcEnc, w, h, glFormat, cb->emuFrameworkFormat);
+                cb->hostHandle = rcEnc->rcCreateColorBufferDMA(rcEnc, w, h, allocFormat, cb->emuFrameworkFormat);
             } else {
-                cb->hostHandle = rcEnc->rcCreateColorBuffer(rcEnc, w, h, glFormat);
+                cb->hostHandle = rcEnc->rcCreateColorBuffer(rcEnc, w, h, allocFormat);
             }
             D("Created host ColorBuffer 0x%x\n", cb->hostHandle);
         }