Android ImageReader.acquireLatestImage returns invalid JPG
I am using Android ImageReader class to receive Bitmaps from MediaProjection.createVirtualDisplay method.
My code so far looks like this:
mProjection.createVirtualDisplay("test", width, height, density, flags, mImageReader.getSurface(), new VirtualDisplayCallback(), mHandler);
mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Image image = null;
try {
image = mImageReader.acquireLatestImage();
final Image.Plane[] planes = image.getPlanes();
final ByteBuffer buffer = planes[0].getBuffer();
final byte[] data = new byte[buffer.capacity()];
buffer.get(data);
final Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);
if (bitmap==null)
Log.e(TAG, "bitmap is null");
} catch (Exception e) {
if (image!=null)
image.close();
}
}
}, mHandler);
The problem is that BitmapFactory cannot decode data[] back to Bitmap, i.e. BitmapFactory always returns null. The only messages I see from logcat come from android_media_ImageReader.cpp and go like this:
D/ImageReader_JNI(1432): ImageReader_imageSetup: Receiving JPEG in HAL_PIXEL_FORMAT_RGBA_8888 buffer.
W/ImageReader_JNI(1432): Image_getJpegSize: No JPEG header detected, defaulting to size=width=3891200
Image object returned by acquireLatestImage is not null but not a valid JPEG either, I tried to check with the following test which fails:
if((buf [0] & 0xFF) == 0xFF && (buf[1] & 0xFF) == 0xD8 && (buf[2] & 0xFF) == 0xFF && (buf[3] & 0xFF) == 0xE0)
Log.e(TAG, "is JPG");
else
Log.e(TAG, "not a valid JPG");
The only think I am suspecting at the moment is that Android 5.0 emulator I am testing against cannot hanlde the API calls.
Any ideas?
The code in answer by @charlesjean works but I would rather not generate each pixel by my self. A better way to get the Image from ImageReader is just to create right sized bitmap and use the method copyPixelsFromBuffer(). Create ImageReader as follows:
mImageReader = ImageReader.newInstance(mWidth, mHeight, ImageFormat.RGB_565, 2);
Then you can get the image from mImageReader using the code below.
final Image.Plane[] planes = image.getPlanes();
final ByteBuffer buffer = planes[0].getBuffer();
int offset = 0;
int pixelStride = planes[0].getPixelStride();
int rowStride = planes[0].getRowStride();
int rowPadding = rowStride - pixelStride * mWidth;
// create bitmap
bitmap = Bitmap.createBitmap(mWidth+rowPadding/pixelStride, mHeight, Bitmap.Config.RGB_565);
bitmap.copyPixelsFromBuffer(buffer);
image.close();
I have described the process of capturing screen using MediaProjection API along with the mistakes most people made when getting image from ImageReader in a blog post which you can read if interested.
I encountered exactly your problem. My ImageReader created as so:
ImageReader.newInstance(mCaptureSize.getWidth(), mCaptureSize.getHeight(), ImageFormat.JPEG, 1);
The ImageReader above should only return compressed images, and these need to be decompressed. I acquireLatestImage(), then pass it through the following:
ByteBuffer bBuffer = planes[0].getBuffer;
bBuffer.rewind();
byte[] buffer = new byte[bBuffer.remaining()];
planes[0].getBuffer().get(buffer);
Bitmap bitmap = BitmapFactory.decodeByteArray(buffer, 0, buffer.length);
The key for me was to rewind the ByteBuffer. Your code should work as so:
mProjection.createVirtualDisplay("test", width, height, density, flags, mImageReader.getSurface(), new VirtualDisplayCallback(), mHandler);
mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Image image = null;
try {
image = mImageReader.acquireLatestImage();
final Image.Plane[] planes = image.getPlanes();
final ByteBuffer buffer = planes[0].getBuffer();
buffer.rewind()
final byte[] data = new byte[buffer.capacity()];
buffer.get(data);
final Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);
if (bitmap==null)
Log.e(TAG, "bitmap is null");
} catch (Exception e) {
if (image!=null)
image.close();
}
}
}, mHandler);
I don't like having to copy the ByteBuffer through an intermediate byte[], but the internal array is protected.
Tested working on 5.0.1 on an HTC
I tested the code of the first answer, but unfortunately it does not work on real device. I make some investigation and the following code solved my problem:
mImgReader = ImageReader.newInstance(mWidth, mHeight, PixelFormat.RGBA_8888, 5);
mSurface = mImgReader.getSurface();// mSurfaceView.getHolder().getSurface();
mImgReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Log.i(TAG, "in OnImageAvailable");
FileOutputStream fos = null;
Bitmap bitmap = null;
Image img = null;
try {
img = reader.acquireLatestImage();
if (img != null) {
Image.Plane[] planes = img.getPlanes();
if (planes[0].getBuffer() == null) {
return;
}
int width = img.getWidth();
int height = img.getHeight();
int pixelStride = planes[0].getPixelStride();
int rowStride = planes[0].getRowStride();
int rowPadding = rowStride - pixelStride * width;
byte[] newData = new byte[width * height * 4];
int offset = 0;
bitmap = Bitmap.createBitmap(metrics,width, height, Bitmap.Config.ARGB_8888);
ByteBuffer buffer = planes[0].getBuffer();
for (int i = 0; i < height; ++i) {
for (int j = 0; j < width; ++j) {
int pixel = 0;
pixel |= (buffer.get(offset) & 0xff) << 16; // R
pixel |= (buffer.get(offset + 1) & 0xff) << 8; // G
pixel |= (buffer.get(offset + 2) & 0xff); // B
pixel |= (buffer.get(offset + 3) & 0xff) << 24; // A
bitmap.setPixel(j, i, pixel);
offset += pixelStride;
}
offset += rowPadding;
}
String name = "/myscreen" + count + ".png";
count++;
File file = new File(Environment.getExternalStorageDirectory(), name);
fos = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, fos);
Log.i(TAG, "image saved in" + Environment.getExternalStorageDirectory() + name);
img.close();
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (null != fos) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (null != bitmap) {
bitmap.recycle();
}
if (null != img) {
img.close();
}
}
}
}, mHandler);