From a3cb77fd1751e68d69ab69e750d2b63d575b9085 Mon Sep 17 00:00:00 2001 From: Emilian Peev Date: Tue, 21 Nov 2023 20:45:38 +0000 Subject: Camera: Add support for active physical crop Allow camera devices to report the source physical camera crop when using the logical camera. Additionally enable also intra-frame lens intrinsics samples. Bug: 297083874 Test: Successful build Change-Id: I58b34e83a14fc791a46bdb9a8b223d5ca013639d --- camera/docs/docs.html | 284 +++++++++++++++++++++++++++ camera/docs/metadata_definitions.xml | 110 +++++++++++ camera/include/system/camera_metadata_tags.h | 4 + camera/src/camera_metadata_asserts.cpp | 6 + camera/src/camera_metadata_tag_info.c | 25 +++ 5 files changed, 429 insertions(+) diff --git a/camera/docs/docs.html b/camera/docs/docs.html index be27c210..e9cc8485 100644 --- a/camera/docs/docs.html +++ b/camera/docs/docs.html @@ -1127,6 +1127,12 @@ >android.statistics.oisYShifts
  • android.statistics.oisSamples
    • +
  • android.statistics.lensIntrinsicsSamples
    • +
  • android.statistics.lensIntrinsicTimestamps
    • +
  • android.statistics.lensIntrinsicSamples
    • @@ -1350,6 +1356,8 @@ @@ -34137,6 +34145,181 @@ is needed.

    + + + + android.statistics.lensIntrinsicsSamples + + + float + x + + + n + + [java_public as lensIntrinsicsSample] + + [synthetic] + + + + + + + + +

    An array of intra-frame lens intrinsic samples.

    + + + + + + + + + +

    3.9

    + + + + + + + + Details + + + +

    Contains an array of intra-frame android.lens.intrinsicCalibration updates. This must +not be confused or compared to android.statistics.oisSamples. Although OIS could be the +main driver, all relevant factors such as focus distance and optical zoom must also +be included. Do note that OIS samples must not be applied on top of the lens intrinsic +samples. +Support for this capture result can be queried via +CameraCharacteristics#getAvailableCaptureResultKeys. +If available, clients can expect multiple samples per capture result. The specific +amount will depend on current frame duration and sampling rate. Generally a sampling rate +greater than or equal to 200Hz is considered sufficient for high quality results.

    + + + + + + + + + + + android.statistics.lensIntrinsicTimestamps + + + int64 + x + + + n + + [ndk_public] + + + + + + + + + +

    An array of timestamps of lens intrinsics samples, in nanoseconds.

    + + + + nanoseconds + + + + + + +

    3.9

    + + + + + + + + Details + + + +

    The array contains the timestamps of lens intrinsics samples. The timestamps are in the +same timebase as and comparable to android.sensor.timestamp.

    + + + + + + + + + + + android.statistics.lensIntrinsicSamples + + + float + x + + + 5 x n + + [ndk_public] + + + + + + + + + +

    An array of intra-frame lens intrinsics.

    + + + + + Pixels in the android.sensor.info.preCorrectionActiveArraySize coordinate system. + + + + + + + +

    3.9

    + + + + + + + + Details + + + +

    The data layout and contents of individual array entries matches with +android.lens.intrinsicCalibration.

    + + + + + + + @@ -38533,6 +38716,106 @@ result metadata to indicate current active physical camera ID.

    + + + + android.logicalMultiCamera.activePhysicalSensorCropRegion + + + int32 + x + + + 4 + + [public as rectangle] + + + + + + + + + +

    The current region of the active physical sensor that will be read out for this +capture.

    + + + + Pixel coordinates relative to + android.sensor.info.activeArraySize or + android.sensor.info.preCorrectionActiveArraySize of the currently + android.logicalMultiCamera.activePhysicalId depending on distortion correction capability + and mode + + + + + + +

    3.9

    + + + + + + + + + Details + + + +

    This capture result matches with android.scaler.cropRegion on non-logical single +camera sensor devices. In case of logical cameras that can switch between several +physical devices in response to android.control.zoomRatio, this capture result will +not behave like android.scaler.cropRegion and android.control.zoomRatio, where the +combination of both reflects the effective zoom and crop of the logical camera output. +Instead, this capture result value will describe the zoom and crop of the active physical +device. Some examples of when the value of this capture result will change include +switches between different physical lenses, switches between regular and maximum +resolution pixel mode and going through the device digital or optical range. +This capture result is similar to android.scaler.cropRegion with respect to distortion +correction. When the distortion correction mode is OFF, the coordinate system follows +android.sensor.info.preCorrectionActiveArraySize, with (0, 0) being the top-left pixel +of the pre-correction active array. When the distortion correction mode is not OFF, +the coordinate system follows android.sensor.info.activeArraySize, with (0, 0) being +the top-left pixel of the active array.

    +

    For camera devices with the +CameraMetadata#REQUEST_AVAILABLE_CAPABILITIES_ULTRA_HIGH_RESOLUTION_SENSOR +capability or devices where CameraCharacteristics#getAvailableCaptureRequestKeys +lists android.sensor.pixelMode +, the current active physical device +android.sensor.info.activeArraySizeMaximumResolution / +android.sensor.info.preCorrectionActiveArraySizeMaximumResolution must be used as the +coordinate system for requests where android.sensor.pixelMode is set to +CameraMetadata#SENSOR_PIXEL_MODE_MAXIMUM_RESOLUTION.

    + + + + + HAL Implementation Details + + + +

    The output streams must maintain square pixels at all +times, no matter what the relative aspect ratios of the +crop region and the stream are. Negative values for +corner are allowed for raw output if full pixel array is +larger than active pixel array. Width and height may be +rounded to nearest larger supportable width, especially +for raw output, where only a few fixed scales may be +possible.

    + + + + + + @@ -40771,6 +41054,7 @@ output format/size combination for Jpeg/R streams for CaptureRequest
  • android.logicalMultiCamera.physicalIds (static)
  • android.logicalMultiCamera.sensorSyncType (static)
  • android.logicalMultiCamera.activePhysicalId (dynamic)
    • +
  • android.logicalMultiCamera.activePhysicalSensorCropRegion (dynamic)
  • HEIC - diff --git a/camera/docs/metadata_definitions.xml b/camera/docs/metadata_definitions.xml index 4bca52cf..e82075de 100644 --- a/camera/docs/metadata_definitions.xml +++ b/camera/docs/metadata_definitions.xml @@ -155,6 +155,9 @@ xsi:schemaLocation="http://schemas.android.com/service/camera/metadata/ metadata android.hardware.camera2.params.ColorSpaceProfiles + + android.hardware.camera2.params.LensIntrinsicsSample + @@ -12142,6 +12145,59 @@ xsi:schemaLocation="http://schemas.android.com/service/camera/metadata/ metadata is needed. + + + n + + + An array of intra-frame lens intrinsic samples. + +
    + Contains an array of intra-frame android.lens.intrinsicCalibration updates. This must + not be confused or compared to android.statistics.oisSamples. Although OIS could be the + main driver, all relevant factors such as focus distance and optical zoom must also + be included. Do note that OIS samples must not be applied on top of the lens intrinsic + samples. + Support for this capture result can be queried via + {@link android.hardware.camera2.CameraCharacteristics#getAvailableCaptureResultKeys}. + If available, clients can expect multiple samples per capture result. The specific + amount will depend on current frame duration and sampling rate. Generally a sampling rate + greater than or equal to 200Hz is considered sufficient for high quality results. +
    +     + + + n + + + An array of timestamps of lens intrinsics samples, in nanoseconds. + + nanoseconds +
    + The array contains the timestamps of lens intrinsics samples. The timestamps are in the + same timebase as and comparable to android.sensor.timestamp. +
    +     + + + 5 + n + + + An array of intra-frame lens intrinsics. + + + Pixels in the android.sensor.info.preCorrectionActiveArraySize coordinate system. + +
    + The data layout and contents of individual array entries matches with + android.lens.intrinsicCalibration. +
    +
    @@ -13676,6 +13732,60 @@ xsi:schemaLocation="http://schemas.android.com/service/camera/metadata/ metadata + + + 4 + + The current region of the active physical sensor that will be read out for this + capture. + Pixel coordinates relative to + android.sensor.info.activeArraySize or + android.sensor.info.preCorrectionActiveArraySize of the currently + android.logicalMultiCamera.activePhysicalId depending on distortion correction capability + and mode +
    + This capture result matches with android.scaler.cropRegion on non-logical single + camera sensor devices. In case of logical cameras that can switch between several + physical devices in response to android.control.zoomRatio, this capture result will + not behave like android.scaler.cropRegion and android.control.zoomRatio, where the + combination of both reflects the effective zoom and crop of the logical camera output. + Instead, this capture result value will describe the zoom and crop of the active physical + device. Some examples of when the value of this capture result will change include + switches between different physical lenses, switches between regular and maximum + resolution pixel mode and going through the device digital or optical range. + This capture result is similar to android.scaler.cropRegion with respect to distortion + correction. When the distortion correction mode is OFF, the coordinate system follows + android.sensor.info.preCorrectionActiveArraySize, with (0, 0) being the top-left pixel + of the pre-correction active array. When the distortion correction mode is not OFF, + the coordinate system follows android.sensor.info.activeArraySize, with (0, 0) being + the top-left pixel of the active array. + + For camera devices with the + {@link android.hardware.camera2.CameraMetadata#REQUEST_AVAILABLE_CAPABILITIES_ULTRA_HIGH_RESOLUTION_SENSOR} + capability or devices where {@link CameraCharacteristics#getAvailableCaptureRequestKeys} + lists {@link CaptureRequest#SENSOR_PIXEL_MODE android.sensor.pixelMode} + , the current active physical device + android.sensor.info.activeArraySizeMaximumResolution / + android.sensor.info.preCorrectionActiveArraySizeMaximumResolution must be used as the + coordinate system for requests where android.sensor.pixelMode is set to + {@link android.hardware.camera2.CameraMetadata#SENSOR_PIXEL_MODE_MAXIMUM_RESOLUTION}. +
    + + The data representation is int[4], which maps to (left, top, width, height). + + + The output streams must maintain square pixels at all + times, no matter what the relative aspect ratios of the + crop region and the stream are. Negative values for + corner are allowed for raw output if full pixel array is + larger than active pixel array. Width and height may be + rounded to nearest larger supportable width, especially + for raw output, where only a few fixed scales may be + possible. + + +
    diff --git a/camera/include/system/camera_metadata_tags.h b/camera/include/system/camera_metadata_tags.h index 4fea968e..a7bf9550 100644 --- a/camera/include/system/camera_metadata_tags.h +++ b/camera/include/system/camera_metadata_tags.h @@ -453,6 +453,8 @@ typedef enum camera_metadata_tag { ANDROID_STATISTICS_OIS_TIMESTAMPS, // int64[] | ndk_public | HIDL v3.3 ANDROID_STATISTICS_OIS_X_SHIFTS, // float[] | ndk_public | HIDL v3.3 ANDROID_STATISTICS_OIS_Y_SHIFTS, // float[] | ndk_public | HIDL v3.3 + ANDROID_STATISTICS_LENS_INTRINSIC_TIMESTAMPS, // int64[] | ndk_public | HIDL v3.9 + ANDROID_STATISTICS_LENS_INTRINSIC_SAMPLES, // float[] | ndk_public | HIDL v3.9 ANDROID_STATISTICS_END, ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES = @@ -540,6 +542,8 @@ typedef enum camera_metadata_tag { ANDROID_LOGICAL_MULTI_CAMERA_START, ANDROID_LOGICAL_MULTI_CAMERA_SENSOR_SYNC_TYPE, // enum | public | HIDL v3.3 ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_ID, // byte | public | HIDL v3.4 + ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_SENSOR_CROP_REGION, + // int32[] | public | HIDL v3.9 ANDROID_LOGICAL_MULTI_CAMERA_END, ANDROID_DISTORTION_CORRECTION_MODE = // enum | public | HIDL v3.3 diff --git a/camera/src/camera_metadata_asserts.cpp b/camera/src/camera_metadata_asserts.cpp index 9091b5bf..6f559f3f 100644 --- a/camera/src/camera_metadata_asserts.cpp +++ b/camera/src/camera_metadata_asserts.cpp @@ -750,6 +750,10 @@ static_assert(static_cast(ANDROID_STATISTICS_OIS_X_SHIFTS) == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_STATISTICS_OIS_X_SHIFTS)); static_assert(static_cast(ANDROID_STATISTICS_OIS_Y_SHIFTS) == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_STATISTICS_OIS_Y_SHIFTS)); +static_assert(static_cast(ANDROID_STATISTICS_LENS_INTRINSIC_TIMESTAMPS) + == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_STATISTICS_LENS_INTRINSIC_TIMESTAMPS)); +static_assert(static_cast(ANDROID_STATISTICS_LENS_INTRINSIC_SAMPLES) + == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_STATISTICS_LENS_INTRINSIC_SAMPLES)); static_assert(static_cast(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES) == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES)); static_assert(static_cast(ANDROID_STATISTICS_INFO_HISTOGRAM_BUCKET_COUNT) @@ -842,6 +846,8 @@ static_assert(static_cast(ANDROID_LOGICAL_MULTI_CAMERA_SENSOR_SYNC_TYPE) == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_LOGICAL_MULTI_CAMERA_SENSOR_SYNC_TYPE)); static_assert(static_cast(ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_ID) == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_ID)); +static_assert(static_cast(ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_SENSOR_CROP_REGION) + == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_SENSOR_CROP_REGION)); static_assert(static_cast(ANDROID_DISTORTION_CORRECTION_MODE) == static_cast(::aidl::android::hardware::camera::metadata::CameraMetadataTag::ANDROID_DISTORTION_CORRECTION_MODE)); static_assert(static_cast(ANDROID_DISTORTION_CORRECTION_AVAILABLE_MODES) diff --git a/camera/src/camera_metadata_tag_info.c b/camera/src/camera_metadata_tag_info.c index 41a6b0ac..f6c963b3 100644 --- a/camera/src/camera_metadata_tag_info.c +++ b/camera/src/camera_metadata_tag_info.c @@ -746,6 +746,10 @@ static tag_info_t android_statistics[ANDROID_STATISTICS_END - { "oisXShifts", TYPE_FLOAT }, [ ANDROID_STATISTICS_OIS_Y_SHIFTS - ANDROID_STATISTICS_START ] = { "oisYShifts", TYPE_FLOAT }, + [ ANDROID_STATISTICS_LENS_INTRINSIC_TIMESTAMPS - ANDROID_STATISTICS_START ] = + { "lensIntrinsicTimestamps", TYPE_INT64 }, + [ ANDROID_STATISTICS_LENS_INTRINSIC_SAMPLES - ANDROID_STATISTICS_START ] = + { "lensIntrinsicSamples", TYPE_FLOAT }, }; static tag_info_t android_statistics_info[ANDROID_STATISTICS_INFO_END - @@ -887,6 +891,9 @@ static tag_info_t android_logical_multi_camera[ANDROID_LOGICAL_MULTI_CAMERA_END { "sensorSyncType", TYPE_BYTE }, [ ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_ID - ANDROID_LOGICAL_MULTI_CAMERA_START ] = { "activePhysicalId", TYPE_BYTE }, + [ ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_SENSOR_CROP_REGION - ANDROID_LOGICAL_MULTI_CAMERA_START ] = + { "activePhysicalSensorCropRegion", + TYPE_INT32 }, }; static tag_info_t android_distortion_correction[ANDROID_DISTORTION_CORRECTION_END - @@ -3393,6 +3400,12 @@ int camera_metadata_enum_snprint(uint32_t tag, case ANDROID_STATISTICS_OIS_Y_SHIFTS: { break; } + case ANDROID_STATISTICS_LENS_INTRINSIC_TIMESTAMPS: { + break; + } + case ANDROID_STATISTICS_LENS_INTRINSIC_SAMPLES: { + break; + } case ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES: { break; @@ -3736,6 +3749,9 @@ int camera_metadata_enum_snprint(uint32_t tag, case ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_ID: { break; } + case ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_SENSOR_CROP_REGION: { + break; + } case ANDROID_DISTORTION_CORRECTION_MODE: { switch (value) { @@ -6723,6 +6739,12 @@ int camera_metadata_enum_value(uint32_t tag, case ANDROID_STATISTICS_OIS_Y_SHIFTS: { break; } + case ANDROID_STATISTICS_LENS_INTRINSIC_TIMESTAMPS: { + break; + } + case ANDROID_STATISTICS_LENS_INTRINSIC_SAMPLES: { + break; + } case ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES: { break; @@ -7074,6 +7096,9 @@ int camera_metadata_enum_value(uint32_t tag, case ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_ID: { break; } + case ANDROID_LOGICAL_MULTI_CAMERA_ACTIVE_PHYSICAL_SENSOR_CROP_REGION: { + break; + } case ANDROID_DISTORTION_CORRECTION_MODE: { enumName = "OFF"; -- cgit v1.2.3