From 18162c622ab4ac8e51a21b9a5ae9b40aff5b478e Mon Sep 17 00:00:00 2001 From: Eino-Ville Talvala Date: Mon, 9 Feb 2015 15:47:27 -0800 Subject: [PATCH] Camera2: Add camera pose and distortion fields - android.lens.info.poseRotation - android.lens.info.poseTranslation - android.lens.info.intrinsicCalibration - android.lens.info.radialDistortion Change-Id: Ia71e525b84bb9e252437a58dea5c8fadde5d7d68 --- camera/docs/docs.html | 553 +++++++++++++++++++++++++-- camera/docs/metadata_properties.xml | 170 ++++++-- camera/include/system/camera_metadata_tags.h | 6 +- camera/src/camera_metadata_tag_info.c | 22 +- 4 files changed, 689 insertions(+), 62 deletions(-) diff --git a/camera/docs/docs.html b/camera/docs/docs.html index 4ad5d985..1fef74bf 100644 --- a/camera/docs/docs.html +++ b/camera/docs/docs.html @@ -498,9 +498,13 @@
  • android.lens.facing
  • android.lens.opticalAxisAngle
    • + >android.lens.poseRotation
  • android.lens.position
    • + >android.lens.poseTranslation +
  • android.lens.intrinsicCalibration
    • +
  • android.lens.radialDistortion
  • @@ -520,6 +524,14 @@ >android.lens.opticalStabilizationMode
  • android.lens.state
    • +
  • android.lens.poseRotation
    • +
  • android.lens.poseTranslation
    • +
  • android.lens.intrinsicCalibration
    • +
  • android.lens.radialDistortion
    • @@ -11876,43 +11888,44 @@ device screen.

    - + - android.lens.opticalAxisAngle + android.lens.poseRotation float x - 2 + 4 - [system] + [public] -
    degrees. First defines the angle of separation between the perpendicular to the screen and the camera optical axis. The second then defines the clockwise rotation of the optical axis from native device up.
    -

    Relative angle of camera optical axis to the -perpendicular axis from the display

    +

    The orientation of the camera relative to the sensor +coordinate system.

    + + Quarternion coefficients + -

    [0-90) for first angle, [0-360) for second

    @@ -11922,15 +11935,32 @@ perpendicular axis from the display

    -

    Examples:

    -

    (0,0) means that the camera optical axis -is perpendicular to the display surface;

    -

    (45,0) means that the camera points 45 degrees up when -device is held upright;

    -

    (45,90) means the camera points 45 degrees to the right when -the device is held upright.

    -

    Use FACING field to determine perpendicular outgoing -direction

    +

    The four coefficients that describe the quarternion +rotation from the Android sensor coordinate system to a +camera-aligned coordinate system where the X-axis is +aligned with the long side of the image sensor, the Y-axis +is aligned with the short side of the image sensor, and +the Z-axis is aligned with the optical axis of the sensor.

    +

    To convert from the quarternion coefficients (x,y,z,w) +to the axis of rotation (a_x, a_y, a_z) and rotation +amount theta, the following formulas can be used:

    +
     theta = 2 * acos(w)
+a_x = x / sin(theta/2)
+a_y = y / sin(theta/2)
+a_z = z / sin(theta/2)
+
    +

    To create a 3x3 rotation matrix that applies the rotation +defined by this quarternion, the following matrix can be +used:

    +
    R = [ 1 - 2y^2 - 2z^2,       2xy - 2zw,       2xz + 2yw,
+           2xy + 2zw, 1 - 2x^2 - 2z^2,       2yz - 2xw,
+           2xz - 2yw,       2yz + 2xw, 1 - 2x^2 - 2y^2 ]
+
    +

    This matrix can then be used to apply the rotation to a + column vector point with

    +

    p' = Rp

    +

    where p is in the device sensor coordinate system, and + p' is in the camera-oriented coordinate system.

    @@ -11939,20 +11969,19 @@ direction

    - + - android.lens.position + " rowspan="3"> + android.lens.poseTranslation float x - 3, location in mm, in the sensor coordinate - system + 3 - [system] + [public] @@ -11962,11 +11991,11 @@ direction

    -

    Coordinates of camera optical axis on -device

    +

    Position of the camera optical center.

    + Meters @@ -11974,11 +12003,178 @@ device

    + + Details + + + +

    As measured in the device sensor coordinate system, the +position of the camera device's optical center, as a +three-dimensional vector (x,y,z).

    +

    To transform a world position to a camera-device centered +coordinate system, the position must be translated by this +vector and then rotated by android.lens.poseRotation.

    + + + + + + + + + + + android.lens.intrinsicCalibration + + + float + x + + + 5 + + [public] + + + + + + + + + +

    The parameters for this camera device's intrinsic +calibration.

    + + + + + Pixels in the android.sensor.activeArraySize coordinate + system. + + + + + + + + + + + + + Details + + + +

    The five calibration parameters that describe the +transform from camera-centric 3D coordinates to sensor +pixel coordinates:

    +
    [f_x, f_y, c_x, c_y, s]
+
    +

    Where f_x and f_y are the horizontal and vertical +focal lengths, [c_x, c_y] is the position of the optical +axis, and s is a skew parameter for the sensor plane not +being aligned with the lens plane.

    +

    These are typically used within a transformation matrix K:

    +
    K = [ f_x,   s, c_x,
+       0, f_y, c_y,
+       0    0,   1 ]
+
    +

    which can then be combined with the camera pose rotation +R and translation t (android.lens.poseRotation and +android.lens.poseTranslation, respective) to calculate the +complete transform from world coordinates to pixel +coordinates:

    +
    P = [ K 0   * [ R t
+     0 1 ]     0 1 ]
+
    +

    and with p_w being a point in the world coordinate system +and p_s being a point in the camera active pixel array +coordinate system, and with the mapping including the +homogeneous division by z:

    +
     p_h = (x_h, y_h, z_h) = P p_w
+p_s = p_h / z_h
+
    +

    so [x_s, y_s] is the pixel coordinates of the world +point, z_s = 1, and w_s is a measurement of disparity +(depth) in pixel coordinates.

    + + + + + + + + + + + android.lens.radialDistortion + + + float + x + + + 3 + + [public] + + + + + + + + + +

    The correction coefficients to correct for this camera device's +radial lens distortion.

    + + + + + Coefficients for a 6th-degree even radial polynomial. + + + + + + + + + + + + + Details + + + +

    Three cofficients [kappa_1, kappa_2, kappa_3] that +can be used to correct the lens's radial geometric +distortion with the mapping equations:

    +
     x_c = x_i * ( 1 + kappa_1 * r^2 + kappa_2 * r^4 + kappa_3 * r^6 )
+y_c = y_i * ( 1 + kappa_1 * r^2 + kappa_2 * r^4 + kappa_3 * r^6 )
+
    +

    where [x_i, y_i] are normalized coordinates with (0,0) +at the lens optical center, and [-1, 1] are the edges of +the active pixel array; and where [x_c, y_c] are the +corrected normalized coordinates with radial distortion +removed; and r^2 = x_i^2 + y_i^2.

    + + @@ -12467,6 +12663,299 @@ is changing.

    + + + + android.lens.poseRotation + + + float + x + + + 4 + + [public] + + + + + + + + + +

    The orientation of the camera relative to the sensor +coordinate system.

    + + + + + Quarternion coefficients + + + + + + + + + + + + + Details + + + +

    The four coefficients that describe the quarternion +rotation from the Android sensor coordinate system to a +camera-aligned coordinate system where the X-axis is +aligned with the long side of the image sensor, the Y-axis +is aligned with the short side of the image sensor, and +the Z-axis is aligned with the optical axis of the sensor.

    +

    To convert from the quarternion coefficients (x,y,z,w) +to the axis of rotation (a_x, a_y, a_z) and rotation +amount theta, the following formulas can be used:

    +
     theta = 2 * acos(w)
+a_x = x / sin(theta/2)
+a_y = y / sin(theta/2)
+a_z = z / sin(theta/2)
+
    +

    To create a 3x3 rotation matrix that applies the rotation +defined by this quarternion, the following matrix can be +used:

    +
    R = [ 1 - 2y^2 - 2z^2,       2xy - 2zw,       2xz + 2yw,
+           2xy + 2zw, 1 - 2x^2 - 2z^2,       2yz - 2xw,
+           2xz - 2yw,       2yz + 2xw, 1 - 2x^2 - 2y^2 ]
+
    +

    This matrix can then be used to apply the rotation to a + column vector point with

    +

    p' = Rp

    +

    where p is in the device sensor coordinate system, and + p' is in the camera-oriented coordinate system.

    + + + + + + + + + + + android.lens.poseTranslation + + + float + x + + + 3 + + [public] + + + + + + + + + +

    Position of the camera optical center.

    + + + + Meters + + + + + + + + + + + + Details + + + +

    As measured in the device sensor coordinate system, the +position of the camera device's optical center, as a +three-dimensional vector (x,y,z).

    +

    To transform a world position to a camera-device centered +coordinate system, the position must be translated by this +vector and then rotated by android.lens.poseRotation.

    + + + + + + + + + + + android.lens.intrinsicCalibration + + + float + x + + + 5 + + [public] + + + + + + + + + +

    The parameters for this camera device's intrinsic +calibration.

    + + + + + Pixels in the android.sensor.activeArraySize coordinate + system. + + + + + + + + + + + + + Details + + + +

    The five calibration parameters that describe the +transform from camera-centric 3D coordinates to sensor +pixel coordinates:

    +
    [f_x, f_y, c_x, c_y, s]
+
    +

    Where f_x and f_y are the horizontal and vertical +focal lengths, [c_x, c_y] is the position of the optical +axis, and s is a skew parameter for the sensor plane not +being aligned with the lens plane.

    +

    These are typically used within a transformation matrix K:

    +
    K = [ f_x,   s, c_x,
+       0, f_y, c_y,
+       0    0,   1 ]
+
    +

    which can then be combined with the camera pose rotation +R and translation t (android.lens.poseRotation and +android.lens.poseTranslation, respective) to calculate the +complete transform from world coordinates to pixel +coordinates:

    +
    P = [ K 0   * [ R t
+     0 1 ]     0 1 ]
+
    +

    and with p_w being a point in the world coordinate system +and p_s being a point in the camera active pixel array +coordinate system, and with the mapping including the +homogeneous division by z:

    +
     p_h = (x_h, y_h, z_h) = P p_w
+p_s = p_h / z_h
+
    +

    so [x_s, y_s] is the pixel coordinates of the world +point, z_s = 1, and w_s is a measurement of disparity +(depth) in pixel coordinates.

    + + + + + + + + + + + android.lens.radialDistortion + + + float + x + + + 3 + + [public] + + + + + + + + + +

    The correction coefficients to correct for this camera device's +radial lens distortion.

    + + + + + Coefficients for a 6th-degree even radial polynomial. + + + + + + + + + + + + + Details + + + +

    Three cofficients [kappa_1, kappa_2, kappa_3] that +can be used to correct the lens's radial geometric +distortion with the mapping equations:

    +
     x_c = x_i * ( 1 + kappa_1 * r^2 + kappa_2 * r^4 + kappa_3 * r^6 )
+y_c = y_i * ( 1 + kappa_1 * r^2 + kappa_2 * r^4 + kappa_3 * r^6 )
+
    +

    where [x_i, y_i] are normalized coordinates with (0,0) +at the lens optical center, and [-1, 1] are the edges of +the active pixel array; and where [x_c, y_c] are the +corrected normalized coordinates with radial distortion +removed; and r^2 = x_i^2 + y_i^2.

    + + + + + + + @@ -25007,6 +25496,10 @@ duration.

    Entry is required for the depth capability.