Android 10 Camera -- 3A Modes and State Transition
说明：转载自 Android Camera，不定期添加自己的学习思考。
3A Modes and State Transition
While the actual 3A algorithms are up to the HAL implementation, a high-level state machine description is defined by the HAL interface to allow the HAL device and the framework to communicate about the current state of 3A and trigger 3A events.
When the device is opened, all the individual 3A states must be STATE_INACTIVE. Stream configuration does not reset 3A. For example, locked focus must be maintained across the configure() call.
Triggering a 3A action involves simply setting the relevant trigger entry in the settings for the next request to indicate start of trigger. For example, the trigger for starting an autofocus scan is setting the entry ANDROID_CONTROL_AF_TRIGGER to ANDROID_CONTROL_AF_TRIGGER_START for one request; and cancelling an autofocus scan is triggered by setting ANDROID_CONTROL_AF_TRIGGER to ANDROID_CONTRL_AF_TRIGGER_CANCEL. Otherwise, the entry will not exist or be set to ANDROID_CONTROL_AF_TRIGGER_IDLE. Each request with a trigger entry set to a non-IDLE value will be treated as an independent triggering event.
At the top level, 3A is controlled by the ANDROID_CONTROL_MODE setting. It selects between no 3A (ANDROID_CONTROL_MODE_OFF), normal AUTO mode (ANDROID_CONTROL_MODE_AUTO), and using the scene mode setting (ANDROID_CONTROL_USE_SCENE_MODE):
- In OFF mode, each of the individual auto-focus (AF), auto-exposure (AE), and auto-whitebalance (AWB) modes are effectively OFF, and none of the capture controls may be overridden by the 3A routines.
- In AUTO mode, AF, AE, and AWB modes all run their own independent algorithms, and have their own mode, state, and trigger metadata entries, as listed in the next section.
- In USE_SCENE_MODE, the value of the ANDROID_CONTROL_SCENE_MODE entry must be used to determine the behavior of 3A routines. In SCENE_MODEs other than FACE_PRIORITY, the HAL must override the values of ANDROID_CONTROL_AE/AWB/AF_MODE to be the mode it prefers for the selected SCENE_MODE. For example, the HAL may prefer SCENE_MODE_NIGHT to use CONTINUOUS_FOCUS AF mode. Any user selection of AE/AWB/AF_MODE when scene must be ignored for these scene modes.
- For SCENE_MODE_FACE_PRIORITY, the AE/AWB/AFMODE controls work as in ANDROID_CONTROL_MODE_AUTO, but the 3A routines must bias toward metering and focusing on any detected faces in the scene.
Auto-focus settings and result entries
| Main metadata entries | |
|---|---|
ANDROID_CONTROL_AF_MODE |
Control for selecting the current autofocus mode. Set by the framework in the request settings. |
| AF_MODE_OFF | AF is disabled; the framework/app directly controls lens position. |
| AF_MODE_AUTO | Single-sweep autofocus. No lens movement unless AF is triggered. |
| AF_MODE_MACRO | Single-sweep up-close autofocus. No lens movement unless AF is triggered |
| AF_MODE_CONTINUOUS_VIDEO | Smooth continuous focusing, for recording video. Triggering immediately locks focus in current position. Canceling resumes continuous focusing. |
| AF_MODE_CONTINUOUS_PICTURE | Fast continuous focusing, for zero-shutter-lag still capture. Triggering locks focus once currently active sweep concludes. Canceling resumes continuous focusing. |
| AF_MODE_EDOF | Advanced extended depth of field focusing. There is no autofocus scan, so triggering one or canceling one has no effect. Images are focused automatically by the HAL. |
ANDROID_CONTROL_AF_STATE |
Dynamic metadata describing the current AF algorithm state, reported by the HAL in the result metadata. |
| AF_STATE_INACTIVE | No focusing has been done, or algorithm was reset. Lens is not moving. Always the state for MODE_OFF or MODE_EDOF. When the device is opened, it must start in this state. |
| AF_STATE_PASSIVE_SCAN | A continuous focus algorithm is currently scanning for good focus. The lens is moving. |
| AF_STATE_PASSIVE_FOCUSED | A continuous focus algorithm believes it is well focused. The lens is not moving. The HAL may spontaneously leave this state. |
| AF_STATE_PASSIVE_UNFOCUSED | A continuous focus algorithm believes it is not well focused. The lens is not moving. The HAL may spontaneously leave this state. |
| AF_STATE_ACTIVE_SCAN | A scan triggered by the user is underway. |
| AF_STATE_FOCUSED_LOCKED | The AF algorithm believes it is focused. The lens is not moving. |
| AF_STATE_NOT_FOCUSED_LOCKED | The AF algorithm has been unable to focus. The lens is not moving. |
ANDROID_CONTROL_AFTRIGGER |
Control for starting an autofocus scan, the meaning of which depends on mode and state. Set by the framework in the request settings. |
| AF_TRIGGER_IDLE | No current trigger. |
| AF_TRIGGER_START | Trigger start of AF scan. Effect depends on mode and state. |
| AF_TRIGGER_CANCEL | Cancel current AF scan if any, and reset algorithm to default. |
| Additional metadata entries | |
|---|---|
| ANDROID_CONTROL_AF_REGIONS | Control for selecting the regions of the field of view (FOV) that should be used to determine good focus. This applies to all AF modes that scan for focus. Set by the framework in the request settings. |
Auto-exposure settings and result entries
| Main metadata entries | |
|---|---|
ANDROID_CONTROL_AE_MODE |
Control for selecting the current auto-exposure mode. Set by the framework in the request settings. |
| AE_MODE_OFF | Autoexposure is disabled; the user controls exposure, gain, frame duration, and flash. |
| AE_MODE_ON | Standard autoexposure, with flash control disabled. User may set flash to fire or to torch mode. |
| AE_MODE_ON_AUTO_FLASH | Standard autoexposure, with flash on at HAL’s discretion for precapture and still capture. User control of flash disabled. |
| AE_MODE_ON_ALWAYS_FLASH | Standard autoexposure, with flash always fired for capture, and at HAL’s discretion for precapture. User control of flash disabled. |
| AE_MODE_ON_AUTO_FLASH_REDEYE | Standard autoexposure, with flash on at HAL’s discretion for precapture and still capture. Use a flash burst at end of precapture sequence to reduce redeye in the final picture. User control of flash disabled. |
ANDROID_CONTROL_AE_STATE |
Dynamic metadata describing the current AE algorithm state, reported by the HAL in the result metadata. |
| AE_STATE_INACTIVE | Initial AE state after mode switch. When the device is opened, it must start in this state. |
| AE_STATE_SEARCHING | AE is not converged to a good value and is adjusting exposure parameters. |
| AE_STATE_CONVERGED | AE has found good exposure values for the current scene, and the exposure parameters are not changing. HAL may spontaneously leave this state to search for a better solution. |
| AE_STATE_LOCKED | AE has been locked with the AE_LOCK control. Exposure values are not changing. |
| AE_STATE_FLASH_REQUIRED | The HAL has converged exposure but believes flash is required for a sufficiently bright picture. Used for determining if a zero-shutter-lag frame can be used. |
| AE_STATE_PRECAPTURE | The HAL is in the middle of a precapture sequence. Depending on AE mode, this mode may involve firing the flash for metering or a burst of flash pulses for redeye reduction. |
ANDROID_CONTROL_AE_PRECAPTURE_TRIGGER |
Control for starting a metering sequence before capturing a high-quality image. Set by the framework in the request settings. |
| PRECAPTURE_TRIGGER_IDLE | No current trigger. |
| PRECAPTURE_TRIGGER_START | Start a precapture sequence. The HAL should use the subsequent requests to measure good exposure/white balance for an upcoming high-resolution capture. |
| Additional metadata entries | |
|---|---|
| ANDROID_CONTROL_AE_LOCK | Control for locking AE controls to their current values. |
| ANDROID_CONTROL_AE_EXPOSURE_COMPENSATION | Control for adjusting AE algorithm target brightness point. |
| ANDROID_CONTROL_AE_TARGET_FPS_RANGE | Control for selecting the target frame rate range for the AE algorithm. The AE routine cannot change the frame rate to be outside these bounds. |
| ANDROID_CONTROL_AE_REGIONS | Control for selecting the regions of the FOV that should be used to determine good exposure levels. This applies to all AE modes besides OFF. |
Auto-whitebalance settings and result entries
| Main metadata entries | |
|---|---|
ANDROID_CONTROL_AWB_MODE |
Control for selecting the current white-balance mode. |
| AWB_MODE_OFF | Automatic white balance is enabled; 3A controls color transform, possibly using more complex transforms than a simple matrix. |
| AWB_MODE_INCANDESCENT | Fixed white balance settings good for indoor incandescent (tungsten) lighting, roughly 2700K. |
| AWB_MODE_FLUORESCENT | Fixed white balance settings good for fluorescent lighting, roughly 5000K. |
| AWB_MODE_WARM_FLUORESCENT | Fixed white balance settings good for fluorescent lighting, roughly 3000K. |
| AWB_MODE_DAYLIGHT | Fixed white balance settings good for daylight, roughly 5500K. |
| AWB_MODE_CLOUDY_DAYLIGHT | Fixed white balance settings good for clouded daylight, roughly 6500K. |
| AWB_MODE_TWILIGHT | Fixed white balance settings good for near-sunset/sunrise, roughly 15000K. |
| AWB_MODE_SHADE | Fixed white balance settings good for areas indirectly lit by the sun, roughly 7500K. |
ANDROID_CONTROL_AWB_STATE |
Dynamic metadata describing the current AWB algorithm state, reported by the HAL in the result metadata. |
| AWB_STATE_INACTIVE | Initial AWB state after mode switch. When the device is opened, it must start in this state. |
| AWB_STATE_SEARCHING | AWB is not converged to a good value and is changing color adjustment parameters. |
| AWB_STATE_CONVERGED | AWB has found good color adjustment values for the current scene, and the parameters are not changing. HAL may spontaneously leave this state to search for a better solution. |
| AWB_STATE_LOCKED | AWB has been locked with the AWB_LOCK control. Color adjustment values are not changing. |
| Additional metadata entries | |
|---|---|
| ANDROID_CONTROL_AWB_LOCK | Control for locking AWB color adjustments to their current values. |
| ANDROID_CONTROL_AWB_REGIONS | Control for selecting the regions of the FOV that should be used to determine good color balance. This applies only to auto-whitebalance mode. |
General state machine transition notes
Switching between AF, AE, or AWB modes always resets the algorithm’s state to INACTIVE. Similarly, switching between CONTROL_MODE or CONTROL_SCENE_MODE if CONTROL_MODE == USE_SCENE_MODE resets all the algorithm states to INACTIVE.
The tables below are per-mode.
AF state machines
| mode = AF_MODE_OFF or AF_MODE_EDOF | |
|---|---|
| State | Transformation cause |
| INACTIVE |
| mode = AF_MODE_AUTO or AF_MODE_MACRO | |||
|---|---|---|---|
| State | Transformation cause | New state | Notes |
| INACTIVE | AF_TRIGGER | ACTIVE_SCAN | Start AF sweep Lens now moving |
| ACTIVE_SCAN | AF sweep done | FOCUSED_LOCKED | If AF successful Lens now locked |
| ACTIVE_SCAN | AF sweep done | NOT_FOCUSED_LOCKED | If AF successful Lens now locked |
| ACTIVE_SCAN | AF_CANCEL | INACTIVE | Cancel/reset AF Lens now locked |
| FOCUSED_LOCKED | AF_CANCEL | INACTIVE | Cancel/reset AF |
| FOCUSED_LOCKED | AF_TRIGGER | ACTIVE_SCAN | Start new sweep Lens now moving |
| NOT_FOCUSED_LOCKED | AF_CANCEL | INACTIVE | Cancel/reset AF |
| NOT_FOCUSED_LOCKED | AF_TRIGGER | ACTIVE_SCAN | Start new sweep Lens now moving |
| All states | Mode change | INACTIVE |
| mode = AF_MODE_CONTINUOUS_VIDEO | |||
|---|---|---|---|
| State | Transformation cause | New state | Notes |
| INACTIVE | HAL initiates new scan | PASSIVE_SCAN | Start AF sweep Lens now moving |
| INACTIVE | AF_TRIGGER | NOT_FOCUSED_LOCKED | AF state query Lens now locked |
| PASSIVE_SCAN | HAL completes current scan | PASSIVE_FOCUSED | End AF scan Lens now locked |
| PASSIVE_SCAN | AF_TRIGGER | FOCUSED_LOCKED | Immediate transformation if focus is good Lens now locked |
| PASSIVE_SCAN | AF_TRIGGER | NOT_FOCUSED_LOCKED | Immediate transformation if focus is bad Lens now locked |
| PASSIVE_SCAN | AF_CANCEL | INACTIVE | Reset lens position Lens now locked |
| PASSIVE_FOCUSED | HAL initiates new scan | PASSIVE_SCAN | Start AF scan Lens now moving |
| PASSIVE_FOCUSED | AF_TRIGGER | FOCUSED_LOCKED | Immediate transformation if focus is good Lens now locked |
| PASSIVE_FOCUSED | AF_TRIGGER | NOT_FOCUSED_LOCKED | Immediate transformation if focus is bad Lens now locked |
| FOCUSED_LOCKED | AF_TRIGGER | FOCUSED_LOCKED | No effect |
| FOCUSED_LOCKED | AF_CANCEL | INACTIVE | Restart AF scan |
| NOT_FOCUSED_LOCKED | AF_TRIGGER | NOT_FOCUSED_LOCKED | No effect |
| NOT_FOCUSED_LOCKED | AF_CANCEL | INACTIVE | Restart AF scan |
| mode = AF_MODE_CONTINUOUS_PICTURE | |||
|---|---|---|---|
| State | Transformation cause | New state | Notes |
| INACTIVE | HAL initiates new scan | PASSIVE_SCAN | Start AF scan Lens now moving |
| INACTIVE | AF_TRIGGER | NOT_FOCUSED_LOCKED | AF state query Lens now locked |
| PASSIVE_SCAN | HAL completes current scan | PASSIVE_FOCUSED | End AF scan Lens now locked |
| PASSIVE_SCAN | AF_TRIGGER | FOCUSED_LOCKED | Eventual transformation once focus good Lens now locked |
| PASSIVE_SCAN | AF_TRIGGER | NOT_FOCUSED_LOCKED | Eventual transformation if cannot focus Lens now locked |
| PASSIVE_SCAN | AF_CANCEL | INACTIVE | Reset lens position Lens now locked |
| PASSIVE_FOCUSED | HAL initiates new scan | PASSIVE_SCAN | Start AF scan Lens now moving |
| PASSIVE_FOCUSED | AF_TRIGGER | FOCUSED_LOCKED | Immediate transformation if focus is good Lens now locked |
| PASSIVE_FOCUSED | AF_TRIGGER | NOT_FOCUSED_LOCKED | Immediate transformation if focus is bad Lens now locked |
| FOCUSED_LOCKED | AF_TRIGGER | FOCUSED_LOCKED | No effect |
| FOCUSED_LOCKED | AF_CANCEL | INACTIVE | Restart AF scan |
| NOT_FOCUSED_LOCKED | AF_TRIGGER | NOT_FOCUSED_LOCKED | No effect |
| NOT_FOCUSED_LOCKED | AF_CANCEL | INACTIVE | Restart AF scan |
AE and AWB state machines
The AE and AWB state machines are mostly identical. AE has additional FLASH_REQUIRED and PRECAPTURE states. So rows below that refer to those two states should be ignored for the AWB state machine.
| mode = AE_MODE_OFF / AWB mode not AUTO | |||
|---|---|---|---|
| State | Transformation cause | New state | Notes |
| INACTIVE | AE/AWB disabled |
| mode = AE_MODE_ON_* / AWB_MODE_AUTO | |||
|---|---|---|---|
| State | Transformation cause | New state | Notes |
| INACTIVE | HAL initiates AE/AWB scan | SEARCHING | |
| INACTIVE | AE/AWB_LOCK on | LOCKED | Values locked |
| SEARCHING | HAL finishes AE/AWB scan | CONVERGED | Good values, not changing |
| SEARCHING | HAL finishes AE scan | FLASH_REQUIRED | Converged but too dark without flash |
| SEARCHING | AE/AWB_LOCK on | LOCKED | Values locked |
| CONVERGED | HAL initiates AE/AWB scan | SEARCHING | Values locked |
| CONVERGED | AE/AWB_LOCK on | LOCKED | Values locked |
| FLASH_REQUIRED | HAL initiates AE/AWB scan | SEARCHING | Values locked |
| FLASH_REQUIRED | AE/AWB_LOCK on | LOCKED | Values locked |
| LOCKED | AE/AWB_LOCK off | SEARCHING | Values not good after unlock |
| LOCKED | AE/AWB_LOCK off | CONVERGED | Values good after unlock |
| LOCKED | AE_LOCK off | FLASH_REQUIRED | Exposure good, but too dark |
| All AE states | PRECAPTURE_START | PRECAPTURE | Start precapture sequence |
| PRECAPTURE | Sequence done, AE_LOCK off | CONVERGED | Ready for high-quality capture |
| PRECAPTURE | Sequence done, AE_LOCK on | LOCKED | Ready for high-quality capture |
Enabling manual control
Several controls are also involved in configuring the device 3A blocks to allow for direct application control.
The HAL model for 3A control is that for each request, the HAL inspects the state of the 3A control fields. If any 3A routine is enabled, then that routine overrides the control variables that relate to that routine, and these override values are then available in the result metadata for that capture. So for example, if auto-exposure is enabled in a request, the HAL should overwrite the exposure, gain, and frame duration fields (and potentially the flash fields, depending on AE mode) of the request. The list of relevant controls is:
| Control name | Unit | Notes |
|---|---|---|
| android.control.mode | enum: OFF, AUTO, USE_SCENE_MODE | High-level 3A control. When set to OFF, all 3A control by the HAL is disabled. The application must set the fields for capture parameters itself. When set to AUTO, the individual algorithm controls in android.control.* are in effect, such as android.control.afMode. When set to USE_SCENE_MODE, the individual controls in android.control.* are mostly disabled, and the HAL implements one of the scene mode settings (such as ACTION, SUNSET, or PARTY) as it wishes. |
| android.control.afMode | enum OFF | means manual control of lens focusing through android.lens.focusDistance. |
| android.control.aeMode | enum OFF | means manual control of exposure/gain/frame duration through android.sensor.exposureTime / .sensitivity / .frameDuration |
| android.control.awbMode | enum OFF | means manual control of white balance. |