VisionCamera 4.0 adds Skia Frame Processors and a CameraX rewrite
VisionCamera 4.0.0 was released on 2024-04-22. The release notes name three features as the most anticipated: Skia Frame Processors, a CameraX rewrite on Android, and GPS location tags for photos and videos.
Back to CameraX on Android
In the move from V2 to V3, VisionCamera switched from CameraX to the low-level Camera2 API to gain more control over the sensor and output streams. According to the author, Camera2 behaved inconsistently across devices, leading to issues such as crashes on Samsung phones, rotated photos and slow frame processors on Huawei. CameraX has since added a stream sharing API that covers all of VisionCamera's features, so the Android codebase has been fully rewritten to CameraX. Stream sharing also means photo, video, frameProcessor, preview and codeScanner can now all be enabled at the same time.
Skia Frame Processors
A new useSkiaFrameProcessor hook creates a drawable Frame Processor. The frame extends a SkCanvas, so all Skia.* APIs can be used, and the frame can be rendered with optional shaders via frame.render(shader):
const frameProcessor = useSkiaFrameProcessor((frame) => {
'worklet'
// draw frame itself
frame.render()
// draw red rectangle into center of frame
const centerX = frame.width / 2
const centerY = frame.height / 2
const rect = Skia.XYWHRect(centerX, centerY, 150, 150)
const paint = Skia.Paint()
paint.setColor(Skia.Color('red'))
frame.drawRect(rect, paint)
}, [])
The feature relied on a series of react-native-skia PRs made with help from @wcandillon.
Other additions
- GPS/EXIF location tags, with a
useLocationPermissionhook, anenableLocationprop, and a$VCEnableLocationPodfileoption to disable location APIs photoQualityBalanceprop (speed,balancedorquality)previewprop to enable or disable the previewandroidPreviewViewTypeprop (surface-viewortexture-view)takeSnapshot(..)API for instant snapshots of the video/preview streams- Photo HDR and 10-bit video HDR on Android
yuvandrgbpixel formats always supported- FPS Graph rewritten in JS
- Support for legacy Camera1 devices and minSdk lowered from 23 to 21
- Split into
VisionCamera/Core,VisionCamera/ReactandVisionCamera/FrameProcessorssubmodules/subspecs
Removals
The qualityPrioritizaion and enablePrecapture parameters of takePhoto(..) are removed, as are the native pixel format and enableGpuBuffers.
Fixes and performance
Bug fixes cover onStarted/onStopped firing too early, the shutter sound playing in silent mode, black or crashing cameras on some devices, camera flipping, and automatic fallback to a supported videoCodec (e.g. h265 -> h264). Concurrent configure { ... } abort calls and some JNI calls were optimized. Documentation was updated for V4, Skia Frame Processors and takeSnapshot.
Links
Source: github.com