VisionCamera V5 rewritten to Nitro with new Constraints API
Marc Rousavy has published an overview of VisionCamera V5, which the post describes as a new foundation for the React Native camera library.
Nitro rewrite
V5 has been fully rewritten to Nitro Modules, replacing most of the hand-written JSI/C++ layer. The Frame Processor implementation is now almost entirely plain Swift and Kotlin, cutting about 3,000 LOC. A CameraDevice is now a HybridObject whose properties are read lazily through native calls, which the post says improves startup performance and reduces camera latency.
Constraints API
The Formats API has been removed. Developers now express intent through constraints={...}, which the Camera negotiates into a supported configuration and enables. Constraints are prioritized by array order. On iOS this filters AVCaptureDevice.Format, and on Android it probes CameraInfo.isSessionConfigSupported(...). The resolved configuration is reported via onSessionConfigSelected={...}, and resolveConstraints(...) resolves constraints without starting a Camera. Because outputs are now taken into account, higher resolutions such as 8K Photos on iOS are possible.
Outputs and photos
Outputs are now separate HybridObjects (for example usePhotoOutput() and useVideoOutput()) passed through outputs={[photoOutput, videoOutput]}, with methods like capturePhoto(...) living on the output. capturePhoto(...) returns an in-memory Photo that can be displayed with react-native-nitro-image without touching the file system.
Camera controls
- Fully manual exposure, focus and white balance, e.g.
setFocusLocked(...) - A full DynamicRange API covering SDR, HDR and Log, including
HLG_BT2020andappleLog focusTo(...)options:modes,adaptiveness,autoResetAfterandresponsiveness; focus also works in<SkiaCamera />- Conversions between Preview-, Frame- and Camera- coordinates
Detection, depth and RAW
The Barcode Scanner now uses MLKit on both iOS and Android and ships separately as react-native-vision-camera-barcode-scanner. Core adds an iOS-only CameraObjectOutput for QR codes, faces, bodies and more. V5 also adds Depth Frame Processors and RAW capture in Adobe DNG or Apple ProRAW formats.
Imperative API and Multi-Cam
An imperative API, used under the hood by <Camera />, is exposed for advanced use. Multi-Cam sessions streaming from two or more devices are available via VisionCamera.createCameraSession(true) when VisionCamera.supportsMultiCamSessions is true.
Modular packages
V5 is split into react-native-vision-camera, react-native-vision-camera-barcode-scanner, react-native-vision-camera-location, react-native-vision-camera-resizer, react-native-vision-camera-skia and react-native-vision-camera-worklets. The resizer is a GPU compute shader (Metal on iOS, Vulkan on Android) measured at about 5x faster than vision-camera-resize-plugin. The worklets package uses react-native-worklets instead of react-native-worklets-core, allowing Reanimated SharedValues to be updated from Frame Processors. Native Frame Processor Plugins are now fully typed Nitro Modules, and custom outputs can conform to the public NativeCameraOutput protocol to plug into a session without forking.
A new documentation site is available at visioncamera.margelo.com. Installation:
npm i react-native-nitro-modules react-native-nitro-image
npm i react-native-vision-camera@5
Links
Source: margelo.com