Skip to content

VisionCamera V5 rewritten to Nitro with new Constraints API

updated

Marc Rousavy has published an overview of VisionCamera V5, which the post describes as a new foundation for the React Native camera library.

Nitro rewrite

V5 has been fully rewritten to Nitro Modules, replacing most of the hand-written JSI/C++ layer. The Frame Processor implementation is now almost entirely plain Swift and Kotlin, cutting about 3,000 LOC. A CameraDevice is now a HybridObject whose properties are read lazily through native calls, which the post says improves startup performance and reduces camera latency.

Constraints API

The Formats API has been removed. Developers now express intent through constraints={...}, which the Camera negotiates into a supported configuration and enables. Constraints are prioritized by array order. On iOS this filters AVCaptureDevice.Format, and on Android it probes CameraInfo.isSessionConfigSupported(...). The resolved configuration is reported via onSessionConfigSelected={...}, and resolveConstraints(...) resolves constraints without starting a Camera. Because outputs are now taken into account, higher resolutions such as 8K Photos on iOS are possible.

Outputs and photos

Outputs are now separate HybridObjects (for example usePhotoOutput() and useVideoOutput()) passed through outputs={[photoOutput, videoOutput]}, with methods like capturePhoto(...) living on the output. capturePhoto(...) returns an in-memory Photo that can be displayed with react-native-nitro-image without touching the file system.

Camera controls

  • Fully manual exposure, focus and white balance, e.g. setFocusLocked(...)
  • A full DynamicRange API covering SDR, HDR and Log, including HLG_BT2020 and appleLog
  • focusTo(...) options: modes, adaptiveness, autoResetAfter and responsiveness; focus also works in <SkiaCamera />
  • Conversions between Preview-, Frame- and Camera- coordinates

Detection, depth and RAW

The Barcode Scanner now uses MLKit on both iOS and Android and ships separately as react-native-vision-camera-barcode-scanner. Core adds an iOS-only CameraObjectOutput for QR codes, faces, bodies and more. V5 also adds Depth Frame Processors and RAW capture in Adobe DNG or Apple ProRAW formats.

Imperative API and Multi-Cam

An imperative API, used under the hood by <Camera />, is exposed for advanced use. Multi-Cam sessions streaming from two or more devices are available via VisionCamera.createCameraSession(true) when VisionCamera.supportsMultiCamSessions is true.

Modular packages

V5 is split into react-native-vision-camera, react-native-vision-camera-barcode-scanner, react-native-vision-camera-location, react-native-vision-camera-resizer, react-native-vision-camera-skia and react-native-vision-camera-worklets. The resizer is a GPU compute shader (Metal on iOS, Vulkan on Android) measured at about 5x faster than vision-camera-resize-plugin. The worklets package uses react-native-worklets instead of react-native-worklets-core, allowing Reanimated SharedValues to be updated from Frame Processors. Native Frame Processor Plugins are now fully typed Nitro Modules, and custom outputs can conform to the public NativeCameraOutput protocol to plug into a session without forking.

A new documentation site is available at visioncamera.margelo.com. Installation:

npm i react-native-nitro-modules react-native-nitro-image
npm i react-native-vision-camera@5

Source: margelo.com