Skip to content

VisionCamera 4.0 adds Skia Frame Processors and a CameraX rewrite

updated

VisionCamera 4.0.0 was released on 2024-04-22. The release notes name three features as the most anticipated: Skia Frame Processors, a CameraX rewrite on Android, and GPS location tags for photos and videos.

Back to CameraX on Android

In the move from V2 to V3, VisionCamera switched from CameraX to the low-level Camera2 API to gain more control over the sensor and output streams. According to the author, Camera2 behaved inconsistently across devices, leading to issues such as crashes on Samsung phones, rotated photos and slow frame processors on Huawei. CameraX has since added a stream sharing API that covers all of VisionCamera's features, so the Android codebase has been fully rewritten to CameraX. Stream sharing also means photo, video, frameProcessor, preview and codeScanner can now all be enabled at the same time.

Skia Frame Processors

A new useSkiaFrameProcessor hook creates a drawable Frame Processor. The frame extends a SkCanvas, so all Skia.* APIs can be used, and the frame can be rendered with optional shaders via frame.render(shader):

const frameProcessor = useSkiaFrameProcessor((frame) => {
  'worklet'

  // draw frame itself
  frame.render()

  // draw red rectangle into center of frame
  const centerX = frame.width / 2
  const centerY = frame.height / 2
  const rect = Skia.XYWHRect(centerX, centerY, 150, 150)
  const paint = Skia.Paint()
  paint.setColor(Skia.Color('red'))
  frame.drawRect(rect, paint)
}, [])

The feature relied on a series of react-native-skia PRs made with help from @wcandillon.

Other additions

  • GPS/EXIF location tags, with a useLocationPermission hook, an enableLocation prop, and a $VCEnableLocation Podfile option to disable location APIs
  • photoQualityBalance prop (speed, balanced or quality)
  • preview prop to enable or disable the preview
  • androidPreviewViewType prop (surface-view or texture-view)
  • takeSnapshot(..) API for instant snapshots of the video/preview streams
  • Photo HDR and 10-bit video HDR on Android
  • yuv and rgb pixel formats always supported
  • FPS Graph rewritten in JS
  • Support for legacy Camera1 devices and minSdk lowered from 23 to 21
  • Split into VisionCamera/Core, VisionCamera/React and VisionCamera/FrameProcessors submodules/subspecs

Removals

The qualityPrioritizaion and enablePrecapture parameters of takePhoto(..) are removed, as are the native pixel format and enableGpuBuffers.

Fixes and performance

Bug fixes cover onStarted/onStopped firing too early, the shutter sound playing in silent mode, black or crashing cameras on some devices, camera flipping, and automatic fallback to a supported videoCodec (e.g. h265 -> h264). Concurrent configure { ... } abort calls and some JNI calls were optimized. Documentation was updated for V4, Skia Frame Processors and takeSnapshot.

Source: github.com