Skip to content

React Native 0.71-RC0 Android outage postmortem published

updated

With React Native 0.71 now available, the React Native team has published a postmortem of the incident that broke Android builds for all React Native versions when the first 0.71 release candidate was released on November 4th, 2022. The contributors who handled the incident met to go over what happened, what was learned, and what actions will be taken to avoid similar outages.

What happened

Version 0.71.0-rc0 introduced a change to improve build times by publishing artifacts to Maven Central instead of building them from source (described in RFC#508). Older project templates declared the Android dependency as:

implementation("com.facebook.react:react-native:+")

The + is a Gradle Dynamic version, which tells Gradle to pick the highest available version. The team had removed all + dependencies from the 0.71 template, but projects on older versions still used them. Once 0.71.0-rc.0 appeared on Maven Central as the highest version, builds on earlier versions (for example 0.68.0) began pulling 0.71.0-rc.0 artifacts, and the version mismatch caused them to fail.

Mitigation and resolution

  • The community found and shared a manual workaround on November 4th that pinned React Native to a specific version.
  • Over November 5th and 6th, the release crew shipped patch releases for previous versions down to 0.63 that applied the fix automatically.
  • The team asked Sonatype to remove the offending artifacts; the issue was fully resolved on November 8th when they were removed from Maven Central.

Key timeline points (GMT/UTC +0): 0.71-RC0 was released at 5:06 PM on Nov 4th, the first build issue report came at 6:20 PM, and the community identified the issue at 7:45 PM. Workarounds were communicated and Expo deployed a fix to its users at 9:39 PM. The last patched release, 0.63.5, shipped at 12:47 AM on Nov 7th, and the incident issue was closed on Nov 10th.

Lessons learned

  • Incident response: the community found a workaround in under 2 hours, but it took 48 hours to recognize the larger scope. The team will revisit when to rely on developer-applied workarounds versus automatically deployed fixes, and look at options for better visibility into ecosystem health.
  • Release support policy: covering more than 90% of developers required patching down to 0.63, which the team attributes to a historically difficult upgrade experience it is looking to improve. The team plans to communicate its support policy and will consider deprecating older versions of React Native on npm.
  • Testing and 3rd party libraries: the team plans to invest further in release and testing infrastructure, is encouraging and supporting 3rd party library testing as part of releases, and is adding new channels and roles in the Core Contributors Discord Server. It has also started working more closely with Callstack on create-react-native-library, whose newer version is fully compatible with 0.71 projects while remaining backward compatible.

The team apologized for the disruption and thanked Sonatype, the community, and the release crew. In the post's words: "Releasing a newer version of React Native should never have an impact on users on older versions."

Source: reactnative.dev