How Airbnb grew from a Rails monolith into services on Kubernetes
Airbnb started as a single Ruby on Rails app and, as the engineering team grew past a few hundred people, split it into Java services. Today it runs on AWS and Kubernetes, with a React and TypeScript web app, native iOS and Android apps, and data tools it built and open sourced, like Airflow and Superset.
Business intelligence app that Airbnb created and open sourced.
How they built it
Monorail: when the monolith became the bottleneck
For years Airbnb ran on Monorail, its Ruby on Rails monolith. Around 2015, with more than 200 engineers shipping about 200 commits a day, deploys were blocked for about 15 hours a day on average because of reverts and rollbacks. A single shared database meant a change in one place often broke something unrelated.
Deploy trains stretched to hours and nobody was sure who owned what. In the words of engineer Jessica Tai, Monorail deploys were a huge pain point that was dragging down developer productivity.
Airbnb then broke Monorail into services. Java and Ruby were both allowed, but many services went to Java because multithreading let them run requests in parallel, which the single-threaded Ruby monolith could not. Every service described its API in Thrift, which generated Java and Ruby clients and standard metrics and alerts.
By 2018 more than 250 services and 1,000 endpoints were on the framework. Deploys went from hours to minutes, weekly deploys rose from 3,000 to 10,000, and the search page got about 3 times faster.
Web on React and TypeScript, mobile back to native
On the web, Airbnb moved away from fully server-rendered Rails pages to React, rendered on the server by its own Node service, Hypernova. Later it made TypeScript the official language for frontend web work and wrote ts-migrate to convert millions of lines at once.
On mobile, Airbnb bet on React Native and then announced in 2018 that it would sunset it and focus on native development. The apps are now written in Swift and Kotlin, with in-house libraries like Epoxy for building screens and Lottie for animations.
AWS, Kubernetes and a GraphQL layer over everything
Airbnb runs on AWS with self-managed Kubernetes clusters, and even spreads distributed databases across three clusters, each in a different availability zone, so a bad deploy only hits one zone. Its main relational fleet is MySQL, hundreds of clusters handling millions of queries per second, recently upgraded from 5.7 to 8.0.
To tame the sprawl of services, Airbnb built Viaduct, a GraphQL layer that gives every team one place to read any data. Its huge JVM monorepo of Java, Kotlin and Scala code now builds with Bazel, after a move off Gradle that took 4.5 years.
Several tools other companies now rely on came out of Airbnb's data team. Maxime Beauchemin started Airflow there in October 2014 to schedule data pipelines, and it was open sourced in 2015. Superset, a business intelligence app, came from the same place.
More recently Airbnb released Chronon, a platform that serves data to machine learning models.
