Making sense of Uber engineering
Table of Contents
- Uber’s Backend
- Monolithic vs. Microservices Architecture
- Web and Mobile Frontend
- Platform or The Tech Backbone
- Closing Thoughts
If I have to breakdown Uber engineering in the most simplistic way, I would probably include the following pieces:
- A user-facing mobile app on iOS and Android
- A lean informative website
- An all-powerful backend system
- A large data store/ database
- Maybe, independent engines for AI and data analytics/ processing
Talking specifically of Uber’s backend, you would probably imagine that it would have independent modules or sections that tackle say, dynamic pricing, matching drivers & riders, processing & managing user profile data among others.
Uber engineering did look like this, but 7-8 years ago. Even then, the individual apps were huge and had complex workflows.
If you try to visualize Uber engineering now, you probably have to picture a tree.
The tree trunk would be common set of technologies used across teams and each tree branch would be an individual engineering team exercising its independence in terms of tech choices, processes, etc.
The entire system is simply gigantic. To give you a quick flavour, Uber now has 4000 or more independent apps or microservices each tasked with a specific function!
For all practical purposes, a microservice is simply an independent piece of code performing a specific task.
Effectively, this means there is an independent service responsible for
- Processing and maintaining your profile data
- Maintaining types of available rides basis each location
- Mapping your location and that of nearby cabs on a custom map
- Creating a set of possible drivers for a specific ride
- Computing the price of your ride (dynamically)
And so on. You get the idea.
But why did Uber move away from an all-powerful app (monolithic) architecture to one with independent services (microservices)?
Monolithic vs. Microservices Architecture
Look, building 1000s of services may sound quite fancy but it brings in a host of challenges:
- Each service is built and managed by an independent team. It has its own codebase and documentation. In such a case, discovering the right service for usage becomes a major challenge.
- You need to closely monitor the interdependency of services. If not handled properly, failure can very easily cascade down.
- You also need to ensure there is an easy, reliable and uniform way for services to communicate with each other.
So there must be some huge positives that come along with building microservices that prompted Uber to make this move. Let’s look at some of these benefits:
- Since they build each service independently, each team gets immense flexibility in the choice of technology, architecture, tools among others while building these services.
- Testing and experimentation can be done independently without affecting, in any way, parts that are working fine. Code deployments and updates happen independently for each service.
- Plans for fixing bugs, reducing technical debt or in fact, the entire engineering roadmap can be taken independently by each team without worrying about what other teams are doing.
- In effect, it speeds up development time and gives immense flexibility to engineering teams.
All of the above parameters are super critical when you are trying to scale up your application (to the levels of Uber).
Now that we understand the broad philosophy behind Uber engineering architecture, let’s try to split the entire ecosystem into independent logical pieces.
Web and Mobile Frontend
On the web and mobile front, Uber’s priorities are user experience and accessibility.
Accessibility is often misunderstood when it comes to tech products.
At a fundamental level, accessibility promises the same experience to every user, regardless of circumstance. This means you ensure the same user experience for people with or without disabilities.
Now, you traditionally think of someone having a hearing or vision impairment as disabilities. However, having a slow internet connection also falls under the bucket of disability/ impairment.
Uber’s mobile apps are written natively, using Swift for iOS and Java for Android. Typically, native apps are built when you want to leverage phone features to the fullest and achieve maximum performance.
In fact, Uber goes one step further and implements its own architecture called Riblets, as opposed to the traditional MVC used by mobile apps. This allows Uber to provide increased availability to users, meaning any kind of experimentation happens in isolation and doesn’t affect the core flows/ core user experience.
The set of backend services, internally called Marketplace, powers most of Uber engineering. Real-time transactions, the matching system, dynamic pricing, and all other core modules sit here.
Any request made by any user goes through Marketplace and hence, it must be capable of handling the highest amount of traffic. The Marketplace needs to be immune to failure despite taking so many hits (user requests). Hence, Availability is the topmost priority for Marketplace. Uber also has to ensure speed since all transactions happen in real-time.
Uber traditionally used Python a lot and continues to use it primarily for data processing. Go and Java are chosen for high-performance reasons while Node allows to explore concurrency/ parallel operations.
Platform or The Tech Backbone
Platform comprises everything that enables Uber engineers to code and build apps. It serves as a sort of support ecosystem to the other parts.
A few critical parts are:
- Infrastructure – Uber uses a mix of cloud service providers and multiple data centers. If one of them fails, requests are routed to another. Effectively, there are active backups running at all points in time.
- Data storage – You would wonder whether Uber uses a SQL or NoSQL DB. Is it MySQL, Postgres or Mongo? Well, in order to meet its own standards of high availability and speed, Uber uses a mix of at least 3-4 different DBs for different use cases. For example, Cassandra is used for speed, Schemaless for long-term data storage and Hadoop warehouse for distributed storage.
- Communication among services – One of the biggest tasks when building 1000s of microservices is to build channels to ensure smooth communication among these services. Uber has built its own framework, Hyperbahn and in fact, open-sourced it so that the community at large could benefit. Uber is quite active in terms of open-source contributions (21 till date) and you can view all of them here.
- Logging – There is so much useful/ critical data Uber has to deal with on a daily basis that it has built independent services to ensure efficient pushing, transport and temporary storage of this data.
- Development and Deployment – Ensuring uniformity and ease of development across its engineering organization, Uber uses a set of popular tools like Phabricator
, OpenGrok and has even built its own internal deployment system.
There’s absolutely no doubt that Uber engineering is no less than a masterpiece. I have barely scratched the surface when it comes to understanding the complexities of Uber’s tech.
However, this is sufficient to prepare us for the next post where we discuss what you should (and should not) be building for your first product or feature.
I document my journey and write regularly tech & remote work on Twitter, you can follow me there.
This post is the first in the series on First Lessons in Product Building.