How Facebook scales to a billion users in 24 hrs..(1/2)

(Image Credit – rappler.com)

The first in this two post series examines Facebook’s corporate philosophy in architecting systems for web scale. Part two will look at their technology stack in more depth. 

Last week, buried in the news of all the tumult in the stock markets, we had what I believe, a major milestone for web scale architectures and practices. A billion people logged into Facebook on a single day (Monday – Aug 26, 2015), marking the maximum number of users (one-seventh of the planet’s population) on any platform ever in a 24 hr period. An incredible number any way you want to slice it and dice it.

Sociologists will no doubt examine the tremendous impact the defining Social Network is having on humans & it’s becoming an absolute necessity in the daily lives of billions . As technologists however, we are left the more mundane job of understanding how any one platform can scale to support an astounding number of users & do so with panache.

The idea ultimately is to understanding how one can bring such innovation (albeit incrementally) into the more run of the mill enterprise.

Lets parse this in context of the bigger picture –

Firstly, Facebook (among an elite grouping of other luminaries – Google, Amazon et al) has figured out how to build platforms (that host an ecosystem of services – social in FB’s case) that provide massively scalable applications while providing an engaging user experience. It is no surprise that the average Facebook user spends around 40+ minutes in the USA and 20+ minutes outside the USA (source – Wikipedia).

(Source – Facebook)

Secondly, Facebook’s achievements are all the more amazing since they did not have the early mover advantage that MySpace did. Their recent earnings report for Q2 2015 shows continued strength across a raft of metrics their business is gauged across – increase in users, revenue per user, advertising revenue as well as revenue from mobile users.

So, what makes Facebook different from an overall strategy & DNA ? In my opinion, four important factors –

1. Like the other web scale giants, Facebook (FB) is a realtime data driven enterprise. Right from the projects they leverage to the algorithms they create that drive interaction on their site –  It all comes down to data and it’s ingestion, processing, logging, analytics & insights driven from it. Facebook’s IT Architecture and Technology stack is built around managing an entire data lifecycle as we will see in the next post.  They also leverage Data Science in a big way. FB’s Data Science team regularly posts on all their latest research & insights. Areas they spend a lot of time include Identity Research (creating and testing models of people’s online identity to power next-generation products and to gain deeper insights into how people interact with digital technology), Economic Research (pricing,forecasting etc), In house analytics, Product Science & Statistical Research. An outstanding approach to digitizing FB’s offerings in a way that supports superior customer interactions and constantly creates new value.The Data Science team even maintain a Facebook page where they publish their recent research and insights at –  https://www.facebook.com/data
2. Facebook is one of the world’s largest open source companies. Everything they do from their web application development to back end systems uses Open Source. Further they have created projects like Hive, Cassandra, Scribe and Thrift to name a few popular ones. They also contribute millions of lines of code to other open source ecosystems (e.g Apache Hadoop). Thus FB adopts a model of constant incubation & invention of open technologies.
3. In fact, FB takes their open source philosophy to somewhat extreme lengths – to creating an open source hardware platform. They incubated the Open Compute Project which is a highly credible attempt to create a lower cost, highly efficient and highly scalable datacenter. OpenCompute aims to create an open source standard that spans server designs,rack specifications, power systems, high end networking equipment like switches, storage & cooling equipment. Open Compute is rapidly evolving into a formidable ecosystem with participation from Apple, Microsoft, Cisco, Goldman Sachs, Fidelity and Bank of America. In fact, BoFA has been vocal about their vision of running commodity white box servers in their datacenters and dynamically reconfiguring these via an OpenStack controller to perform compute, storage and networking functionalities as dictated by business workloads. The charter for the Open Compute project lays out very ambitious goals and over time, the industry will move to a model where a range of bespoke datacenter equipment can be assembled, at much lower cost, using a LEGO style approach as long as the components themselves as OCF (Open Compute Foundation) spec compliant.
   
   
4. Building for the future by inculcating innovation into the organizational DNA. This is done by generating new ideas, being unafraid to cannibalize older (and even profitable) ideas and constant experimenting across new businesses. FB is famous for not having a review board that designers and engineers go present to with PowerPoint slides – prototypes and pilot projects are directly presented to executives – even CEO Mark Zuckerberg. Facebook pivoted in a couple of years from weak mobile offerings to becoming the #1 mobile app company (more users access their FB pages using mobile devices running iOS & Android compared to using laptops).  FB’s culture is almost Steve Jobs-ian in this respect.

A good chunk of FB’s success is owed to a contrarian mindset in terms of creating a technology platform & generating a huge competitive advantage from it via continuous improvement.

The next post will be a technical one in nature; we will examine Facebook’s generic technology stack (from publicly available information) and take a brief look at the individual projects that are brought together to support such massive horizontal scalability.