Thursday, June 4, 2015

The story of database scaling at YouTube and Dropbox

I have recently been studying postgreSQL. Coming from the MySQL Replication background and studying postgres, it was natural to just flow into their replication module and I couldn't help compare the replication modules of two of the biggest open source databases. I have also been studying ways to scale a growing database and trying to figure which of these two databases is better at scaling- in terms of ease of use, stability and performance. I have always been interested in scale and distributed systems and I am a big time follower of the architectures of the biggest web companies.

As evident from my last few posts, I have also been reading a lot about entrepreneurship lately. Combining all of this, when I come across a resource that tells me how a YouTube or Dropbox grew in scale, what are the problems they faced while their user-base grew and how they overcame it, gives me immense joy. Recently I came across two videos that I want to share.

The first is the story of YouTube presented pretty well by one of their earliest developers. The video clearly depicts the technical fine prints, the desperation to keep their growing site live and the common problems scale tends to bring up. There are times when you think something is so well presented that you don't want to reproduce it in your own words and thence I will just leave you with this video. Have a look at the amazing presentation.

The second video that I want to share today is the story of Dropbox. The speaker begins with a simplest architecture the founders started with and goes on iteratively to show how they kept improving their back-end. Note that Dropbox is a write intensive workload and hence this is a real fun video. Have a look.

Lastly, if you have more such references, do share along.

No comments:

Post a Comment