Reddit's Scalability

An interesting scalability presentation from one of the founders of reddit [a text based summary of the presentation]. One of the interesting parts is that the restrictions of a relational database impeded the user experience and the speed or ability to make changes or updates to the website.

They keep a Thing Table and a Data Table. Everything in Reddit is a Thing: users, links, comments, subreddits, awards, etc. Things keep common attribute like up/down votes, a type, and creation date.

The Data table has three columns: thing id, key, value. There's a row for every attribute. There's a row for title, url, author, spam votes, etc. When they add new features they didn't have to worry about the database anymore. They didn't have to add new tables for new things or worry about upgrades. Easier for development, deployment, maintenance.

The price is you can't use cool relational features. There are no joins in the database and you must manually enforce consistency. No joins means it's really easy to distribute data to different machines. You don't have to worry about foreign keys are doing joins or how to split the data up. Worked out really well. Worries of using a relational database are a thing of the past.

I haven't worked on a site that has to scale that level. At my previous employer it was not unusual for the javascript components I wrote to have to deal with two million ajax calls a day which was humbling to see. Get that wrong and it is an immediate bad user experience.

With my current employer we have the challenges of SOA and scaling across disparate applications that have their own needs for data consistency. Doing that in a transparent and consistent manner is the difficult part.
Permalink, Reddit's Scalability, May 2010, cam

More Reading on Scalability

Most Popular on South Sea Republic

The articles that have been viewed the most:

Most Popular Restaurants in Phoenix

Phoenix Eats Out is the restaurant review site for Phoenix, Scottsdale and Old Town Scottsdale which lists the modernist and contemporary restaurants, taverns and bars in the greater Phoenix area. This is the list of the most popular restaurants pages from phoenixeatsout.com that have been viewed the most; My personal favourite restaurants in Phoenix are AZ88, Postinos, Bomberos with Grazie, Humble Pie, Orange Table, The Vig, Fez and others coming close behind. View the complete list with the photo-journalistic style images on phoenixeatsout.com

Most Popular Hikes in Arizona

Arizona is an outdoor state and has lots of hiking in the city and around the state. Phoenix is unusual for most cities in having several large mountains in the center of the city with great hiking. Anyone who comes to Phoenix has to do the Echo Canyon trail on Camelback and the Summit Hike on Squaw Peak or Piesta Peak. The views of the city, suburbs and surrounding mountains are wonderful from Camelback and Piesta Peak. For more experienced hikers there is the McDowell Mountains in North Scottsdale that has several difficult and strenuous hikes in Tom's Thumb and Bell Pass. Alternatively, you can hike the highest mountain in Arizona. At 12,600 feet Humphrey's Peak is a long and difficult hike.

Alternate Australian Constitutions

Between 2004 and 2009 this site, southsearepublic.org, was a constitutional blog based on scoop which focused on Australian and global constitutional issues. One of the strongest aspects of it was the development of constitutions by those involved in the blog. These constitutions are the outcome: The constitutions were built using principles from Montesquieu's separation of powers, the enlightnment's universal political rights and the ancient Athenian technology of sortition and choice by lot.

Archives For South Sea Republic

South Sea Republic started in 2004 as an Australian constitutional blog in 2004 based on scoop software. It was an immigrative outgrowth of Kuro5hin. The archives for each year since then; The articles are ordered by views.

Who Is Cam Riley

Cam Riley I am an Australian living in the United States as a permanent resident. I am a software developer by trade and mostly work in Java and jump between middleware and front end. I originally worked in the New York area of the United States in telecommunications before moving to Washington DC and working in a mix of telecommunications, energy and ITS. I started my own software company before heading out to Arizona and working with Shutterfly. Since then I have joined a startup in the Phoenix area and am thoroughly enjoying myself.

I do a lot of photography which I post on this website, but also on flickr. I have a photo-journalistic website which lists the modernist and contemporary restaurants in phoenix. I have a site on the Australian Flying Corps [AFC] which has been around since the 1990s and which I unfortunately lost the .org URL to during a life event; however, it is under the www.australianflyingcorps.com URL now. The AFC website has gone through several iterations since the 90s and the two most recent are Australian Flying Corps Archives(2004-2002) and Australian Flying Corps Archives(2002-1999) which are good places to start.

Websites Worth Reading

Websites of friends, colleagues and of interest;