Etsy's Continuous Deployment Process

One of the reasons why Engineering is so important to a modern technical operation is that despite all the departments and the specialties it is engineering that has the skills to automate everything in such a way it is reproducible and consistent.

QA for instance is often great at poking around apps and running through steps, but it is JUnit and Front End functional test frameworks like Selenium that are eroding that to a smaller role. We are seeing similar things in the other areas of operations.

Programming is becoming a more and more important skill, not a lesser one. Programming is hard, scripting is also hard, and putting it all together into a package where continuous deployment is possible is harder still. I love this suggestion of Kent Beck's for getting there.

We are not there despite Engineering providing more and more scripts and automated processes. So I am jealous of reading this on how Etsy performs their continuous deployment process;

what's the smallest number of steps, with the smallest number of people and the smallest amount of ceremony required to get new code running on your servers? This isn't a trivial question. Even if you're on a slow release cycle, and have a push engineer, what happens if there's an emergency push needed? Does it go through your normal process, or is there a fast-lane? Do your fast-lane deployments get logged? Are they measured for speed? Is everyone aware that it happened the way they would for a normal deployment, or is it your dirty little secret?

It's not hard to get started. If you currently have a bunch of shell scripts that move everything in place, wrap those up with a single shell script. The most important thing is that it's ONE easy step. This might require changing your process. Try to remove or replace special cases. The less thought it takes to deploy, the more you can focus on getting stuff done.

Sadly we are still at the point where deployment is a big deal. We have solved at the dev integration level, that is all scripted, and we are moving that to the QA level, we still have to get it through stage and production. Baby steps, but we will get there. Etsy spun their own and used as their requirements;

Web based
Logged (When, What and Who)
Adaptable to our network
Run from a central location
Announced in our IRC and email
Transparent in regards to its actions
Integrated with our graphing/monitoring tools

That is pretty achievable and quickly. Since we use Bamboo for our continuous integration and deployments, it is probably easy to combine that into one interface and skip bamboo for the deployment aspect while still using ant for the deployment step.
Permalink, Etsy's Continuous Deployment Process, May 2010, cam

More Reading on Software engineering

Most Popular on South Sea Republic

The articles that have been viewed the most:

Most Popular Restaurants in Phoenix

Phoenix Eats Out is the restaurant review site for Phoenix, Scottsdale and Old Town Scottsdale which lists the modernist and contemporary restaurants, taverns and bars in the greater Phoenix area. This is the list of the most popular restaurants pages from phoenixeatsout.com that have been viewed the most; My personal favourite restaurants in Phoenix are AZ88, Postinos, Bomberos with Grazie, Humble Pie, Orange Table, The Vig, Fez and others coming close behind. View the complete list with the photo-journalistic style images on phoenixeatsout.com

Most Popular Hikes in Arizona

Arizona is an outdoor state and has lots of hiking in the city and around the state. Phoenix is unusual for most cities in having several large mountains in the center of the city with great hiking. Anyone who comes to Phoenix has to do the Echo Canyon trail on Camelback and the Summit Hike on Squaw Peak or Piesta Peak. The views of the city, suburbs and surrounding mountains are wonderful from Camelback and Piesta Peak. For more experienced hikers there is the McDowell Mountains in North Scottsdale that has several difficult and strenuous hikes in Tom's Thumb and Bell Pass. Alternatively, you can hike the highest mountain in Arizona. At 12,600 feet Humphrey's Peak is a long and difficult hike.

Alternate Australian Constitutions

Between 2004 and 2009 this site, southsearepublic.org, was a constitutional blog based on scoop which focused on Australian and global constitutional issues. One of the strongest aspects of it was the development of constitutions by those involved in the blog. These constitutions are the outcome: The constitutions were built using principles from Montesquieu's separation of powers, the enlightnment's universal political rights and the ancient Athenian technology of sortition and choice by lot.

Archives For South Sea Republic

South Sea Republic started in 2004 as an Australian constitutional blog in 2004 based on scoop software. It was an immigrative outgrowth of Kuro5hin. The archives for each year since then; The articles are ordered by views.

Who Is Cam Riley

Cam Riley I am an Australian living in the United States as a permanent resident. I am a software developer by trade and mostly work in Java and jump between middleware and front end. I originally worked in the New York area of the United States in telecommunications before moving to Washington DC and working in a mix of telecommunications, energy and ITS. I started my own software company before heading out to Arizona and working with Shutterfly. Since then I have joined a startup in the Phoenix area and am thoroughly enjoying myself.

I do a lot of photography which I post on this website, but also on flickr. I have a photo-journalistic website which lists the modernist and contemporary restaurants in phoenix. I have a site on the Australian Flying Corps [AFC] which has been around since the 1990s and which I unfortunately lost the .org URL to during a life event; however, it is under the www.australianflyingcorps.com URL now. The AFC website has gone through several iterations since the 90s and the two most recent are Australian Flying Corps Archives(2004-2002) and Australian Flying Corps Archives(2002-1999) which are good places to start.

Websites Worth Reading

Websites of friends, colleagues and of interest;