We had another couple of great external speakers talking at Digg HQ. The open-source master Doug Cutting stopped by the Digg office to talk about the growth of the Hadoop platform. He talks in detail about Avro. Avro is an RPC and serialization framework that has some interesting differences compared to the popular Thrift framework.

Digg's Andrew Bayer has just written a blog describing how we use Git, Hudson, Selenium, Puppet and Gerrit to manage continuous deployment at Digg.

Andrew describes how we get developer commits to production quickly and safely using a combination of automated packaging and staging, web based code review and automated testing (unit and selenium)

Read the full blog here.

We recently started inviting external speakers to talk at Digg HQ on a variety of Technical subjects. Jack Dorsey the founder of Twitter and Square stopped to talk about the growth of Twitter and the resulting engineering and cultural challenges. He talks about applying the lessons learned there to his latest venture, Square. Fascinating stuff.

We've recently started using Drupal 6 at Digg.com for all our content needs. So far so good. Everything from our jobs page, to our site tour, to the Open Source site we launched three days ago, is managed via Drupal. Read more about our use of Drupal on the Digg Blog.

André Maurois (1885-1967) wrote that "The effectiveness of work increases according to geometric progression if there are no interruptions." At Digg we struggle between the clear benefits of uninterrupted work and the need to be agile in our communication.

digg has been growing like crazy, creating space problems for our development department, resulting not only in excessive density, but in our team being spread across several floors. these space issues have caused significant challenges for our emerging agile development environment, stifling informal communication and making it difficult to organize teams into cohesive groups.

i'm currently working on re-engineering many of the development processes at digg. we're adopting a number of practices from the agile world that complement the type of development that we do. these practices include: build automation, automated deployment, daily scrums, short releasable time-boxed iterations, simple design, refactoring, and just-in-time specification.

we've decided to be agile about our adoption of our new agile processes, introducing them incrementally, measuring the results along the way and iterating as necessary.

as we worked through the cascading dependencies of our adoption path, it quickly became clear that automated testing had to be a cornerstone practice if we were going to make a success of the others.

i started working at digg.com a few months ago. on my first day i was handed a brand new macbook pro. a reasonable person would have been delighted, but i was filled with dread. i had started a new high-pressure job, and the only tool at my disposal was a mac. not only that, i shuddered at the very thought of becoming an iPerson.

my most recent experience with apple had been extended subjection to a dog-slow powerpc running mac os 7. after which, i have always assumed macs to be fisher-price type devices, designed for those too misguided or incompetent to operate even a second-rate device like a wintel box.