Version almost final

lecture: Contain yourselves

The Way We Build Systems is (Still) Wrong

Event_large

In this talk, I'll offer my contrarian view on the latest hyped containerization technologies, why I think the new approach is flawed and how we can usefully blend the old and new prevailing wisdom to build systems that actually scale.

Building systems is hard and tedious work.

Building scalable systems is much harder.

Don't despair, though, we hear; there are tons of amazing services
nowadays which will help you and make building scalable systems easy,
quick, error-free and totally DevOps-compliant, as well.

Docker and other containerization tools are the latest hot, new
technology. Everyone is talking about them: the Twitterati, major
analyst firms and, as HackerNews will tell you, If it's good enough
for Google, it's good enough for you. We seem to have quickly reached
universal agreement: containers are the future, everyone needs
Microservices, and if you're not Cloud-Native your business or project
is dead already. These technologies are supposed to solve all the
issues system engineers face when building scalable systems.

Unfortunately, as so often, the reality is far different than the hype.

Having been a member of the system engineering team at Booking.com, a
small Amsterdam hotel reservations startup turned world’s largest
hotel booking site, I’ve had the opportunity to experience massive
growth of our systems infrastructure. We went from less than tens of
MySQL servers to several thousand database servers, achieved 60% year
over year growth in site visitors, infrastructure needs *and* brand
new employees I've had the opportunity to see what works inside a
company that builds at scale, and a lot of what we're hearing will be
the next big thing just isn't going to work.

Automation, configuration management, A/B testing, quick deployments,
failing fast, and no-blame reviews work well and allow for a
high-growth environment. For these techniques to work however, we need
strong, cross-functional teams. We need systems engineers, developers
and product owners working closely together defining requirements and
delivering results. Many of these lessons have been codified by the
DevOps movement and have been applied by other industry players as
well.

The problem with Docker and many other container solutions, however,
is that they actually repudiate these lessons and the marketing is
leading people down the wrong path.

Instead of cross-functional teams working together on delivering
working software, we’re seeing the return of the old and dreaded “It
works on my machine, ship it!” attitude: It somehow worked on the
developer’s machine, we packed it up into a Docker image and now it’s
ready in production.

This talk tries to clarify the different requirements of development
vs. production operations. Development is completely results driven.
It doesn’t matter how we got there, as long as we reached the
destination. Operations is more focused on the process of getting to
the destination. It needs to be repeatable and well understood.

Combining the two is the challenge all successfully growing companies
have to face.

Container technologies and, specifically, their current messaging
about use cases falls short. Right now, the hype trifecta is not yet
ready for helping companies scale their systems and maintain them.
It's actually leading you to waste time building infrastructure you do
not need.

In this talk, I'll offer my contrarian view on the latest hyped
technologies, why I think the new approach is flawed and how we can
usefully blend the old and new prevailing wisdom to build systems that
actually scale.