Continuous Integration is for Losers

How to lose $113,462 a year, decrease productivity, and create a culture of waste

Everyone knows that Continuous Integration was created by Gandalf in 1926. It was thought to be the best way to protect Narnia after Voldemort lost the Ring. Charles Xavier protested, but the TSA wanted the illusion of security.

Note: I may have been off by a year or two, origins are always a bit fuzzy. Martin Fowler may tell a different story. History aside, this is what it looks like now:

Developers have set up a Continuous Integration server, aka CI, because it’s An Internet Best Practice™. Code is checked in regularly and pushed to CI.

But,

  • Time is spent managing yet another environment, often completely different from a dev box and the production servers
  • Tests are flakey, so dev’s spend hours trying to figure out clever workarounds
  • There are false negatives on feature branches because of Work In Progress
  • There are still fails in Master
  • No one runs the entire suite anymore
  • Everyone ignores the pulsating CI dashboards spread around the office
  • Other developers still spend anywhere from a few minutes to a few hours chasing down unrelated fails after merging master into their feature branch

But I thought CI was supposed to make things better? It’s benefits are touted to be:

  1. Dramatically less bugs
  2. Faster development cycles
  3. Immediate developer feedback
  4. Less integration issues
  5. Higher quality code

But aren’t those just the merits of Test Driven Development? Advocates of CI tend to lump it in with TDD. These things are orthogonal. These independent, eight-sided, practices remind me a lot of Agile and Scrum. (Damn, is anything sacred?)

You have well intentioned people piggybacking on something they don’t understand in hopes of carving out their own niche to make things better. Altruistic motivations. I applaud the effort, but not all help leads to good results.

What actually happens is three dev’s and an architect struggle with a Jenkins box at random intervals for over a year and half. Each session never amounts to that much time, an afternoon here and a night or weekend there, but the cumulative effect of all those “necessary” hours easily amounts to tens of thousands of dollars. All for what?

What are we trying to solve with CI? Faster development cycles? Higher quality code?

Ok! Now we’re getting somewhere. We want higher quality code, but:

CI does nothing for the actual quality of the code.

That is a human problem. Just because it passes the build doesn’t mean it’s good code. And, it’s here that CI can actually have a negative impact. If you ever do get bad developers to test, a passing build just adds fuel to their fire, because “The build is green. Just merge it. We can refactor it later.” This. Actually. Happens.

Seriously. It feels like you’re moving fast, but anything can be gamed. Keeping the build green doesn’t actually measure the productivity or velocity of your team. It just gives the illusion of moving fast and a warm and fuzzy false sense of security.

So, what is the problem? What’s actually causing the problem? Or, what were we trying to solve, or prevent, with CI?

YES!! Those are the exact questions you should always be asking. Always. A culture with a relentless pursuit for continual improvement. A culture that periodically questions everything. A culture that’s not afraid to strip away the bullshit and get down to the real root of the problem. That’s the culture you want to create.

That’s my goal for this article. I want you to be better. I have high hopes for you. I want things to run more smoothly in your organization. I want your developers and teammates to ship the most valuable features with the highest quality code in the shortest amount of time. I want your users to have a better experience and to be excited to use your product. I just want people to be happy.

Disclaimer: CI actually is good for testing third party API’s. And I know there’s a lot of fantastic engineers working on CI software. Please, this was not a knock to you or the value you add to the world. I just want people primed for what they’re about to get themselves into.

An oft quoted line when discussing CI with other engineers was: “Until you’ve seen it work, you won’t appreciate its value.”

Ok... but I’d make the argument that you already have a high functioning team and they never needed CI in the first place. They had the motivation and the drive to deliver value to customers, consistently over a long period of time.

“Good developers don’t need CI.”

(I can’t wait for that quote to be bastardized in the years to come.)

A tool can be a nice way to help facilitate a solution, but rarely can it be a solution itself. That’s my beef with CI. Everywhere I’ve seen it used, it’s either a prevention strategy or the solution itself.

A piece of technology is never the solution, just a piece of it. More tech doesn’t solve tech problems. It can actually just create more of them. You may have issues with junior developers being quick to add technology they don’t fully understand to a problem they don’t fully understand. This is a huge issue. “Hey we can use a javascript plugin for this. Or SQL functions for that! Look at all this cool tech!” That is the mentality that drives products into the ground.

When people finally lose the expectation that technology-related problems are solved with more technology, that’s the day we defeat Skynet, and Arnold Schwarzenegger can get back to being Governor again.

Start solving the problem, then apply technology. It’s the order in which you apply technology that matters most because:

Technology only accelerates the direction you were already moving.

The change needs to happen first, then tech can be applied. Bandages aren’t first applied to gunshot wounds, but they are used after the surgery’s over. Yes, bandages can be used to stop the bleeding, but at some point you’re still going to need some medical attention, aka a worthwhile product culture.

Agree? Disagree? Good! I just want people thinking about the actual problem they’re trying solve.

So, what is the culture you want to create? I’d love to hear about it.

---

And, if CI is running wonderfully at your company. Please! Send me an email. What did it take to get you there? In retrospect, was it truly worth it? Were there other ways to solve the same problem? I look forward to hearing from you.

Further Reading >>