Chris James - Software Engineer and other things

How to go fast

02 March 2019

This article is for anyone who works in software and wants to keep life simple, less stressful but still keep their employer happy. You'll see how by making simple choices and sticking to well-understood, proven principles you can ship software and not burn everyone out along the way.

The business have arbitrarily given your team a year to make a website thing and they've got many grandiose ideas. Don't be tempted to spend lots of time planning up front imagining big architectures and backlogs around all this ambition. This is the path to failure, premature optimisation and broken promises.

The only promise you'll make is you will showcase what you've done every 2 weeks and accept steering from the business on actual problems and not feature decisions.

We know going fast is important but so is the quality of the system we're making and the mental health of our team. We will go really fast but we will always be doing it in a sustainable way.

We don't want to be stressed out about big-bang releases, having to do big scary refactors or obsessing over irrelevant estimates for work that we may not ever actually do.

Iterate

Start with hello world, fully deployed with continuous delivery.

We always want a small and focused vision for the team to be working on. Developing small things is easy and likely to succeed.

Thinking about delivering tons of features causes teams to over-complicate and makes decision making difficult.

With lots of features up in the air there's the temptation to expand the team and this also slows things down. See: The Mythical Man-Month

Instead we work on the next most important thing, based on business expertise, research, developer know-how and feedback from our users.

We will carry on continuously delivering small features iteratively indefinitely.

Rules and principles

1 Small co-located team

Small teams deliver quicker than big teams because the lines of communication are small and the amount of work being done at once has to be small. Large teams working on lots of things breeds confusion and complexity.

Having multiple small teams working on clearly separated things can work but requires good coordination. It's a lot easier to define those teams on architectural lines once you have a mature product. I think it's a mistake to take a new project with lots of unknowns and try and divide it amongst many teams because the architecture needs to be very flexible at this point and therefore the teams wont be able to work independently, they will be coupled.

In my experience being able to very easily talk to someone is important. The informal chats you have by just turning to your colleague is invaluable in breaking down the barriers of sharing ideas and building relationships. To me this is why I prefer to work in a co-located team because it minimises friction in communication.

If it is possible make sure the decision makers, such as the product owner be sitting with the team too. I wrote how work would be showcased every 2 weeks as a ritual but what's even better is quickly trying out an idea and getting immediate feedback from the stakeholders. This levels up the teams collective knowledge of the domain and massively reduces waste.

Pair programming most of the time

Pairing helps you to write better software and helps cross-pollinate skills, ideas and domain knowledge. It will help grow your team and will improve relationships which will in general make everyone happier!

Don't mandate it from 9-5 though as it becomes very tiring very quickly. People need time and space to think.

No branches and no pull requests

You have a small team that are pairing. You should be talking to each other all the time so you don't need pull requests to check code is OK. You're better off shipping it and getting feedback. You should be able to trust two people to get things right most of the time.

Pull requests are great for open source projects where you don't have implicit trust in every contributor. In a small team you don't need it and you need to take advantage of very tight feedback loops by reducing ceremony in writing software.

A big theme of these rules is removing barriers to feedback loops and this means we can change software easily. Someone shipping code that isn't ideal is not the end of the world, just fix it and move on - don't introduce complicated processes that slows everything down

Have a minimal deployment pipeline

Maintaining pipelines and various non-live environments can become burdensome. At least at the start of the project all you really need is:

  1. Run tests
  2. Deploy to live
  3. Run smoke tests

If you find yourself trying to justify a non-live environment ask yourself

Most of the time you probably wont have good answer to these questions.

Continuously deliver to live on green builds.

This gives you:

This has been written about a lot.

Accelerate book cover

And is proven to be how high-performing teams work

Manually shipping a week or two weeks worth of work takes time and is risky; releasing small things constantly is easy and less risky.

Write tests

They enable you to iterate quickly with confidence. Cant do continuous delivery without them and you cant go fast if you have to manually check things all the time.

Monolith, not microservices

Writing distributed systems is a pain in the arse and I'd rather avoid that pain until I actually need to distribute my system. YAGNI in other words.

Refactoring in-process method calls is far easier than changing API calls. Deploying one application is much easier to deploy than orchestrating many system deployments etc etc.

I've worked on microservice projects in the past but my last project was a monolith and we moved so much faster because we cut out a huge number of issues to contend with.

In order to do microservices you have to think how you will break your system up into small components which leads nicely to...

Don't design up front

Just ship some code, if you make something useful and keep refactoring and iterating a design will emerge from reality rather than developers arguing over a whiteboard.

Refactor continuously

Bad code makes you go slow. Bad code tends to get worse. So refactor continuously, don't have a refactor sprint or something dumb. Don't ask for permission, if it's bad just fix it.

Half the perceived problems of monoliths come from people writing bad code. If you maintain the quality of the code real abstractions based on real usage will emerge and then if you need to break up your monolith there will be obvious ways to do.

Avoid writing an SPA, embrace progressive enhancement

SPAs bring a mountain of complexity that most projects don't need. You're probably not building Gmail so stop pretending you need to write an SPA. You can get React on your CV at 10% time.

Instead prefer to use progressive enhancement techniques by writing semantic HTML, decorated with CSS and a sprinkle of JavaScript to enhance the experience a little when necessary. Even better, don't write any client-side JavaScript if you can.

Don't obsess over pixel-perfect design

Writing HTML and CSS can be challenging and very time-consuming if you have a designer being overly precious about their design looking pixel perfect in every browser.

We are writing code for users not designers and most of the time they wont care if it looks a bit different in Firefox compared to Chrome.

Save time, effort and stress by challenging what is good enough for your users.

Simple designs are simple to implement, elaborate designs are complicated to implement. If your designer is giving you difficult to implement web pages, remind them we're trying to go fast.

Ship the first 80%, polish it later

Related to the above it is a total waste of time to really perfect a certain feature in respect to design and UX if it turns out that it is a feature that isn't needed.

I have seen a team spend weeks polishing a search feature on a website in respect to its design and its JavaScript only for it to be thrown away a few months later because the fundamental idea was wrong in the first place.

Ship enough to get feedback and prove the idea, once you've done that you can then spend time polishing it.

The people who write the software are the ones that are on support for it

This responsibility must be coupled with the empowerment to make the system maintainable.

If the people creating the system are the ones who have the threat of getting called at 3am because it is down they are more likely to ship more robust code.

Use mature, proven tools

It doesn't matter if you go fast with shiny thing if you then leave and no one can build or understand your code anymore.

Get a consensus in your team of what you'd all enjoy writing with that is appropriate for the task at hand. Don't pick up a new programming language that is unfamiliar to everyone - remember we're trying to go fast.

It must be simple to build the software

There should be a minimal amount of setup required to run and test the project. It is totally unacceptable to be unable to build a project just because it hasn't been touched for a few months or a key person has left. If it takes an individual more than an hour to setup the project you have failed hard.

With technologies like Docker it is trivial to configure a project so it has all it's 3rd party dependencies such as databases available locally.

It should not be reliant on a particular setup of a computer (beyond installing a few key things such as the JVM, Go or whatever). It must not be reliant on a 3rd party system for it to run & test locally.

To help enforce this, configure whatever CI tool you are using to automatically build the software every morning; even if no one has committed to it.

If the build is red, stop

The tests must always be passing. If you have unreliable tests, stop and fix them. Flaky tests have cost me many hours of my time in the past.

Actually sit with users and watch them work

What the business thinks the users needs vs what the user actually needs is often very different. I have been lucky enough to witness this first hand where we saved ourselves a ton of work by actually watching a user at work and we were able to identify a very cheap and simple solution that they actually want vs the extremely costly solution the business proposed.

User stories are a conversation starter

If a user story prescribes exactly what software to write, who made that decision? Why? How did they know they were correct?

This kind of approach of a "tech lead" or whatever deciding how to write the software creates disillusioned developer robots who have no sense of ownership and stories that are "blocked" because they have not been blessed by some over-controlling leader.

User stories should describe a user problem and when it is picked up the team figures out a way to do it and then they execute that plan, simple as that. This has a higher chance of success because more people are thinking about the problem and makes people feel involved.

Each ticket have some kind of measure of success that should be validated when enough data/research is gathered.

Understand that we are all software developers

In my imaginary company if you consider yourself a "backend" developer and consequently get blocked because the "frontend" developer is sick consider yourself fired. Same if the UX designer is away and you don't want to take a punt at how a particular interaction should pan out.

You should have an appreciation of all aspects of software development. That's not to say you have to be an expert in everything but being blocked because a particular person is away is unacceptable.

You should feel empowered enough to give whatever task a try. It doesn't matter if it's not perfect because it's better to ship something that is 70% good and then when the expert returns you can then iterate on the software to make it better.

When I said pairing earlier I didn't just mean "backend developers" or whatever. Everyone should be involved. If a UX designer is interviewing users it is their responsibility to bring along someone else to get experience of this skill and learn about the users. This kind of knowledge sharing is vital to make individuals that feel invested in the project and the team.

Have 10/20% time

Allocate time for everyone to work on whatever they want and be strict about it.

We work in a creative profession and we need some space to breathe, learn things and try out different ideas.

This is a great perk for the team, can help avoid burnout and can help bring new ideas that can bring great improvements to your systems down the line.

It's also important because it allows everyone to spend time developing their skills, not just people who happen to have a lot of free time and little responsibility. 10% time is important for diversity.

Don't plan extensively beyond 6 weeks

Having a vision of the future is fun, but keep it a vague and aspirational. Remember a team focused on one or two features is far more productive than one which is worrying about 10 imaginary requirements.

No Jira, no estimates, no timekeeping, no burn-up charts

All a massive waste of time, causes friction and I haven't seen a single burn-up chart in my career deliver any kind of actual useful information.

Remember we have committed to showing real working software frequently, not imaginary charts which are at best educated guesses. Finger in the air guesses are fine, adding up story points for tickets that have no hope of being actually looked at in the next 3 months is worthless.

Individuals and interactions over processes and tools

By all means have a simple, physical wall with the columns of ready for dev, in dev & done but much more than that it just becomes silly.

Wrapping up

The way to move fast is not imagine the perfect product, architecture, design a backlog and then execute it.

The way you move fast is by iterating quickly with low-effort, proven methodologies and communicating with each other as you learn through your iterations.

Optimise your processes so that your team can fearlessly and quickly change software. This way you will be able to build a good product guided by constant feedback loops.

There is no one true way to create great teams who can ship great products but in my experience so far these principles seem to stand up well. Your way might be different and that's fine but ask yourself if your team is as good as it could be. Retrospection is always important to help your team adapt to new conditions.

Footnotes

This article has been translated into Korean

Work 2018

30 December 2018

This post is a summary of work stuff over the past year which covers

The team and the way we work

I rejoined Springer Nature (SN) August 2017 on the exciting premise of being in an "innovations team" where we would attempt to maybe make scientific publishing a bit better using technology.

I was a little sad to be moving from a role mainly writing Go to Kotlin but programming languages aren't everything.

What mainly drew me to the role was the understanding that we would be empowered to work in the way we felt was best.

I am a big believer in "agile"; the actual agile, not the project management nightmare. Stuff like

Individuals and interactions over processes and tools

and

Responding to change over following a plan

We are a small team of 3 developers, UX and a BA. We work very collaboratively with little ceremony. To give an idea our "stand up" is having a chat about what were working on and our general thoughts on the work. By that definition we have at least 3 standups a day.

The responsibilities within the team are more or less shared and we rely on each other's relevant expertise to fill in the blanks when needed. I wont be blocked if our UX is away at a meeting, I'll just take a punt knowing it probably wont be that bad.

We talk a lot and we iterate quickly and actually talk to real users, rather than fighting through many layers of hierarchy.

With all of us feeling empowered and involved it means ideas come to us quickly and if we think we have a good idea we'll discuss it and try it out. We dont worry about 2 week long inceptions and prioritising backlogs; if we think its the best thing for us to try now, we'll do it.

If we get things wrong, we dont sweat it; we can change software. Someone very clever said

A prototype is worth a thousand meetings

The other key to us moving quickly is really acknowledging the rule that the last 10% of work takes 90% of the time and just avoid doing it (at least in the short term).

We often say

Perfect is the enemy of good

We would rather ship something that we hope is good, get feedback and iterate on it rather than tying ourselves in knots over perfection.

In terms of the way we work, I couldn't be happier.

But what are we working on?

The work

We were connected with a journal who had some exciting ideas around data and community building. We thought we could create a minimal viable product (MVP) peer-review system (PRS) to experiment around these topics to hopefully learn how SN can help in these areas.

We thought Trello was cool and we had actually interviewed someone who managed a journal entirely in Trello so it seemed like a nice way to bootstrap a "system" quickly.

Our first iteration was a simple HTML form for an author to submit a manuscript which would create a card in Trello. An editor could then inspect the details and then manage the whole process of peer review manually themselves (for now); when they were happy with the manuscript if they dragged it into the column "Ready for publication" our web hook would listen to that event and then create an XML payload for publishing. We turned this around in about a week and felt really rad.

From there we kept iterating, adding a simple peer reviewer search and inviting mechanism (still in Trello).

Eventually though Trello's limitations in terms of UI were becoming burdensome. It was great to prototype and we learned some interesting things from the users about their processes but ultimately we needed to control the UI. We didnt throw it all out and start again though; we just kept iterating, creating new web pages to address our new needs until eventually we didnt need Trello at all.

We "launched" with our MVP submission system for the journal in September. Since then we have carried on iterating based on user feedback and have been working to try and make the arduous quality assurance process for a manuscript a more pleasant experience.

Innovation vs invention and politics

What exactly is an innovations team? Some would think that we should be using blockchain terraform kubernetes machine-learning wizzpops to change the world. I have often wondered why what we build isn't massively transformative.

The talk Building a winning product and UX strategy from the Kano Model by Jared Spool gives a lot of clarity for me.

The real tension is between invention and innovation. Innovation is where we improve a customer's experience so that they are "delighted". It's moving the needle from basic expectation to something that's awesome.

Invention is when you are launching moonshots at ideas that are wild and likely to fail but if they work, could radically change the business.

When I think about how software teams should be working, they should all be empowered to be innovating. This to me is what agile is all about. You should be shipping a product, collecting feedback and as a team understanding it as best you can and then trying out new ideas to move the needle.

This is exactly how we work and I feel this is how every team should work. You dont work on a project (because there's no such thing as projects in software), you work on a product. You start small and keep iterating for as long as the product lives.

As your product grows so does your team, in terms of improving the way it works, how it collects information and ideas around a domain so that it can deliver better ideas quicker.

Maybe software wouldn't be so expensive if we treated it properly rather than shipping on a particular date and then letting it rot; then re-writing it all over again forever and ever.

My conclusion for all of this, is the whole concept of an innovations team is an anti-pattern. Which is a bit weird to say given that was what was sold to me when I rejoined SN.

The joy of monolith

Until this year I have mainly worked on microservices systems for a good 4-5 years. I really like the approach of microservices in terms of creating small systems within bounded contexts and then gluing them together.

I've always understood that the cost of this is in terms of increased complexity in terms of continuous integration (CI), deployments and making sure your system actually works with unreliable network calls and evolving APIs so they dont break each other (e.g. consumer driven contracts)

This year we monolith'd our system and it has been incredibly liberating. I sometimes sit in meetings with other teams as they discuss their technical stuff and I had forgotten how much of a pain building distributed systems is.

This has really cemented the idea that you should always start monolith when building a system.

YAGNI is getting ignored too much these days, every recruitment email tells me how its a "greenfield microservices project" - sounds awful!

It's far easier and quicker to build your monolith and then you get to see the actual areas in your system that could be separate services when the API is a bit more mature.

A lot of the perceived downsides of monoliths are often fixed with just good programming practices.

At some point a system does get too big and you'll need to break it up, but maybe we should worry about building a successful system before breaking it up?

Kotlin

After writing Kotlin for about 18 months I conclude it's alright.

It feels like a nice happy medium between Scala and Java, but I do miss a number of Scala features such as for comprehensions and pattern matching.

These features being missing often makes Kotlin code more complicated than the Scala equivalents which flies in the face of the opinion of Kotlin being simpler than Scala. Abstractions can and should bring simplicity when used correctly.

Scala for better or worse gives you a lot of abstraction power which most developers will make a mess with if I'm being honest; often myself.

The Kotlin compiler is still quite slow; especially compared to Go but its much faster than Scala.

Ad-hoc Kotlin-isms

I'm personally not a fan of the ad-hoc syntax it has added for commonly known problems. It feels like to me they have really tried to avoid acknowledging monads and the like but have invented different things to learn instead.

I've run into occasions where I've wanted to refactor code from a T? (nullable T) to a List<T> which is harder than it should be.

In Scala the abstraction is clearly modeled so you can call map on them in a consistent way. So if you change the type there's very few code changes to make.

In Kotlin if it is nillable you do things like myThing?.let{it + 1}. But with lists its myList.map{it + 1}. To me it's annoying that they're different. A nullable thing should be seen as a collection of one or zero items in it.

Ecosystem

From what I read a lot of people writing Kotlin still want to feel the comfort of Java land in respect to relying heavily on frameworks like Spring. I really dont want to use a massive framework and neither does anyone in our team.

Thankfully SN has invested a lot of time and effort in allowing some developers to create a lovely library called http4k which offers a simple API to create web servers.

I still feel Go is probably my weapon of choice for a general purpose language but it's certainly not a deal-breaker for me to write Kotlin.

Writing Learn Go with tests

SN is nice enough to have 10% time. In March I thought it would be fun to do a TDD guide of a "Hello, world" function in Go.

I stumbled on a nice approach of starting with a simple function and iterating on it, adding new features of Go as the software grows and gets refactored which results in quite a nice flow of learning.

It got a lot of attention on the social media thing, getting top of hacker news and lots of internet points in Github stars.

This led to me feeling very motivated and since then I've written a lot of content covering most of the Go programming language syntax by TDD-ing useful functionality.

You can find it all on Github

For me the most amazing thing is some members of Chinese Go community have translated it into Chinese!

A few chapters have been contributed by other members from the community and a lot of my terrible grammar has been fixed. Overall there are 39 different contributors to the project.

Using gitbook it has a website and I used Pandoc to generate an epub file from the markdown that the content is written in.

The feedback from the community has been invaluable and very rewarding. Committing knowledge to paper is challenging and trying to explain what you know helps solidify your ideas and I am sure the whole experience has improved me as a developer.

I am immensely proud of this work and intend to press on with it to add more content in 2019.

Summary