Shaher Jamal Eddin Blog ← All posts
Software

Software projects don't need more managers, they need a checklist!

A checklist: the boring tool that can improve IT operations & rollouts speed and quality if utilized correctly.

Global systems rollouts & activation, why are they slow ?

Many who have worked on global systems development are familiar with this scene when it comes to rollout and activations: bottlenecks, endless escalations and project delays. For example after 1 year of a pilot, only 20% of markets have the new solution. It became a normality. The product and project managers wonder why, they were told the new system should be easier to adapt, faster and great for end users. Now all they have are delays, quality and end-users acceptance issues. This is expensive especially when it comes to global systems keeping the hidden cost of delays that no one talks about in mind!.

In my view it does not have to be this way, I was lucky to have also experienced it when it works as efficiently as an assembly line, a rollout takes days, users are happy and some wonder how can this work so well. What was the difference ? using a checklist well. It's really that simple (until it is not!).

checklist usage in aviation vs software caricature

So checklists sound like a simple concept that most people agree to use, yet it is not taken seriously as a comprehensive practice within software rollout and operations as it could be in comparison to other industries like aviation where checklists are a mandatory part of the practice. That's what I suggest to try out in our lovely software world.

The hidden costs of delays - for no good reason!

Global systems usually hold high business and impact value by the sheer number of users and markets they operate in, being slow or not taking full advantage of speed is a missing opportunity to say the least.

This is particularly painful to witness when it's due to lack of adapting easy practices (as having a checklist) in comparison to other potential reasons like development bottleneck or delays due to dependencies that are much more complicated to resolve. In these cases, the complexity of sequence and things to be aware of is the root cause of delay, I wouldn't want to guess how many rollouts were delayed because of a missing system permission during a rollout project, waiting for a feedback from an expert that is on holiday (and only she or he have the knowledge in their mind!) and the list of trivial things that go wrong can go on and on.

A checklist - to off-load the mental weight into a document

It's probably obvious that checklists could be a simple and cheap solution, perhaps you are also acquainted that they are quite popular and have a track of success in some industry like in aviation / medical services, perhaps you already read the "checklist manifesto by Atul Gawande" and wondering what's the key message here ?

Firstly, I believe that the software industry treats system changes less seriously than others for the mere fact that changes are easy to be made and reverted, however we all witnessed many instances where wrong configurations on production brought major services down globally, and those failures probably caused countless (hidden) delays and failures that could add up in many corporations probably costing millions of dollars.

Secondly, a checklist simplicity may work against it, a system operator might underestimate its importance. A checklist? naah .. I've got it in my head, or think the next person will surely remember the five steps to activate this or that, not accounting to the future changes and learnings that will occur in the next activations and how mistakes will accumulate to make the system rollout a nightmare and end up with escalations and major delays in one year time.

In a global system rollout or feature activation, it's a fact that most of the technical work was already done at the time the rollout starts, now it's a matter of carrying out the steps (the recipe), so checklists are great because they off-load mental capacity (steps that are too complicated to be left to memory) from one expert and makes it available at a finger tip to other members in the change management / rollout teams. Additionally they act as a knowledge base to accumulate notes / risks / points needing attention evolving around a specific system activation. They ought to be treated a bit more seriously.

It sounds easy, until it is not!

What makes a checklist good

This all sounds pretty easy, so let's have a checklist then, and our rollouts will be fast. Well, not so fast! Having a checklist is just the starting point, in my experience those checklists must have certain characteristics to have efficient and successful rollouts and activations attributed to them, here are some key factors:

  • include phases: consider the time before the rollout starts, what to do in batch as preparation for the system activation, what permissions to request, who should know what core steps, and also what to do at the end, what to check on the next days? and who to inform about the activation.

  • collaborative approach is a must: a checklist must be believed in, in order to gather and adapt it with knowledge from real working conditions by the change management (rollout) team.

  • it must be written in simple language: without unnecessary jargon, this is especially important for new members joining the train of the rollout, the focus should be around actions (DO-VERIFY), in organized manner around the following questions:

    • what should be done
    • by whom it should be done
    • how long does it take to be done
    • notes and considerations (for exceptional scenarios)

    Illustration of a simple checklist content

A successful story for a checklist probably starts with a simple checklist with some steps (totally unfinished, it looks like a draft), and by the end of the rollout process, that initial list should look like a buzzing busy document due to much more additional content such as notes, additional steps and warnings by the end of the project.

The risks

Of course, like anything in the software development-sphere (and in life!).. there're always pros and cons, and risks. The risks of adapting a checklist centered approach exist, but I think they are minimal if handled in a conscious way from the start, to avoid the downsid:

  1. Obssesing with its completeness don't be obsessed with a completion of a checklist before using it, it may never be finished 100%, simply start with as good a state as your project allows.

  2. Blind Trust It shouldn't be thought of as a "top-down" approach to read and trust what is on there 100%, it should be challenged and adapted as the team use it more and more.

  3. Jeopardizing other quality measures don't skip other quality steps (like all sorts of testing incl. manual ones) simply because a step on a checklist was done. It is always good to test from a user point of view (especially in case of critical processes and systems).

the AI angle, it has to be there

Discussions around tools and frameworks feel almost incomplete nowadays if we don't bring up AI and how it relates to it. In this case, it's very easy, a cherry on top benefit of having a checklist, is that it can be fed to an AI workflow relatively straight forwardly via an .md file, perhaps it's another way to divide labor in the future for certain tasks that would make sense to be done without human involvement (when explicitly specified).

Last note

So having a checklist and building a collaborative approach around it, and taking seriously is all I'm suggesting here. I finally wrote this! it's been something I advocate for verbally in countless meetings since I had my first smooth rollout and saw it working in actions (thanks an awesome checklist we developed). Now I can use this article the next time, I want to promote using a checklist in our next global rollout, so probably tomorrow :)