What is a problem statement?

A problem statement in technical documents like a software design document is a paragraph that explains why some piece of code needs to be written, referring to both technical reasons (e.g., the current approach is not performant enough) and also looking at this from a business point-of-view: the code is not performant enough as we have just signed a client who is going to double our current traffic levels, or the current code is causing high database CPU usage which is leading to instability in the system.

Why would I want to write a problem statement?

The first reason would be to ensure that we are delivering business value with our work and not just doing CV-driven development. There is always scope for that kind of exploration work, but it all needs to tie back into how and why we are doing it.

Writing a problem statement should be done before starting coding, researching or any activity that could create a bias to one of the solutions. As such, it provides an equal opportunity for multiple solutions. It’s much easier to fabricate a problem statement that fits some findings and removes the feasibility of any other options. Therefore, writing the statement before considering solutions ensures that we limit this possibility.

How can writing an unbiased problem statement help exploration?

By having a technology-agnostic and business value-focused problem statement, we create space for exploring technologies that might not have been our first choice but could solve the problem in a much more facile way. Because the problem statement does not tie one down to any technology, this also helps remove any bias we have for technology and helps avoid situations like:

  • we could discredit an option due to our lack of experience
  • we could promote an option due to experience with the technology, however, based on the technology facts alone it would be a sub-optimal option

It is paramount to make a decision considering our strengths and weaknesses. However, that should not disqualify any options from being considered, as we can always look to transform weaknesses into strengths.

As humans, we are heavily biased by our experiences. If we let that drive and shape our work, we will keep repeating the same patterns - good and bad. Whilst there is a lot of value in using well-established practices, they also need to be challenged - otherwise, we will never improve.

In addition to our biases, if we start being emotionally involved in a solution - either by writing code for it or drafting up detailed designs (putting in significant effort) - we will suffer from the sunk cost fallacy on top of our biased experience due to having seen that solution potentially working.

Sunk cost fallacy

The Wikipedia page succinctly describes this: [...] is a cost that has already been incurred and cannot be recovered. In programming, this will come into effect after we have dedicated a small amount of time to a PoC, which may not be working yet, but we FEEL compelled to continue since we have already put some effort in.

This is a very expensive situation and something that I believe happens more often than we would like to admit! Furthermore, by having written some code for an approach, we feel much more committed to it and are much more willing to write off any of the inconveniences that came up and ignore warning flags that an uninvolved observer would have noticed and challenged.

Conclusion

I believe that by taking some upfront time to formulate a problem statement for the piece of work that is due, one can drastically improve the quality of the resulting solution and ensure alignment on both the business impact that work will produce and the technical direction chosen.

By postponing, even briefly, the writing of code until one has locked down the problem statement and considered multiple potential approaches - one should try not to jump into writing code or further solutionising. The focus should be on developing a few high-level options (if possible!) and doing small spikes for them before fully committing to a direction.