Understanding why people underestimate software complexity

Software gets complex very fast over time. Developers and business people alike tend to underestimate how costly software development is.

To illustrate the point we make up a fictious restaurant table reservations system. It should be a sufficiently generic problem domain that the reader can follow easily. Let's see what such an apparently "trivial" example described by a few sentences morphs into from a complexity perspective when we have a deeper look at the problem.

Basic assumptions

The restaurant has a few tables. Each table has a count of seats. The count can vary from table to table because there are small and big ones. Only the restaurant staff uses the software. It means that guests call or send messages to the restaurant and the staff types in the reservations into the system. That's it! Sounds easy!

Complexity-driving factors

What makes software development so costly is caused mainly by two factors:

Requirements explosion at the beginning

The amount of requirements is factors higher than people ever imagined because of the many corner cases the software needs to account for. This phenomenon is especially big at the inception of a software project. I categorize those requirements as the essential ones to make something meaningful with the software.

Software often contains a lot of bogus requirements which were imagined by some stakeholder but are not necessary for achieving a business goal. I put them into the requirements creep category.

Developers undoubtedly see them at latest when they fall across the unspecified else branch of an if statement. It famously results in a fellow developer showing up in his colleague's office asking what he should do in this else branch.

We can take our reservation example and think of requirements that will come up pretty soon otherwise the software won't be production-ready.

Hidden requirements

Tables aren't only reserved in a yes/no fashion. There's a timely component baked into a reservation. A table gets reserved for a specific date (including time) range. Further a guest with some attributes like name, and email or phone number reserves the table. So instead of just having a table reservation toggle we need a form to input all the required data. More than "you just need to reserve..."! Time considerations make every system considerably more complex.

What happens when guests don't show up to claim their reservation? The restaurant needs to make tables eligible for other guests. Most business concepts have a lifecycle. Objects that are created eventually need to be deleted or archived or marked as "deleted". Whatever concept the system uses for deletion.

The same applies for guests who call in advance and cancel the reservation. The reservation needs to be deleted.

"Deletion" is a perfect example of a symmetry feature. By looking for symmetries in your business logic you can find hidden requirements. Some examples are creation/deletion, arrival/departure, free/occupied, idle/busy, and infinitely more.

What happens when guests show up and occupy the reserved table? The staff needs to know which table is free so tables don't get doubly-reserved. We need a mechanism to mark tables as "not elegible for reservation". I call those tables "occupied".

Since the staff needs see which tables are free there's some "unspoken" requirement that we need to provide a user interface that shows free tables. It can be a web page, tablet app or the like.

A fact that we have taken for granted until now was that the there are tables in the system already. We need a mechanism to configure the tables with their seat count, some key/ID. This kind of data is called the master data. Without it we can't start using the software to manage reservations.

Requirements creep over time

Stakeholders don't know the requirements up-front. Business cases, the environment like technology or laws, people using the software and even stakeholders change over time.

There's a lack of understanding of the domain at beginning. Everybody involved in such a software development project builds up domain knowledge over time. I don't know which car I want until I bought it. Until software is out in production nobody can be certain if it fulfills the user's need.

That's why requirements creep in slowly over time. We figure them out as we go.

Let's think of some requirements that could creep into this system over time.

Additional requirements

Let's get creative now! A waiting list is needed.

The restaurant is high-class and one of the best ones in town therefore it's in high demand. The owner wants to have a waiting list to avoid empty tables due to reservation cancellations.

To kinda make sense of this feature we need to think deeper into this feature. Do we have one global waiting list or one per table? A guest usually reserves a table for a certain amount of people. To save some complexity we can model the system with one global waiting list. The software does not allocate waiting guests to tables automatically but leaves this task to the staff.

A waiting list only makes sense if waiting guests are informed sufficiently long in advance to take the turn from another guest. Therefore the owner sets a cancellation policy of three days before the reservation date. Also to incentivize people to respect the cancellation policy they need to pay a deposit online when reserving. Our software isn't involved into the deposit though. The staff handles that.

When a guest cancels the reservation the system tells the staff which guest to inform via phone if there's someone on the waiting list. The waiting list gets smaller by one.

The staff needs to be able to type in the guest into the waiting list for a table if the table is already reserved and be able to tell the guest that they are on the waiting list because it's fully booked. If not everything is as "normal". Some user interface needs to model this logic.

Most likely there are more intrinsic requirements that we haven't dug out but we can see the direction this goes to. This waiting list feature turned out to create a requirement explosion as we saw at the beginning. An innocent one-liner requirement turned into a tens of requirements essay.

Summary

Just by taking a seemingly trivial example as a table reservation system we can see how complex it can get quickly. Without even getting fancy in terms of exotic features the corner cases account for a big bump in complexity.

You can find hidden requirements more easily if you search for symmetry features. Say you have creation of something. Ask for the deletion part!

The next time you are involved in a software development project remember these forces that will drag you into the requirements hell.

Back