Understanding why people underestimate software complexity

Software complexity grows rapidly over time. Developers and business people alike often underestimate how costly software development is.

To illustrate this, let's use a fictitious restaurant table reservations system. It is a sufficiently generic problem domain for the reader to follow easily. Let's examine how an apparently "trivial" example, described by a few sentences, evolves from a complexity perspective when we have a deeper look at the problem.

Basic assumptions

The restaurant has a few tables. Each table has a specific number of seats, which can vary because there are small and large ones. Only the restaurant staff uses the software. Guests call or send messages to the restaurant, and the staff enters reservations into the system. The initial description sounds simple enough.

Complexity-driving factors

Two main factors drive software development costs:

Requirements explosion at the beginning

The number of requirements is many times higher than people initially imagine because of the many corner cases the software needs to handle. This phenomenon is most pronounced at the inception of a software project. I categorize those requirements as the essential ones to make something meaningful with the software.

Software often contains numerous unnecessary requirements imagined by a stakeholder but not needed to achieve a business goal. I put them into the requirements creep category.

Developers inevitably encounter them when they reach an unspecified else branch of an if statement. This famously results in a fellow developer showing up in their colleague's office asking what they should do in this else branch.

We can take our reservation example and think of requirements that will soon arise; otherwise, the software won't be production-ready.

Hidden requirements

Tables aren't only reserved in a yes/no fashion. A reservation has a time component. A reservation specifies a date and time range for a table. A guest with attributes such as a name, email, or phone number reserves the table. This requires more than simply toggling a reservation status. Time considerations add significant complexity to any system.

What happens when guests don't show up? The restaurant needs to make tables eligible for other guests. Most business concepts have a lifecycle. Objects that are created eventually need to be deleted, archived, or marked as "deleted," using whatever concept the system supports.

Similarly, guests who call in advance to cancel require that the reservation be deleted.

"Deletion" is a perfect example of a symmetrical feature. By looking for symmetries in your business logic, you can find hidden requirements. Some examples are creation/deletion, arrival/departure, free/occupied, and idle/busy, with infinitely more.

What happens when guests arrive and occupy their reserved table? The staff needs to know which tables are free to prevent double-bookings. We need a way to mark tables as "unavailable for reservation." We can mark these tables as "occupied."

Since the staff needs to see which tables are free, there is an unspoken requirement for a user interface that displays table availability. This could be a web page, a tablet app, or the like.

We have also taken for granted that tables exist in the system. We need a mechanism to configure the tables with their seat count and a unique key/ID. This kind of data is called master data. Without it, the software cannot be used to manage reservations.

Requirements creep over time

Stakeholders don't know all the requirements upfront. Business cases, the environment (like technology or laws), the people using the software, and even the stakeholders themselves change over time.

At the beginning, there is a lack of domain understanding. Everyone involved in a software development project builds domain knowledge over time. I don't know which car I want until I've bought it. Until software is in production, nobody can be certain if it fulfills the users' needs.

This is why requirements creep in over time. We figure them out as we go.

Let's think of some requirements that could creep into this system over time.

Additional requirements

Consider the following additional requirement: a waiting list is needed.

The restaurant is high-class and one of the best in town, so it is in high demand. The owner wants a waiting list to avoid empty tables due to reservation cancellations.

To properly understand this feature, we need to think deeper. Should there be one global waiting list or one per table? A guest usually reserves a table for a certain number of people. To save complexity, we can model the system with one global waiting list. The software does not allocate waiting guests to tables automatically but leaves this task to the staff.

A waiting list only makes sense if waiting guests are informed with adequate advance notice to take a cancelled reservation. Therefore, the owner sets a cancellation policy of three days before the reservation date. Furthermore, to incentivize guests to respect the cancellation policy, they must pay a deposit online when reserving. Our software is not involved in processing the deposit; the staff handles that.

When a guest cancels a reservation, the system tells the staff which guest on the waiting list to inform via phone. The waiting list is reduced by one entry.

The staff needs to be able to add a guest to the waiting list for a table that is already fully booked and inform the guest of their position on the list. The user interface must model this exceptional logic.

Most likely, there are more intrinsic requirements that we haven't uncovered, but we can see the direction this is going. This waiting list feature has created its own requirements explosion, just as we saw at the beginning. An innocent one-line requirement has turned into an essay of tens of requirements.

Summary

Just by taking a seemingly trivial example like a table reservation system, we can see how quickly it becomes complex. Without even adding exotic features, the corner cases account for a significant increase in complexity.

Searching for symmetrical features can help uncover hidden requirements. For example, for every "create" action, a corresponding "delete" action must exist.

The next time you are involved in a software project, remember these forces that can drag it into requirements hell.

Back