Everybody agrees that Excel Hell exists. But what caused it?
By Hiran de Silva
There is something fascinating about the long-running attack on Excel in corporate planning.
For years we have been shown pictures of spreadsheet chaos.
Hundreds of spreadsheets.
Broken links.
Multiple versions.
Endless consolidation.
People emailing files backwards and forwards.
Nobody quite knowing which number is correct.
The famous phrase Nine Circles of Excel Hell captured this beautifully.
And here is the surprising part.
I agree with them.
That mess exists.
I have seen it.
I have been hired to sort it out.
So my argument has never been that Excel Hell is imaginary.
My question is much more fundamental:
Why does Excel Hell exist?
Because before we prescribe a cure, surely we ought to diagnose the disease.
And that is where this story becomes very interesting.
The original culprit was Excel
Go back to the earlier versions of the Excel Hell argument and the implication was fairly unmistakable.
The problem was Excel.
The spreadsheet was inherently unsuitable for serious corporate planning.
The humour in Nine Circles of Excel Hell depended upon that premise. Dante wrote about Hell centuries before Excel existed, but the joke was that somehow he might have been anticipating the horrors of corporate spreadsheets.
It was entertaining.
It was memorable.
And it made a very convenient sales proposition.
Excel creates the problem. Our planning system solves it.
Except there is an awkward problem with that explanation.
Some of us had been building large corporate Excel systems without experiencing those problems.
I made a very good living doing precisely that.
I wasn’t eliminating Excel.
I was changing the architecture around it.
Instead of allowing hundreds of spreadsheets to become hundreds of little independent databases, I separated the working document from the data.
The spreadsheets became clients.
A relational database became the central Digital Librarian.
Excel could GET the information it needed.
Excel could PUT information back.
Everybody could work from the same centrally controlled data.
Suddenly many of the supposed limitations of Excel disappeared.
Consolidation didn’t require opening hundreds of spreadsheets.
Version control didn’t require chasing files.
Audit trails didn’t depend upon filenames.
Security didn’t depend upon hoping somebody hadn’t forwarded a workbook.
The problem wasn’t necessarily Excel at all.
It was how Excel had been architected.
That creates a problem for the original Excel Hell story.
If the same Excel can produce both chaos and an enterprise system, Excel itself cannot be a sufficient explanation for the chaos.
So the argument has had to evolve.
Perhaps the problem is the users
A different explanation has increasingly appeared in discussions around spreadsheets.
Perhaps Excel itself isn’t incapable.
Perhaps ordinary spreadsheet users simply don’t know how to build these systems.
Now we hear arguments about junior people, departmental thinking, lack of governance, lack of systems knowledge and people working inside their own little boxes rather than seeing the organisation as a whole.
That explanation is certainly closer to the real issue.
But I think even that explanation is unfair.
The users aren’t stupid.
Consider Tim.
Tim is the conventional Excel professional in the story I have been developing.
Tim is intelligent.
Tim works hard.
Tim learns.
Tim watches Excel experts.
Tim follows LinkedIn.
Tim watches YouTube.
Tim learns the fashionable new Excel technologies.
Tim is doing exactly what we encourage ambitious professionals to do.
And yet Reg is producing something fundamentally different.
Reg has re-engineered the process.
Tim is improving spreadsheets.
Reg is asking why the organisation has hundreds of independent spreadsheets containing fragments of what is really one corporate dataset.
Tim asks:
How can I make this spreadsheet better?
Reg asks:
Why is the data in this spreadsheet in the first place?
That is an enormous difference in thinking.
And here is the uncomfortable question.
If Tim has spent ten years diligently learning modern Excel and nobody has shown him this alternative, is that really Tim’s fault?
The missing suspect: Excel education
This is where I think an important part of the story has been overlooked.
Perhaps the problem isn’t Excel.
Perhaps it isn’t stupid users either.
Perhaps it is what spreadsheet users have been taught to believe Excel is.
Look at mainstream Excel education.
We teach functions.
We teach formulas.
We teach PivotTables.
We teach Power Query.
We teach dynamic arrays.
We teach LAMBDA.
We teach dashboards.
All useful things.
But overwhelmingly, we teach people how to do more sophisticated things inside the spreadsheet.
The spreadsheet remains the universe.
And when we have 100 departments, we create 100 increasingly sophisticated spreadsheet universes.
Then somebody has to consolidate them.
And eventually somebody publishes a white paper about spreadsheet chaos.
What if Tim had been taught something different ten years ago?
What if, alongside formulas and PivotTables, somebody had shown him that Excel can be a client to a relational database?
What if somebody had explained that the data used by hundreds of spreadsheets could live in one centrally structured location?
What if GET and PUT were as familiar to advanced Excel users as XLOOKUP and Power Query?
Tim might have become Reg years ago.
That is why I don’t regard Tim as incompetent.
Tim has been following the map he was given.
The interesting question is whether the map has been pointing in the wrong direction.
Then the argument changes again: spreadsheets versus cloud
More recent versions of the spreadsheet replacement story introduce another fascinating shift.
Now the comparison increasingly becomes something like:
Spreadsheets versus the cloud.
But wait a minute.
Those aren’t opposites.
Excel can use cloud infrastructure.
Excel can connect to centrally managed databases.
Excel can participate in client-server architecture.
Excel can consume centrally governed information and write information back.
So comparing spreadsheets with cloud technology is rather like comparing a telephone with the telephone network.
The useful comparison would be between two architectures.
For example:
Excel connected to centrally governed cloud data
versus
a specialist planning application connected to centrally governed cloud data.
Now we have a meaningful comparison.
We can examine security.
We can examine scalability.
We can examine workflow.
We can examine cost.
We can examine maintainability.
We can examine user experience.
We can examine development speed.
We can examine governance.
And different organisations may legitimately reach different conclusions.
But simply putting “spreadsheet” on one side and “cloud” on the other creates a straw man.
It quietly assumes that the spreadsheet must remain the isolated file that caused Excel Hell in the first place.
It doesn’t have to.
And then comes the Excel add-in
This is where the story potentially becomes wonderfully circular.
Many enterprise platforms provide ways for users to continue working through Excel.
Why?
Because people like Excel.
Finance professionals know Excel.
There are things Excel does exceptionally well.
So after years of explaining why spreadsheets are problematic, vendors discover that customers still want to use spreadsheets.
The interesting question then becomes:
What does the Excel integration actually do?
If Excel can retrieve centrally controlled information from the planning platform, we have a client-server architecture.
If Excel can also write information back, the resemblance becomes even stronger.
And if whole budgets can be submitted from Excel into the central system, we have arrived somewhere very interesting indeed.
Because conceptually we are getting remarkably close to the architecture I have been advocating for decades.
Excel is the working interface.
The data lives centrally.
Excel GETs.
Excel PUTs.
The central system governs the data.
Which raises an awkward question.
Why does the central database have to belong to the planning software vendor?
Why couldn’t the organisation implement that architecture itself?
Technically, it can.
And suddenly we arrive back where we started.
The real problem may be know-how
Imagine owning a modern smartphone but believing the only way to send somebody a message is to type the message, print it, put it in an envelope and post it.
We wouldn’t conclude that the smartphone lacked communication capability.
We would conclude that the user had never been shown the Send button.
Something remarkably similar has happened with spreadsheets.
For decades, millions of people have behaved as though the natural way for one spreadsheet to communicate its information to another person is to send them the spreadsheet.
Email the workbook.
Copy the workbook.
Rename the workbook.
Save another version.
Consolidate the workbooks.
Then complain about spreadsheet proliferation.
But Excel has been capable of participating in client-server architectures for decades.
The spreadsheet doesn’t have to travel.
The data can travel.
That distinction changes everything.
So what actually created Excel Hell?
We have now travelled an interesting circle.
First:
Excel is the problem.
Then:
Perhaps Excel isn’t the problem; the users don’t know enough.
Then:
Spreadsheets are the problem because modern cloud systems are better.
Then:
Actually, let’s connect Excel to our cloud system.
And finally:
Excel works rather well when it is connected to centrally governed data.
Which brings us back to the question we should perhaps have asked at the beginning.
If Excel connected to centrally governed data works, was Excel ever the fundamental problem?
I don’t think that is the most interesting conclusion.
The more interesting question is:
Why weren’t spreadsheet users routinely taught to work this way?
That takes us away from a technology argument and into an education argument.
Tim and Reg
This is why Tim and Reg matter.
Tim isn’t stupid.
Tim isn’t lazy.
Tim isn’t technologically backward.
Tim is the product of the Excel world around him.
He has spent years learning how to make increasingly sophisticated spreadsheets.
Reg has made a conceptual jump.
Reg has stopped thinking about the spreadsheet as the system.
The spreadsheet is merely the working interface.
The organisation’s information belongs somewhere else.
Once Reg understands that, an entirely different world becomes possible.
And that is why Reg can disrupt established processes so dramatically.
The difference between Tim and Reg isn’t necessarily intelligence.
It isn’t even Excel skill in the conventional sense.
It is architecture.
It is knowing that the Send button exists.
Excel fights back
That is why I think the Nine Circles story deserves another examination.
Not because spreadsheet chaos doesn’t exist.
It absolutely does.
Not because specialist planning applications have no value.
Of course they do.
The interesting issue is much simpler.
For years, we have been shown the symptoms of Excel Hell.
Broken links.
Multiple versions.
Manual consolidation.
Poor governance.
Duplicated data.
Spreadsheet proliferation.
But whenever somebody shows you symptoms and then immediately offers to sell you the cure, there is one question worth asking before reaching for the chequebook:
What actually caused the disease?
Because if the diagnosis is wrong, you may be replacing the wrong thing.
Perhaps Excel Hell was never fundamentally about Excel.
Perhaps it was about an architectural idea that millions of spreadsheet users were never taught.
And if that is true, the solution to Excel Hell may not begin by replacing the spreadsheet.
It may begin by teaching Tim what Reg already knows.
Add comment