Why does everybody hate programming (and why this must change)?

DSLs and Subject Matter Experts

As you probably know, I spend my time developing domain-specific languages. In virtually all cases these languages are intended for use by people who are not trained programmers. They might be engineers, healthcare professionals, tax law experts or subject matter experts in various other fields. However, because their expertise is often the backbone of software systems, this expertise has to make it into the software somehow. I believe — and can demonstrate through successful projects — that we can and should build languages and tools that allow subject matter experts to contribute this expertise directly to software as opposed to just writing informal documents that are then (mis-)interpreted by “real” developers and implemented in software. This approach to software development is described in the Subject Matter First manifesto that I recommend you check out and then sign.

Everybody Hates Programming

But I wanted to get to a different (though of course related) point here. I am currently involved in a DSL project in the healthcare domain. We are in the early phases of the project, exploring whether a DSL is the right approach (of course it is, IMHO :-)). We spend our time discussing the details of the domain with subject matter experts, searching for suitable abstractions and notations, and of course building prototypes. During all these activities I am hearing the following sentence (in various guises) over and over again: This sucks, it’s too much like programming! Sometimes it’s phrased as “this is too complex” or it is “too technical”. But whatever the phraseology is, it expresses a distinct aversion against everything that smells like “programming”.

It’s really funny. These same subject matter experts are able and willing to work with arbitrarily complex and crappy GUI tools, with bad error messages, lots and lots of clicking, unsuitable abstractions that require byzantine contortions to express things and don’t even have ubiquituous undo. They are willing to use low-level database tools to shoehorn non-trivial structures into relational tables. They are perfectly ok with writing long and detailed specification documents or spreadsheets without error checking, code completion and goto definition — and without any ability to check consistency if they make a change one location in the document, breaking seven other places. And let’s not even talk about running or testing things. Yet it’s their responsibility to get things right!

So why is this the case? Why is it a problem? What can we do? I try to provide some answers in the rest of the article.

Why does Everybody Hate Programming

Sure, actual programming is not something for subject matter experts, otherwise they would have become programmers. But nobody expects them to understand complicated type systems, to care about performance, memory efficiency, scaling or security, or build their own general, reusable and composable abstractions — which is what programming is really about. But we do expect subject matter experts to be able to precisely specify how “their” domain works and then validate the correctness of their specification. Because only they can and should determine what is correct behaviour in the domain. And often they do specify these things in the aforementioned monster specification documents. Just with bad tool support.

So why is it? First, introducing something new always runs into resistance. Change is hard. It takes mental capacity, time and effort. And the approach is legitimately different from what subject matter experts are used to, so education and training is needed. But in today’s business climate, most organizations already run at > 90% (perceived) capacity, so making that change in parallel to routine work (which of course does not stop) is painful. So you fight against it by calling the new approach “complex” or “technical”.

Also, there may have been earlier attempts where subject matter experts were paired up with programmers, attempting to code the subject matter directly in a programming language. And as we all know, writing code often does require taking care about all those -ilities mentioned above. It’s very hard to concisely and readably express subject matter in a general purpose language — things are actually “technical” or evolve into “complex”. So some subject matter experts might be burned by past experience.

A final reason is that, unless you study computer science or some kind of engineering, most folks have not encountered even the basics of how programmers think and how to talk to computers in a structured way in their professional education. So they really have no experience in this kind of work. Which is a major issue. See below.

Why is this a problem?

Because it holds organizations back. We hear everywhere and all the time that everything is becoming digital. In almost all domains, computation has become, is becoming or soon will become a cornerstone. Yet, the people who have the expertise in the domain can’t really “talk” to these computers. Instead they play a game of telephone with the computer, intermediated through documents and developers.

That is inefficient in terms of both effort and quality. It’s also frustrating for everybody involved, subject matter experts and software engineers alike. In the end, it hurts competitiveness, blocks innovation and is sabotages business agility.

We really need to change! This formal specification thing that I propose domain experts should do when they use a DSL to express subject matter, this flavour of “programming light” if you will, this needs to be ubiquituous for every professional. It should be cool, too, not something peope hate!

What can we do?

First of all, publicize the idea, and explain the difference between programming and formal-subject-matter-specifying. This is why we wrote the Subject Matter First manifesto.

But we as software engineers must also ensure that we do keep the complexity of specification/modeling tools for subject matter experts low. Don’t force them into using general-purpose languages and tools, but instead build DSLs that are tailored to the domain. Use suitable abstractions. Use notations that aren’t unnecessarily terse: higher-order functions on collections can be reformulated as structures that look like half sentences, nested ifs can be expresssed as tables, and so on.

But even if we do this, it is still necessary that everybody (yes, everybody!) understands the basic concepts needed to talk to computers, such as values, functions, variables, the notion of state and concepts such as modules or specialization, as well as the idea of testing. I am not suggesting that everybody has to become a programmer and learn a programming language at school or university (as was fashionable years ago). But everybody has to understand these basics of thinking (and expressing oneself) like a programmer, at least for simple cases. This way, they everybody has the opportunity to take up DSLs and related tools in order to efficiently talk to computers. This is the thing that really is as important and reading, writing or math. It must be taught at school, and it must be a part of every university curriculum.

Further Reading

If you’re interested in a more extensive and more systematic discussion of this topic, check out this draft paper I have been writing recently. Or this attempt at teaching these basics to non programmers.