Will AI Replace Models and DSLs?


Just as everybody else I have played a bit with CharGPT recently. Initially I was quite impressed. The first couple of questions I have asked were answered reasonably. I asked about some of my favourite bands, made it come up with some regexes I needed, had it explain some superficial stuff about aerospace and physics and even asked the machine to write a poem about flying. Everything was kinda correct and -- from my perspective -- quite impressive for a language model.


When I asked ChatGPT how to land an airplane in strong crosswind, I got suspicious: while it suggested using the aileron and rudder to "keep the airplane aligned" with the runway (duh!) it did not mention the specific techniques used in crosswind. Even multiple "why" and "how" prompts did not help. It became obvious that there's no semantic model behind the replies, only language statistics.


Other people pointed out that ChatGPT can't calculate, and that, when you ask about scientific papers, it often got only half of the citations correct and made the rest up -- all while sounding quite convincing and plausible. I tried myself: indeed, ChatGPT suggested that my book, DSL Engineering, was written by Jean Bezivin :-) There's certainly a degree of societal risk if machines are able to produce confident-sounding, superficially plausible falsehoods.


So why am I talking about this? Why does the guy feel compelled to talk about AI? Because I am often confronted with statements along the lines of, "hey, why do you continue working on formal models of things, won't we just be able to ask an AI in the future?". I don't think that will be the case.


Of course deep learning will continue to improve and will get better and better -- GPT4 is rumored to be "mind-bending". But I think it will not (soon, anyway) be able to be deterministically correct. It will perform great when you need a mostly correct answer in most cases -- just like humans, whom they strive to replace :-) But when you want to perform salary and tax calculations or describe a drug trial, you really want be correct, always.


"Come on, traditional software isn't always correct either.", you might reply. True. But at least you *can* use other means of verifying what it does beyond testing. You can review. You can analyse. You can potentially proof correctness -- the formal methods communite is also making great progress, it's just not as sexy as AI. Deep learning on the other side can't be subjected to these ways of building confidence. Expressed in the self-aware words of ChatGPT:


"[A] limitation is that deep learning algorithms can be complex and difficult to interpret, making it challenging to understand why they produce certain results. This can be problematic in industries where transparency and explainability are important, such as finance and healthcare."




So for situations where the 95%-most-of-the-time correctness is good enough, AI will replace lots of traditional approaches. However, in cases where higher levels of correctness are needed, it is important that review, analysis and proof are exploited effectively. Which brings me to ... surprise ... DSLs. You can use those to express complex subject matter precisely, and in a way that can be analysed effectively by (various degrees of) formal methods.


So here is my prognosis (entering the danger zone ... :-). While traditional programming won't ever go away, we will see an increasing split into those problems where AI is good enough (in terms of correctness) and those where we need (degrees of) guarantees of correctness. For those, formal models, plus user-friendly and domain-specific language to create them, will flourish.