Is Test Driven Development the only future for software development?

! Warning: this post hasn't been updated in over three years and so may contain out of date information.

Evolution imageThis blog post was inspired by a brief twitter conversation between myself and Mark Seeman on how, in the future, developers will likely just write tests and how the computer will generate the code to make those tests pass.

In the beginning, there was the computer, and it was huge; but simplistic in its abilities. White-coated folk fed it its simple instructions via toggle switches on a front panel. As computers became both smaller and faster, those toggle switches gave way to keyboards, the form of instructions that the computer could accept became more complex, and the concept of defining applications via programming languages were born.

Over the years, those programming languages, and the tools used to utilise them, have grown in sophistication. Ever larger tasks can now be undertaken by small teams of developers. As those tasks have grown in complexity, the simple approach of just using a programming language to express an application design has had to change too. These days, smart developers actually start with tests and the code follows. We have left the age of programming-language-driven development and entered an age of test-driven development (TDD).

Grand though the above might sound, we are only in the very early days of TDD. The tests drive (or at least guide) the developer, rather than driving the implementation and the tests often badly capture the requirements. As Mark Seemann put it in the tweet that sparked the conversation, “I envision a future of software development where tests are the specification, and working software is auto-generated to pass all tests”. Reading this, you might imagine a future where an intelligent computer takes those tests and generates the code from them. This misses the point though, as all that would happen there is that the AI becomes the developer. To really understand how this statement might come about, simply take a look at the life on our planet: the answer lies in their DNA.

Life on earth is extremely diverse and it is like this for a good reason: the environments within which life exists is itself incredibly diverse. Yet at the heart of a hyperthermophile bacteria‘s ability to survive temperatures well above 100°C; a giant redwood‘s ability to grow to 85m tall; and the amazing abilities of the human brain is simply variation in DNA. Variations in DNA supply variation in individual organisms and some variations fit an environment’s needs better than others. Life has evolved its complexity simply through environments imposing survival requirements and those best-fit individuals coping with the challenges imposed by those environments. Those best-fit individuals survive to pass on their DNA.

What does any of the above have to do with software development? The answer is evolutionary algorithms. It is possible to give an evolution system a set of requirements and allow it to evolve the solution. The downside to this idea at the moment is that such systems are really rather slow. However, one thing the last 70 years have taught us is that computers have been getting – and will continue to get – ever faster. In the last few years, we have gone from PCs having just one CPU core, to four or even eight (virtual) cores being the norm. Even if we imagine that number doubling just once a decade, in 100 years time, that puts the number of cores in a device at around 4000! Traditional software development techniques can find it hard to utilise so much parallel computing power, but being able to evolve thousands of solutions in parallel might speed up the creation of software solutions through evolution by many orders of magnitude.

Of course if we are to use evolution to create software, we need ways of specifying requirements in a way that the mindless algorithms within the evolution system can understand. As hopefully you have worked out by now, those requirements would likely be best expressed as precisely defined tests.

So imagine a future of software development: gone are the programming languages and development tools of today. In their place are tools for expressing and managing large numbers of requirements tests, an evolution-based solution generator and a test environment for testing the solutions against the requirements. True TDD would have arrived. It would be the only form of software development that made sense, for it would provide more robust solutions at speeds that “intelligently designed” systems created by humans could never hope to match.

If you use TDD these days to develop your code, you should be proud of the fact that you are possibly a pioneer of the entire future of software application development. Oh and of course, evolution tells us that the dinosaurs who aren’t smart enough to adopt this way of development are destined for extinction… 😉

8 thoughts on “Is Test Driven Development the only future for software development?

  1. I share your enthusiasm for tests, but this version of the future would be a bit of a disappointment for me.

    Let’s evolve an implementation of Add(a: int, b: int): int … first some tests:

    > test: Add(1, 2) = 3
    > test: Add(34,-2) = 32


    Okay, thanks computer, Add() is ready to go!

    > Add(34,1)

    Hmm, I see we needed to add another test case…

    (And so on :))

    Managing the Byzantine set of test cases even for something as simple as Add() would be pretty onerous. Even if the computer could infer the full set of tests from our examples, there are an infinite number of functions that can fit a given set of points.

    We might find some improvement in forming the tests from logical predicates:

    > For all a, where a: int
    > For all b, where b: int
    > Add(a, b) = a + b

    But, this isn’t a test any more (we’ve ditched the empirical measurement) – it is a theorem about Add(), and the role of the computer has turned around to start assisting with proofs.

    And, the computer isn’t writing the code any more, we are…

    (Someone with more background and familiarity with this approach might pick holes in this observation, of course.)

    I’d love it if the future ‘feels’ like TDD, but at least as a naive observer, it seems like proof systems are laying the groundwork for that today, rather than the test-driven approach that you and I are using.

    On the other hand, I’m quite happy writing code, and adding another indirection between what I express and what the computer executes seems like a leap I’m unprepared to make with the currently-available tools. Wrapping hand-written code with tests a-la TDD is something of a sweet spot, don’t you think?

    Thanks for the thought-provoking article.

  2. @Nicholas,

    Are you not making the mistake of thinking in terms of basic unit testing & how that would be used to express requirements? Writing a test, then another test with slightly different values is like using Intel 4004 assembler rather than a modern programming language.

    For example, taking your add example, we know that if x is 10, add(34,1) is the same as add(add(3x, 0x), add(4, 1)). In other words, to prove add(a, b) works:

    1. We need to test it handles adding the small 2D array of values of a in -9 .. 9 and b in -9 .. 9.
    2. Then for any two numbers, we can split them in to two sets of digits, use the proven add() to add the digits, handle the overflows and compute the expected result.
    3. Provide the test method with a set of data and have it repeatedly apply rule 2 to the numbers to determine the expected result and compare it with the actual results.

    Using this, we can prove add() for every single number combination with just two tests. And all I’m doing is using features in NUnit already. I’d be very surprised if testing frameworks in 50, 100 or even more years time will be substantially more advanced than NUnit!

  3. @ “Douglas Adams”, you are right. Evolutionary algorithms have been around for a long time. In the 50s, computers were too slow to use this method to evolve an application and the same still applies today. One day though, computers might be powerful enough for this idea to become viable.

  4. @mnem,

    Biological evolution is arguably inefficient. The shark and dolphin both evolved the same energy efficient body pattern through independent random chance. We have lost the “code” to provide us with gills, so we can’t just turn that feature back on for our summer holidays. We aren’t constrained like this when designing (evolving even) application evolution systems. We could for example – use libraries of previously evolved building blocks that can be plugged together to speed up the application creation process.

  5. Good article, and I see the nearest step to bring this future: use IDE to generate the code after the tests. It’s still your privilege to make smarter solution, but in most cases it should do the work. Let’s focus on expressing clear ideas in tests!

Comments are closed.