After chatting with some friends about what is needed to produce really robust, reliable systems, I thought I’d share my thoughts on softwareengineering (as distinct from coding or programming) as an activity that involves science, craft and artisanship and the role that test-driven-development (tdd) plays in this.
For the techies amongst you, what do you think? Agree? Disagree?
Engineered solutions are those where we care about resilience, safety, security, reliability. By ‘caring’ I mean everything from ‘thinking about’ and ‘considering’ all the way through to ‘ensuring’ and ‘guaranteeing’. The further to the right you are on this spectrum, the more you require an engineering process to get there.
An engineering process requires scientific feedback and relies upon evidence that supports or refutes a hypothesis.
If you want well designed solutions (i.e. with characteristics of predictability, resilience, safety, security, recoverability) then you require an engineering process to create these solutions. That engineering process absolutely requires the application of the scientific process to test hypotheses, discover facts about the system and to provide feedback into the design process.
Test-driven development is the canonical example of how to implement this in practice.
There are aspects of software development that are craft- or artisan-like, and aspects that are engineering, and both capabilities are necessary.
The process of identifying potential solutions to problems is quite craft-like: it’s subjective, born of experience, judgement and learned heuristics, and pattern recognition. These potential solutions still require the hypothesis-testing that is the bedrock of a scientific, engineering process, however.
The other characteristic of ‘software development’ that is often overlooked (because of a focus on either craft or engineering) is that of curation of - and care for - a system or product after it has been deployed: identifying bugs and unexpected behaviour, remediating these, improving the character of the code (e.g. maintainability, clarity, etc).
I have always compared this aspect of software development to gardening: the need to do the weeding, fertilising, turning the soil on a regular or continuous basis.
Of course the application of craft (identifying potential solutions) and engineering (testing and proving those solutions) applies just as much to this software gardening as it does to the initial design, but it’s a continuous, ongoing process able to adapt to the changing seasons of the maturity of a system and the environment in which it operates.”