Yup. Dijkstra was one of the early champions of Software Engineering; he is said to have almost single-handedly coined the phrase “Software Crisis” which was one of the rhetorical underpinnings of the discipline. His vision of it was as a branch of applied mathematics.
However, realizing the Software Engineering vision in practice turned out to require some compromises, for instance in order to get money from funding authorities to establish a Software Engineering Institute.
The cumulative effect of these compromises was, as Dijkstra later described, to turn away almost completely from the “applied mathematics” vision of Software Engineering, and to remake it into a sort of Taylorist conception of software development, with notions of “factories” and “productivity” and “reliability” taking center stage.
“Reliability” in particular must have been a heartache to Dijkstra: if software was mathematical, it could not break or wear out or fall apart, and the whole notion of “defects” imported from manufacturing had absolutely no place in thinking about how to write correct software. (See this paper by Priestley for interesting discussion of this dichotomy.)
Dijkstra lost, and his ideas today have little real influence on the discipline, even though he is given lip service as one of its heroes.
For my part, I don’t necessarily think Dijkstra was “right”. His ideas deserve careful consideration, but it’s far from clear to me that software development is best seen entirely or even primarily as a branch of applied mathematics. But maybe that’s only because my own mathematical ability is deficient. :)
One of the problem is that Software Engineering is very broad.
Some parts of it (making a btree library, a neural network, finding the shortest path in a graph, …) are very “mathy” and Dijkstra conception holds fully on them. Those parts are mostly done mathematically, with proofs on both the way the algorithm works and its complexity (worst-case, average-case, CPU and memory). Some other parts, like building a UI, are much harder (not totally impossible, but significantly harder) to address in a “mathy” way, and much easier to address in a “taylorist” way.
The other issue is about the way the software will be used. If you’re making embedded software for a plane, a failure can mean hundred of deaths, so you can afford spending more time doing it a more rigorous (mathematical) way. When you’re doing something not so critical, but with a final customer switching his mind every two days and asking a new feature for yesterday, you end up being much less rigorous, because the goal is no longer “to be perfect” but “to be made fast and still be good enough”.
Like in industry, if you’re making a critical part of plane, you won’t necessary use the same process than if you’re making cheap $10 watches. And yes, it’s quite sad to have to do cheap $10 watches, at least personally, I hate being forced to write “quick and dirty” code, but in the world as it now, the one who pays decide, not the one who codes...
The distinction I find useful is between “computer science”, perhaps better called “the study of the characteristics and limitations of algorithmic computations”; and “software engineering”, which is supposed to be “the systematic application of science, mathematics, technology and engineering principles to the analysis, development and maintenance of software systems, with the aim of transforming software development from an ad hoc craft to a repeatable, quantifiable and manageable process” (according to one definition from a university).
The former strikes me as a healthy discipline, the latter much less so.
to be made fast and still be good enough
Well, assume that’s the goal. When people take decisions which result in both “slow” and “not good enough”, we would say that they are irrational. In practice, “quick and dirty” code often results in being actually “slow and dirty”.
The methods we now use to build software are by and large quite irrational.
The aim of “software engineering” could be better described as “instrumentally rational approaches to creating software in the pursuit of a variety of (arbitrary, i.e. not necessarily epistemically rational) goals”.
The problem, then, is that software engineering has failed to offer approaches that are instrumentally rational in that sense: it gives advice that doesn’t work, based on opinion that turns out not to be backed by good empirical evidence but on academic politics and ideology.
Software engineering is about as sensible a phrase—and as sensible a discipline—as “fiction engineering”. Writing a program is far more like making a work of art than making most other “manufactured” products, and hiring a team of programmers to write a large program is like hiring a team of writers to write a novel the length of the collected works of Stephen King and finish in four years. Both work about as well as you’d expect. (I’ve done coding and I’ve done fiction writing, and the mental effort they require of me feels exactly the same.)
(The only example of successful “writing by committee” that I can think of is television series—they almost always have writing teams, rather than relying on individuals. I suspect that “software engineering” could probably learn a lot from the people who manage people who write TV scripts.)
I suspect that many other forms of engineering aren’t nearly as problematic as software engineering. For example, the Waterfall model, when applied to other engineering disciplines, often produces useable results. The big problem with software design is that, as in fiction writing, requirements are usually ill-defined. You usually don’t find out that the bridge that you’re halfway done building was in the wrong place, or suddenly needs to be modified to carry a train instead of carrying cars, but in software development, requirements change all the time.
Another issue is that software products are extremely diverse compared to the products of other types of industries. GM has been making automobiles for over one hundred years, and that’s all it makes. It doesn’t have the challenge of needing to make cars one year, jet airplanes the following year, and toasters the year after that. A company like Microsoft, however, makes many different types of software which have about as much in common with each other as a dishwasher has in common with a jet airplane.
As the joke goes, if cars were developed like software, we’d all be driving 25 dollar vehicles that got 1000 miles per gallon… and they would crash twice a day.
There is a very healthy (and mathematical) subdiscipline of software engineering, applied programming languages. My favorite software-engineering paper, Type-Based Access Control in Data-Centric Systems, comes with a verified proof that, in the system it presents, data-access violations (i.e.: privacy bugs) are impossible.
This is my own research area ( http://www.cs.cmu.edu/~aldrich/plaid/ ), but my belief that this was a healthy part of a diseased discipline is a large part of the reason I accepted the position.
I happen to know something about this case, and I don’t think it’s quite as you describe it.
Most of the research in this area describes itself as “algorithm engineering.” While papers do typically prove correctness, they don’t provide strong theoretical time and space bounds. Instead, they simply report performance numbers (often compared against a more standard algorithm, or against previous results). And the numbers are based on real-world graphs (e.g. the highway network for North America). In one paper I read, there was actually a chart of various tweaks to algorithm parameters and how many seconds or milliseconds each one took (on a particular test case).
This is probably the best of both worlds, but of course, it is only applicable in cases where algorithms are the hard part. In most cases, the hard part might be called “customer engineering”—trying to figure out what your customer really needs against what they claim to need, and how to deliver enough of it to them at a price you and they can afford.
Yup. Dijkstra was one of the early champions of Software Engineering; he is said to have almost single-handedly coined the phrase “Software Crisis” which was one of the rhetorical underpinnings of the discipline. His vision of it was as a branch of applied mathematics.
However, realizing the Software Engineering vision in practice turned out to require some compromises, for instance in order to get money from funding authorities to establish a Software Engineering Institute.
The cumulative effect of these compromises was, as Dijkstra later described, to turn away almost completely from the “applied mathematics” vision of Software Engineering, and to remake it into a sort of Taylorist conception of software development, with notions of “factories” and “productivity” and “reliability” taking center stage.
“Reliability” in particular must have been a heartache to Dijkstra: if software was mathematical, it could not break or wear out or fall apart, and the whole notion of “defects” imported from manufacturing had absolutely no place in thinking about how to write correct software. (See this paper by Priestley for interesting discussion of this dichotomy.)
Dijkstra lost, and his ideas today have little real influence on the discipline, even though he is given lip service as one of its heroes.
For my part, I don’t necessarily think Dijkstra was “right”. His ideas deserve careful consideration, but it’s far from clear to me that software development is best seen entirely or even primarily as a branch of applied mathematics. But maybe that’s only because my own mathematical ability is deficient. :)
One of the problem is that Software Engineering is very broad.
Some parts of it (making a btree library, a neural network, finding the shortest path in a graph, …) are very “mathy” and Dijkstra conception holds fully on them. Those parts are mostly done mathematically, with proofs on both the way the algorithm works and its complexity (worst-case, average-case, CPU and memory). Some other parts, like building a UI, are much harder (not totally impossible, but significantly harder) to address in a “mathy” way, and much easier to address in a “taylorist” way.
The other issue is about the way the software will be used. If you’re making embedded software for a plane, a failure can mean hundred of deaths, so you can afford spending more time doing it a more rigorous (mathematical) way. When you’re doing something not so critical, but with a final customer switching his mind every two days and asking a new feature for yesterday, you end up being much less rigorous, because the goal is no longer “to be perfect” but “to be made fast and still be good enough”.
Like in industry, if you’re making a critical part of plane, you won’t necessary use the same process than if you’re making cheap $10 watches. And yes, it’s quite sad to have to do cheap $10 watches, at least personally, I hate being forced to write “quick and dirty” code, but in the world as it now, the one who pays decide, not the one who codes...
The distinction I find useful is between “computer science”, perhaps better called “the study of the characteristics and limitations of algorithmic computations”; and “software engineering”, which is supposed to be “the systematic application of science, mathematics, technology and engineering principles to the analysis, development and maintenance of software systems, with the aim of transforming software development from an ad hoc craft to a repeatable, quantifiable and manageable process” (according to one definition from a university).
The former strikes me as a healthy discipline, the latter much less so.
Well, assume that’s the goal. When people take decisions which result in both “slow” and “not good enough”, we would say that they are irrational. In practice, “quick and dirty” code often results in being actually “slow and dirty”.
The methods we now use to build software are by and large quite irrational.
The aim of “software engineering” could be better described as “instrumentally rational approaches to creating software in the pursuit of a variety of (arbitrary, i.e. not necessarily epistemically rational) goals”.
The problem, then, is that software engineering has failed to offer approaches that are instrumentally rational in that sense: it gives advice that doesn’t work, based on opinion that turns out not to be backed by good empirical evidence but on academic politics and ideology.
Software engineering is about as sensible a phrase—and as sensible a discipline—as “fiction engineering”. Writing a program is far more like making a work of art than making most other “manufactured” products, and hiring a team of programmers to write a large program is like hiring a team of writers to write a novel the length of the collected works of Stephen King and finish in four years. Both work about as well as you’d expect. (I’ve done coding and I’ve done fiction writing, and the mental effort they require of me feels exactly the same.)
(The only example of successful “writing by committee” that I can think of is television series—they almost always have writing teams, rather than relying on individuals. I suspect that “software engineering” could probably learn a lot from the people who manage people who write TV scripts.)
See also: The Source Code is the Design
The relevant question is the extent to which the other kinds of engineering have similar character.
I suspect that many other forms of engineering aren’t nearly as problematic as software engineering. For example, the Waterfall model, when applied to other engineering disciplines, often produces useable results. The big problem with software design is that, as in fiction writing, requirements are usually ill-defined. You usually don’t find out that the bridge that you’re halfway done building was in the wrong place, or suddenly needs to be modified to carry a train instead of carrying cars, but in software development, requirements change all the time.
Being able to actually move the bridge when you’re half-way done probably has something to do with it...
That’s probably true.
Another issue is that software products are extremely diverse compared to the products of other types of industries. GM has been making automobiles for over one hundred years, and that’s all it makes. It doesn’t have the challenge of needing to make cars one year, jet airplanes the following year, and toasters the year after that. A company like Microsoft, however, makes many different types of software which have about as much in common with each other as a dishwasher has in common with a jet airplane.
As the joke goes, if cars were developed like software, we’d all be driving 25 dollar vehicles that got 1000 miles per gallon… and they would crash twice a day.
There is a very healthy (and mathematical) subdiscipline of software engineering, applied programming languages. My favorite software-engineering paper, Type-Based Access Control in Data-Centric Systems, comes with a verified proof that, in the system it presents, data-access violations (i.e.: privacy bugs) are impossible.
This is my own research area ( http://www.cs.cmu.edu/~aldrich/plaid/ ), but my belief that this was a healthy part of a diseased discipline is a large part of the reason I accepted the position.
I happen to know something about this case, and I don’t think it’s quite as you describe it.
Most of the research in this area describes itself as “algorithm engineering.” While papers do typically prove correctness, they don’t provide strong theoretical time and space bounds. Instead, they simply report performance numbers (often compared against a more standard algorithm, or against previous results). And the numbers are based on real-world graphs (e.g. the highway network for North America). In one paper I read, there was actually a chart of various tweaks to algorithm parameters and how many seconds or milliseconds each one took (on a particular test case).
This is probably the best of both worlds, but of course, it is only applicable in cases where algorithms are the hard part. In most cases, the hard part might be called “customer engineering”—trying to figure out what your customer really needs against what they claim to need, and how to deliver enough of it to them at a price you and they can afford.