OK, suppose you are writing Google Maps, from scratch. Is the above the first thing you’re going to worry about?
Actually yes—you usually start with drawing a tiled raster map, it’s way easier than a vector one. A raster map is just a bunch of IMG tags side by side. But let’s go with your scenario of vector drawing, it will serve just fine and maybe I’ll learn something:
And the “postcondition” is that the canvas should receive drawing commands to display something in the right line style for a road segment, against a background of the right color, at the right scale.
So the test says “calling this code must result in this exact sequence of calls to the underlying API”? Hah. I have a method like this in one of my maps, but as far as I can remember, every time I tweaked it (e.g. to optimize the use of the different canvas abstractions in different browsers—SVG, VML, Canvas) or fixed bugs in it (like MSIE drawing the line off by one pixel when image dimensions have certain values) - I always ended up changing the sequence of API calls, so I’d need to edit the test every time which kinda defeats the purpose. Basically, this kind of postcondition is lousy. If I could express a postcondition in terms of what actually happens on the screen, that would be helpful, but I can’t. What does TDD give me here, apart from wasted effort?
Or, that I’ve heard of, a tiny little IDE called Eclipse?
Eclipse was developed test-first? I never heard of that and that would be very interesting. Do you have any references?
Gamma described the Eclipse “customized Agile” process in a 2005 keynote speech (pdf). He doesn’t explicitly call it test-first, but he emphasizes both the huge number of unit tests and their being written closely interleaved with the production code.
Eclipse was developed test-first? I never heard of that and that would be very interesting. Do you have any references?
Look for write-ups of Erich Gamma’s work; he’s the coauthor with Kent Beck of the original JUnit and one of three surviving members of the Gang of Four. Excerpt from this interview:
Erich Gamma was the original lead and visionary force behind Eclipse’s Java development environment (JDT). He still sits on the Project Management Committee for the Eclipse project. If you’ve never browsed the Eclipse Platform source code, you’re in for a real treat. Design patterns permeate the code, lending an elegant power to concepts like plug-ins and adapters. All of this is backed up by tens of thousands of unit tests. It’s a superb example of state of the art object oriented design, and Erich played a big part in its creation.
Even with this kind of evidence I prefer to add a caveat here, I’m not entirely sure it’d be fair to say that Eclipse was written in TDD “start to finish”. It had a history spanning several previous incarnations before becoming what it is today, and I wasn’t involved closely enough to know how much of it was written in TDD. Large (application-sized) chunks of it apparently were.
So the test says “calling this code must result in this exact sequence of API calls”?
That’s one way. It’s also possible to draw to an offscreen canvas and pixel-diff expected and actual images. Or if you’re targeting SVG you can compare the output XML to an expected value. Which method of expressing the postcondition you use is largely irrelevant.
The salient point is that you’re eventually going to end up with a bunch of isolated tests each of which address a single concern, whereas your main vector drawing code, is, of necessity, a cohesive assemblage of sub-computations which is expected to handle a bunch of different cases.
You only need to change the test if one such behavior itself changes in a substantial way: that’s more or less the same kind of thing you deal with if you document your code. (Test cases can make for good documentation, so some people value tests as a substitute for documentation which has the added bonus of detecting defects.)
Without tests, what tends to happen is that a change or a tweak to fix an issue affecting one of these cases may very well have side-effects that break one or more of the other cases. This happens often enough that many coding shops have a formal or informal rule of “if the code works, don’t touch it” (aka “code freeze”).
If your test suite detects one such side-effect, that would otherwise have gone undetected, the corresponding test will have more than paid for its upkeep. The cost to fix a defect you have just introduced is typically a few minutes; the cost to fix the same defects a few days, weeks or months later can be orders of magnitude bigger, rising fast with the magnitude of the delay.
Those are benefits of having comprehensive unit tests; the (claimed) added benefit of TDD is that it tends to ensure the unit tests you get are the right ones.
Again, this whole domain could and should be studied empirically, not treated as a matter of individual developers’ sovereign opinions. This thread serves as good evidence that empirical study requires first dispelling some misconceptions about the claims being investigated, such as the opinion you had going in that TDD requires first writing a huge test harness.
Wha? I’m not even sure if you read my comment before replying! To restate: the only reason you ever modify the method of drawing a line segment is to change the sequence of emitted API calls (or output XML, or something). Therefore a unit test for that method that nails down the sequence is useless. Or is it me who’s missing your point?
The cost to fix a defect you have just introduced is typically a few minutes; the cost to fix the same defects a few days, weeks or months later can be orders of magnitude bigger, rising fast with the magnitude of the delay.
For the record, I don’t buy that either. I can fix a bug in our webapp in a minute after it’s found, and have done that many times. Why do you believe the cost rises, anyway? Maybe you’re living in a different “world” after all? :-)
Thanks for the links about Eclipse, they don’t seem to prove your original point but they’re still interesting.
I can fix a bug in our webapp in a minute after it’s found
It’s still relevant that “a minute after it’s found” might be months after it’s introduced, possibly after thousands of customers have silently turned away from your software.
For the record, I don’t buy that either. I can fix a bug in our webapp in a minute after it’s found, and have done that many times. Why do you believe the cost rises, anyway?
For the record, cousin_it was entirely right to be wary of the rising-cost-of-defects claim. I believed it was well supported by evidence, but I’ve since changed my mind.
You want behaviour to be nailed down. If you have to go back and change the test when you change the behaviour, that’s a good sign: your tests are pinning down what matters.
What you don’t want is to change the test for a behaviour X when you are making code changes to an unrelated behaviour Y, or when you are making implementation changes which leave behaviour unaltered.
If you’re special-casing IE9 so that your roads should render as one pixel thicker under some circumstances, say, then your original test will remain unchanged: its job is to ensure that for non-IE9 browsers you still render roads the same.
Why do you believe the cost rises, anyway?
It’s one of the few widely-agreed-on facts in software development. See Gilb, McConnell, Capers Jones.
The mechanisms aren’t too hard to see: when you’ve just coded up a subtle defect, the context of your thinking (the set of assumptions you were making) is still in a local cache, you can easily access it again, see where you went off the rails.
When you find a defect later, it’s usually “WTF was I thinking here”, and you must spend time reconstructing that context. Plus, by that time, it’s often the case that further code changes have been piled on top of the original defect.
they don’t seem to prove your original point
I wasn’t the one with a point originally. You made some assertions in a comment to the OP, and I asked you for a clarification of your argument. You turned out to have, not an argument, but some misconceptions.
I’m happy to have cleared those up, and I’m now tapping out. Given the potential of this topic to induce affective death spirals, it’s best to let others step onto the mat now, if they still think this worth arguing.
Well, this frustrates me, but I know the frustration is caused by a bug in my brain. I respect your decision to tap out. Thanks! Guess I’ll reread the discussion tomorrow and PM you if unresolved questions remain.
No. Just no. I’d guess that different minor versions of Firefox can give different screenshots of the same antialiased line. And that’s not counting all these other browsers.
Actually yes—you usually start with drawing a tiled raster map, it’s way easier than a vector one. A raster map is just a bunch of IMG tags side by side. But let’s go with your scenario of vector drawing, it will serve just fine and maybe I’ll learn something:
So the test says “calling this code must result in this exact sequence of calls to the underlying API”? Hah. I have a method like this in one of my maps, but as far as I can remember, every time I tweaked it (e.g. to optimize the use of the different canvas abstractions in different browsers—SVG, VML, Canvas) or fixed bugs in it (like MSIE drawing the line off by one pixel when image dimensions have certain values) - I always ended up changing the sequence of API calls, so I’d need to edit the test every time which kinda defeats the purpose. Basically, this kind of postcondition is lousy. If I could express a postcondition in terms of what actually happens on the screen, that would be helpful, but I can’t. What does TDD give me here, apart from wasted effort?
Eclipse was developed test-first? I never heard of that and that would be very interesting. Do you have any references?
Gamma described the Eclipse “customized Agile” process in a 2005 keynote speech (pdf). He doesn’t explicitly call it test-first, but he emphasizes both the huge number of unit tests and their being written closely interleaved with the production code.
Look for write-ups of Erich Gamma’s work; he’s the coauthor with Kent Beck of the original JUnit and one of three surviving members of the Gang of Four. Excerpt from this interview:
Even with this kind of evidence I prefer to add a caveat here, I’m not entirely sure it’d be fair to say that Eclipse was written in TDD “start to finish”. It had a history spanning several previous incarnations before becoming what it is today, and I wasn’t involved closely enough to know how much of it was written in TDD. Large (application-sized) chunks of it apparently were.
That’s one way. It’s also possible to draw to an offscreen canvas and pixel-diff expected and actual images. Or if you’re targeting SVG you can compare the output XML to an expected value. Which method of expressing the postcondition you use is largely irrelevant.
The salient point is that you’re eventually going to end up with a bunch of isolated tests each of which address a single concern, whereas your main vector drawing code, is, of necessity, a cohesive assemblage of sub-computations which is expected to handle a bunch of different cases.
You only need to change the test if one such behavior itself changes in a substantial way: that’s more or less the same kind of thing you deal with if you document your code. (Test cases can make for good documentation, so some people value tests as a substitute for documentation which has the added bonus of detecting defects.)
Without tests, what tends to happen is that a change or a tweak to fix an issue affecting one of these cases may very well have side-effects that break one or more of the other cases. This happens often enough that many coding shops have a formal or informal rule of “if the code works, don’t touch it” (aka “code freeze”).
If your test suite detects one such side-effect, that would otherwise have gone undetected, the corresponding test will have more than paid for its upkeep. The cost to fix a defect you have just introduced is typically a few minutes; the cost to fix the same defects a few days, weeks or months later can be orders of magnitude bigger, rising fast with the magnitude of the delay.
Those are benefits of having comprehensive unit tests; the (claimed) added benefit of TDD is that it tends to ensure the unit tests you get are the right ones.
Again, this whole domain could and should be studied empirically, not treated as a matter of individual developers’ sovereign opinions. This thread serves as good evidence that empirical study requires first dispelling some misconceptions about the claims being investigated, such as the opinion you had going in that TDD requires first writing a huge test harness.
Wha? I’m not even sure if you read my comment before replying! To restate: the only reason you ever modify the method of drawing a line segment is to change the sequence of emitted API calls (or output XML, or something). Therefore a unit test for that method that nails down the sequence is useless. Or is it me who’s missing your point?
For the record, I don’t buy that either. I can fix a bug in our webapp in a minute after it’s found, and have done that many times. Why do you believe the cost rises, anyway? Maybe you’re living in a different “world” after all? :-)
Thanks for the links about Eclipse, they don’t seem to prove your original point but they’re still interesting.
It’s still relevant that “a minute after it’s found” might be months after it’s introduced, possibly after thousands of customers have silently turned away from your software.
For the record, cousin_it was entirely right to be wary of the rising-cost-of-defects claim. I believed it was well supported by evidence, but I’ve since changed my mind.
You want behaviour to be nailed down. If you have to go back and change the test when you change the behaviour, that’s a good sign: your tests are pinning down what matters.
What you don’t want is to change the test for a behaviour X when you are making code changes to an unrelated behaviour Y, or when you are making implementation changes which leave behaviour unaltered.
If you’re special-casing IE9 so that your roads should render as one pixel thicker under some circumstances, say, then your original test will remain unchanged: its job is to ensure that for non-IE9 browsers you still render roads the same.
It’s one of the few widely-agreed-on facts in software development. See Gilb, McConnell, Capers Jones.
The mechanisms aren’t too hard to see: when you’ve just coded up a subtle defect, the context of your thinking (the set of assumptions you were making) is still in a local cache, you can easily access it again, see where you went off the rails.
When you find a defect later, it’s usually “WTF was I thinking here”, and you must spend time reconstructing that context. Plus, by that time, it’s often the case that further code changes have been piled on top of the original defect.
I wasn’t the one with a point originally. You made some assertions in a comment to the OP, and I asked you for a clarification of your argument. You turned out to have, not an argument, but some misconceptions.
I’m happy to have cleared those up, and I’m now tapping out. Given the potential of this topic to induce affective death spirals, it’s best to let others step onto the mat now, if they still think this worth arguing.
Well, this frustrates me, but I know the frustration is caused by a bug in my brain. I respect your decision to tap out. Thanks! Guess I’ll reread the discussion tomorrow and PM you if unresolved questions remain.
Why not? There are automated tools to take snapshots of the screen, or window contents.
No. Just no. I’d guess that different minor versions of Firefox can give different screenshots of the same antialiased line. And that’s not counting all these other browsers.