Programming experience vs. whether you got the correct results (Here “experienced” means “professional or heavy user of programming” and “moderate” means “occasional user of programming”):
[pollid:528]
Did you think this was fair as a quick test?
[pollid:529]
I got this right, but ended up having to invent notation to keep track of the indirection in the last segment. I think it’s likely a decent test of whether you’re likely to quickly pick up an intuitive head for pointer math and a very basic variable name-value distinction, but it won’t capture other forms of abstraction that’re necessary for programming: loops, types, conditional branching, Boolean logic. You could probably get away with dropping conditionals (I get the impression they’re fairly intuitive), but I’ve had trouble teaching the others in the past.
Has a bit of an old-school feel to it, too; I’d expect the results to correlate better with talent for C than they would with, say, Python.
I didn’t. Instead, I just kept taking the least-condition-laden part of the instruction, replacing it with a number, and repeating the operation on the newly simplified sentence.
So, as of 2013-06-30 20:42 (UK summer time) it’s 13:4 for “experienced programmers”, 10:2 for “moderate programmers”, and 7:2 for “non-programmers”. The “moderate programmers” are beating the “experienced”, and the “non-” are well within the margin of error.
Now, of course LW is a hive of scum and villainy^H^H^H^H^Hplace where even the non-programmers tend to be pretty programmery, and as Morendil points out the people who chose to take the test may be atypical somehow—but, still, this looks to me like evidence that this isn’t a very effective test for discriminating between people with aptitude for programming and people without.
Yes, I agree that the poll results aren’t too encouraging. It might still be interesting to try the test on a more general population, but without a follow-up to see who eventually learned to program it would be hard to tell how accurate (if at all) it was.
I suspect this is more confusing because of the way it’s written—especially the last step which I’d imagine is where most people are falling down—than because of it really being complicated.
Answered “moderate programmer, incorrect”. I got the correct final answer but had 2 boxes incorrect. Haven’t checked where I went wrong, although I was very surprised I had as back in grade school I got these things correct with near perfection. I learned programming very easily and have traditionally rapidly outpaced my peers, but I’m only just starting professionally and don’t feel like an “experienced” programmer. As for the test, I suspect it will show some distinction but with very many false positives and negatives. There are too many uncovered aspects of what seems to make up a natural programmer. Also, it is tedious as hell, and I suspect that boredom will lead to recklessness will lead to false negatives, which aren’t terrible but are still not good. May also lead to some selection effect.
The indirection syntax should be rewritten to be left to right. As it is, it’s a good fit for C’s idiosyncratic type syntax, but needlessly obtuse otherwise.
In the box whose number is in box 6, write the sum of the number that’s in the box whose number is in box 4, and the number that’s in box 5.
Take the number in box 5. Where n is the number in box 4, take the number in box n, and the number in box 5, and add them. Where m is the number in box 6, write the result in box m.
The indirection syntax should be rewritten to be left to right.
I don’t actually think so. The final answer is simply f(g(h(x))), which is a perfectly normal thing to see in programming.
That said, I still think it’s a bad test. It involves no reasoning whatsoever—merely following instructions carefully. I’m a reasonably good programmer, but sometimes a bit sloppy (that’s why I write tests). So, I ended up with the correct final answer but a wrong number in one of the boxes.
I wasn’t sure whether to score myself as a moderate programmer or non-programmer. One CS class in college, three in Udacity, read GEB, use algorithmic approaches in day to day life, but not on a computer.
I think there’s a category missing between moderate and non-programmer. I got halfway through Eloquent JavaScript and maybe a tenth of the way through Learn Python the Hard Way. I have way, way less exposure to programming than palladias but almost infinitely more than the modal person. I suggest novice/glancing familiarity. I did the poll as moderate.
Poll for test takers:
Programming experience vs. whether you got the correct results (Here “experienced” means “professional or heavy user of programming” and “moderate” means “occasional user of programming”): [pollid:528]
Did you think this was fair as a quick test? [pollid:529]
I got this right, but ended up having to invent notation to keep track of the indirection in the last segment. I think it’s likely a decent test of whether you’re likely to quickly pick up an intuitive head for pointer math and a very basic variable name-value distinction, but it won’t capture other forms of abstraction that’re necessary for programming: loops, types, conditional branching, Boolean logic. You could probably get away with dropping conditionals (I get the impression they’re fairly intuitive), but I’ve had trouble teaching the others in the past.
Has a bit of an old-school feel to it, too; I’d expect the results to correlate better with talent for C than they would with, say, Python.
This is also the case for myself. I would be very impressed by anyone who did not have to do this.
The trick is to evaluate right to left.
I opted for doing this and also checking the answer once, as opposed to using notation.
I didn’t. Instead, I just kept taking the least-condition-laden part of the instruction, replacing it with a number, and repeating the operation on the newly simplified sentence.
Ditto.
I didn’t invent notation, but I did write
number whose number [redacted] box whose number = [redacted]
so that I could keep track as I worked from bottom to top.
I’m a new user with −1 karma who therefore can’t vote, so I’ll combat censorship bias like this:
Moderate programmer, correct
Yes
So, as of 2013-06-30 20:42 (UK summer time) it’s 13:4 for “experienced programmers”, 10:2 for “moderate programmers”, and 7:2 for “non-programmers”. The “moderate programmers” are beating the “experienced”, and the “non-” are well within the margin of error.
Now, of course LW is a hive of scum and villainy^H^H^H^H^Hplace where even the non-programmers tend to be pretty programmery, and as Morendil points out the people who chose to take the test may be atypical somehow—but, still, this looks to me like evidence that this isn’t a very effective test for discriminating between people with aptitude for programming and people without.
Yes, I agree that the poll results aren’t too encouraging. It might still be interesting to try the test on a more general population, but without a follow-up to see who eventually learned to program it would be hard to tell how accurate (if at all) it was.
I suspect this is more confusing because of the way it’s written—especially the last step which I’d imagine is where most people are falling down—than because of it really being complicated.
Answered “moderate programmer, incorrect”. I got the correct final answer but had 2 boxes incorrect. Haven’t checked where I went wrong, although I was very surprised I had as back in grade school I got these things correct with near perfection. I learned programming very easily and have traditionally rapidly outpaced my peers, but I’m only just starting professionally and don’t feel like an “experienced” programmer. As for the test, I suspect it will show some distinction but with very many false positives and negatives. There are too many uncovered aspects of what seems to make up a natural programmer. Also, it is tedious as hell, and I suspect that boredom will lead to recklessness will lead to false negatives, which aren’t terrible but are still not good. May also lead to some selection effect.
The indirection syntax should be rewritten to be left to right. As it is, it’s a good fit for C’s idiosyncratic type syntax, but needlessly obtuse otherwise.
I don’t actually think so. The final answer is simply f(g(h(x))), which is a perfectly normal thing to see in programming.
That said, I still think it’s a bad test. It involves no reasoning whatsoever—merely following instructions carefully. I’m a reasonably good programmer, but sometimes a bit sloppy (that’s why I write tests). So, I ended up with the correct final answer but a wrong number in one of the boxes.
I wasn’t sure whether to score myself as a moderate programmer or non-programmer. One CS class in college, three in Udacity, read GEB, use algorithmic approaches in day to day life, but not on a computer.
I’d say moderate.
I think there’s a category missing between moderate and non-programmer. I got halfway through Eloquent JavaScript and maybe a tenth of the way through Learn Python the Hard Way. I have way, way less exposure to programming than palladias but almost infinitely more than the modal person. I suggest novice/glancing familiarity. I did the poll as moderate.