Owen_Richardson

Karma: 188

Owen_Richardson Sep 25, 2011, 7:13 PM
3 points
in reply to: [deleted]’s comment on: Review: Michel Thomas French (Direct Instruction)
Interestingly enough, the study with the highest effect size in the meta-analysis (2.44) involved non-basic skills. Actually I think I’ll just type up the summary:

This study analyzes the use of the Earth Science videodisc program with elementary education majors who traditionally have had negative attitudes towards science teaching. One group received the DI program and the other group received the traditional approach [random assignment, of course] during a one-semester science course. The DI group had significantly higher posttest knowledge scores (91%) and higher confidence in their understanding of science knowledge and ability to teach science.

Cited as:

“Vitale, M. & Romance, N. (1992). Using videodisc instruction in an elementary science methods course: Remediating science knowledge deficiencies and facilitating science teaching. Journal of Research in Science teaching, 29, 915-928.”

Not that I’ve dug up the original paper myself yet.

But one of my favorites was a study that didn’t use random assignment, but actually compared the performance of two groups of high school students: AP kids (doing whatever they normally do to study), and kids with performance previously in the lower two quartiles (taught through a videodisc course on “Chemistry and energy”). Both groups then took the same test.

Results as a researcher reported informally outside the study: “The experimentals whumped the AP students on all topics related to what was covered by the videodiscs of our course.”

(This one wasn’t included in the meta-analysis, so I’ll have to try to dig up the reference later.)

Owen_Richardson Sep 25, 2011, 6:32 PM
1 point
in reply to: jsalvatier’s comment on: Review: Michel Thomas French (Direct Instruction)
Thank you for your offer of help with feedback (I’ll def take you up on that) and papers (there are some papers referenced in “Research on Direct Instruction” I might like to get my hands on), and the sympathy on my ma.

I’m interning at a DI school in Baltimore (City Springs). Currently working with the kindergarteners on the language program (I’m supposed to move on to also doing math and reading soon, and teach older kids as well).

The National Institute For Direct Instruction (NIFDI) placed me here. It usually takes a minimum of two years for someone to get really good at presentation, but they figure I should be able to do it in one.

They’re just setting up a program for talented DI teachers to learn design by becoming coauthors on new programs, and that’s obviously where I’m aiming to go next year right after the annual summer DI conference.

Anyway, thing about the internship is that they’ve never had an unpaid foreign intern floating around before, so I end up as the third teacher in the room (the usual set-up at City Springs is a two teacher team. One of them is technically just a ‘paraprofessional’, but their instructional responsibilities are the same at a DI charter school). So I have to make sure I’m actually working on the things I need to be working on rather than getting side-tracked into some not really DI-relevant task.

Owen_Richardson Sep 25, 2011, 6:13 PM
1 point
in reply to: [deleted]’s comment on: Review: Michel Thomas French (Direct Instruction)
Yes, but we’re not talking about long-term research here. It wouldn’t be hard at all to get a bunch of volunteers interested in learning a language, and randomly assign them to one of the various different treatments popular in the industry (MT, Pimsleur, Rosetta, traditional classroom instruction, whatever). It would take less than a year, probably.

(Various choices would be up to the experimenter, like whether they wanted to control time so all groups spent, like, an hour a day or whatever, or examine the time students chose to spend themselves as one of the dependent variables, or whatever. Obviously it would be best to do it both ways.)

Obvious, easy, and valuable. Just nobody with the resources has the incentive or the perspective to see that it should be done.

Anyway, I’d expect to see a pretty high failure rate even for the MT courses. Just a significantly lower failure rate than any of the other courses.

Owen_Richardson Sep 25, 2011, 6:01 PM
1 point
in reply to: [deleted]’s comment on: Review: Michel Thomas French (Direct Instruction)
Yes, and I should make clear that Solity didn’t say, ‘The MT courses work well, wherever they work well, due to approximating DI’. He presented DI more as one of many interesting little connected pieces (many of which were pretty much fluff), rather than as an overarching explanation.

The interpretation that, “If dalmatians are metaphorically the gold standard, then the MT courses are mangy mutts in an industry where everyone else is painting black spots and pinning floppy ears on chickens,” is mine.

Owen_Richardson Sep 25, 2011, 5:06 PM
0 points
in reply to: [deleted]’s comment on: Review: Michel Thomas French (Direct Instruction)
Yeah, and all of what Cainntear was talking about later in that post with “teach confusable things separately” is covered in Chapter 10 of ToI, “Introducing coordinate members to a set”.

[And yes, there’s a typo in the table on the first page. Awareness of abstract feature “C” rules out example 6. It doesn’t have to wait until feature “(D)” is brought to attention.]

I really think both the places where MT follows DI principles, and the places where he fails, should jump out to anyone familiar with both.

Which makes me wonder about how familiar Solity was with ToI, because I don’t remember his exploration of the samenesses and differences as very good. How much of that recollection is due to my lack of familiarity at the time, I’m not sure, but I don’t think all of it. Don’t think I’m gonna reread it any time soon, though.

Owen_Richardson Sep 25, 2011, 4:06 AM
1 point
in reply to: jsalvatier’s comment on: Review: Michel Thomas French (Direct Instruction)
No, although I’d like to. I’ve just been really inactive the past couple weeks. Settling in to the internship and making sure I’m actually learning what I’m supposed to be learning there is still taking most of my energy during the week, and then I found out my mom had cancer (she just had a little bit, they got it out completely, and the chances of it coming back are apparently ‘virtually nil’ with just five weeks of radiation… but still, totally killed my productivity for one weekend), and then lazing around with a cold the next weekend. Yeesh.

The most I’ve been doing is poking at a post tentatively titled “A dry introduction to the empirical evidence on DI’s effectiveness”, essentially a summary of “Research on Direct Instruction”, since I was feeling like maybe the best thing to do would be to take a step back and present a better explanation of why people should be interested in the theory before explaining the theory itself. (Yes? No? Maybe?)

MT was never supposed to be presented as evidence itself, but as an explanation for the initial inspiration for strategies for the “what we can do for DI / what DI can do for us” thing. Obviously, it would be way better to wait until many other things are explained before I tackle that one, but I might have to try a bit anyway just to keep the confusion down since it’s already out there...

Anyway, I’ll go ask Misha if he wants to work on the ‘top-level post on DI’ project now, and if he could use me for it.

Owen_Richardson Sep 25, 2011, 3:37 AM
3 points
on: Review: Michel Thomas French (Direct Instruction)
I should also note that MT was not merely poor at meta-teaching how to use his eponymous ‘Method’ itself; He was actively secretive about it.

I believe the reason he claimed was something like fear of the establishment stealing it and claiming credit? That doesn’t seem to make much sense to me. Was that the ‘real’ reason, or a rationalization for some traumatic after-effect of his war experience, or what? Not really a question I’d expect a high chance of success or high returns on answering.

Owen_Richardson Sep 25, 2011, 3:20 AM
0 points
on: Review: Michel Thomas French (Direct Instruction)

Summary: Nice for beginners and people with bad learning experiences …

Actually, this strikes me as a bit weakly worded. I think the MT courses are the best resource currently available for an English speaker looking to start learning French or Spanish, by a significant margin.

Unfortunately there are no scientific studies comparing the effectiveness of various different ‘teach yourself’ programs and traditional classroom instruction, so I can’t find any direct evidence on that question beyond my own anecdotal experience.

But still, what with that and the much more indirect evidence available, I’d still be very surprised if this wasn’t true.

Owen_Richardson Sep 25, 2011, 3:02 AM
1 point
on: Review: Michel Thomas French (Direct Instruction)

However, according to several DI proponents the reason MT works so well is that it applies (an approximation of) DI techniques.

Wait, ‘several DI proponents’? Are you sure? Because I know of no-one in the DI world who is aware of MT (unless I were to count myself as properly ‘in the DI world’, which I do not yet).

The only place I found the connection was in the book “The Learning Revolution” by Jonathan Solity (2008). This was where I found the first reference to DI period, setting me off down this long path. However, it isn’t really the focus of Solity’s book itself, and I wouldn’t recommend the book as useful to anyone who already knows about DI, especially not if they already know about MT.

Owen_Richardson Sep 25, 2011, 2:24 AM
0 points
in reply to: Mike Bishop’s comment on: What Direct Instruction is
Yes, Project Follow-Through had some problems, but I don’t think it’s likely that those problems provided a systematic bias towards DI sufficient to explain away the huge differences as non-significant, especially since similar results were replicated in many smaller studies that were in a situation where better random assignment etc was possible.

“Research on Direct Instruction” (Adams and Engelmann, 1996) goes into much better detail on Follow-Through and those other experiments.

Actually, it basically covers three different types of studies:
- Those dealing with the relative effectiveness of DI compared to other models (in a meta-analysis)
- Those pinning down the internal details of DI theory, validating unique predictions it makes (about the effect specific variations in sequencing, juxtaposition, wording, pacing, etc should have on student performance). Only one prediction ever came out differently than expected: That a sequence of examples starting with negatives would be more efficient at narrowing in on a concept for the learner. It was found that while this did hold with more sophisticated older learners, more naive younger students simply interpreted the, ‘This is not [whatever]’ to mean, ‘This is not important, so don’t attend to this’.
- Those demonstrating ‘non-normative’ outcomes. For instance, calling Piagetian developmental theory into question.
You should be able to find the book at a local university library. Could you get your hands on it? I’d love to hear what you think after reading it!
What links here?
- Owen_Richardson's comment on Review: Michel Thomas French (Direct Instruction) by [deleted] (Sep 25, 2011, 4:06 AM; 1 point)

Owen_Richardson Sep 24, 2011, 9:02 PM
0 points
on: Review: Michel Thomas French (Direct Instruction)
Wow, nice work!

Just one note for now: On the MT courses being approximations: Yeah, the way I usually think of it now references an article by Zig Engelmann called “The Dalmatian and Its Spots” (contextual prologue here).

To summarize the most here-relevant message of the article:

Thinking that programs with certain features [eg, some focus on ‘phonemic awareness’ and ‘phonics’ in a reading program] will be successful because research shows successful programs have those features is like thinking something with spots must be a dalmatian because research has shown dalmatians have spots.

So I would say that the MT courses are, “Essentially dalmatians, not just spotted. Some mutt, some mange, but dalmatian enough to suffice for many practical purposes”.

(Although this metaphor obviously shouldn’t be phored too meta :P)

Meetup : BALTIMORE: Anyone wanna meet up?

Owen_RichardsonSep 12, 2011, 2:57 AM

4 points

3 comments1 min readLW link

Owen_Richardson Sep 8, 2011, 1:06 AM
1 point
in reply to: [deleted]’s comment on: What Direct Instruction is
Hmmm, there could be lots to reply to in that post, but I’ll try to keep it brief...

Can you give me a few specific examples of actual tasks that your students have problems with most commonly? Like, show me exactly what the students are presented with.

With that, I might be able to do a transformed task analysis, and develop an example cognitive routine.

Actually, factoring is used as one illustration of a cognitive routine in Theory of Instruction. I’ll scan that section when I get time.

Owen_Richardson Sep 8, 2011, 12:56 AM
1 point
in reply to: [deleted]’s comment on: What Direct Instruction is
Thanks! I did think it sounded annoying for commenter, and I don’t want to try the general audience’s patience much further at this point. Hence why I’m just asking a few people what they think of it in the comments.

Being able to calibrate myself objectively is an extremely attractive idea, though.

Owen_Richardson Sep 8, 2011, 12:39 AM
0 points
in reply to: [deleted]’s comment on: What Direct Instruction is
DI is a theory of instruction, not of learning.

If you’re interested in judging in greater detail how DI might offer any ideas on AI that are both useful and original, the place to start would be Inferred Functions of Performance and Learning (Engelmann and Steely, 2004), which does attempt to set out a theory of learning (and the logically necessary things that must be going on inside of any system that performs a given behavior, whether learned or unlearned).

Please see this comment.

Owen_Richardson Sep 8, 2011, 12:25 AM
0 points
in reply to: PhilGoetz’s comment on: What Direct Instruction is
Yes, this post is just an introduction to the very basics of DI (technically to just the very basics of one half of it, the ‘stimulus-locus analysis’).

Theory of Instruction goes into detail on those fundamental principles and how they apply to teaching the most basic concepts. It then shows how the basic concepts can be built up into more complex ones, and therefore how more complex ones can be analyzed to reduce them into parts for teaching.

Once you understand the details, you’ll probably just say, “Oh, right, reductionism. Of course that also applies here.”

Owen_Richardson Sep 7, 2011, 4:25 AM
0 points
in reply to: gjm’s comment on: Another treatment of Direct Instruction getting more into the technical details of the theory
Considering that you didn’t even try to see if I was making a bluff by offering to bet me one cent against my $2000...

S=probability of scientology involvement

2000S<0.01(1-S)

2000S<0.01-0.01S

2000.01S<0.01

S<0.000004999

Again, assuming I didn’t make any embarrassingly simple math errors, that’s an over 99.9995% confidence that the ‘scientology-related’ hypothesis was wrong.

Not that this is factoring in the hassle for both of us of setting up the judging and so on, but still, right? :P

Owen_Richardson Sep 7, 2011, 4:05 AM
0 points
in reply to: khafra’s comment on: What Direct Instruction is
Oh no, I know DAMN well I could’ve done WAY better if I’d been less stupid in the first place! Although if I had to communicate with my past self, I think the best thing I could have told him would be just to put a note at the beginning of the original post saying explicitly that it was a draft with many, many problems, but that I was pretty damn sure DI was a super-important topic to bring to the attention of LW, so if anyone would be so super-cool nice as to give me some feedback on how to make it more presentable...

There’s no way I could communicate the things I’ve learned so far to him more effectively than his resulting experience would teach him.

Uh, does this seem like an interesting idea?

Owen_Richardson Sep 7, 2011, 3:43 AM
2 points
in reply to: gwern’s comment on: What Direct Instruction is
Most excellent Gwern!

I have a proposition!

So, I’ve begun writing a new post, “A dry presentation of some empirical evidence on DI’s effectiveness”. (An attempt to replace that intended function of my original post with as high-quality a replacement as Misha’s post was for the intended function of the ‘theory sketch’ section.)

KPier very kindly offered to help me with editing, so I sent her the first seven-ish paragraphs I had written. She found one change to recommend, somewhat ambivalent herself over which way was best. I wasn’t sure either, and found myself wondering what I’d decide in the end.

Then I started wondering about what differences in responses there might be between a post where she made all the final decisions, and a post where where I did.

And then I thought… double-blind experiment! (Woot woot, raise the empirical roof. :P)

Here’s my idea:

I finish writing the post, get the ‘her final’ and ‘my final’ versions, and then make a post linking to both versions and explaining the experiment.

I’ll just label them version A and version B (flip a coin to avoid any weird bias I may have on As and Bs, not that I’d anticipate much), and ask the reader to follow one or the other (by flipping a coin to avoid any weird bias they may have; Mostly just to make sure the sample sizes for each version are equalized.)

Then people record their impression and give me their feedback (without directly quoting the text), and I have to try and discriminate which readers got which.

Does that sound like a neat idea? If it works well, it seems like it might even end up being worth creating an automated system for setting up and running such experiments (without all the coin flipping and link following), for people to use with appropriate posts.
What links here?
- Owen_Richardson's comment on What Direct Instruction is by [deleted] (Sep 7, 2011, 4:05 AM; 0 points)

Owen_Richardson Sep 6, 2011, 10:57 PM
1 point
in reply to: [deleted]’s comment on: What Direct Instruction is
Some other thoughts: perhaps you could give me some examples of specific teaching goals you have, and specific problems you often encounter?

Honestly, I suspect most of the problems high-school students have are due to lack of mastery of the basics. That they are weak enough on such things as adding/subtracting/multiplying/dividing fractions and working with exponents that they are likely to make mistakes on those even if they aren’t having their cognitive resources split between trying to track that shaky foundation and learn the details of the new thing you’re presenting to them.

If we could develop some systematic diagnoses, corrections, and practice materials (practice to mastery!) for just fractions and exponents, I think we might be able to hugely improve any tutoring you [or anyone else!] attempt.

ETA: If the lowest hanging fruit in improving your own skills is to “stop doing stupid shit”, then it follows that the lowest hanging fruit in improving your teaching is to figure out how to get your students to “stop doing stupid shit”. :P

Owen_Richardson

Meetup : BALTIMORE: Any­one wanna meet up?

Meetup : BALTIMORE: Anyone wanna meet up?