In previous posts, I have summed up my position on Khan Academy as follows: Khan Academy teaches only one part of mathematics—procedures—and that isn't the most important part. Writing about mathematics, developing a disposition for mathematical thinking, demonstrating a conceptual understanding of mathematical topics are all more important than procedures. That said, procedures are still important, and Khan Academy provides one venue where students can learn them. In the end, I think every young person should have an account there. Even if only one in a thousand or ten thousand benefit, that would be a terrific outcome. (I pretty much said that verbatim in Shrewsbury on Tuesday in my flipped workshop. Ask Derek.) (There is a response from Khan in Updates at the bottom)
I'm adding a new plank to my KA platform: no teacher or administrator should use or support the use of Khan Academy videos without watching the first episode of Mystery Teacher Theater 2000 or MTT2K. (Tip of my hat to Dan Meyer for leading me to this story.)
In a send-off of the Comedy Central classic Mystery Science Theater 3000, two teacher-educators sit in front of a Khan Academy video on multiplying and dividing positive and negative integers and offer their critical commentary. Dave Coffey and John Golden are the hosts here (they really do need at least one talking robot), and they clearly are not big fans of Mr. Khan or his patron Mr. Gates.
The two teachers systematically dissect the video, noting a variety of missteps. There are a few unquestionable errors of mathematics: Khan uses incorrect terminology at a couple of points. Khan is also inconsistent in his language about positive and negative numbers (using plus when he means positive, or minus when he means negative), which is perhaps a lesser sin, but poor practice and misleading for students. He's also inconsistent in his use of symbols, sometimes writing "+4", sometimes writing "4", never explaining why he does or doesn't. He making the kind of mistakes that would reduce his score on the Mathematical Quality of Instruction observational instrument, used in the Gates-funded Measures of Effective Teaching Project.
Coffey and Golden are probably most savage when Khan makes these outright mistakes, but I think the true fuel of their satire is their broader critique of Khan's approach. Khan teaches students to memorize a small set of procedural rules for dealing with multiplying negative numbers, with essentially zero effort expended to explain conceptually what the symbolic manipulations represent. In fact, in the final minute of the video, Khan says verbatim, "In your own time, think about why these rules apply."
For many math teachers, the most important work to do is to get kids to think about why the rules apply, to help them derive them where applicable, and to help them contextualize them when derivations are impractical.
Khan Academy pulled down the video satirized in MTT2K, Episode 1 within a day or so of publication. It will be interesting to see if they simply fix the outright errors, or if they address some of the broader pedagogical concerns. (Update: I think they do, see Kudos for Khan)
No teacher or department head or district administrator should put Khan Academy into action without watching this satirical critique first. There are so many media sources putting Khan on a pedestal, and there are many educators steeped in mathematical teaching who have serious concerns about his approach. Anyone assigning Khan videos to students should know about these critiques of the limitations of Khan's approach. All too often, discussions of Khan Academy deal with levels of abstraction beyond the actual videos, and Coffey and Golden bring the conversation down to individual pedagogical moves that Khan makes.
There seems to be a brewing movement for a summer of MTT2K, and I think that would be fabulous. Maybe it would provoke some reflective thinking by Khan's team, but even if Khan Academy ignored them, it would be good fun, a great professional learning experience for creators, and a terrific set of resources for all math teachers to think carefully about Khan's presentations and their own. So let's see some videos on YouTube under the #MTT2K hashtag!
Update 22-6-2012: An astute commenter, MaryDoom, notes that I made an error in the text. The post has been corrected to say that the video is about multiplying and dividing positive and negative integers, not fractions. Mary's comment has some good thoughts on the issue of errors and production.
Update 23-6-2012: Khan rapidly uploaded three new video, which address many of the critiques in MTT2K Episode 1. I offer Kudos for Khan here and launch the MTT2K prize.
Update 23-6-2012: Sal Khan responded with the following comment: "We at the Khan Academy really appreciate the feedback from you and your colleagues regarding videos that could use improvement. We think we are just at the beginning stages in our quest to make a truly valuable resource for the world and are eager to improve as much as possible (especially our older videos originally made for my cousins). We do strongly disagree that we only emphasize procedural understanding at the cost of conceptual. We think it is important to do both so that students have access to worked examples and deeper conceptual understanding, intuition, and proofs." I appreciate Khan's response, and support his intentions. And it's on me to do some more research and see if I need to change my strongly worded position.