This is part of a series of posts that will build into my final paper for the Motivation course I am taking this semester. I want to emphasize that this a rough draft and welcome comments, especially ones that point out flaws in my logic or understanding of the motivational theory under consideration. I’m going to try and use my “blogging” voice here rather than my “boring academic voice” that I use in my official paper, but I apologize in advance if I don’t entirely succeed.
In this case, let’s consider that badges here are operationalized as a reward system instead of an assessment system. For the purposes of this exercise, I’m going to rely on existing learning sites with badge systems that I’ve seen in use, namely the Khan Academy, which is largely an automatic reward system based on levels of interaction with the site’s content, such as viewing tutorials or taking quizzes.
Under Skinner’s (1950) model of operant conditioning, the observed behavior of the student viewing a tutorial or taking a quiz is the behavior that we would like to encourage. We want the student to persist in that behavior, and therefore learn more from viewing more tutorials and taking more practice quizzes. In this model, the behavior of interacting with the site leads to a consequence, or reinforcer, of earning a badge. This positive reinforcer of the earned badge, in turn, leads to the strengthened or repeated behavior of the student’s continued interaction with the site’s content. Behavioral theorists have also identified that the timing of the reinforcer has a great deal to do with how effective it is at encouraging the desired behavior–an idea known as the reinforcement schedule. Reinforcers can be on a continuous reinforcement schedule, for example, and be presented every time the desired response is demonstrated. The Khan Academy largely employs a fixed-ratio reinforcement schedule in which badges are awarded after a set number of responses. This type of reinforcement schedule predicts that there will be a drop in persistence, especially once the set number of responses occurs and no reinforcer appears (i.e. I already earned the badge for watching tutorials five days in a row, so I am very unlikely to watch five days in a row again since I’ve now earned my badge for that behavior). The schedule that results in the most persistence is called a variable-ratio reinforcement schedule in which the reinforcer is presented at intermittent times after the behavior is demonstrated (think slot machines). While Khan Academy has largely a fixed-ratio reinforcement schedule, they also advertise that some very “rare” badges can be earned in ways that are not clear, thus employing the variable-ratio schedule. By also awarding these rare badges, the intermittent nature of the reward would predict an increase in persistence because it is unclear which action will lead to the jackpot (perhaps overcoming the problem with the fixed-ratio schedule of the other badges? Not really sure on that one.). While this would predict the more persistence than the other reinforcement schedules, it still predicts that gradually response will drop off.
There is one important assumption that is made in predicting how badges might impact learning behavior–that the reward of the badge is actually a positive reinforcer. For some, a digital badge may mean very little and therefore not function as a reinforcer at all. In this case, a behavioral view would predict a lower rate of persistence than for individuals for whom the badge was seen a positive reward.
Additionally, if we assume that the awarding of a badge functions as a positive reinforcement, there is an additional prediction to me made about whether or not the potential harm of using a reward system outweighs the potential learning benefits. The use of rewards has been shown to be highly detrimental for intrinsic motivation especially (Deci, Koestner, & Ryan, 1999), and considering that the majority of users in these open course systems are there voluntarily (or are intrinsically motivated to visit the site and engage with the content), is it worth using badges to possible decrease the motivation that brought the learner to the Khan Academy in the first place? This is the typical argument leveled against behavioral learning techniques: those gold stars may not only be motivating in the short-term, but harmful in the long-term.
But would different outcomes be predicted based on different theories of motivation and learning? More on that soon!
References
Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125, 627-668.
Skinner, B. F. (1950). Are theories of learning necessary? Psychological Review, 57, 193-216.
Greetings, Andrea! Looks like you and I have been poking around the same topic!
I, too, have been curious about Khan’s badge system. With all this talk about “gamification,” I’ve been curious about the effects of this system.
I checked on Twitter and the like – it appears that many students are working to “beat” the Khan system through “cheats”, etc. – JavaScript codes that help students rack up mileage in “watching” lectures (sometimes more than can be watched within an hour) and/or browser plug-ins that answer questions correctly. I’ve been working to contact Khan teachers to test their experiences – they appear varied.
In terms of research – I’ve read that extrinsic motivators can actually cause deficient learning – prompting superficial learning strategies. Intrinsic motivation is preferred for more meaningful encoding. Extrinsic motivators can also “hurt” learners during heuristic exercises – students with rewards made about 2x as many mistakes as nonrewards students on problems that involved creative problem solving, in one study (I’ll look for the research on that; Harlow’s monkey study, though, experienced the same situation). But the students’ abilities were not harmed in algorithmic tests (and Deci had actually postulated that the rewards group would have an advantage during algorithmic exercises). I think that’s what Khan relies on – the fact that its modules and exercises rely on algorithmic learning – that’s great in terms of applying procedures, but awful in terms of conceptual learning. I wonder to what degree the rewards harm students’ ability to work on creating a rich, conceptual understanding of mathematics.
In terms of harming intrinsic motivation, though – students that are intrinsically motivated are generally less at risk for the damaging effects of rewards than other groups (I need to find that study, as well). My worry is that the system reinforces counterproductive behavior in individuals that tend towards performance-seeking and performance-avoidance goal orientations – that Khan rewards indiscriminately – if you perform the “action” indicated in the badge, you get the badge; this, though, will potentially lead to students that employ a “minimax” tactic – doing the least to gain the largest rewards. When “learning” is seen as an obstacle to the “end” of a reward, these students – as is common in students that lack intrinsic motivation – would be more prone to picking easy modules to avoid failure and challenge – to easily win “streaks,” and such, as well.
But the teachers that generally use this system either downplay or ignore the badges, and plenty of teachers have praised the system for motivating students to spend their time on task. I haven’t heard that students consider them a “status” symbol – except for maybe in the Los Altos “Sun Badges” post, which is the largest indicator of this event. I don’t know how common it is, with others, though.
My main concern, again, is the effect such motivators have on meaningful learning – that of all the things Khan could have done to promote intrinsic motivation, they chose instead to promote this breed of badge system, which doesn’t encourage meaningful learning for students that are performance-goal driven. Extrinsic goals have been positively correlated with shallow learning – they encourage algorithmic thinking and narrow, potentially forgettable, mental sets. So transfer of learning is likely harmed. The problem sets on Khan generally relate to a similar algorithm and don’t require creative problem solving – mental sets are likely similarly suffocated.
Just thoughts… I’ll work to revisit my research on this and get back to you… Wishing you the best, Layla.
Hi Layla,
Thanks you so much for this detailed and thoughtful response. That’s interesting research you cite about the extrinsic motivators causing deficient learning…I would love to look at that, though at this point I am looking less at achievement/learning and more at predictions about motivation to learn in general. I am wondering if a behavioral model might not be the most appropriate theory for looking at this, but rather, as you started to discuss, the performance vs. master goal idea, or multiple goal theory might better encompass the differences in how learners are positioning themselves in relation to the badges (i.e. cheating, etc). Khan Academy is just one example of a badging system and proponents seem to advocate for a differing “assessment and accreditation” system which is somehow different from just rewarding certain behaviors. Nonetheless, Alfie Kohn among others rightly note that grades are often positioned as an extrinsic reward, so it becomes very hard to disentangle them. At this point, I am just trying to wrap my head around various theoretical frameworks that might work in understanding how a badging system might work in practice. After that, it is a matter of exploring and testing what is actually happening. It seems as if badges were employed as a nod to digital games with the assumption that it would improve learning without any hard evidence that it actually does. Additionally, the badges are part of a larger learning system, like Khan Academy, that may not be the most pedagogically sound. Teasing out which aspects of the entire system (badges, content delivery, etc.) are contributing to learning and motivation is important, too. The next theory I’ll be exploring in regards to predictions about badge systems will be self-determination theory and should be on the blog soon.
I remember checking out Deci & Ryan’s CET theory regarding motivation, as well – it involves the “informational” versus “control” aspects of the reward orientation… It’s more subtle, maybe, than self-determination theory; it discusses how the informational aspect of the reward might offset other causes.
Also, though, with the badge system – because it’s automated – I wonder if the students view it as “controlling” – I’d worry more about the social pressure among peers causing the control element. As far as the “informational” aspect – for these badges, since they are more about reinforcing interface time with the Khan system over learning aspects, I wonder how helpful these badges are to feedback in learning.
I don’t know. Khan has spent so much time developing its game dynamics, and it seems like they’re just tacked onto their lectures, which are not terribly engaging, themselves.
Several Khan teachers mentioned points as a motivating factor for the kids – perhaps moreso than the badges, and I don’t even quite know what points mean to the kids.
The badge system, on its present reward system, seems odd to me. Of course if you’re trying to get a badge with a 50-question streak, you’re going to prefer a module with less challenging questions. It’s just odd… I don’t see why the badges have to be “unlocks,” in terms of doing it once and then being done with it – but, then, I imagine those dynamics might be better received by younger folk that are simply practicing their times tables and such.
Khan does many things well – like allowing students to see their progress and track it. And they’ve now started this “coaching” business, where peers can help one another out.
I, personally, would hope that a badge system could be created that would allow students to choose their own personal goals.
I don’t know. I’ve been trying to wrap my mind around the problem, as well – and there are just SOOOO many factors. It’s a headache-and-a-half.
The recent Daniel Pink book, “Drive”, discusses the damaging effects of extrinsic motivators on performance – I think it also mentions learning, though I’m not sure… RSA Animate did a video on it that’s fun to watch: http://www.youtube.com/watch?v=u6XAPnuFjJc
More anon, getting interrupted… My best to you, Andrea, L.