Badges - or how to complete the course

Task Discussion

• 
  Jennifer Claro   June 1, 2011, 5:14 p.m.

   

  Hi Everybody! 

  Here are the ideas about badges that we have expressed in the badge discussion forum and also in our PiratePad chat of May 15. (Have I missed any?)

  Rebecca: My feeling is that for this course the badge system should be kept very simple. I like the idea of each learner producing four artifacts, one for each of the four main topics of the course. I guess my question is, is the badge awarded for completion or for "demonstrating understanding of the topic." My preference would be to see a badge awarded for completion, with the learning artifacts linked to the badge.

  Monica: do people agree about the preference for course completion badges linked to artifacts? I mean, people have already been producing great artifacts, and so this approach would simply have people keep doing what they've been doing, with no extra time demands. We're already at WK 5 so some may already have all four artifacts!

  Joe: I'm hesitant about the "badge" idea in general.  I think there are other ways to acknowledge, assess, and accredit learning that feel more organic, and like less of a competition.  For example, profile pages could show the number of forum posts and the percentage of these that have been "voted up" or "starred".  In addition to forum posts, users could point to external work (e.g. blog posts) that they want to have as part of their portfolios, and these could be commented on or "starred" by other users… I think it's good to look at the facts (how much does someone contribute? How useful are these contributions to peers?), and insofar as badges do that - and help people meet their own goals in the process - they're definitely cool.  From this point of view, a "badge" is just an analytic - a cold hard fact.

  Nate (from PiratePad chat May 15): I think we should at least have one badge, the course-completion badge. I don't think we have to have it dependent on sub-badges… applying for the badge is almost an afterthought if you have some blog posts (or distributed whatever) you can link to as evidence… a badge is the accreditation we get for the work we did, not the work itself, as I understand it. I want a little link to put on my website to a badge I "earned". Note: Some of us agreed with this on May 15.

  So above we have 4 ideas on how badges could work. Joe likes the idea of starring posts and this is an idea that we haven’t discussed yet. Others mention linking artifacts to badges, or one single badge awarded at the end. It could be 4 posts, one on each topic. Many of us have made strong posts on one or more topics already.

  I’d like to suggest that we make a kind of portfolio of our own (or everyone’s all together?) posts that we think contribute well to our understanding of the topic. We could all acknowledge the contributions made, and agree that these posts qualify for posts counted towards the final course completion badge. Do we want one final course completion badge with 4 (or more) artifacts linked to it? It seems like that is the common ground of the ideas expressed above. Joe’s idea about starring posts is also interesting and worthy of discussion. It might be too late for this course to start starring posts tho. And some of our main posts are external, in blogs. No one would be able to star them.

  What do you think? I think we should discuss this here and try to come to a decision together. The main question seems to be if we get a course completion badge (just for completing the course?) or if we need 4 artifacts in order to get the final course completion badge. Please tell me if I have understood this correctly. I think that we need to decide this soon as we only have a few weeks left.

  Thanks for your help!

  Cheers,

  Jennifer

• 
  Joe Corneli   June 1, 2011, 6:47 p.m.
  In Reply To:   Jennifer Claro   June 1, 2011, 5:14 p.m.

  Hi Jennifer: By the way, I think the idea of "linking work to badges" would be nicely handled by the multi-star features I proposed, in http://new.p2pu.org/en/groups/introduction-to-the-field-of-computer-supported-co/content/how-to-participate-in-this-class/#1829 or thereabouts.  I mention this because it helps emphasize that stars and badges aren't an either/or proposition.  -Joe

• 
  Jennifer Claro   June 1, 2011, 11:48 p.m.
  In Reply To:   Joe Corneli   June 1, 2011, 6:47 p.m.

  Hi Joe,

  I think your multi-star idea is great! Very useful. Do you have any vision of how they could work with badges? Also, do you know if P2PU can be set up to utilize stars? How long would it take? I think we should definitely explore this.

  Maybe we should ask what everyone thinks about both stars and badges?

  Cheers,

  Jennifer

• 
  Stian Haklev   June 2, 2011, 3:27 a.m.
  In Reply To:   Jennifer Claro   June 1, 2011, 11:48 p.m.

  Not sure what you mean with stars, and how exactly that would work, but I know that P2PU currently has no functionality built-in for "starring" messages etc. 

• 
  Joe Corneli   June 2, 2011, 7:36 a.m.
  In Reply To:   Stian Haklev   June 2, 2011, 3:27 a.m.

  Hi Stian, I don't know if you saw my May 27, 2011, 10:51 a.m.,  May 28, 2011, 3:09 a.m, and May 27, 2011, 12:03 p.m posts in this forum, but that would explain what I mean by stars.   I've also posted about this in the Lighthouse tracker, here.

  As to Jennifer's question about how they would work with badges, I just figured that each badge could have a "marker" associated with it, and in order to associate a given posts to a certain badge, one would simply add the associated marker on the post.  This would more or less save the trouble of "challenging" the badge with a description of one's work, since the work one's most interested in and proud of would automatically be indexed on one's profile page.

  Joe 

• 
  Joe Corneli   June 2, 2011, 8:31 a.m.
  In Reply To:   Joe Corneli   June 2, 2011, 7:36 a.m.

  I added this to Uservoice, since Zuzel said that this is one of the only ways to get ideas implemented by the dev team.

• 
  Nate Otto   June 2, 2011, 1:33 p.m.
  In Reply To:   Joe Corneli   June 2, 2011, 7:36 a.m.

  Hey, Joe:

  I think multi-stars are an interesting idea, but I don't think they fulfill the purpose I see badges as fulfilling. The point of badges are to stand in for accreditation by a recognized authority, so that people looking at the badge don't need to dig deep into the structure of the courses you took and evaluate for themselves if you "really know" the material from Course X (though since P2PU is open, it would be possible to dig in as much as one likes). A badge is a portable stamp of approval, not embedded in p2pu's profile system. It can be presented wherever the earner wants, on their own portfolio, for example.

  A multi-star or tagging based system of making the links between content within p2pu might be useful to advance course engagement and other goals, but I don't think it does for us what badges can do. You did mention that they're not exclusive systems, so I'm sure you know all this. Tagging posts with the badge you're working toward with it would indeed help organize evidence for badge approval, but also keep in mind much of what we write is not in p2pu anyway, so we would still have to link to to those blog posts, etc. in our badge submissions.

• 
  Jennifer Claro   June 2, 2011, 4:50 p.m.
  In Reply To:   Joe Corneli   June 2, 2011, 7:36 a.m.

  Hi Joe and Nate,

  So after getting a badge, it would show up beside the post that got you the badge? This sounds cool but would work only if we were going for the 4-badge idea, not the course completion 1 badge idea. But I suppose it could work for the 1 badge idea if we say that this post contributed to getting the final course completion badge. I think that is what Rebecca was thinking of; four posts that would collectively get one badge. I think we need to firm up the 4-badge or 1-badge idea soon.

  What do you think?

  Jennifer

• 
  Stian Haklev   June 3, 2011, 11:48 p.m.
  In Reply To:   Jennifer Claro   June 2, 2011, 4:50 p.m.

  So I think there are two ideas in play here, maybe overlapping. One would be to be able to tag certain posts or other contributions as contributing to certain pre-specified badges. This would be similar to what we do now, by linking to evidence of contributions etc. I am quite sure that we are planning to build this kind of functionality into P2PU - exactly how it will be designed and functioned, I don't know yet. 

  The other idea is to allow more general starring and tagging of posts - even having them aggregated across studygroups, for example having "Building P2PU" be an aggregator for meta-posts about P2PU from all other study groups. This is quite an interesting idea, and we might experiment with that.

  However, for this course, none of this functionality currently exists, nor will it come into existence during the next few weeks. The only thing that exists is the experimental badge platform at badges.p2pu.org, where you can manually link to evidence of existing contributions - and of course this evidence will always be available, so anyone who clicks on the badge will be able to go back and look at your "portfolio" of contributions.

  Stian

• 
  Monica Resendes   May 13, 2011, 9:49 a.m.

  Hi all,

  I was hoping to get a discussion started up on this page as to the possibilities, utility, efficacy of badges as forms of evaluation / assessment in this course.

  As you can see, Stian and I had outlined some basic options for badges to represent course "success" or completion. As the course is emerging, however, I'm not so sure about our initial ideas on how the badges would work or fit into the course structure.This course was very experimental from the get go, so the fact that we need to have this conversation is, in many ways, no big surprise.

  (Here's a link to the "P2PU Open Badge System Framework" paper for your reading pleasure- or at least the most recent one I found. If there is a more recent version, please add it here!)

  So, I'm wondering what you all think -- how could evaluation / assessment tools and methods be useful and utilized in a course that deals less with skills-based activities and more with open-ended issues and problems like this one?

  What modes of evaluation could you see as being effective for this course? (if any?)

  How would this course need to be re-structured in order to accomodate or enable measures of evaluation?

  Can we imagine other criteria or assessment measures we could employ for this course to give us some notion of course 'success' or 'completion'?
  (this may be less of a concern for participants of courses like this and more for people evaluating the efficacy of the course itself when considering guidelines for course design or delivering info on P2P course success rates)

• 
  Rebecca Cober   May 13, 2011, 3:06 p.m.
  In Reply To:   Monica Resendes   May 13, 2011, 9:49 a.m.

  Hi Monica -

  Could you please repost the link? It doesn't work. I'm interested in reading more about the badge system.

  Cheers!

  Rebecca

• 
  Monica Resendes   May 13, 2011, 3:22 p.m.
  In Reply To:   Rebecca Cober   May 13, 2011, 3:06 p.m.

  Hi Rebecca

  try this link - I've also fixed the one above in my original post. Thanks for pointing this out!

• 
  Rebecca Cober   May 22, 2011, 12:08 p.m.
  In Reply To:   Monica Resendes   May 13, 2011, 9:49 a.m.

  Hi Stian and Monica -
  
  Okay, so I've finally read through the doc that Monica posted. I'm really impressed with the amount of work that was put into the doc. It is really well written and demonstrates an innovative approach to assessment. I did find the description of post-secondary evaluation a little one dimensional. In my experience, I have had to demonstrate my learning through writing, participation, group work, etc.
  
  My feeling is that for this course the badge system should be kept very simple. I like the idea of each learner producing four artifacts, one for each of the four main topics of the course. I guess my question is, is the badge awarded for completion or for "demonstrating understanding of the topic." My preference would be to see a badge awarded for completion, with the learning artifacts linked to the badge. Assessment is a really challenging undertaking and is extremely time consuming. I think the learning artifacts should speak for themselves - personally, I am more comfortable with that option.
  
  If the learning artifacts will be evaluated, then how will "success" be evaluated? The rubric link that I looked at seemed more like a checklist than a rubric with different levels of achievement in various categories. Who will perform the evaluation? What if there are conflicting opinions?
  
  I like the idea of a group wiki as described above, but I'm wondering if we are too late in starting this. Are there other ways that learners can collaborate and contribute?

  Well, those are my thoughts. What does everyone else think?

  Rebecca

• 
  Jennifer Claro   May 22, 2011, 4:49 p.m.
  In Reply To:   Rebecca Cober   May 22, 2011, 12:08 p.m.

  Hi Rebecca,

  Thank you for your thoughtful post. I agree with you on the completion vs. assessment nature of the badges. Unless we could come up with a system that everyone agrees with, do the badges have any value?

  I haven't seen this kind of assessment in action before, and badges are new to me, and they may be an interesting new way to assess work, but we are already halfway thru this course and I'm worried about time. Most of us are struggling just to get the readings done and to contribute something to the discussion. Re-introducing badges at this halfway point may create pressure.

  Last week in our group meeting we came up with several great initiatives: making a permanent space for CSCL-ers to meet and discuss/post, writing a manifesto... our notes from that meeting are fantastic. None of that was discussed yesterday. There was also no discussion of badges yesterday, just a statement that Stian and Monica wanted to stick with badges. If this is truly a peer-to-peer environment, I think this should be discussed and agreed upon by all of us. Last week we got rid of badges for the most part! And we had an amazing generative discussion. 

  Personally, I have to prepare for a presentation that I'm making in 2 weeks, so I will not be very active in here until that presentation is ready. This is an interesting time in here, and may be pivotal. The badge system is controversial, how will it be decided? 

  Thank you again for your insightful comments.

  Cheers,

  Jennifer

• 
  Monica Resendes   May 22, 2011, 6:55 p.m.
  In Reply To:   Rebecca Cober   May 22, 2011, 12:08 p.m.

  Hi Rebecca and Jen and everybody,

  you raise an interest point re: collaboratively determining criteria for 'success' / a system of evaluation. Since there is no established practice for integrating or utilizing badges in a course like this, it is no surprise at all we are all bumping into some prickly issues. However I think even having this discussion is extremely useful. Perhaps we can use this conversation to hopefully arrive at a consensus about how to handle badges for this course - do people agree about the preference for course completion badges linked to artifacts? I mean, people have already been producing great artifacts, and so this approach would simply have people keep doing what they've been doing, with no extra time demands. We're already at WK 5 so some may already have all four artifacts! Also, perhaps specifying one main reading and listing the other 2 or 3 as suggested or recommended from here in would help people feel a little less lost amongst the resources?  Moreover, if others are more interested in playing around with the badges, voting up processes, etc., we could discuss this as well, with more of an exploratory and experimental spirit.
  In terms of the course overall, along with engaging the readings, we have generated a number of great spaces for further work - the Food Court, the Manifesto, the Zotero group, the opportunity for a group wiki. If people think it would be useful, we could create a space either in the current WK page that has teh readings, or we could create new page under the Tasks for setting up the agenda for the weekly meeting. People could list what they want to discuss, and, depending on the number and coherency of the items people list, we could designate time for whole group discussion (regarding readings, other matters of interest to everyone) and then separate smaller group discussions based on people's interest. At the least, people interested in varous things can talk together and schedule other meetings or email each other about whatever facet they want to work more on.

  So, to sum up, I wonder if people could put their two cents in about:

  1.) Badge consensus - completion or

  Hi Rebecca and Jen and all readers,

  you raise an interest point re: collaboratively determining criteria for 'success' / a system of evaluation. Since there is no established practice for integrating or utilizing badges in a course like this, it is no surprise at all we are all bumping into some prickly issues. However I think even having this discussion is extremely useful. Perhaps we can use this conversation to hopefully arrive at a consensus about how to handle badges for this course - do people agree with Jen and Rebecca about the preference for course completion badges linked to artifacts? I mean, people have already been producing great artifacts, and so this approach would simply have people keep doing what they've been doing, with no extra time demands. We're already at WK 5 so some may already have all four artifacts! Also, perhaps specifying one main reading and listing the other 2 or 3 as suggested or recommended from here in would help people feel a little less lost amongst the resources?  Moreover, if others are more interested in playing around with the badges, voting up processes, etc., we could discuss this as well, with more of an exploratory and experimental spirit.
  In terms of the course overall, along with engaging the readings, we have generated a number of great spaces for further work - the Food Court, the Manifesto, the Zotero group, the opportunity for a group wiki. If people think it would be useful, we could create a space either in the current WK page that has teh readings, or we could create new page under the Tasks for setting up the agenda for the weekly meeting. People could list what they want to discuss, and, depending on the number and coherency of the items people list, we could designate time for whole group discussion (regarding readings, other matters of interest to everyone) and then separate smaller group discussions based on people's interest. At the least, people interested in varous things can talk together and schedule other meetings or email each other about whatever facet they want to work more on.

  So, to sum up, I wonder if people could put their two cents in about:

  1.) i.) Badge consensus - completion or artifact-based assessment?

      ii.) Interest in experimenting with badge functionality?

  2.) Meeting Agenda page

  Thanks all!

   

  2.)

• 
  Stian Haklev   May 23, 2011, 4:20 a.m.
  In Reply To:   Jennifer Claro   May 22, 2011, 4:49 p.m.

  Hi Jennifer,

   

  I actually really wanted to discuss badges and explain better what I had in mind, as well as leave time to discuss the generation of OERs etc. However, with almost everyone arriving 15 minutes late and leaving 15 minutes early, there was really not much time :( I did want us to also focus on the readings, to not let the weekly call only be used for meta-discussions, and I thought the Skype experiment was very successful.

   

  I agree that we should be democratic, but I also want to see lot's of people experiment with different things. Honestly, I personally am not that interested in a manifesto. That's no problem - I am not going to stop you from doing it at all, I will support you in any way I can. And me and Monica would like to experiment with badges, even though maybe it'll only be her giving me one, and me giving her one :) Hopefully that won't have a negative effect on the rest of the class.
• 
  Jennifer Claro   May 23, 2011, 7:15 a.m.
  In Reply To:   Monica Resendes   May 22, 2011, 6:55 p.m.

   

   

  Hi Monica and Stian,

  Thank you for your messages. I think it's better if I don't contribute to the badge discussion as it’s not likely that I’ll be able to produce much in the next couple weeks at least anyway because of this presentation. Sorry about that, I will try to contribute when possible, but realistically there are only 24 hours in a day. So maybe everyone else should decide on what to do about the badge issue. I do hope it will be a group decision.

  I think the meeting agenda page is a great idea. We could all suggest things, and possible even make a brief agenda. The first week I thought we were going to talk about the readings, but we talked about course organization and badges. The second week there was no plan and (for me) that was the best plan! It was very open and organic, topics came up all by themselves and by the end of the meeting (which was a lot of fun, on top of being very productive :) we had decided to try to become a permanent group and we had the start of a manifesto. Wow! We also discussed badges and mostly got rid of them.

  Then this week we were back to badges, with no discussion, after we had already discussed the issue the previous week, and we had all seemed to agree. So it was disorienting to come to this meeting and see that what we had previously decided on had been cancelled or ignored or whatever. However, Monica’s ideas about badges (in her message above) are great and so are Rebecca’s, and I really hope that this issue can be resolved by group consensus.

  The good things about the last meeting were that we discussed the readings! :)

  I want to thank the two of you again for all your hard work. It’s a great course and group resolution of issues like this for me are a good measure of just how peer-to-peer we are in here. This is all about collaboration right? :)

  Thanks again for your great support and all your hard work. This really is a great course.

  Thanks again,

  Jennifer

  P.S. – I wasn’t late for the meeting, I was an hour early :)  I didn’t realize that the time had changed from the previous 2 meetings. Neither did Joe, we were both an hour early, so we had a great chat while we waited for the gang to show up. I won’t be able to make the next meeting if it’s at 6:00 p.m. (7:00 a.m. here), or not much of it, as my children wake up at about that time and I am breakfast chef on weekends. Should we post again on Whenisgood to find a good time for everyone?

• 
  Stian Haklev   May 23, 2011, 7:51 a.m.
  In Reply To:   Jennifer Claro   May 23, 2011, 7:15 a.m.

  Hi Jennifer,

  it's good to hear how you felt. We are definitively groping our way in the darkness on this - there are lot's of people with very different ideas about how this should work. Part of the confusion might have been because I was gone for two weeks, so I unfortunately missed those two Etherpad meetings (although I of course read up on the logs).

  I think this concept of democracy in a course is quite interesting. Monica and I put in a lot of time to plan the course before it started, and we had a vision of how it would look - and part of that would be that we focused on providing people with at least a basic introduction to CSCL issues. We were also very interested in thinking about what it constitutes to complete a course. 

  Of course, once P2PU courses start running, we have a lot less constraints than other courses, and can tear up everything we decided and change it. It's exciting to see what emerges when a lot of people get together and create something together. On the other hand, we were also a bit worried that so much time was spent "creating" something, and maybe not enough on the actual discussions and ideas. (Other people might feel completely differently about this).

  Another interesting discussion is the concept of doing something as a whole group, versus enabling individuals and subgroups to do things. Part of the problem of Knowledge Building, which I think is rarely acknowledged, is that it assumes a common idea about which ideas to pursue, or at least which questions... So if I am in a class at OISE, and really interested in looking at Knowledge Building and Open Education, and nobody else in that class have an interest in that, I can write all I want, but I'll be doing it by myself. To what extent "should" I then do what the class does? 

  To what extent are the course organizers, once the course begins and they supposedly are more just "normal peers", individual actors, and to what extent are they the unelected representatives of the will of the masses? Theoretically, nothing is stopping anyone from publishing another bi-weekly newsletter, if they are unhappy with the one we put out, or organizing another meeting time, organized in a different way. (Being an organizer, there is almost no difference in the kind of functionality I have available on P2PU from normal participants)... If nobody does any of this, can I assume that the meeting I organize is for me to set up as I see fit, or is it still an outgrowth of the will of the community? 

  I hope you'll forgive me, I'm not writing this against you at all, I just think they are very interesting questions, and writing and thinking aloud helps me get more to grips with them.

  Stian

• 
  Stian Haklev   May 23, 2011, 7:52 a.m.
  In Reply To:   Stian Haklev   May 23, 2011, 7:51 a.m.

  BTW: I am sorry about the change of time. 6PM EST is 6AM here, so 5AM would have been a bit too early for me. I did post a note about it, as well as mention it in the bi-weekly, and on the task page. But I know it's easy to disregard when you are used to something. Yes, we can try for another whenisgood - there might also be people who have not been able to participate at that time all along, who can know make their voice heard.

• 
  Jennifer Claro   May 23, 2011, 7:04 p.m.
  In Reply To:   Stian Haklev   May 23, 2011, 7:51 a.m.

   

  Hi Stian,

  Wow, you raised a lot of issues! Many of them interesting from both pedagogical and democratic points of view. You mention the tension that arises between group and individual needs. I’d say that getting a fair balance between the two is essential in any group endeavour, and that's exactly what we are doing by discussing issues like how we all see badges (for example). Another tension might be between the needs of the organizers and the needs of the participants. There seems to be a strong need for badges coming from at least one of the organizers, and there is likely a good reason for this. Perhaps the idea of using badges in an online course is something that is new and an organizer is eager to try it out. In this case, I think that badges should have been integrated from the beginning of the course, not in the middle, where we are now. We could have discussed various posts both in blogs and in our shared space and which of these met the criteria for a badge. It seems a bit late now to be getting the badge idea going. For those of us who haven’t done any badge-work yet it means 1 badge per week for the next 4 weeks, when time is already an issue.

  When we realized (a week ago) that badges were becoming an issue, we discussed them together and came to a much less stringent view of how badges would be awarded, and how many were necessary. You weren’t there, but you said you read the logs, so you know that this was a group decision that we all participated in. If this was not something that you saw as flexible, then it shouldn’t have been discussed. However, in a peer-to-peer community, I think that flexibility and respect for the needs/wishes of everyone is a core element.

  This brings up democracy and how it might function in an online learning community. The organizers have done an amazing job of putting together a great course. I think everyone agrees on this. On the other hand, I don’t know if anyone sees organizers as “unelected representatives of the will of the masses” but I certainly don’t. One person one vote, organizer or participant. Yes, you two do more of the “grunt work” in here like the biweeklies (which are awesome, as we all acknowledge) and scheduling and organizing, but that’s what organizers do, isn’t it? As far as control/power goes, I see organizers as peers with one vote each, the same as everyone else in here. So of course I want to hear what you want to do, how you see the course going. But if it’s not a democracy in here you’d better let me know now :) I thought that was the underlying meaning of “peer-to-peer”.

  And of course we can chip in on the grunt work as well. But I think this has to be volunteer. Asking or even suggesting taking on further responsibilities like writing a biweekly or organizing may make people feel obliged to do it, when they haven’t even had time to do the readings, or post anything of their own. 

  Basically I think the course is going really well. I think we are all going to complete the course and learn a lot in the process. I see our most productive times as being in the weekly chats and in our posts in our shared space and in our blogs. So what about badges? Several ideas about badges are floating around now, Stian’s, Monica’s, Rebecca’s… plus our ideas that we shared in our discussion last week. There are some common themes, and those common themes would be the most democratic place to start.

  There, I’ve had my say, and it’s taking a lot of time to sort out these issues that have been raised. I think they are important issues and are integral to how we all see a peer-to-peer course functioning. The next discussion is on Saturday and we could take up the badges issue then. I’ll post about this in the meeting agenda Conversation Wish List (great title! :) Maybe we could firm up what the two useful questions of Monica’s mean, and then answer them together on Saturday? Like a referendum? The questions she asked were to put in our 2 cents on these topics;

  1. Badge consensus - completion or artifact-based assessment?
  2. Interest in experimenting with badge functionality?

  Maybe these could be explained more fully? Then we could discuss them on Saturday. What do you think? What does everyone think? I’m speaking only for myself here, not being an unelected speaker for the masses. Should we decide together how badges should work? And yes, let’s use Whenisgood to decide a good time for everyone. Do we use the same one as before or a new one?

  Sayonara for now, and thanks again Monica and Stian for all your hard work. 

  Cheers,

  Jennifer

• 
  Joe Corneli   May 24, 2011, 3:35 p.m.
  In Reply To:   Jennifer Claro   May 23, 2011, 7:04 p.m.

  My 2p: I think it's very interesting to look at how different people are engaging in the course.  It's not entirely clear that we're building one shared artifact, and the badge proposal seems to fit in with that style of work (each person posting content to his/her blog).  There are other ways of participating though, e.g. posting in the forums, or working on a concrete shared artifact.  Given this, it might make sense for each person to identify their own satisfaction criteria, based on how and how much they want to push themselves.  

  Aside from this, I'm hesitant about the "badge" idea in general.  I think there are other ways to acknowledge, assess, and accredit learning that feel more organic, and like less of a competition.  For example, profile pages could show the number of forum posts and the percentage of these that have been "voted up" or "starred".  In addition to forum posts, users could point to external work (e.g. blog posts) that they want to have as part of their portfolios, and these could be commented on or "starred" by other users (compare the way each encyclopedia article in PlanetMath has a discussion forum attached to it). To me, badges seem likely to create a sort of instrumental approach to courses (much like grades).  Take a look at the short article on MOTIVATION that ships with Emacs!

  To sum up, I think it's good to look at the facts (how much does someone contribute? How useful are these contributions to peers?), and insofar as badges do that - and help people meet their own goals in the process - they're definitely cool.  From this point of view, a "badge" is just an analytic - a cold hard fact.  Nothing to argue with there.  But there's a different between a fact like this, and a goal to be pursued.

• 
  Stian Haklev   May 25, 2011, 9:01 p.m.
  In Reply To:   Joe Corneli   May 24, 2011, 3:35 p.m.

  Thanks for all your input, people. This is the first P2PU course I've ever organized, and I've learnt a huge amount from the experience, and it has raised many issues that I will probably be pursuing for a long time to come. There are fundamental issues of teaching and learning that people have debated long before computers, but there are also new and interesting issues that arise from various aspects of the "openness" and "peer-to-peer". Also, because this is such a new model, people tend to have very different ideas of what different concepts mean - and there is no established model to lean on, which is exciting, but can also be exhausting :) 
  
  When it pertains to badges, I think we've spent enough time discussing that in this course. It's clear that there is no deep-felt need for them for most of the participants, and I really don't want to impose something that is not welcome. I am also not sure of the effect badges would have on the learning or the community (and of course, badges can be used in many ways, they are just a tool, like Etherpad). I still have a desire to try them out and see how it would work, but I am very happy to wait with that until the next course -- me and Monica are planning for a course on analysis of collaborative learning, probably this fall (you are all welcome to join!) At that time, we'll try to integrate badges from the very beginning, and it will also be easier technically, since more of the infrastructure will be integrated with the core P2PU system, etc. 

  But for now, I would much rather spend more time this week discussing the articles, which I find really fascinating! I hope people take some time to read at least the Suthers (2008) article, and post something on their blogs or the forums. I propose that we try to use about half an hour this weekend on the course readings (ideally a combination of Skype, and group discussion on Etherpad). The other half an hour is open to people who want to discuss the creation of common products etc - here you guys really have to take the lead. 

  I was going to put out a whenisgood, but I am a bit worried that it is getting late, and that many people won't get around to answering. I've confirmed that the hotel I'll be staying at has wifi, and I'm happy to wake up at 5AM, so we can do the regular time, if people are happy with that? Might be easier for this week? 

  PS: We've confirmed two excellent external speakers, and we'll announce the details of that very soon.

  Stian

• 
  Jennifer Claro   May 26, 2011, 1:09 a.m.
  In Reply To:   Stian Haklev   May 25, 2011, 9:01 p.m.

  Thank you for your considerate reply, Stian. So, badges are out now? This isn’t what we discussed as a group. We agreed to keep badges but we made the guidelines less stringent. I think everyone was happy with our decision. If not, let’s find out. I’m not happy with tossing the badges idea unless everyone else is too.

  We need to discuss some other things too.

  1. Are we making a group document? If so, in what form? What are our options? We had better get started soon if we are going to make one.
  2. Are we making a long-term group? If so, are we making it here in P2PU? How many of us are committed to this idea and are willing to spend a couple hours a week on initial startup and upkeep?
  3. Our manifesto hasn’t been discussed since we started it. Marcy suggested looking at another manifesto to be sure we are doing it right. Maybe we could look at it and see if there are any parts that we should add.

  As far as badges go, I think we need to answer a few basic questions:

  1. Do we want badges?
  2. If so, how many? How will they be assessed?
  3. Have any of us already made posts that could be considered for badges? This is what Monica suggested and I think it’s very realistic.

  This could be discussed in our weekly meeting or in here. I’ll post my answers below.

• 
  Jennifer Claro   May 26, 2011, 1:18 a.m.
  In Reply To:   Jennifer Claro   May 26, 2011, 1:09 a.m.

   

  1. 1. Are we making a group document? If so, in what form? What are our options? We had better get started soon if we are going to make one.

     My answer: I’d like to make a group document. I don’t want to post pages on someone else’s wiki. We could make our own wiki, or any kind of group document, and each of us could contribute to the part that we are interested in. If several people are interested in the same topic, that topic could be broken down into sub-topics. This could also be a way to get a badge (if we keep the badge idea).

  2. Are we making a long-term group? If so, are we making it here in P2PU? How many of us are committed to this idea and are willing to spend a couple hours a week on initial startup and upkeep?

     My answer: I want to make a long-term group, there doesn’t seem to be one out there yet. But there has to be enough of a base to get it going. I can volunteer a couple hours a week for as long as necessary, years if needed. I think if three or more of us are committed, that would be enough.

  3. Our manifesto hasn’t been discussed since we started it. Marcy suggested looking at another manifesto to be sure we are doing it right. Maybe we could look at it and see if there are any parts that we should add.

     My answer: I think we could look at a few manifestos in our free time and see if there is anything missing from ours that we want to add. We could consider adding the manifesto to our group document and signing it (well, putting our names on it anyway J)

     Sorry, got to get to class. Will write about badges later.

     Cheers,

     Jennifer

• 
  Joe Corneli   May 26, 2011, 4:08 a.m.
  In Reply To:   Stian Haklev   May 25, 2011, 9:01 p.m.

  Hi Stian: just to be clear, my concerns about the badge idea would apply across P2PU, not to the implementation in this course.  For the moment, I'm impressed again with the idea from a famous Canadian about how "the medium is the message".  We shape our conversations to the platform!  That's why I'm more interested in super low level things like tags and starred posts: integrating organizational features into the platform itself would be a very low-cost way to send messages about quality and relevance.  But that should be debated and refined (somewhere) too, at an appropriate moment.  I'm not trying to oppose the work that Erin and others have been doing on badges, but I don't think the relevant discussion about the "paragogy" wrapped up with badges has happened yet.  (I'd probably know about it if it had.)

  I think the Knowledge Building readings from ICSCL would be relevant to that discussion, and I think it's a good example for people in the course to discuss -- not so much should we have badges or not, but how do they work from a learning standpoint!  Can we use the Knowledge Building principles to structure a discussion about how badges work?  If the badge or whatever other accreditation system is  in line with principles like these, at least we'll know it has some theoretical grounding.  And we wouldn't feel like we had drifted off-topic!

  Joe

• 
  Nate Otto   May 26, 2011, 1:17 p.m.
  In Reply To:   Stian Haklev   May 25, 2011, 9:01 p.m.

  Stian, if you want to try badges, go for it.

  You're organizing the course after all, so you can decide. We would probably be applying for them, linking up our evidence, voting, etc, already if you had set some up. It's perfectly fine to also award diplomas to people who don't go through the badge hoops though, so you can accomodate people who don't feel the need to use the badge for self-promotion.

  In my opinion, the 4 subject badges = 1 course meta-badge structure is a little complicated to start with. Why not just set up a course badge, and we can link evidence from the subject areas and vote on it all at once?

  I'm reading Suthers now, and I have this Saturday off, so we'll get deep into it in conversation, I hope!

• 
  Jennifer Claro   May 26, 2011, 4:34 p.m.
  In Reply To:   Nate Otto   May 26, 2011, 1:17 p.m.

   

  “You’re organizing the course, so you can decide?” This really goes to the heart of it, doesn’t it? Whose course is this? Is it our course, all of us, participants and organizers? Is it the organizers’ course? Are organizers our peers, meaning our equals, or do they have some kind of extra decision power that we don’t have?

  I’ve only been in one online course before (out of nine online courses all together, all at OISE) where the organizer made decisions without considering the expressed wishes of the group, and it had a very negative effect on us. I don’t think it’s the role of an organizer to make unilateral decisions for the group. With a topic (currently being badges) that has such a wide variety of interpretations, I think we must try to find the points in common, and go from there. I don’t think an organizer should say, “We’re doing badges this way” when we have already decided on another way, and I don’t think an organizer should cancel badges unilaterally. I don’t think we should have top-down decisions in a peer-to-peer course!

  What I am trying to say is that I think this must be a group decision, that all topics that are controversial must be decided in a way that all voices are heard, and various options considered. Joe’s post on needing a theoretical basis for badges and linking them to KB is interesting, novel, and worthy of discussion. Rebecca’s ideas of badges for completion rather than as a form of assessment are valid and useful. Monica has several ideas about how to use badges; they are productive and actually see us as possibly having completed some badges already. Nate has expressed his opinion above. I do not think that these valid ideas can be dismissed, just like that.

  So, where are we now? For me, this is a pinnacle moment in this class, and the time when it will be decided that either all voices are heard and opinions considered or we agree that this space is indeed not democratic. My vote is for democracy, a true peer-to-peer group where we make major decisions together, where we see our organizers as one of us, not above us. Decentralized, distributed (democratic) power is one of the strengths of many online learning groups, and I think it would be our strength too.

• 
  Joe Corneli   May 26, 2011, 9 p.m.
  In Reply To:   Jennifer Claro   May 26, 2011, 4:34 p.m.

  Jennifer:

  I don't think P2PU (or this course) is a democracy.  Maybe a kind of anarchy.  But whatever the model is at present, surely the question of what model is best is a subject for debate.  And I think it's valuable if these debates are held in a philosophical spirit and tone.  For example, here's how Martha Nussbaum describes Socrates, who seems to have some points in common with you.

  Socrates has a passion for argument. He doesn’t like long speeches, and he doesn’t make them. He also doesn’t like authority. He takes nothing on trust, not from the poets, not from the politicians, not from any other source of cultural prestige and power. He questions everything, and he accepts only what survives reason’s demand for consistency, for clear definitions and for cogent explanations. This also means that Socrates and his interlocutor are equals: the fact that he is a philosopher gives him no special claim, no authority. Indeed, he practices on himself the same techniques of examination and refutation he practices on others. If he is one step ahead of his interlocutors at times, it is only because he knows what he does not know, and they sometimes fancy that they have answers—which soon fall to bits.

  With that as a background, I wonder if you aren't being a bit hard on Stian.  It might help bring the dialog to a fruitful resolution if you rephrase your concerns in a more general way - perhaps expanding the critique, but phrasing things in a way that positions you and your interlocutors as equals (which I think is the goal you're aiming for).  In such a circumstance, would top-down decisions even be an option?  I think there's something fundamental to consider here.

  To conclude with a quote from a more contemporary philosopher who said something very Socratic: 

  The opinions I have formed and the conclusions I have arrived at are not final. I may change them tomorrow. I have nothing new to teach the world. Truth and non-violence are as old as the hills.

• 
  Jennifer Claro   May 26, 2011, 9:14 p.m.
  In Reply To:   Joe Corneli   May 26, 2011, 9 p.m.

  Hi Joe,

  Awesome post, thank you. Great quotes from great leaders, who saw themselves as equals with everyone.

  I am being hard on Stian because I think he sees organizers as above participants, that we are not peers. Twice in the past week we've been handed decisions that seem to have been made by one person, or at best two, if Monica contributed to them too. We have already discussed badges and decided as a group how to deal with them. Stian has twice ignored what we have written about badges, and yes, I agree that I am being blunt about putting this on the table. But this unilateral decision making is anathema to peer-to-peer ideology, it recognizes a higher authority in here when we specifically rejected that in our manifesto. Are we not all equals?

  I am trying to make a case for democracy (actually anarchy would be even better, but I wonder if others might see it as too radical). Thank you very much for your contribution. It was great to see Socrates and Gandhi on the same page.

  Cheers,

  Jennifer

• 
  Joe Corneli   May 26, 2011, 10:19 p.m.
  In Reply To:   Jennifer Claro   May 26, 2011, 9:14 p.m.

  I am confident from previous conversations with Stian that his views are rather different from those you've just ascribed to him.  But supposing he did feel that organizers had "more of a say" or something like that -- they do most of the work, after all.  Are you justified in being hard on him, simply because you see things differently?

  You said that "unilateral decision making is anathema to peer-to-peer ideology" -- But how does one decide what to put into the public discourse, except unilaterally?  Maybe unilateral decision making is the foundation of peer-to-peer discourse.  It could be!

  Finally, all of this underscores what I said above, which I will reiterate and expand here: if you accept that this is peer learning, and that part of the basis for peer learning is equality, then the kinds of injustices you are perceiving simply cannot happen.  I realize that might sound really hokey!!!  I'm not suggesting that you deny your feelings (they're real), or that things can't go wrong (they most certainly can).  I'm simply suggesting that by reminding yourself that peers are equals, you can't be dispossessed of your equality.  Think it over!

• 
  Jennifer Claro   May 27, 2011, 1:56 a.m.
  In Reply To:   Joe Corneli   May 26, 2011, 10:19 p.m.

  Hi Joe,

  Thank you for your reply. It seems that this is turning into a Jennifer vs. Stian argument when what I had intended was instead a member of the group, including both participants and organizers, protesting against an action (actually two actions) of an organizer. I don't want this to become personal - it is the action that I object to, not the person, and I've said everything that I could say. I think it's best if I stop protesting now, as I seem to be the only one who objects to these actions. I don't agree with an organizer or anyone deciding something for the group that is not in line with what the group decided previously. But I've said that already, and I'm the only one protesting, so enough already.

  I'm very glad to hear that Stian's views may be different from what's been happening in here, but I don't understand how one's views and one's actions can be so disparate. Either you think you can make the rules or you don't. I had thought that the meaning of peer-to-peer meant that in here we would be seen as equals, equal to each other and equal to the organizers. Perhaps that is my misunderstanding. I'd really like to know what is meant by peer-to-peer, if I got the meaning wrong.

  Thank you to everyone for listening. 

  Cheers,

  Jennifer

• 
  Nate Otto   May 27, 2011, 3:20 a.m.
  In Reply To:   Jennifer Claro   May 27, 2011, 1:56 a.m.

  Jennifer, with respect,

  I'm sorry you feel decisions are being made without your or everybody's input and that my earlier comment today was part of that. I tried to reply from my phone at lunch today, but this site does not work very well from a mobile browser, and I ended up losing my comment into the ether.

  I think you're misinterpreting what Stian posted in this thread. What you took to be a unilateral decision, I think, was his attempt to react to feedback in this thread. P2PU marks a radical experiment in removing the hierarchy between teachers and students, and I think Stian is embodies this philosophy well. However, there are mechanical responsibilities that only course organizers can do, such as configure the badge tool (and it even seems like they need to ask a certain P2PU officer to set a badge up for them in this beta version of the software.) I wish he had configured the badges he and Monica envisioned before the course had even begun so we had a chance to build evidence toward them instead of not knowing what we should do with this space and trying to negotiate it ourselves. Now that we are here, however....

  As far as my personal opinion on badges go, I want to try out having a badge on my website that I can link up in my portfolio. That was part of why I signed up for the course; it was in the course description. I think it's an important part of how we are experimenting with CSCL ourselves through introducing ourselves to the field.

  I'd also like to move on from discussing the mechanics of the course so much, now that it's more than half over and focus on the subject matter.

  Stian: for the conversation Jennifer mentioned you were off the net for, see the wk3 chatlog and sort by the badges topic: http://ottonomy.net/temp/cscl-intro/interactive-chatlog-wk3.html
  (And while you're on that page, note the vast disparity between the amount of text devoted to "class mechanics" vs subject matter "stahl" + "CoPs") As I'm reading Suthers (2008), I think a lot of the activity we're seeing in the class is the "negotiation" phase of altering the course environment ("context") for different affordances, and that this may be interrupting actual learning or artifact-creation.

• 
  Joe Corneli   May 27, 2011, 3:51 a.m.
  In Reply To:   Jennifer Claro   May 27, 2011, 1:56 a.m.

  I think it's easy to see how a misunderstanding could arise between you and Stian: he missed two meetings in a row (because he was ill and/or traveling if I remember correctly).  So anything from those meetings that he "ignored" (your word) or failed to completely understand (more likely) was probably due to him not being there.

  Seriously, it's safer to assume that there's a garden variety misunderstanding here than to imagine you've just caught onto a major moral failing, which nevertheless is what it sounds like you feel!  I've tried to suggest above that by reasserting your equality, you have a perfect antidote (here) against the sort of authority that you so dislike and disapprove of.  Seeing and treating peers as equals is up to you as much as it is up to anyone else!

  Finally, I think there are tremendous benefits to be had from turning your protest into a question to explore.  It could be a very interesting one: what is the role of "authority" in peer courses or within P2PU?  Certainly such authority exists (there's an executive director of the organization, a board of directors, and each course has organizers), so how to proceed? 

• 
  Joe Corneli   May 27, 2011, 5:20 a.m.
  In Reply To:   Nate Otto   May 27, 2011, 3:20 a.m.

  Hi Nate:

  Thanks for connecting this conversation back to the readings!  

  As I'm reading Suthers (2008), I think a lot of the activity we're seeing in the class is the "negotiation" phase of altering the course environment ("context") for different affordances, and that this may be interrupting actual learning or artifact-creation.

  The view from paragogy is that "context is a decentered center", that, in fact, shaping the context is the one thing we're all working on.  I would agree that this can interrupt artifact creation (if we mean an artifact that's smaller than the context as a whole), but maybe it supports learning! (albeit in a somewhat different way from concretely discussing articles etc.).

  Check this quote from Suthers (p. 11): "[There was] a shift in the role of the graph representation from object of discourse in the face-to-face condition to medium of discourse in the CMC condition."

  OK, so we don't at present have a graph to look at, but we're definitely using text as a medium of discourse.  A lot!  Had we been meeting in person to discuss the articles, the medium would "get out of the way" and probably not even be noticeable.  (The fish doesn't see the water it's swimming in.)  But here, the medium is very noticeable (a new website, some new tools, patterns of communication with text).  The more practically-oriented our discussions are, the more rooted they will be in media (e.g. the medium of badges).  But to be practical, I think we should also frequently revisit and reassess our goals!  A keen awareness of media seems necessary, but not sufficient, for success in peer courses.

• 
  Rebecca Cober   May 27, 2011, 10:51 a.m.
  In Reply To:   Joe Corneli   May 26, 2011, 4:08 a.m.

  Hi Joe -

  I would be a fan of tagging and starring posts. I think if each person's P2PU personal page was a portfolio of sorts, linking to work that they have done in the course, that would be a very rich way of demonstrating participation and learning.

  It would be interesting to use KB principles to discuss badges - perhaps a debate graph (not used to debate, but to cluster and add to ideas) would work? Or we could try the "pass on" KB circle idea, where only one person speaks at a time, and we build on each others ideas.

  Another interesting point to consider might be, how do teachers who use KF in KB environments assess their students' learning? Do students assess peers? Do students assess themselves?

  Thanks for your interesting comments!

  Rebecca

• 
  Joe Corneli   May 27, 2011, 12:03 p.m.
  In Reply To:   Rebecca Cober   May 27, 2011, 10:51 a.m.

  Hi Rebecca:

  Thanks for the encouragement!  I just had an idea during my supervision meeting today, that people could indicate in their user profile what kind of feedback they're looking for - and this would make a certain set of tags available to peers.  So instead of just giving a "star"   there could be a drop down menu that would give all sorts of options: or--- with the available selection and the choice of meanings determined by the posting user (perhaps in addition to free-form tags).

  This way, people could quickly and easily get feedback on the particular dimensions they're interested in -- in other words, whatever they've specified as goals they're working on!

  I'd love to talk more about the idea of using KB ideas in this context, comparing this sort of tagging-based evaluation openmindedly with badges.  Sometime when you're not busy with (other) research maybe we can have a chat about it... or during the next group call when we're both there.

  Joe

• 
  Jennifer Claro   May 27, 2011, 4 p.m.
  In Reply To:   Nate Otto   May 27, 2011, 3:20 a.m.

  Thanks for your note. I agree, let's "move on from discussing the mechanics of the course so much, now that it's more than half over and focus on the subject matter".

• 
  Stian Haklev   May 28, 2011, 3:09 a.m.
  In Reply To:   Jennifer Claro   May 27, 2011, 4 p.m.

  Hi all, I was away for a while (spent 24 hours on a train, am in Hangzhou now for a Chinese conference on computers and learning). This has been an interesting discussion, although I have a feeling that we are still kind of "talking past each other". I think one of the biggest differences between my views and Jennifer's, is that I don't think we have to do everything as a group. I see this course as a platform - my initial idea was to have a minimum that we would all agree upon (and ideally - because we had prepared much of it before the course began - you would mostly agree with because you signed up for the course :)), which would include the core weekly readings etc. But beyond that, everyone will have very different interests and goals with this course, and this is fine. I personally wanted to try out one specific vision of badges, and I would set it up - as a normal course participant, and ask anyone interested to join me in trying it out. If Jennifer or Nate wanted a very different vision, nothing stopping them from sending an email to Erin, asking her to set them up, and trying them out... 

  Similarly, I have very little interest in a manifesto, or even a common document to come out of this course. However, I have absolutely nothing against it, and would willingly support anyone who wanted to work on it - including giving some time in the weekly call etc (which is what I proposed). But I don't want to have to be part of it, just because we took a vote or something. 

  So instead of sitting around arguing forever about what we should all reach consensus about doing as a group - why don't you all go off and DO the things you want to do? And invite others to join you? I'm not the only one who can create a task, or invite people to a group phone call - all of those tools are available to everyone! 

  And I agree that there has been too much talk about meta-issues. It's something we hadn't expected, so it's a very interesting lesson learnt. No idea how to deal with it though. We do want there to be flexibility and participation, but on the other hand, there is some value in people spending a lot of time before the course starts in setting up a structure, and then trusting them and being able to devote most of the course time to the actual readings etc. I know that's why I would often sign up for a course, rather than just doing the readings myself. 

  The readings and topics in CSCL are really rich, and we are just beginning to scrape the surface. I hope many of you were able to do some of this week's readings, and I hope to see more blog posts, more people engaging with the ideas others have posted, and a lively discussion tomorrow morning (for me)!

  So let me rephrase the badge question again: I am quite worn out from this long discussion, and I don't want to spend any more energy discussing it. I would (as a participant) choose to focus my time on the readings, and the discussions. If anyone else want to resurface the issue of badges, come up with a rubric for one, or four or whatever badges, I'm happy to forward that to Erin and have her set it up. 

  Stian

• 
  Joe Corneli   May 28, 2011, 7:39 a.m.
  In Reply To:   Stian Haklev   May 28, 2011, 3:09 a.m.

  I think it's interesting that there has been a lot of meta-discussion here, but almost no discussion in Shaping P2PU, where such discussion is welcome and unambiguously appropriate.  To me this points to the need for tags on new.p2pu.org.  Shaping P2PU could just become an aggregator for all meta discussions -- and someone who only wanted to focus on discussions about readings instead, say, could easily do that.  (For comparison, using the above link, look at how nicely the "tag" tag allows people to focus on tickets about tags in Lighthouse!)

  At the same time, I think meta-discussion about CSCL is appropriate for this course, although, per Stian, I think it's fine for people to focus on whatever they are interested in.  The problem is when the medium doesn't support "focusing" very well (e.g. think about how we read - or don't read - high-traffic mailing lists).

  For this reason, I think that "do what you will" is not a complete philosophy of computer-mediated communication.   

• 
  Marcy Murninghan   May 28, 2011, 5:07 p.m.
  In Reply To:   Stian Haklev   May 28, 2011, 3:09 a.m.

  Where are we supposed to go? I'm having trouble finding easily the PiratePad link, or, for that matter, an easy way of logging on where everyone is. I'm presuming we're meeing at 5.00 pm EST today (Sat)??

• 
  Stian Haklev   May 28, 2011, 5:17 p.m.
  In Reply To:   Marcy Murninghan   May 28, 2011, 5:07 p.m.

  Hi Marcy, we're here: http://piratepad.net/cscl-meeting5

• 
  Joe Corneli   May 31, 2011, 10:01 p.m.
  In Reply To:   Joe Corneli   May 27, 2011, 12:03 p.m.

  BTW, it seems like Gmail is a couple days behind me... admittedly, they have to implement whereas I was just brainstorming. 

  

  From http://gmailblog.blogspot.com/2011/05/3-labs-graduations-1-retirement.html.