To get started in this course, send an email message to Greg Wilson to say 'hi'; he'll send you back these three chapters from the book Making Software:

1. Mark Guzdial: "Why Is It So Hard to Learn to Program?"
2. Michele A. Whitecraft and Wendy M. Williams: "Why Aren't More Women in Computer Science?"
3. Andrew Begel and Beth Simon: "Novice Professionals: Recent Graduates in a First Software Engineering Job"

Please also have a look at this post on the Software Carpentry blog, which compares Greg's attempt to teach online with these research-based best practices. For your first task, write a blog post of your own that compares what you do (or what's been done to you :-) with those best practices, then add a comment below with a link to your blog post.

Thanks,

Greg

Links (extracted from people's comments)

• Gregory Brown
• Julie Pichon
• Jaelle
• MrSteve (and further thoughts)
• Duane Griffin (and further thoughts)
• Laura B
• Terri
• Leopoldo Teixeira
• Ethan White
• Neil Chue Hong
• Eric G
• Andrew Cox
• Heather Payne
• Luis Pedro Coelho
• rglmrn
• Kerri
• Darlene Flores

1. Comments by Greg Wilson are in italics with the initials 'GW'; please add lots of your own.
2. Some ideas for things people could tackle as their third task are gathered together at the bottom; please add more.

Common Threads

One theme came up repeatedly is that these recommendations are made with a traditional (battery-farmed) learning model in mind, and it's hard sometimes to relate them to free-range learning. In particular, we typically have brief or intermittent contact with our learners through one-day workshops, drop-in lessons spaced over a few weeks, or self-paced online learning. GW hasn't found any work (yet) on what teaching and learning practices work best in those situations—who'd like to go looking for some as their third task in this course?

Greg Wilson

• Background
  • Teaching scientists how to program (and do other things with computers) for 14 years.
  • Mostly crash courses (9-5, five days straight)
• Space learning over time.
  • Nope: it's mostly been contract teaching, so we cram.
  • Can see that people's brains are full around the middle of day 3.
  • Shifting to two-day workshops + six weeks of online follow-up.
• Interleave worked examples with problems.
  • Don't do it.
  • Doubt students would do the exercises: most are surfing for solutions to specific problems of their own rather than diving deep.
• Use graphics.
  • Yes, in slides.
• Connect concrete and abstract.
  • Try to, but no idea how to tell if we succeed.
• Use pre-questions to introduce a new topic.
  • Nope. On reflection, I think most learners would find this frustrating.
• Use quizzes to re-expose students to key content.
  • Don't; should; doubt students would do them.
• Teach students to use delayed judgments to identify content that needs further study.
  • Can't.
• Use quizzes to identify content that needs to be learned.
  • Don't.
• Ask deep explanatory questions.
  • Don't.

Terri Ko

• Background
  • Didn't absorb as much when first using a computer for note-taking—had to re-train herself to read & comprehend what she was typing.
  • Has taught in university classrooms.
• Space learning over time.
  • Mentees pretty much "learn and do".
  • Don't come back to subjects much (that she notices).
• Interleave worked examples with problems.
  • Yes in the classroom, not so much as a mentor.
  • Tends to throw Google Summer of Code students in at the deep end.
• Use graphics.
  • Classroom yes; almost never as a mentor.
  • Web doesn't currently make this easy.
• Connect concrete and abstract.
  • Try, but it's hard.
  • Game dev students seem best at it—not sure what conclusion to draw.
• Use pre-questions to introduce a new topic.
  • Rarely in classroom, occasionally when mentoring (to get an idea of learners' background knowledge).
• Use quizzes to re-expose students to key content.
  • Yes in class, never with mentees (but do ask them to write up learning summaries periodically).
• Teach students to use delayed judgments to identify content that needs further study.
  • Nope (and some classroom learners seem upset by the idea).
• Use quizzes to identify content that needs to be learned.
  • Yes in the classroom
    • GW: is it used for assessment, or to identify content that needs to be learned?
  • No with free-range learners.
• Ask deep explanatory questions.
  • Occasionally in the classroom, but there it's mostly the students doing the asking.
• Reflections
  • Many of these ideas would feel forced outside the classroom.
  • But gamification does seem to work (with some people, in some contexts).
  • EPW: gamificaiton has worked to keep me engaged in things online and I've often wondered if Software Carpentry couldn't improve on "interleaving" and "rexposing through quizes" by having a weekly problem sent out to a list with an online system for checking the answer and recording how many a user has answered successfully like Project Euler (http://projecteuler.net).

Neil Chue Hong

• Background
  • Teaching scientists and engineers (typically at the graduate or post-grad level).
• Space learning over time.
  • Learners are typically working at 200% of capacity, so there's usually no time for training at all, much less for revisiting things.
• Interleave worked examples with problems.
  • Always.
• Use graphics.
  • Lots (though coming up with a good visualization is often hard).
  • JP: Using flow charts to add a visual element is a great idea!
• Connect concrete and abstract.
  • Skipped.
• Use quizzing to promote learning.
  • Yes.
• Help students allocate study time efficiently.
  • As above, this is very hard.
  • Helps to take learners out of their usual environment.
• Ask deep explanatory questions.
  • Not clear what the "deep questions" are in software development.
  • GW: would you like to work with other course participants to try to compile some for your third task?
• Reflections
  • Would like to try using quizzes more.
  • Main focus has to be on finding time.

Mr Steve

• Space learning over time.
  • Not good at this.
• Interleave worked examples with problems.
  • Have thought about giving students examples to read, but have rarely done it.
  • Instead, ask students who are writing good code to share their work with others.
  • Mentions "40 mathematical shapes" problem (turtle geometry).
  • Also thinking about Etoys Challenge: give students a fixed set of tiles for building programs, ask them to solve a particular task.
  • Quotes Minsky: "You don't really know something unless you know it multiple ways."
• Use graphics.
  • Yup (though more with math than programming).
• Connect concrete and abstract.
  • Can only think of one example.

Luis Pedro Coelho

• Background
  • Teaching programming to scientists
• Space learning over time.
  • Did well with Python programming
  • Poorly with other things like version control—only came up once.
• Interleave worked examples with problems.
  • Did some (mostly reviewing homework exercises in class).
• Use graphics.
  • Not much.
• Connect concrete and abstract.
  • Tried to—had both a lecture and a practical class, and tried to focus more on concepts in the former and examples in the latter.
    • GW: But what the recommendations are suggesting is "make the connections explicitly and frequently".
• Use quizzing to promote learning.
  • Multiple choice homework every week, ungraded in-class quiz by show of hands at the end of week 1.
  • Thinks it worked well.
  • GW: But was this assessment, or to aid learning?
• Help students allocate study time efficiently.
  • Nope.
• Ask deep explanatory questions.
  • Refrained from doing this—was focusing on programming skill, rather than computational thinking.

Leopoldo Teixeira

• Background
  • Interested in teaching at the NGO where he volunteers.
  • Not much experience teaching, so will discuss experience as a learner.
• Space learning over time.
  • Rarely done in class, but did it on his own before exams.
• Interleave worked examples with problems.
  • Used in most programming classes he took, but sometimes there wasn't time to solve the problems in class.
  • And students often skip classes devoted to problem solving.
  • Echoes GW's experience: problem-solving tutorials at U of Toronto are mostly empty.
  • EPW: I combine presentation of material and problem-solving together during the same periods and have typically had near %100 attendence.
• Use graphics.
  • Agrees with the recommendation, but has seen a lot of bad slides in his life :-)
• Connect concrete and abstract.
  • His research group recommends doing a concrete example before introducing the abstract concept.
  • Doesn't comment on how well this has worked for him.
• Use quizzing to promote learning.
  • Hasn't seen it at all.
• Help students allocate study time efficiently.
  • Not in general, though some teachers did help him specifically.
  • But as per #2, students often don't seem interested.
• Ask deep explanatory questions.
  • Has been asked to reflect on the big picture
  • GW: Not clear if this is the same as instructor explicitly asking questions that required "deep answers".

Jaelle

• Background
  • Thoughts as both learner and facilitator.
• Space learning over time.
  • Difficult in free-range setting where students can drop in and out.
  • Remind them using RSS, etc.?
  • Has noticed that websites aimed at helping people learn new life skills often use reminders in various forms.
• Interleave worked examples with problems.
  • Can offer examples, but no guarantee people will work through them.
• Use graphics.
  • As a learner, usually prefer text, but want visuals to help with difficult content.
• Connect concrete and abstract.
  • Seems straightforward.
  • GW: do you regularly do this, and/or has it regularly been done for you by your instructors?
• Use quizzing to promote learning.
  • Difficult in a free-range setting.
• Help students allocate study time efficiently.
  • Haven't seen any good suggestions to ensuring that students follow up.
• Ask deep explanatory questions.
  • Can be asked, but no way to ensure students follow up.
• Reflections
  • Facilitators' inability to require students to do something in free-range setting undermines many of these recommendations.

Julie Pichon

• Background
  • Wondered if "free-range" specifically means "online".
    • GW answered "No, but many people doing free-range learning now are online."
  • Teaches sporadically as a volunteer, usually to groups of 6-10.
• Space learning over time.
  • Difficult in a 5-6 week course, but most concepts build on each other.
• Interleave worked examples with problems.
  • Regularly shows students short working programs, ask them to figure out what they do.
  • Is this review rather than worked examples?
  • Thinks alternating is important (and thinks she should go into more detail than she does).
• Use graphics.
  • Doesn't really do this—is usually projecting code.
  • GW: which raises an interesting question: does the visual nature of programming environments like Scratch help learning as much as diagrams?
• Connect concrete and abstract.
  • Not doing, or not doing well—students have trouble re-using ideas in new contexts.
  • EPW: I like this way of evaluating whether you're being successful with this or not. I do it a lot in class, but now that you mention this criteria for success I'd say that 1/3 of my students have a very difficult time re-using ideas in new contexts, which suggests that I definitely have more work to do.
• Use pre-questions to introduce a new topic.
  • Doesn't do this.
  • Wonders if previewing an unknown programming concept would lead to confusion.
• Use quizzes to re-expose students to key content.
  • Doesn't do this either (though asking students to figure out what small programs do might count as quizzes?)
• Teach students to use delayed judgments to identify content that needs further study.
  • Nope.
• Use quizzes to identify content that needs to be learned.
  • Doesn't do this—considering that most students don't take time to do homework, unlikely it would work.
• Ask deep explanatory questions.
  • Only does this a little bit.

Andrew Cox

• Space learning over time.
  • Hadn't considered doing this, but many skills build on each other.
  • Thinks it's often enough to know that a particular tool or library exists (then go find the docs).
  • Notes that "...few studies have examined acquisition of complex bodies of structured information."
• Interleave worked examples with problems.
  • Points out that one of the problems with learning to program is knowing if your code is well-formed, and thinks examples help with this.
• Use graphics.
  • Speculates that videos may have the same beneficial effect as graphical diagrams.
• Connect concrete and abstract.
  • Doesn't really know how to connect this.
• Use quizzing to promote learning.
  • Likes the idea, will give it a try
  • But how do we validate that the students' work is correct?
  • Just checking output isn't good enough.
• Help students allocate study time efficiently.
  • "...the key takeaway for me."
  • Likes the idea of having students rate how much they know about the question before answering it.
• Ask deep explanatory questions.
  • Thinks that having students write blog posts answering deep questions will work.
• Reflections
  • Do classroom methods apply to learning software development? Absolutely.
  • How does learning programming differ from learning history?
    • Most problems are unique.
    • Wide range of correct answers.
    • Need to be aware of a broad range of technologies as well as fundamentals.
    • GW: I actually think most historians would claim all these are true of their field :-)
  • Hardest thing about online learning is validating what the student has actually done or learned.
  • Andrew's OK with this: it's up to them to learn.

Duane Griffin

• Quotes Greg Wilson's observations ("students won't revisit material" and "no practical way to assess")
• Thinks student motivation is the key to the first.
• Thinks peer assessment might help with the second (mentions "a suitably modded MeetBot")
• On the teaching side, has always used unstructured mentoring.

Ethan White

• Background
  • "Figuring out how to teach well as a professor at a research university is largely a self-study affair."
  • Currently teaching programming and data management to biologists.
  • Read stuff in a textbook or watch some online videos before class.
  • 5-10 minutes of review.
  • 45 minutes on exercises.
• Space learning over time.
  • Doing OK, but not systematic.
• Interleave worked examples with problems.
  • Doing well.
  • Using Online Python Tutor.
• Use graphics.
  • Yes, and Online Python Tutor does graphical representations of running programs.
• Connect concrete and abstract.
  • Yes.
• Use quizzing to promote learning.
  • Not convinced this makes sense in this context.
  • Does have one mid-way quiz intended to provide delayed re-exposure.
  • EPW: To be clear, I'm not convinced that using pre-quizzing to introduce new topics makes sense in this context. I think quizzing to re-expose students is a good idea and I'd like to try doing more of it than the single mid-way quiz.
• Help students allocate study time efficiently.
  • Yes, mostly for students who ask questions.
  • "By looking at their code and talking to them it typically becomes clear where the 'illusion of knowing' is creeping in and causing them problems and I think I do a fairly good job of breaking that cycle and helping them focus on what they still need to learn."
• Ask deep explanatory questions.
  • Gives students a large individual project as a capstone.
  • GW: This is not quite the same as explicitly asking questions that require deep explanatory answers—I wonder if it has the same effect?
  • EPW: Yes, I agree. I guess like Luis Pedro Coelho my feeling is that the concept of "deep explanatory questions" is a little strange in a skills course. As such, the analog for me of being pushed to think deeply about a concept is being pushed to tackle problems that: 1) aren't defined for you; and 2) are more complex than any sort of homework problem.

Heather Payne

• Background
  • Teaching people (primarily women, and soon girls) how to program in a free-range setting.
• Space learning over time.
  • Run workshops twice a month, priced accessibly, so that people can come frequently.
  • Hit the same fundamental ideas in different contexts, which sort of meets this criterion.
• Interleave worked examples with problems.
  • Yes: alternate 15-minute "theory" explanations with 15-30 minute practical work.
• Use graphics.
  • We have slides.
• Reflections
  • Think it's more important to teach in a way that inspires passion.
  • Use an inspiring space and build a community around what you're doing.
  • GW: I don't think anyone would argue for the contrary :-)
  • "I wonder what would happen if we spent more time thinking about the things that are often ignored: things like people and space and community and fun and creativity and a-ha moments."
  • GW: would you like to go look for an answer to that question as your third task, and share your findings with us?

Gregory Brown

• Background
  • Mendicant U's courses are entirely project-based and discourse-oriented.
  • Expect 15-20 hr/week for 3 weeks.
• Space learning over time.
  • Doesn't really fit MU's format.
  • Running follow-up courses probably isn't practical.
• Interleave worked examples with problems.
  • Hard to provide those with open-ended projects.
  • "...many of the problems students are trying to solve fall outside of the nicely packaged set of examples that we've been compiling over time. This is to some extent the nature of the course..."
• Use graphics.
  • MU courses are entirely text (email, IRC, etc.)—would like to improve on this.
  • Have to consider limited/intermittent connectivity, etc.
• Connect concrete and abstract.
  • We do it into both directions.
  • Would be happy to share thoughts.
• Use pre-questions to introduce a new topic.
  • "We provide a review of entrance exam problems for each of our students, and through that process, we do get an opportunity to talk with students about things...that will resurface during our sessions. However, our entrance exams aren't especially representative of the format and contents of our courses."
• Use quizzes to re-expose students to key content.
  • Have informal reviews rather than formal quizzes.
  • Students can always revisit old reviews, but even over three weeks, it seems like students are only able to keep track of a limited amount of information.
• Teach students to use delayed judgments to identify content that needs further study.
  • Hard to do (largely as a result of the compressed timeline).
• Use quizzes to identify content that needs to be learned.
  • "To the extent that we don't use 'tests'...it's because we have the time and resources to do even better...The only discomfort I have with that claim is that I haven't yet found a way to evaluate/measure whether our approach is objectively better than formal testing for the goals that we are trying to accomplish."
• Ask deep explanatory questions.
  • This is the whole basis of our approach.
• Reflections
  • Hard to apply IES recommendations because our philosophy is so different.
  • Hope to find other ideas through this course.

Eric Gric

• Limited, unsystematic experience as teacher (one-on-one or one-on-few)
• Discusses the mentoring model at Mendicant University.
• "Course completion rate has increased as we have more closely integrated the entrance exam and pre-course mentoring with the core course."
• Difficulty applying many recommendations because of one-off nature of Mendicant encounters.
• Interleaving worked examples and problem-solving exercises fits his intutions.

RGLMRN

• A university student, developing challenges for the Trisfera P2PU course.
• Reflections based primarily on experience as a learner.

Kerri

• Is providing an after-school webmaking class
• Background is web development, not education
• Space learning over time: had 13 weeks with about 1.5 hours week
• Interleave worked examples: "Our students were presented with a short presentation at the beginning of every sessions and were asked to find examples of what they were shown."
• Graphics: not much.
• Quizzing: "I noticed that the students would warm up and be ready for new concepts once they spent 10 minutes trying to answer something on their own."
• Connect abstract and concrete: related concepts to real life (seemed most appropriate with high school students).
• Ask deep explanatory questions: again, relate to real life.

Ideas for Task #3

• What teaching and learning practices have been shown to work well in our free-range learning contexts?
• What are the "deep questions" in programming that we should be raising and answering?
• What effect does inspiring passion, a cool environment, etc., have on what and how much people actually learn?
• Please add more here.