New Report: Analysis of Student Backgrounds in Medical Neuroscience MOOC

In the Spring 2014 semester, Duke University offered the second session of Medical Neuroscience, a Massive Open Online Class (MOOC) available on the Coursera platform.  The class is taught by Leonard White, Associate Professor of Community & Family Medicine and Neurobiology.  Medical Neuroscience is a very challenging course; an in-person version of the course is taught to first year medical students.  This is considered to be a graduate-level course and students typically need some background in the topic.  The commitment required in this course is also significant; students reported spending an average of over 10 hours a week on the course, which lasted 12 weeks.

Because this is an advanced-level course, we wanted to evaluate whether students needed to have significant formal training in the subject matter in order to be successful learners.  One of the challenges associated with MOOCs is that, unlike in a traditional university-level class, students have a wide range of prior knowledge.  Some students are working professionals in the area, while others are completely new to the topic.  In order to design courses optimally, it is important to evaluate whether students are equally successful regardless of prior training.  To accomplish this, we compare how medical doctors and non-doctors performed in Medical Neuroscience.  The complete details of our analysis can be found in the full report, available HERE.

While we expected there to be significant differences between doctors and non-doctors in terms of their learning gains and course experiences, we found this was generally not the case.  The doctors and non-doctors were equally as likely to report learning gains and positive benefits from taking Medical Neuroscience.  We therefore turned to some of the qualitative data in the post-course surveys to understand why.  The most likely reason is that the non-doctors were already knowledgeable about the topic of neuroscience.  For example, we found that many non-doctors were taking the course as part of their formal medical education:

I’m a medical student, and went through our neurology course at the same time. Your videos and material were far superior, and I relied on them to an extraordinary amount, as did many in my class.

Taking this course in conjunction with undergraduate studies in medical sciences at a local university was incredible.

Furthermore, we found several students who, while they had no formal training in neuroscience, they had a passion for the subject and had previously self-trained in this area. One such student wrote:

This course finally gave me the neuroscience training I’ve wanted ever since I had a brain hemorrhage at age 12 and became completely fascinated by neuroscience. I became a violinist instead of a neurologist, but I’m having a blast becoming an armchair neurologist!

Download the full report HERE, or copy and paste the following URL in to your browser: dukespace.lib.duke.edu/dspace/handle/10161/8928

Using an Android Tablet with Active Stylus To Create Screencasts Easily and Inexpensively

henry greensideBy Professor Henry Greenside
Department of Physics, Duke University

Introduction: what is a screencast and what is a screencast good for?

The enormous success of Khan Academy‘s free online tutorials for students ranging from elementary school to college has made many educators aware of the benefits of using short (5-10 minute-long) screencasts to supplement or to flip a class, so that students spend more of their class time actively asking questions and solving problems, and less time listening passively to the instructor.

A screencast is a video file that displays in real-time what someone is writing on some screen, usually with an audio explanation of what is being written. Watching a screencast is similar to watching a video of an instructor writing on a board in front of a class but a screencast can be more effective pedagogically since a student sees only what is being written on the screen, and so is not distracted visually by details of the instructor or of the classroom. Screencasts can also be more effective than recording a PowerPoint-like presentation since a screencast allows a student to see step-by-step how an instructor assembles or creates complicated ideas, say by writing out and describing successive steps of some mathematical derivation, or by drawing some complicated diagram part by part, or by rewriting a sentence to be more grammatical and clear. In contrast, PowerPoint-like slides involve preassembled information, say a slide of several typeset equations or a slide containing a polished figure from a textbook, and it is then more difficult for student to understand the many details of such a slide since all the details are visually presented at the same time, or to reproduce some of the details by writing out the math or by drawing a figure on his or her own.

In this post, I discuss how one can create screencasts easily, inexpensively (less than $550 for all hardware and software as of the date of this post), and with good quality by using an Android tablet, specifically a Samsung Galaxy Note 10.1 2014 tablet, with an Android application called LectureNotes.

I learned to use this combination of software and hardware in the context of creating screencasts for my spring 2014 course Physics 162L (“Electricity, Magnetism, and Light”) of thirty students, and I found the screencasts to be a useful way to supplement the classes and the assigned textbook reading. I also found the screencasts to be a useful way to answer some student questions outside of class, e.g. it was quick and easy for me to answer some student questions by my creating and posting brief screencasts that could include drawing sketches or doing some mathematics; this was much better than trying to provide an answer verbally via email.

My hope is that this post will give Duke instructors a sense of how easy it is to create a screencast using current tablet technology and software, and so encourage them to create screencasts for their own classes. In a separate later post, I will discuss some of my thoughts about the educational aspects of using screencasts compared to other ways of making a class more active such as using clickers (which involve asking a class to discuss some question in groups and then poll all students anonymously to find out if the students have understood some key point) or by using pre-lecture online questions related to assigned reading.

Why I chose a Samsung tablet with an active stylus for screencasting

Before discussing the key steps required to create and share a screencast, I would like to discuss briefly what hardware and software options are available for creating screencasts and why I chose specifically a Samsung Galaxy Note 10.1 2014 tablet with a so-called active stylus.

The active stylus that comes with a Samsung Galaxy Note 10.1 2014 tablet. One writes with this on the tablet’s screen just as one would write with a pen on paper. Like many active styluses, this stylus comes has a single mouse-like button (the gray ribbed bar just right of center) that can be clicked to bring up menu options on the tablet’s screen that can be selected with the stylus. Although this stylus has internal circuitry to tell the tablet where the tip of the stylus is positioned, it does not use a battery and so one does not have to worry about the stylus becoming inactive.

Since screencasts involve recording what you write and draw on some touch-sensitive electronic screen, a key detail that determines the ease of creating a screencast and the quality of the screencast is what you use to write on the screen. The best current technology is to use a so-called “active stylus” which is a pencil-like object that contains circuitry that actively and continuously transmits to a tablet or computer the precise location of the tip of the stylus, even when the tip is not in contact with the screen. Active styluses have two benefits over “passive” styluses such as your fingertip, which communicate with a screen just by direct mechanical touches. First, an active stylus allows you to draw fine precise lines on the screen, basically with the same quality as writing on paper with a pen that has a fine point. Second, an active stylus lets you rest your hand on the surface of the tablet while you are writing, just as you would rest your hand on a sheet of paper while writing. (The active transmitter in the stylus allows the tablet to ignore any input to the screen except what is coming from the stylus.) This is an important detail since you can then write in a way that is comfortable for many minutes. With a passive stylus, you have to hold your hand above the screen while writing (so that the edge of your hand does not accidentally trigger some software command or cause unintended drawing), which is tiring and awkward when creating a screencast that is many minutes long.

Active styluses are also better for sketching since they transmit to the tablet or computer how firmly you are pressing down on the screen as you draw, which allows you to vary the thickness of your pen strokes smoothly and naturally, from thin to thick.

Active styluses are a more expensive technology and so most tablets reduce their costs by using passive styluses, most often a fingertip or you can purchase for $10 or so a pencil-like stylus that has a rather thick rubber-like ball that mimics the tip of a finger. Writing with a passive stylus is much like writing with a fat crayon while holding your hand in the air above a table, one simply cannot write or draw as clearly, quickly, or comfortably as one does with an active stylus. Passive styluses are especially awkward to use when writing a lot of mathematical symbols that require fine strokes since one is forced to write big letters to overcome the thick strokes, and so one can not fit much information on the screen and the writing does not look attractive.

So a simple conclusion based on my trying different hardware and software combinations for creating screencasts is that you should use an active stylus since that allows you to write with precision and to rest your hand comfortably on the tablet screen without side effects. At the time of this post, the best hardware solutions for creating screencasts are the Surface Pro tablets produced by Microsoft and several Android-based tablets, for which Samsung is arguably the market leader. The tablets sold by these companies come with an active stylus and have been designed to use their active stylus in a consistent useful way. For example, for the Surface Pro tablets and for the Android tablets, anywhere you can type in text with a keyboard you have the option to input text by writing the text with your stylus and having the tablet convert your handwriting to typed letters. (The conversion process is remarkably accurate, and both of these systems go a step further and can convert hand-drawn mathematical expressions of considerable complexity to a high-quality typeset form.)

The Android tablets with active styluses have the advantage over the Surface Pro products of being less expensive ($500 versus $1200), thinner (so easier to write on), more lightweight (1.2 pounds vs 2.2 pounds) and having superior and more rapidly improving screencasting software (the benefit of many Android vendors competing in a market of millions of users, versus a Surface Pro community of tens of thousands of people). But Android tablets have the disadvantage of being much less powerful than a Surface Pro tablet (which is basically a fully powered Windows 8.1 laptop disguised as a tablet), and so the Surface Pro tablet is the better choice for Duke instructors who, in addition to creating screencasts or as part of creating a screencast, need to run programs like Mathematica or Matlab (which are not available on Android or Apple tablets), who need to write and run substantial computer programs, or who need to run other kinds of software like spreadsheets, databases, or video editing programs.

Note: for those of you who already own an Apple iPad and who want to create screencasts, it is possible to do so using excellent applications like “Explain Everything”. But creating quality screencasts on an iPad is more difficult since Apple’s tablet operating system iOS currently does not support active styluses. This means you have to buy your own active stylus (and possibly purchase extra hardware to attach to your iPad so that the iPad can detect the active stylus), and it means that each stylus company and each iPad program are figuring out independently, and often inconsistently, how some vendor’s stylus will work with some other vendor’s program; it can well be the case that a stylus that works for one iPad program will not work well (or at all) with some other program. Also, you cannot input text via the stylus in any text box, the way you can for the non-Apple tablets that come with active styluses.

For my Physics 162L class, I was only interested in creating screencasts that would involve my drawing and writing so I did not need the greater power of the Surface Pro tablets. I also wanted to buy a tablet that I could use for academic needs when I was not creating screencasts (say for taking notes in seminars, for reading and annotating PDF files of journal articles, and for sharing research videos and plots with research colleagues) and so I wanted to avoid the substantially heavier and thicker Surface Pro tablet since I would sometimes be carrying a laptop and tablet around. After some trial and error, I found that a complete package for creating screencasts for my Physics 162L class could be put together as follows:

  1. A 32-GB Samsung Note 10.1 2014-edition tablet (current price about $530).
    Note: I found that one needs at least 32 GB of memory to store software and screencast videos, a tablet with 16 GB or less of storage will be impractical for screencasting.
  2. Three related Android applications called LectureNotes, LectureVideos, and PDFView, all made by the same company Acadoid Developer (total cost for the three applications is about $10).
    LectureNotes is actually the top Android program for taking handwritten notes but the author added video recording features that made the same program one of the best screencasting programs available on any tablet, Android, Apple, or Microsoft.
  3. An external microphone for recording my voice during a screencast.
    It turns out that the internal microphone of most tablets picks up the sounds of the stylus striking or rubbing against the surface and so one needs an external microphone to avoid these sounds. I found that a $3 Samsung microphone with earbuds (model EH64AVFWE, designed for use with a Samsung phone) recorded my voice as clearly as a $50 external microphone and so was a good way to record voice.
  4. A cloud storage account such as DropBox or Google Drive for storing files.
    Such accounts are free (at least for users with modest storage needs of say 5 gigabytes) and are needed as a practical way to transfer files from a tablet to a Duke computer or personal laptop, either for editing or for posting on the Internet for your class to download.

The total current cost of this hardware and software is just under $550 as of June 2014.

Preparing to create a screencast

Preparing to create a screencast is much more work than creating the screencast itself (see the next section “Creating a Screencast”). The preparation is rather similar to preparing to give a lecture that involves mainly writing on a board in that you need to organize your thoughts and to strategize about how to present your information visually a screen at a time. One substantial way that screencasts differ from giving a lecture is that you have to be much more careful about what you draw and what you say since the screencast captures all the details permanently. This means also that you need to practice giving the screencast several times in advance before creating a final version.

My experience, and the experience of many others including Salman Khan, is that it is better to create screencasts of at most ten minutes in duration (and preferably shorter, say 4-8 minutes long) that focus on one example or on one well defined point at a time. (Flipping an hour-long class then would correspond to creating 5-8 short screencasts that students can watch in sequence.) A screencast of 4-8 minutes corresponds to explaining the material of just two to three PowerPoint-like slides so I prepare for a screencast by writing out what I plan to include in the screencast on two to three blank sheets of paper (turned sideways with the long edges along the top and bottom, to match the preferred orientation of the tablet’s screen for recording video). I then place these sheets near the tablet, to remind me what to write on each screen and what to say.

Since a screencast is a standard video file (say in the MP4 video format), it is straightforward to edit the file to make modest corrections. But in practice, I found (as some experienced screencasters had suggested to me) that it is quicker and easier to just create a new screencast from scratch if some mistake is made. In fact, I found that to create a 4-8 minute long screencast consisting of 2-3 successive screens of information, it was easiest and best to record the screencast in roughly 2-minute long separate video files, say corresponding to one screen at a time, and then use the ability of the LectureNotes program to combine the short separate videos into a single final screencast file. This means that fixing some mistake typically involved just creating a new 2-minute recording, which was indeed easy and straightforward to do.

I should note that some screencasting software like Panopto (this is the software that Duke instructors have been using to create screencasts that involve using several different programs such as PowerPoint or Matlab) is able to generate bookmarks and a table of contents of the bookmarks for the video file so that students can quickly jump to different parts of a long video. But creating short successive screencasts serves the same purpose, and the name of the screencast file can indicate what is the content of the screencast. The LectureNotes screencasting software that I used and describe below has the ability to glue several successive short video files into a single longer file so you can always create one larger file from shorter files if need be.

Another requirement for preparing to create a screencast is to find a quiet environment (no background noises) where no one will interrupt you and you won’t be visually distracted. So you should tell your family members or friends not to interrupt you, close your office door, and silence all nearby phones. It is also helpful to have a large well-lit desk so you can spread out the written sheets that will serve as your screencast guide and so that you can write comfortably on the tablet with your stylus.

The final step is to prepare your tablet for recording your voice by plugging your external microphone into the headphone jack of the tablet and placing the microphone on the surface of your desk within a foot or so of your face and where you won’t bump the microphone’s cable accidentally.

How to create a screencast

Representative home screen of an Android-based Samsung Galaxy Note 10.1 2014 tablet

Now that you have organized your thoughts on several sheets of paper and practiced in your mind what you plan to write and what you plan to say during your screencast, you can start the recording of your screencast. When you turn on an Android tablet like the Samsung Galaxy Note, you get a home screen that might look something like the image to the right.

To start the LectureNotes program that will record and generate your screencast, you touch with the stylus or with your finger the LectureNotes icon on the left bottom of the screen, at which point the homescreen is replaced with the LectureNotes program.

Screen of the LectureNotes program for creating a screencast,with some of the most important icons labeled. Clicking the video camera icon on the far upper right causes the program to start recording all activity on the screen and all sounds from the microphone. Once touched, the video icon also displays the elapsed recorded time so you can see how long you have been talking.

To start recording the screencast, you touch with the stylus the video camera icon in the upper right, which starts blinking and displaying the elapsed time in minutes and seconds. You then start writing anywhere on the screen under the top icon bar while also talking into the microphone.

Creating a screencast using the LectureNotes program by writing with a stylus on the screen and speaking into an external microphone (not shown).

At any time, you can change the color or thickness of the line created by your stylus by touching the pen icon (this is the first icon to the right of the words 162/template) to bring up a menu of pen colors and thicknesses. If you make a mistake in writing or drawing something, you can undo recent pen strokes by clicking the undo icon (looks like left curved arrow) once or several times in a row. You can indicate a region of interest on the screen to the viewer by clicking the pointer icon, at which point a pointing arrow appears just under the tip of the stylus and this arrow moves around the screen without leaving any trace, just as if you were moving the dot of a laser pointer around a screen. To move to a new blank screen (similar to a blank slide) or to refer to some already created screen, you can move back and forth between screens by clicking on the right-pointing solid black triangle (third icon from the right), and on the the adjacent left-pointing triangle. (I don’t describe the use of the other icons shown since they are associated with features of the program related to note-taking or to annotating a PDF file rather than to creating a screencast.)

When you have finished creating your screencast, you touch the movie camera icon a second time which stops recording activity on the screen , stops recording audio, and saves all information into two files on the tablet, one file for the visual content of the screen and a separate file for the associated audio. You then need to go through a few more straightforward steps to create a final video file for your students to watch.

One next step is to tell the LectureNotes program to merge the two files into a single MP4 video file which is a useful format for displaying audiovisual information on the Internet. This is the one step for which using an Android tablet, even a high-end tablet like the Samsung Galaxy Note 10.1 2014, is painfully slow compared to using a Surface Pro tablet or a laptop. The conversion and merging of the files is such that, on this Samsung tablet, one minute of recording takes about one minute of processing time which means that a five-minute long screencast requires about five minutes to create a final MP4 file. The processing is so intense that the tablet is pretty much useless while the LectureNotes program is creating the final MP4 file, you basically need to do something else away from the tablet while the conversion and merging is taking place. I found in practice that it was about six times faster to first upload the video and audio files created by LectureNotes to my desktop computer and then run free public-domain programs like ffmpeg to create the MP4 file. (Please contact me if you would like to see how I do this using a so-called shell script to automate the process, the details are too technical to include here).

The final step is to upload the completed MP4 screencast file to a computer where you can then post it on the Internet for others to enjoy. One does this by touching (with stylus or finger) the “Share File” icon (second from the upper right in screen of LectureNotes above). LectureNotes then presents you with the option to select some cloud-based file system such as Dropbox or to email a copy of the file to yourself. You then tell LectureNotes (again by indicating with your stylus or finger) which file you wish to upload to Dropbox, at which point the file is uploaded via a wifi connection.

Screencast MP4 files created by LectureNotes turn out to be rather compact in size, about one megabyte of storage for each minute of recording if you use the default choices of screen resolution and of number of frames recorded per second (ten). So a five-minute recording leads to an approximately 5 megabyte MP4 screencast file, which takes just a few seconds to upload via Wifi if you have a good connection.

You can see some examples of screencasts that I created for my Physics 162L class.  I should point out that the quality of these videos is set mainly by my limited experience in creating screencasts and by my limited artistic abilities, and not by the capabilities of the LectureNotes software. That software has enough options that one can create HDTV-like crystal clear recordings but at the expense of requiring more processing time and larger video files. For the purposes of creating educational screencasts for my physics class, I found the default choices reasonable and effective.

Conclusions

Rapid evolution in tablet hardware and software over the last two years has made the creation of high quality screencasts easy and inexpensive for Duke instructors. Although I strongly recommend your using a tablet with built-in support for an active stylus, basically any tablet made within the last two years is capable of running easy-to-use inexpensive software that will produce effective screencasts for your classes. The most time consuming part of creating a screencast is the effort needed to prepare a lecture (collecting and organizing thoughts, thinking about how to present the material as several successive slides) rather than actually recording and editing a video file. Duke instructors should therefore simply go ahead and try creating and using screencasts to see if they may be helpful for their classes.

If you would like to see first hand how to use the Samsung tablet and LectureNotes software to create a screencast, please feel free to email me at hsg@phy.duke.edu and I will be glad to show you in person how to do this.

Acknowledgements

I would like to thank Andrea Novicki and Randy Riddle for providing an opportunity for Duke instructors like myself to learn about and to try different teaching approaches, and for providing support for me to purchase and to explore the use of the Samsung tablet for my Physics 162L course. I would also like to thank Prof. Steve Wallace of Duke’s Biomedical Engineering Department for several helpful discussions about how he has been using a Microsoft Surface Pro 2 to create online videos for his engineering classes.

Flipping the Duke Political Science Graduate “Math Camp”

One of the problems faced by many Ph.D. and M.A. programs in the social sciences is how to prepare graduate students for the advanced mathematical and statistical concepts they will be using as part of their research and methods classes.  Many graduate programs, including Duke, have a “math camp” to give them the necessary background to apply math concepts in ways they will be using them in their work.

However, students come to graduate studies with varying background and levels of interest in math.  With only a short time available to get the students started – Duke’s math camp is only two weeks – how can faculty use this time most effectively?

David Siegel, a faculty member in Political Science, has prepared an online, self-paced video course that students can use before they arrive at Duke to ensure they understand basic concepts and the practical use of math in social science research so that the two week math camp can be used for more in-depth work and customized help for incoming graduate students.

Screenshot of video by Siegel

Siegel prepared the video course using a check-out recording kit and consulting from the CIT over the past few months.  Students can use a pdf syllabus containing links to over one hundred short videos that explain how to apply basic mathematical operations in Social Science, recognize and avoid pitfalls in applying mathematics and statistics in research work and attain the basic mathematical literacy required for published research.  The online video lectures are accompanied by a textbook and online video problem sessions. Students can use exercises at the end of each textbook chapter and additional online problem sets for practice.

The course is being used by incoming Political Science graduate students this summer for a “flipped” version of Duke’s “math camp” when they arrive in the Fall.  In addition to making the materials available online for use by other political science departments, Siegel is reaching out to faculty in other disciplines in which graduate math training is not uniform to adapt the online materials to include examples for those disciplines.

Siegel will assess student reactions to the videos and problems sets and the effectiveness of the “flipped” math camp this Fall.

Duke faculty interested in flipping their courses and exploring creative ways to solve student learning problems can contact the Center for Instructional Technology to talk about ideas and campus resources available for their instruction with a consultant.

Learning Objectives in MOOCs

We are learning more and more about who enrolls in Massive Open Online Classes (MOOCs) and how those students behave.  For example, Harvard and MIT recently released de-identified data from their first 16 MOOCs that ran in 2012-2013 (read more about the Harvard and MIT data sets here and access the actual data here).  The data set includes several variables relating to student activities – for example, whether students visited the course website, watched videos, or completed exams.  These types of measures can tell us a lot about what students do, but it is not clear how much they learned as a result of those actions.

We were interested to find out how much students gained in specific learning objectives as a result of participating in a MOOC.  To do this, we asked students to rate their learning gains in five areas that roughly correspond to Bloom’s taxonomy of learning outcomes.  Bloom’s taxonomy is a classification of learning outcomes that includes both lower-level outcomes (remembering, understanding) and a progression towards higher-level outcomes (evaluating, creating).

6700283_orig

 

 

 

 

 

 

 

 

We sent a post-course survey to students in Dorian’s Canelas’ Introduction to Chemistry course asking them to indicate the extent to which the course contributed to their progress on five different learning objectives.  The survey was completed by 382 students, and it should be noted that this does not represent a random sample of students in the course.  Rather, the findings generated from this survey are indicative of what some of the more-engaged students experienced.

Of the students who completed the survey, 62% earned a Statement of Accomplishment.  However, even those who did not earn a Statement reported having a very positive experience in the course.  As shown below, when asked to rate their overall experience with the course, the overwhelming majority of all students rated it highly.

Slide03

The graph below shows student’s self-reported progress on the five learning objectives we asked about.  The percents reported are the percent of students who said that the course contributed “highly” or “very highly” to their progress (other options were “not at all”, “a little”, and “moderately”).

Slide12

As is typical in a traditional class, most students made significant progress on the lower-level outcomes of gaining knowledge and understanding basic concepts.  However, over half the students reported that they made progress with the higher-level outcomes of applying knowledge to other situations and synthesizing information.  Finally, 42% of students made progress learning to conduct their own inquiry, an objective at the top of the taxonomy.  This group does not include only those who completed the course and earned a Statement.  Of those who did not earn a Statement, 33% said that they made significant progress learning to conduct inquiry.

These numbers highlight the need to think carefully about how we define success in a MOOC.  It is increasingly clear that students who do not earn certificates at the end and who do not meet the traditional metrics of completion are still having meaningful engagements with the course material and accomplishing learning gains.

Online Teaching: New Skills for CIT’s Bass Online Apprentices

For the past 10 years, online learning has experienced consistent growth in the U.S. with the number of online students reaching 6.7 million in 2012 (Babson Research Group 2012 Survey of Online Learning). The rapid growth in online education, combined with new teaching models (such as MOOCs), and emerging instructional technologies is transforming the landscape of teaching and learning.

Recognizing the need to help students become knowledgeable in online college teaching, the Graduate School’s Bass Undergraduate Instructional Program, for PhD students, offers fellowships for Bass Online Apprentices (Bass OAs).  Dr. Hugh Crumley, Director of the Certificate in College Teaching, partnered with The Center for Instructional Technology (CIT) to provide online teaching and course development experience for Bass OAs.

Adrian Down, Ph.D. candidate in Ecology at Duke University, is a summer Bass OA with CIT, working with Dr. Mine Çetinkaya-Rundel on her course “Data Analysis and Statistical Inference.”

Adrian explains why he decided to become a Bass OA:

Adrianheadshot

Adrian Down

“I was interested in becoming a Bass OA because the field of online education is growing so rapidly.  Having experience with online education is a valuable skill set in many environments.  I’ve had several job interviews since joining the OA program, not all for academic teaching positions, and all of them have mentioned the online teaching experience.  It’s a skill set that is in demand and transferable across disciplines and professions.”

Adrian explains his role as helping to prepare the course by getting materials ready for  students and choosing a platform to deliver the course. Adrian has also migrated Dr. Rundel’s materials from her MOOC to a format that can be used later in the course.  He’s also responsible for preparing materials and having them ready for the students each week, including lab assignments and evaluations.

What has impressed Adrian about the use of instructional technology in teaching and learning?

“Technology can be a big help or a hindrance to the students, and we try to choose our technological platforms carefully to maximize the students’ learning experience.  We have to knit together many different tools (Sakai, Piazza, YouTube, Webex, RStudio, and others) to deliver all the content to the students, and keeping track of all of those resources, and coordinating them effectively can be a challenge.”

As Adrian is actively learning a new skill set for his future career, he is most surprised at the level of engagement made possible with online teaching.

“I don’t think online teaching is “inferior” to classroom instruction, it’s just different.  In a well-designed course, the instructor can actually have more contact with the student and can more precisely monitor students’ learning progress.  Effective online teaching can take more investment from the instructor, both in terms of course design and in adapting to new and rapidly developing technological platforms, but the rewards can be great for students.”

The Bass OA apprenticeship with CIT provides participants valuable insights into innovative teaching practices by working with Duke faculty, and gives apprentices a “behind-the-scenes” look at the work required to design an engaging online learning experience for Duke students. CIT is pleased to offer this apprenticeship, and for the Bass OAs, it’s an exciting time to prepare for online college teaching!

Read more about the Bass Online Apprenticeship with CIT:

Bass Online Apprentices Share the MOOC Student Experience

Preparing Future Faculty for Online College Teaching

Coursera Forums: Why Students Don’t Like To Have Graded Discussions

In several Coursera classes, including two from Duke University, course instructors and staff have attempted to motivate students to interact in the forums through a participation component to the final grade. In every instance I have observed, this choice resulted in a vocal backlash from some students against the idea. The response at times has been so heated that the course staff have completely removed the participation requirement, while in other courses the weight of the participation grade has been reduced. I’d like to present in this post some of the common objections to graded forums presented by MOOC students, as well as some data from forum participation in Duke courses.

Forums Will Be Unusable

By far the most common objection posed by students, and it appears in several flavors.

  1. High potential volume of posts. Sometimes students will note the numbers involved and follow the arithmetic to a natural, if inaccurate, result. For example, requiring a course with 20,000 enrolled students to each post 5 times throughout the course would theoretically result in 100,000 posts, which would indeed be a large number for such a course.
  2. Low signal-to-noise. Following on the volume argument, students will often comment that a participation grade based on the number of posts would result in a large fraction of the forum filled with meaningless posts, redundant questions and answers, and other minutiae.
  3. Wasted time. Adult learners, which comprise the majority of current MOOC users, are often very conscious of their available time. The combination of the above two arguments usually leads to student concerns about time wasted attempting to read through large forums clogged with irrelevant comments in an effort to contribute their own thoughts. In addition, students who are subscribed to course threads fear that they will be inundated with emails containing meaningless content.

The evidence from 30 Duke course sessions is that forums never reach hundreds of thousands of posts, even in courses with many active forum participants or a grade requirement for a high number of posts per student (See Figure 1).

Figure 1. Over a wide range of values for both volume of forum posts and number of forum users, Duke Coursera students consistently post an average of approximately 6 posts per forum user. This data may suggest that a student’s decision to post in the forums is not strongly impacted by the existing volume of forum posts.

Additionally, the average number of posts per student does not appear to correlate strongly with the number of course students or the total number of posts at the end of the course, perhaps suggesting that forum volume does not significantly affect students’ decision to interact in the forums.

ForumEngagement

Figure 2. The ordinate shows the fraction of students who accessed the course that posted at least once in the discussion forums. The abscissa marks the average number of posts per forum user. Despite the idea that both variables are possible measures of student engagement, there is no clear correlation. A few courses stand out as interesting cases: the purple triangle marks the only session of the English Composition course, which had an exceptionally high participation rate in the forums without a graded requirement. The dark red and green circles mark the two course sessions that had graded forum participation (Medical Neuroscience = red, Healthcare Innovation = green). Medical Neuroscience removed their forum grading component (36 posts/student over 12 weeks for 5% of the final grade) during their second session, which is identified by the light red circle.

Lowered Comment Quality

Related to the usability arguments, students will often complain that participation requirements based on the number of posts rather than their quality will result in frequent posts along the lines of ‘I agree’, ‘You’re right’, and ‘Thank you’. While these types of posts do exist in courses with graded participation, they also exist in those without any posting requirement. Qualitatively, it is not clear from our sample of courses that there is any significant increase of such posts when participation is graded, except in the ironic case of students who object to the requirement and are striving to make their point. A more rigorous analysis of discussion forum content would add clarity.

A couple existing options may increase the quality of forum comments that satisfy a participation requirement. First, the course staff can create a peer-evaluated assignment where each student would copy and paste their highest quality forum post(s) and grade them along a simple, preferably objective rubric. Second, Coursera’s platform currently allows the number of up-votes a student’s posts receive to be included in the grading formula. However, this second option can be affected by the voting culture of a course:  voting based on agreement with a comment rather than based on its quality within the discussion erodes the integrity of this grading approach. Third, there is a new forum grading feature that allows course staff to track the number of posts in each sub-forum. Potentially, students could be instructed to satisfy their forum participation requirement in more targeted ways, such as providing a quality response on a specific topic posed by the instructor and closely moderated by the course staff and community TAs.

Objection to Compulsory Interaction

A few students are concerned about interacting with other students on a social level, and they would prefer to have a course that does not require interaction. Several reasons may be given.

  1. Dislike social interaction. Some students specifically chose online education because they do not like the interaction and social aspects of in-person classes. This group is especially vocal in their rejection of forum participation.
  2. Weak language skills. Given the international nature of Coursera MOOCs, we do have some students who mention their difficulty with English as a reason why they do not wish to post in the forums.
  3. Privacy concerns. Some students are especially concerned about their privacy and protection of their identity. These students will point out that their only two options are to post under their name, or under a generic ‘Anonymous’ title that makes it difficult to understand who is participating in a conversation. Additional complications arise from a platform bug that does not allow a student to see their own anonymous posts on their personal forum summary page. This bug makes it difficult for a student who wishes to remain private to track their completion of the forum requirement. An ‘alias’ option is often suggested as a compromise, although this is not yet possible on the Coursera platform. However, students are able to edit their name into an alias, but are often not aware of this possibility.

An interesting and feasible suggestion that often crops up is to make forum participation a bonus to the final grade, rather than a basic component. In theory, this might make forum participation a rewarded effort rather than a required one. With the current means of grading on Coursera, this suggestion can be implemented.

Congratulations 2014 Trinity Teaching Award winners

CIT congratulates all of the 2014 Trinity College and Alumni Teaching Award winners. Once again this year, CIT co-sponsored the Teaching with Technology Award. This year’s award winner, Denise Comer (Thompson Writing Program) showcases excellence in teaching with technology through her recent delivery of a writing MOOC, contributions to various studies to learn more about pedagogical and technological impacts, and continued commitment to creatively and critically exploring innovative approaches to teaching.

Other award winners include:

We’re excited for all this year’s winners and looking forward to continuing to partner with them on new innovations and projects in the years to come!

Duke CIT Accepting Presentation Proposals for 2014 Showcase

Screen Shot 2014-05-06 at 1.39.12 PM

CIT is seeking presentation proposals for the 2014 Duke Center for Instructional Technology Showcase.  The Showcase, which will be held Monday, October 13 (Fall Break) at the Washington Duke Inn & Golf Club (click for a map), features innovative pedagogical approaches from across Duke’s vibrant teaching and learning community.

To submit a proposal, please complete this short form.

Examples of past presentations can be found on the CIT Showcase website.

Registration, which is free of charge, is also currently open for individuals who wish to attend the Showcase.  Click here to register.

Questions may be directed to cit@duke.edu or (919)660-5806.

MOOC completion and course activities

There has been a lot of discussion about MOOC completion, most of which has focused on completion rates: what percent of people complete a MOOC, and how should we calculate that number?  However, what has drawn less attention, but is potentially more interesting, is what in-course activities impact completion.  Understanding whether or not different course elements in a MOOC affect completion can potentially help us better understand best practices in MOOC design.

We used data from three of Duke’s MOOCs that recently concluded – The History and Future of (Mostly) Higher Education, Introduction to Genetics and Evolution, and Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital – all three of which began in January and ended in March.  A total of 4,973 students successfully completed one of these three classes.

We asked students to complete a survey after the course ended.  A total of 1,979 students across the three courses completed a survey.  Note that this includes both students who received a Statement of Accomplishment and those who did not.  The surveys are roughly split 50/50 between those who completed the course and those who did not.

We were interested to find out what course activities increased the likelihood that someone would finish the course.  We asked students whether they participated in various activities – things like completing optional readings, participating in forums, or editing a course wiki.  The full list of activities we asked about is shown in the table below.  We then ran a statistical model predicting whether a student had completed a course based on their involvement in each of the activities.

Slide1

We found three especially interesting things in our analysis.  First, “Watch lectures” is omitted from most of the models.  This is because watching the lectures perfectly predicts completion in two of the three classes.  Unsurprisingly, no one completed the courses without watching the lectures in either Genetics of Image and Video Processing.  What is interesting is that some students in the Higher Education course did complete the course without watching lectures.  This is likely because the Higher Education course focused on delivering content in a variety of new and innovative ways using different mediums.  Because the “watch lectures” variable is not significant in that model, we believe this suggests that content can be effectively delivered in MOOCs in ways other than only through video lectures.

Second, we find that viewing the course wiki in the Higher Education course was significantly associated with completion.  Students who used the course wiki were 75% more likely to complete the course.  This may be because this course had a very extensive and active course wiki; 73% of survey respondents viewed the wiki at least once.  Our analysis indicates that participating in a student group in this course is associated with a lower likelihood of completion.  However, this is likely not a robust finding since less than 10% of the sample reported participating in a study group.

Third, there is a significant relationship between forum activity and completion, but only in the Genetics class.  In that class, students who participated in the forums were twice as likely to complete the course.  This could be due to the level of difficulty in this course; students who were actively discussing the material with other students may have learned more.  The survey respondents from the Genetics class were especially active in the forums; 46% reported some forum activity.

Note – while taking quizzes is obviously the strongest predictor of course completion, this is essentially a non-finding because the quizzes are the main graded elements of the courses.  Therefore, completing the course means, by definition, that students completed the quizzes.  We kept the variable in the analysis simply to confirm that the analysis was accurately modeling course completion.

Bass Online Apprentices Share the MOOC Student Experience

Massive open online courses (MOOCs) offer the potential for a student learning experience that is collaborative, engaging and global. What is distinctive about the student experience in a massive open online course (MOOC)? What do participants find most challenging about learning in a MOOC, and how has the MOOC experience impacted their learning?

To find out, I asked two students who are a part of the Duke Graduate School Bass Instructional Fellowship Program, for PhD students, led by Dr. Hugh Crumley, Director of the Certificate in College Teaching.  As part of the Bass Online Apprenticeship (OA) fellowship, students enroll in GS 762 Online College Teaching, and participate in an online apprenticeship with the Center for Instructional Technology (CIT), to gain practical experience working with Duke faculty and CIT consultants in online education. One of their course assignments is to experience learning and observe instruction in a massive open online environment, with a class peer, and at the end of the semester reflect and share their MOOC experiences.

 Keri HamiltonKeri Hamilton, a PhD student in Biochemistry, Biophysics and Molecular Biology, and a Bss OA Fellow, signed up for two MOOCs offered by Coursera: Duke‘s Introduction to Genetics and Evolution, and Human Evolution: Past and Future offered by University of Wisconsin-Madison.  In Genetics and Evolution, Keri participated in Google Hangouts on Air, which she found enhanced her learning:

“Being able to record the hangouts allowed students to view them later, and the hangouts had a Q & A feature that allowed students viewing the interactions to ask questions and participate. I really enjoyed this feature, and thought it was a good way to interact with students in real time on a broad level.”

In terms of what positively impacted the user experience, Keri commented on the forums, which made it enjoyable to interact with many different students, read their opinions, and watch collaboration on problems sets or fundamental ideas.  “It made the class feel more interactive,” she said. Keri sums up her student experience:

“Participating in MOOCs has shown me that you can learn new things even in fields that you didn’t think you could take classes in.  It lets you continue to broaden your horizon past your job and research specialties. Take a MOOC! The worst that can happen is you learn something!”

Bass OA Fellow Giuseppe Prigiotti, a Phd Candidate in Romance Studies (Italian), enrolled in the Coursera Duke PrigiottiMOOC: The History and Future of (Mostly) Higher Education.  For Giuseppe, the MOOC experience was “a unique opportunity to envision the future of college education, constructing effective paths to twist online and on the ground learning.” Giuseppe benefited most from the peer assessments. “Writing these three essays, I was obliged to rethink course materials in light of my personal perspective. I want to question my idea and practice of education. I have had many chances to teach in the last 14 years, but I still like to learn, unlearn, and relearn.”

Giuseppe’s commentary captures one of the many benefits of learning in a MOOC –the opportunity to experience innovation and consider the pedagogical possibilities.  Of special note is Giuseppe’s comment on the significance of the Bass OA fellowship, and the important experience it provides:

“The new Bass Online Apprentice Fellowship has been the starting point to discover MOOCs, and that may be beneficial for my future work in academia, as a professor of Italian Culture — hopefully!”

With the added experience of enrolling in a MOOC, reflecting on the experience and how it has impacted their own learning, these Duke Bass OAs (online apprentices) will have a well-rounded set of online teaching skills to add to their fellowship experience, as well as practical, applied experience in the realm of online teaching and learning.