Active learning template for reading assignments
This active learning template uses Perusall - the first innovative online social platform for collaboration at the University of Groningen (UG) - as an example of how to implement a social annotation tool in your course and as part of an active learning course design. The template was developed from a UG pilot project and aimed to provide teachers with an intensive guide in using social annotations tools in reading assignments.
- What is Perusall?
- Aligning with Learning Outcomes
- Course Activities
- Providing instruction, analyzing, and giving feedback to students
- Testing and Assessment
- Whom to contact?
What is Perusall?
Perusall helps every student prepare for every class. Students carry out assignments to annotate texts before class for a part of the course credit. Thus, they are invited to engage with your texts before the lecture. With better-prepared students, the time during the lecture can be used for practicing higher-order skills (creating, evaluating, analyzing) through active learning activities: discussions, tasks, and exercises.
Simply put, Perusall is a social system for tasks that are normally individually completed. Students annotate together, so studying becomes a social, rather than an isolated, activity. Some students may want to be the first to ask questions based on a text; others may like answering questions or responding to fellow students more. The lecturer sees the annotations beforehand so this makes it possible to select good annotations from students who may normally be too shy to speak up in class. By rewarding these annotations it is possible to encourage more students than usual to participate in class discussions. If this is the case, scoring and grading are done automatically thus saving teachers a lot of time. At the same time, Perusall helps make sense of the many annotations by providing a confusion report and overviews of the most upvoted comments and questions. This helps you as a teacher to focus on the most important issues that students come up with. Still curious about the tool itself? Take a look at the following video that can further explain what this looks like:
Aligning with Learning Outcomes
When planning for use, where should you begin? In many courses, studying literature is an essential part of the course but where does it fit into your overall course structure? What do you want your students to focus on with their annotations? You may want to ask students to compare the literature to other known approaches, evaluate the author's viewpoints, phrase sources, or reflect on an original text. It is important, then, to make sure all these goals fit within your overall course structure and more specifically, the intended learning outcomes you ask of students.
But don’t stop there. Students also need to know why they have to annotate and what is expected of their annotations. On what cognitive level should they focus? From the start, you as an instructor can give guiding principles to support your students’ skill development.
Perusall is based on the idea of having weekly assignments students complete before a lecture so that lecture time is used to discuss issues around the content. The assignments should be made for points (course grades) or rewarded in some other way (for example, as a bonus exam question). Without some form of reward, our experience is that students are less likely to carry out the task.
The weekly assignments consist of reading one or more texts and making annotations based on this text. Annotations consist of questions based on the text or responses to the other student questions. Perusall grades the student work automatically and provides an overview of all annotations in a confusion report, thus providing advice to the lecturer on what points are interesting for discussion. The lecturer reviews the annotations, checks and releases student grades, and organizes discussions.
Be sure to plan enough time for this; typically, we advise starting no later than an afternoon before the lecture starts and setting aside two hours of review time for two hours of lecture time. With more experience, this can be shortened to about one hour of review. To plan for the class session, use the confusion report, the most active discussion threads, and the most “upvoted” discussion thread.
The workflow normally looks like this:
Based on your particular course requirements, variations on this template are possible.
Providing instruction, analyzing, and giving feedback to students
Before students start annotating
The first step would be to make a list of the most important topics that you think should be addressed and how you can support them in their learning strategies. What do you need to see from students in order to prove they have reached the learning objectives? On what pages would you expect many annotations? Which steps would you like them to follow?
You can create guiding reading questions for your students that you use in the instructions of the assignments guiding their focus when reading the material. This type of instruction can be given prior to each assignment.
You can also guide collaboration and interaction. Before the first assignment, explain to your students that a Perusall assignment helps them better understand an article when preparing for class. This will make the class more relevant and interesting for them. Encourage them to interact, engage in dialogue, and use argumentation in their annotations. If possible, show them actual examples of what good discussion threads look like.
After students completed their annotations: confusion report
A confusion report is a summary of the questions of students. When the deadline for your assignment has passed and student annotations are gathered, click on the assignment, then click on “Confusion report” to generate the report. It is important to note that Perusall needs around 20 unanswered questions to generate the confusion report.
A helpful tool in the confusion report is to click on the little icon next to the student comments to copy a comment over to your (Powerpoint) presentation.
To view the confusion report, first, click on the assignment.
Now the button for the confusion report will become available and the report will appear once you click on it.
To copy an annotation, click the copy button next to the annotation.
At the bottom of the confusion report, you can find the buttons for printing or emailing the confusion report. Some of the analytics reports are computationally-intensive to generate, so they are updated once a day. The updated-at-time is shown at the bottom of the report.
Next, check whether you have missed big discussion topics (the number behind the conversation indicates the number of annotations) by clicking on XXX where you see the text and the number of Perusall comments.
The topics that you want to discuss, the topics from the confusion report, and the topics you have identified from the annotations together make a list of points to address during your synchronous activity or lecture.
After students completed their annotations: provide feedback and decide on in-lecture activities
After the assignment is finished you can discuss the major discussion points and provide students with feedback during class. By giving feedback on the annotations that were done by the students, you can tweak their responses to make sure they reach the level where you want them to be. Consider selecting the best student examples and use them as the basis for class discussion. You can give short presentations (in Powerpoint, possibly with a few annotations made by your students) to explain some content to your students. However, you can also think about how to activate your students by organizing small group discussions.
Also, don’t forget to give students feedback on how they interacted. Was it what you hoped for or do you want to encourage them to engage in interaction more? Use some good examples of student discussions you liked as a model for other students.
Testing and Assessment
To use Perusall in your course, it is recommended to reward students for their engagement by giving points for it, counting towards the total course mark. It needs some thought to make this work for your particular course.
Perusall grading within a course
First, check your course description in the online course catalog to consider what you have promised the students. It is important that you have written in Ocasys that students are graded on “assignments”, or better, that the Perusall assignments are described since Ocasys is legally binding.
Typically, a very small fraction of the total points for a course is enough to invite students to do their work in Perusall. The system is set up to reward studying, so when students make an effort to annotate, it is easy to reach the full points for it. The Perusall part of the grade is typically arranged to reward students for their effort and engagement--not to summatively evaluate students. Ideally, every student would get full credit for the Perusall part of their course grade! For instance, Perusall assignments count for 10 % of the course grade.
We suggest that you provide students with two documents relating to Perusall. First, you can download and modify a one-page rubric that explains what Perusall is, how it works, and how students are graded. (You may need to edit this document to reflect the scoring settings you have set for your course.) You can also provide students with a set of example annotations with associated quality scores and an explanation for each score to help them get a feel for what sorts of comments and questions they should be posting.
Automatic calculation of student scores
There are two options for grading students. In the default mode, students are graded on quantity, timeliness, distribution, and quality. Based on these criteria, individual scores for students are calculated. In practice, when students make an effort, they will receive full credit for the Perusall assignments. We advise only reviewing student scores that are marginal (say 4 to 6 out of 10). Scores below this number will be generally very poor; scores above this number will be fine. You can manually change the overall student score, or change the points given for a particular annotation.
The second option is to use holistic grading. In addition to the “Annotation content only”, the holistic scoring takes also into account specific behaviors. Research shows that the following behaviors on Perusall predict higher end-of-semester grades and long term mastery of the subject.
Contributing thoughtful questions and comments to the class discussion, spread throughout the entire reading (see some examples)
Starting the reading early
Breaking the reading into chunks (instead of trying to do it all at once)
Reading all the way to the end of the assigned reading
Posing thoughtful questions and comments that elicit responses from classmates
Answering questions from others
Upvoting thoughtful questions and helpful answers
In the grade book for the overall course, you can see the overview of the student grades per assignment. Typically, through feedback on their annotations, students learn what kind of annotations are expected, and thus their scores gradually increase over time.
How does automatic scoring work?
The quality of annotations is determined by a machine learning algorithm. Annotations are scored by a set of defining features. Via a process of repeatedly feeding the system data, and comparing it to data that is scored the weight (importance) of the defining features is determined.
To give an idea of how this works we can describe how Perusall was trained to deal with texts in Dutch. Three UG lecturers were asked to score a total of 1000 annotations on the same text. After scoring the first 100 annotations, they met up to discuss the differences among their grading in order to reach better inter-rater reliability. The algorithm was changed to reflect the agreement between the lecturers, after which the lecturers were asked to do another set of 100 annotations. This was done in 10 sets, hereby training the algorithm. To put this simply, we told the system what we score as good annotations, and the system learned what are defining features of our “good annotations”.
Whom to contact?
Biggs, J., & Tang, C. (2011). Teaching for Quality Learning at University. Maidenhead: McGraw-Hill.
Miller, K., Lukoff, B. King, G., & Mazur, E. (2018). Use of a Social Annotation Platform for Pre-Class Reading Assignments in a Flipped Introductory Physics Class. Frontiers in Higher Education 3, 1-12. https://doi.org/10.3389/feduc.2018.00008
|Last modified:||8 June 03:41 pm|