Feedback Patterns
Submission to the PPP pattern language project
Copyright
© the respective pattern authors.
Permission is granted for the purpose of EuroPLoP 2002
Version 2.7
Editors:
|
Jutta Eckstein
jeckstein@acm.org |
Joseph Bergin
berginf@pace.edu |
Helen Sharp
H.C.Sharp@open.ac.uk |
This pattern language in progress proposes some successful techniques to assist with teaching and learning. For professional educators, these patterns may seem obvious, even trivial, because they have used them so often. But for those newer to teaching, they offer a way for experienced teachers to pass on their experiences.
The pedagogical patterns project is working on collecting many types of patterns that can help teachers teach and students learn. This collection focuses on providing and obtaining feedback.
Patterns are not step-by-step recipes. Each of these offers a format and a process for recording knowledge that can then be used by a variety of different teachers in many different ways.
Most educators and trainers are not taught how to teach. Rather, they often find themselves teaching by accident. Typically, a person with a skill that is in demand, such as a particular programming language, will be asked to teach it. People assume that if the person is good in this programming language, she will be good at teaching it. But knowing the subject matter is very different from knowing how to teach it.
Effectively communicating complex technologies is often a struggle for information technology instructors. They may try various teaching strategies, but this trial and error process can be time-consuming and fraught with error. Advice is often sought from other “expert” instructors, but these individuals are not always readily available. This creates the need to find other ways to facilitate the sharing of teaching techniques between expert and novice teachers.
This is the goal of the Pedagogical Patterns Project (PPP). Pedagogy is a term that refers to the “systematized learning or instruction concerning principles and methods of teaching” [Web59]. Patterns provide a method for capturing and communicating knowledge such as pedagogy. As an example, imagine that you are looking for an effective way to teach message passing to experienced programmers in a week-long industry course. A friend who is teaching a semester-long object technology course to traditional age university students has found an effective technique. He shares it with you without dictating the specific implementation details. This allows you to use your own creativity to implement the technique in a way that is most comfortable for you and most useful for your industry students. This is the essence of patterns to offer a format and a process for sharing successful practices in a way that allows each practice to be used by a variety of people in many different ways.
A collection of patterns could form a repository of techniques for teaching a specific subject such as object technology (OT). Ideally, many of the patterns would have an even broader scope than OT, but all of them would be useful in many different training or learning environments because they are proven teaching techniques.
But even this is not the end of the story. Related patterns can be combined in either a pattern catalog [Bus96] or in a system of patterns [Fow97]. A third possibility is to relate several patterns within a common problem space, the result of which is a language of patterns that provides a resource for solving complex problems. The goal of the project described in this paper is to form pedagogical pattern languages for teaching. This will provide instructors with the ability to share their effective teaching techniques in a common format, to document relationships between the techniques and to form powerful tools known as pattern languages.
This pattern language under construction contains patterns from the Pedagogical Patterns effort [PPP], which were revised and rewritten in Alexandrian form in order to support the integration into a pattern language. The currently available patterns focus on a classroom situation at beginners to advanced level, but their usability is not limited to that. Further patterns will be submitted to future conferences of the PLoP series.
The patterns in this pattern language use a form similar to the one used by Alexander in his book A Pattern Language [CA]. All patterns are written in the you-form, thus directly talking to you, the teacher. In addition to the pattern name, each pattern is divided into several sections. The sections are separated by jjj. It starts by setting the context, which is followed by the forces and the problem in bold font. The next section outlines the solution in bold font including the consequences, limitations and disadvantages. The last section complements the discussion of the solution and it additionally provides examples in italic font as well as further information. References to patterns inside this pattern language are in Capital Letters, references to patterns published elsewhere are in normal font, but followed with the [pointer] to the reference section.
In addition, each pattern is marked with one or two asterisks (*), as in Alexander’s patterns. They show how fundamental we believe the pattern is.
Two asterisks denote patterns that state invariants. We believe that it is not possible to solve the stated problem properly, without referring to the solution that we have given. One asterisk means that we think that we are on the right track, but we believe it will be possible to improve the solution.
The following table lists some problems, which often occur in a teaching environment, and the respective patterns of this language, which address those problems.
|
You want to ensure that the participants understood the topic. |
Feedback, Differentiated Feedback, Try It Yourself, Kind of exam |
|
Participants might have understood the theory, but have never applied it. |
|
|
Participants don’t trust in their own knowledge. |
Explain It Yourself, Peer Feedback, Embrace Correction, Peer Grading, Student Online Portfolio |
|
You want to provide feedback. |
|
|
You want to make the participants less dependent on yourself. |
|
|
You want to ensure that participants learn from their own experience. |
|
|
You want to make the gained knowledge visible. |
|
|
Participants don’t know how to prepare for the exam. |
|
|
You want to ensure fair (individual) grading. |
Fair Grading, Fair Project Grading, Key Ideas Dominate Grading, Grade It Again Sam |
|
You want to grade teams fairly. |
|
|
You want to know if you and the course were useful for the students. |
This pattern was written by Astrid Fricke and Markus Voelter [VF] and revised by Joseph Bergin.
You are teaching a course or seminar. Some of the material is complex and you have students with a variety of abilities and they have varying degrees of attention to devote to your material.
jjj
Your students want to learn the content of the course or seminar. But until you exercise and challenge their understanding neither you nor they will know what level of understanding they have achieved. If they think that they have grasped a topic although they have not, they will not ask questions. They will also be unhappy later when they need what they did not learn.
jjj
Therefore, give the participants activities that exercise their knowledge and then give them feedback on their performance. The feedback should be differentiated and objective. Always start with Positive Feedback First. Criticism should always help the participants to improve the criticized aspect. Be sure to give the feedback in time, later feedback is not effective. Positive feedback can significantly increase motivation.
Feedback helps students learn where their understanding is faulty and thereby correct it.
Feedback takes many forms. Your words, spoken or written, as well as actions you take are all important (see Gold Star, for example). How you give Differentiated Feedback is also as important as the content.
Feedback is intended to foster the growth of the student, not to point out flaws and deficiencies, though that may be sometimes necessary. Feedback helps students understand where their understanding is faulty. Feedback is best when it helps the student want to overcome difficulties and is worst when it makes the student feel bad about him or herself.
jjj
One way of giving feedback is to give the participants exercises to work on; thereby they can see their learning progress. Explicit feedback by you is also possible, for example by asking and answering questions.
In the patterns community, members help one another improve papers (such as this one), using a structured process that always first speaks about what is best in the paper (that which should not be changed in a revision). This is followed by a period of constructive advice to the authors and is closed with additional positive comments. These sessions feel like sharing, not like attacks, even though some of the participants have high levels of ego.
The remaining patterns in this language refine, extend, and implement this one.
This pattern was written by Joseph Bergin.
You are giving Feedback to your student. You have a variety of things to say. You want your students to learn from the feedback you give them and treat it as part of learning.
jjj
If you are negative with your students they may tune you out and not listen. If they are especially sensitive they may be hurt. If they are especially arrogant they may take your comments as an attack and attack back. You want them to feel good about themselves and also to feel that they can do better.
jjj
Therefore, when you give feedback, start and end with positive feedback. Suggestions for improvement are sandwiched between these reinforcing comments.
Even if you have largely negative things to say, you can still start with the things that were well done and should be retained in the future.
Even in the less positive aspects of your feedback you can take a tone that you are giving suggestions for improvement, not just condemning. You can say “This might be made better if you think about …,” rather than “This is bad.” You can also say you don’t understand something, or something in a presentation doesn’t “work” for you.
Some of the other patterns in this language suggest to have your students giving feedback to each other. Make sure they know about this technique and practice it. This can make the giving of feedback easier for both the giver and the receiver.
jjj
The patterns community IS a community largely because we use this technique uniformly in analyzing each other’s work and giving feedback on it. It is a very powerful community builder.
This pattern was written by Joe Bergin.
You are giving Feedback to your students. You want to respect that students are individuals.
jjj
Your students learn differently and at different rates. They understand you with differing degrees of precision. They have different backgrounds that make it easier or harder for them to grasp certain topics.
In spite of all the difference, you want to teach all your students well.
jjj
Therefore, give differentiated feedback whenever possible. The feedback to a student is tailored to the needs of that student.
The best example of this is oral examinations, though these are labor intensive.
One of the worst examples is the correct answers posted after an exam. They may show the correct answers, but may not show the path a student should take to arrive at the best answer.
The best feedback helps the student move forward from where they are. It helps them get over their own misunderstandings or gaps of knowledge. A poor performing student might have been handicapped by Dr. Bad Professor in the past.
You need to analyze why they have answered as they have, but you can give them pointers to places in which to learn rather than specific answers, of course. Therefore this does not need to be overly burdensome.
jjj
Common feedback and answers to frequent questions can be put on a published Course FAQ. If you frequently get the same question you can easily add to your growing FAQ. This saves you time in which to give differentiated feedback.
Joe Bergin teaches a course with a large project (a compiler) graded only at the end. During the term each team submits an interim report every other week. On this report changes from the previous report are highlighted. At the next meeting these are returned with comments, but no grades. Most of the comments are just check marks, meaning OK. There is some short hand used, but mostly the feedback is individual and directed at the individual student.
This pattern was written by Joe Bergin.
You teach a difficult course. It has a lot of ideas and a lot of work for the students. The time is short and there is pressure on the students.
jjj
If your students fall behind or miss early material it will be difficult for them to catch up and therefore difficult to succeed.
jjj
Therefore, give them Early Warning when you see that they are headed for trouble.
This can take many forms. If your course has special pitfalls for the student, you can publish these on your course FAQ. For example, if you GRADE IT AGAIN, SAM, you can point out the trap of spending too much time on re-work of old papers rather than advancing on new.
Advice is best if it points a path to success, not just pointing out the roadblock. The earlier you give the advice, the better chance for success in the student.
jjj
It helps if you give frequent short exams and quickly return the marked papers. Some universities require exams in every course every Friday, for example.
You can speak to a student privately if you think she has potential above her performance. Joe Bergin vividly remembers a respected Professor's quiet word in the elevator, even after 35 years.
This pattern is based on Do You Remember, written by Jutta Eckstein [JE1] and on Lecture-Example-Activity-Student Presentation-Evaluation, written by Martin L. Barrett [MLB] and revised by Jutta Eckstein.
You teach complex material. You want to know if the students understood the topic and also to Build and Maintain Confidence [EMWM].
jjj
Because topics are complex, the students may be able to repeat definitions and other material verbatim without real understanding. They might also not be able to extract the key ideas from the supporting material.
jjj
Therefore, invite the students to express the key ideas using their own words. If a student uses her own words you will be better able to judge the level of real understanding.
You have to take into account, that this might be difficult for introverted people. Therefore take a look at Introvert Extrovert [JB1]. Consider letting them explain the topic by using Round Robin [EMWM]. You can also start to ask the students to explain the topic first to their neighbors.
Consider holding a short debate between student teams on the value of the key ideas.
jjj
Being able to explain a complex topic increases the students’ confidence in their own knowledge.
This pattern is based on In-Line Exercises, written by Kevin Morris Marler [KMM] and revised by Jutta Eckstein.
You have taught a new topic and you want to know if the students really grasp the content. Furthermore you would like the students to be able to evaluate their gained knowledge themselves.
jjj
You often have a difficult time knowing the degree of task competency during the presentation of a topic. Additionally students usually believe they understood the topic, but this is often only true in theory. As soon as they have to accomplish a task that is based on this new topic they realize their lack of understanding.
jjj
Therefore, take a break in the presentation and ask the students to perform an exercise that requires them to understand the new topic.
Since the exercise is inline it will provide immediate feedback to the student as well as let you see the state of understanding of your class. With the focus on providing feedback, keep the exercise simple, short, and to the point. Inline exercises additionally help to keep the students alert.
Besides providing feedback Try It Yourself also helps the student develop confidence in her own ability, as suggested by Build and Maintain Confidence [EMWM]. It furthermore encourages the students to ask questions based on problems that occur when performing the task. And finally it helps you fine-tune your teaching material and style, in synchronization with the capability of the students.
jjj
Try it Yourself can also be used for Experiential Learning [EMWM]. See especially One Concept Several Implementations, Abstraction Gravity, and Solution Before Abstraction.
Try it Yourself is also related to Explore for Yourself [BEMW], however Explore for Yourself focuses on exploring unknown terrain.
This pattern was revised by Joseph Bergin, based on Self Test by Christoph Steindl [CS].
You teach in a way in which you revisit each key idea several times, each time at a deeper level of understanding (Spiral [BEMW]). Or you teach material that depends on earlier material in important ways. Your students need to verify that they have the knowledge at the current level before going deeper.
jjj
Your students want to know if they understand the material. They need early feedback on their understanding and often the formal exam is too late and there is little time for recovery after the exam in any case.
If your students don’t understand what you have presented, they have a poor basis for moving forward. If you don’t understand what they really know, you have a poor basis for designing the next part of the lecture series.
jjj
Therefore let the students apply the theory by answering a self-test after they have heard the theory once before revisiting the theory another time or moving on to the next key ideas.
The students often feel that the theory presented in lectures is either trivial or hard to understand. In both cases they easily lose interest, and you lose their attention. When they are asked to apply the theory, they fail since they missed some critical points. You want to motivate the students to first listen more carefully during the presentation of the theory in the lecture, to let them apply the theory in the accompanying exercise, and to make them aware of the difficulties before the theory is applied in the exercise.
The self-test requires your feedback only when it reveals problems. It should not contribute toward the grade of the student. It is a learning exercise, not one for evaluation. Since it is informal, you can encourage students to explore their incorrect answers with you and each other. The self-test can lead to various team exercises for exploring the theory further.
Both you and your student benefit from Self Tests. The students get feedback on their understanding and you get feedback on the state of the class. This requires, however, that you see the results. While it is possible to make these entirely informal, perhaps with a web page, you need access to the results. One way that is very informal is to have it self administered with answers available immediately afterwards. The students can then seek your help for answers they don’t understand or more specific feedback.
jjj
The Self Test can be realized by using questionnaires. You must prepare the questionnaires and their answers. However, you can reuse the questionnaires to help make the examination at the end of the course. The students will appreciate that since they know the kind of questions. You can also test how much time the students need to answer the questions. You see which questions are difficult to answer or are ambiguous. You can use this feedback to make the final examination more predictable and reliable for the students and for yourself.
This pattern works best if you use a Spiral [BEMW] approach. Your Self Test at the end of one cycle is used to solidify knowledge before you go deeper into the theory in the next cycle. The repetition combined with the reinforcement of the Self Test deepens the knowledge.
This pattern was revised by Jutta Eckstein, based on Fernando Brieto e Abreu’s Peer Review and Corrective Maintenance [FBA] pattern.
The students have produced some artifacts and you want them to learn both how to improve their own artifact, and to help others improve. The artifacts might be developed via Try it Yourself. Furthermore you want to Build and Maintain Confidence [EMWM].
jjj
Typically people assume that learning involves receiving feedback, but this approach is a rather reactive way of learning and ignores the fact that students are knowledgeable too. However, the students are often not confident about the relevance of their experience and furthermore unsure about the value of their own knowledge.
jjj
Therefore, invite the students to evaluate the artifacts of their peers. The students will provide feedback to their peers by drawing on their own experience. Students often expect to learn from the teacher, but if you also want them to learn how to learn they have to be less dependent on the teacher.
As soon as the students have completed their artifacts they will pass the artifact on to the next student or group of students. Every receiver now has the task of understanding the artifact and to provide constructive feedback.
Carefully introduce good ways of providing feedback, to ensure that the feedback will be constructive and not destructive. See Positive Feedback First.
If the artifacts were produced in teams consider one team member accompanying the artifact as an agent. The agent can provide valuable insights for the review team. Give each team the chance to report to the whole group on what they have learned, when evaluating the artifact as well as what the agent has learned from the reviewers.
If the artifacts were produced by individuals instead, ask the students to pass the artifact around. Depending on the size of the whole group, either again give everybody a chance to report on what they have learned, or assign some time so the artifact producer and the reviewer can discuss what they have learned in a dialog. You can provide special feedback forms to facilitate this.
A variation on this pattern, Peer Grading, suggests that it may be appropriate for students to provide part of the grade for other students. This is especially useful in team projects. The portion of the overall grade provided by peers should be small and objectively assigned.
jjj
Students will learn by providing feedback to peers: through critiquing they learn from others' mistakes and insights. This enables them to also learn from their colleagues, which in turn draws the students’ attention away from the teacher. In addition they learn how to critique in a way that will further help their peers to improve their work.
You have to make sure, that the critique is always formulated in a positive way, so that the students who produced the work learn rather than feel offended. You can support the students accepting the feedback (and perhaps admitting mistakes) by admitting yourself that Nobody is Perfect [VF].
Furthermore allowing all participants to produce an artifact as well as to review an artifact reinforces self-confidence. Because even a student who has struggled producing the artifact will be able to give valuable feedback on an artifact produced by another student. This way every student learns that she can contribute something to the final production of an artifact.
For example, writers’ workshops at pattern conferences work in a similar way. While the author is present, a group of other pattern authors evaluate the work following strict evaluation rules. Because, typically, every reviewer is also an author, the whole group will also evaluate her work, with the first author now being in the role of the reviewer.
When the student moves later to the world of work she will be called on to critique the work of others, so this is good early practice under your guidance.
This pattern was revised by Jutta Eckstein, based on Fernando Brieto e Abreu’s Peer Review and Corrective Maintenance [FBA] pattern.
The students have received feedback on their artifact, for example via Peer Feedback.
jjj
Artifacts produced by students are evaluated as delivered, and graded as such. But it is also almost impossible for students to make it right the first time. Furthermore a lot can be learned by revising artifacts.
jjj
Therefore, give the students the chance to improve their artifacts.
Help the students to understand the feedback and allow some time for clarifications. Assist the students if they have problems translating the feedback into action.
Depending on the value of earlier versions of the artifact, you might have to help the student to set the focus right. Instead of focusing on eliminating defect, they should focus on meeting the requirements.
jjj
Of course, if you ask them to revise after grading then you should also Grade it again, Sam so that the grade can benefit from their extra work and improved understanding. You can avoid the extra step by providing the chance for improvement before any grading is done.
This pattern was written and revised by Joseph Bergin[JB2].
You want to provide a forum in which students can present their work to others and get feedback from a wide variety of sources. You realize that the work of prior students can help you teach the current group of students. Students need a way to present their best work -- to potential employers, to each other, and to their professors.
jjj
Your students need feedback from others as well as yourself. They can get excellent feedback from their peers if you can make it easy to obtain. There isn’t always an obvious way to make this happen.
jjj
Therefore provide a means for students to publish their best work, perhaps on the web. The more public this can be, the better it is.
In each course, your students publish one or two examples of their best work on the web, using separate pages. They also build an index of their pages, detailing the courses. This index page can take the form of a resume. Ideally, these pages should be freely available across the web.
The work to be presented can be selected either by yourself, or by the student (or a combination). One option is to have one assignment in each course that is created specifically for web presentation. Current students can use examples of work from prior years to emulate. Ask your students to comment on one another's work.
You will not be able to let your assignments become stale. You will need to update your assignments regularly. At a minimum, you will need one new assignment in each course each term, as students have access to prior year’s published work.
jjj
In arts and craft disciplines, students typically build portfolios over their professional lifetimes, beginning in the earliest courses. For example, students studying fine arts will build a portfolio with examples of their best work in different media--design, watercolor, etc. Computer science shares some of the features of the craft disciplines, so student portfolios may be valuable here as well.
Universities need a consistent way to assess students. This is becoming a requirement of most accrediting agencies.
There are definite advantages if the University can commit to keeping the pages active far beyond the student's graduation. The student advantage should be obvious, but the advantage to the University involves continuing contact with the student as they advance in the profession. Both the Alumni and Admissions offices may find use for the material.
As a side benefit, this pattern also helps reduce plagiarism, as public work is less likely to be copied, due to the increased likelihood of discovery. This is not its main benefit however.
Note: The recommendation is that each piece of student work be a separate page. This is to facilitate faculty links to those pages that you consider of special merit. The student is free, of course, to link back to the faculty page that "confers" the merit. See Gold Star. If there is a worry that students will change pages after faculty links are in place, then student pages need to be placed in a special directory to which the students do not have write/update access. The students can still create the pages, however.
For an example of use, see http://csis.pace.edu/~bergin/Java/groups.html. The course in question is described at http://csis.pace.edu/~bergin/StudentTasks.html. In this course, the project was designed for web presentation and this was required of all students.
Aside: Parents love the idea of their child's work being published.
One possible contraindication: For this to be really effective in the university setting, many professors need to adopt it. It is, however, even valuable in an industrial seminar if students need to produce a product that can benefit from peer evaluation.
This pattern is based on Gold Star, written by Joe Bergin [JB3] and revised by Helen Sharp.
You want to encourage excellent work and to praise a student for work well done.
jjj
Students want and need your praise. Praise can be a prime motivator, and students work best when they feel good about themselves and feel appreciated.
Normally the reward structure is private. In grading you give the student praise, but this loses the opportunity to show other students what you value most highly.
jjj
Therefore, when a student is doing well, or has done something well, praise them publicly for it.
Achievements you might consider rewarding include an especially good job on a project or a special contribution to a class, or to campus life in general. Give a public token of appreciation for work well done. As known by every elementary school teacher, this works for young children. It is often unexpected by young adults, and this alone can account for some of its effectiveness.
The token can be a few simple words spoken in public to the class or it can be an insertion into the student's permanent academic record.
If you are going to do this, it is important to be consistent, and never belittle a student for poor work.
jjj
There are a variety of ways of implementing this pattern. If a course is difficult, then you could try publishing "Gold, Silver and Bronze Medal" winners on the Internet. If your college policy allows you to insert a special note into a student's permanent file when they do something out of the ordinary, then this is worth pursuing. Students value such notes highly.
You might consider publishing the student's work on the course website, in which case this pattern is complementary to STUDENT ONLINE PORTFOLIOS.
At a lower level, you could keep to hand a handful of sticky gold stars (like those used for young children). These could be handed out to students in the class for special work, such as asking key questions. You could choose to make your reward very public or private.
Joe Bergin gives these sticky stars to students at all levels, including Doctoral students. He has noticed that he gives most stars for asking questions.
This pattern is based on Kind of Exam, written by Astrid Fricke and Markus Voelter [VF] and revised by Jutta Eckstein.
You want to examine the understanding gained by your students from Differentiated Feedback.
jjj
You have to examine the knowledge of your students, but not all topics can be examined equally. You have to examine every student uniformly, however every person benefits differently from the usage of her own sensory modality capability.
jjj
Therefore, use different kinds of exams. The exams should respect the Different Approaches [BEMW], which serve the students learning capabilities.
Exams can be written or oral. The written forms serve often better the visuals learners, whereas the oral exams are often preferred by auditory students.
You can differentiate among the participants by testing the skill or by testing how fast they can accomplish the exam.
You can use a project that has to be completed by individual participants or by a team. This approach respects those students with a dominant kinesthetic learning style.
Change the kind of exam occasionally in order to account for the different learning capabilities of the individuals.
jjj
If you want to also take team skills into account you should consider using One Grade for All or Fair Team Grading. To make sure of not focusing on specific skills, but on an overall understanding, Fair Project Grading could serve you well.
An oral exam allows you to respect the individual better, on the other hand with oral exams it is much more difficult to support Fair Grading, because the results are not as easy to prove and compare. Furthermore oral exams might be more challenging for introverted students [JB1].
Written exams have a long tradition in education. Multiple choice questions from a previously published catalog are often preferred because they don’t require a lot of effort from your perspective.
Students are often afraid of exams, you should try to eliminate this fear by using Mock Exams.
Not every exam needs to result in a grade. Often it is sufficient to know if the participant fails or passes, without more detailed information.
This pattern is based on Trial Exam, written by Astrid Fricke and Markus Voelter [VF] and revised by Jutta Eckstein.
You have to make examinations.
jjj
The participants are afraid of the exam and they repeatedly ask you what the scope of the exam will be. Although the students might have the required knowledge, they fear they might not know the topics to be examined. In order to have a meaningful examination you can’t tell them the exam in every detail, however you want to help the students to prepare themselves.
jjj
Therefore, give the students a chance to prepare for the exam by permitting them to take a trial exam. Be sure to ask the same kind of questions with the same difficulty level as in the real exam, otherwise you will hinder rather than help the student prepare for the real exam.
The trial exam could either be written during the class time or provided as a hand out for self preparation and can be either optional or required. If you give these in the classroom, you can also give hints to individuals as needed without disadvantaging other students. This can sometimes release a block in a student and let them progress.
jjj
A Mock Exam does not necessarily mean more work for you, because you can compile it from exercises or old exams and the participants can correct the trial exam on their own with the help of previously prepared solutions. You can also consider former exams as the Mock Exam.
As an additional help, you can give hints on the topics that should be repeated if a especially difficult trial exam question could not be answered successfully.
This pattern was written by Joe Bergin.
You have to give grades to your students. You want to respect that students are individuals.
jjj
You want to be fair in your grading to each individual student. You also want the students to be satisfied of your fairness and of their own accomplishments.
jjj
Therefore, publish your minimum grading standard and stick to it. Think of it as the minimum, however. Each student can do a bit better than the minimum without compromising fairness. Be generous, not miserly with your marks.
Think of your grading standard as a contract. If the students do such and so they will earn at least whatever. Grade so that it comes out a bit better. This has two benefits. You won't feel like you need to listen so much to complaints, since you have built in a cushion already. The students will also feel good about themselves and about you since they did a bit better than they, perhaps, expected.
You can be tough, but you must be fair. Moreover you must be perceived as fair.
jjj
It is worth looking at a grade distribution now and then and comparing it with other evidence you have about student learning. If your grades are lower than what you think justified by the real performance of your students, then you probably need to adjust: both generally and for the individuals involved.
This does not mean rewarding poor work. But this combined with other patterns here, such as Grade It Again Sam, can let you implement both high standards and high levels of student learning.
Remember that you NEVER need to withhold a reward from John to be fair to Mary. (Matthew: 20 in the Christian Bible)
This pattern was written and revised by Joseph Bergin [JB2].
You teach rich courses with a lot of ideas. Some of the ideas are key to the course and others support the key ideas and they have less importance. The difficulty of the idea isn't always directly related to its importance. Your exams attempt to cover most of the material. You also realize that different students will have different levels of understanding of the various topics.
jjj
If your grading scheme weights material according to its difficulty, or gives equal weight to all topics, you may be giving students the wrong impression about which topics are key. You will also be disadvantaging some students who, while not brilliant, understand the key ideas.
jjj
Therefore, the key ideas, not necessarily the hardest material, should be worth the most points in your grading.
This applies to all work, actually, not just exams.
Of course you want to challenge your best students. You can do this by asking difficult questions, but valuing them less than questions covering the key ideas.
jjj
A consequence of this is that you will be perceived as fair by most of your students. You may need to explain your scheme and the reasoning behind it to your best students, however. It is good to do so before hand, so that people know that much effort might be expended for few points. You can always also give a Gold Star for work that is exceptional even though it contributes little to the final grading.
This pattern was written by Joe Bergin Grade It Again Sam [JB2], and revised by Helen Sharp.
Your students have submitted some assignment work, you have graded it, and you want them to think again about the material, to learn from their mistakes as in Embrace Correction, and to increase their grade.
jjj
Everyone makes mistakes, and everyone can learn from their mistakes. An education or training situation should provide a safe environment in which students can make mistakes, and learn from them, but sometimes students fear they will suffer because of the grading structure. You have to grade assignments, but you also want students to learn from their mistakes.
jjj
Therefore, permit your students to change and re-submit an assignment for re-evaluation and re-grading, after you have graded it and provided feedback.
The new grade will be higher than the original but to discourage abuse of the system, you could charge a small 'penalty' so that a perfect score is not attainable. For example, if a student is sloppy he or she may hand in especially poor work, thinking that there is no risk. You might use a 10% penalty on the difference between the initial score and the re-graded score to discourage this. On the other hand, if a student is overly conscientious he may spend too much time on new editions to the detriment of other work. To help avoid this, you could withhold the last 3% of re-grade.
You may need to limit the number of re-evaluations allowed per assignment, for both your sake and the students'. If your class is larger than 30, with no teaching assistants, you may only be able to manage one re-evaluation, due to time constraints. If the potential for improvement is small, the student's time may be better spent elsewhere. The purpose of this pattern is to permit a student to spend additional effort on material with which they have special difficulty.
You should ask students to include all previous attempts at an assignment for each hand-in so that you can see why points were lost; having them all in a folder is useful. It also helps if you ask students to mark the changes with highlights or change bars.
jjj
As an alternative, you could permit your students to resubmit only their lowest grade work to date, rather than just dropping the lowest score, as is commonly done.
A key disadvantage to this pattern is that this approach is time and labor intensive, however students can benefit greatly from it.
This pattern helps you Reduce Risk[JB2]. This pattern can be used almost ubiquitously. The only exception is a course with a major project due at the end which is graded only once. For that situation see Fair Project Grading.
In Joe Bergin’s experience, there are seldom more than three editions, although he doesn't limit the number. Some of his students have grown tremendously with this approach. You end up working with the students who really need your help.
An alternative to this is to give students lots of time before the hand-in date of an assignment, and tell students that you will comment within 24 hours on any material handed in,. Students may get two or three rounds of feedback. Those who take advantage of the system will get a higher grade than those who don't, but they'll also work harder and learn more.
This pattern was written by Jeanine Meyer as Assigning and Grading (short) Team Projects [JM] and revised by Helen Sharp.
You have a short team project to mark, one that has lasted between one class session and three weeks, and does not form a substantial part of the overall course marks. You may have assigned the teams through Team Mixup [TBP].
jjj
You want all team members to benefit equally from the teamwork experience, in terms of grading and learning outcomes. But some team members often put in more work than others. Teamwork can deepen the learning of subject matter because the projects can be more substantial, but the benefits will only accrue to the individuals if each of them puts in an equal amount of effort.
jjj
Therefore, grade the team's work based on a presentation, which may be given by any member of the team. Choose the presenter on the day so that each member of the team has an equal stake in preparing for the presentation and will have prepared equally. Give each team member the same grade. This resembles the real-world situation. You can tell teams to divide up the tasks in any way they think is appropriate, but that everyone must understand everything.
Students will take responsibility for the inter-personal issues and project management issues. Students learn from each other. You may find that individual students demonstrate unexpected talents.
For a project longer than three weeks, look at Fair Project Grading.
jjj
This pattern has been used for a group project to produce web pages as part of an introductory computer information systems class, a group programming project and a group database design project.
This pattern was written and revised by Joe Bergin.
You teach a course in which students work in teams.
jjj
The grade of the individual depends on the work of the team. Different people contribute differently to the work of the team. You want to guard against the non contributing student benefiting in the grading from the work of others. You also want to help encourage students to be contributors, not just penalize those that are not. You need to make the grading fair to the whole team and to each individual.
jjj
Therefore, base part of the grade on the team product, but part of it on individual contributions.
The written artifact of the team may need to be graded as a whole with equal points given to each student, but you can also divide the project explicitly so that each student is responsible for a particular part. Even if this is not the case, you can have individual presentations of the work and grade these individually.
When the report is presented, you can ask for a summary sheet, signed by all members, that details the contributions of each. This can take the form of "All members worked equally on all parts." Or "John did ..., Mary did..." or whatever seems appropriate to the students. Make sure that they know this requirement at the beginning of the project, otherwise the students might believe that they have the opportunity to slack off and hide behind the other team members.
However: keep in mind that differences in kind of contribution may be interpreted by students as differences in quality. This may be entirely unjustified. For example, in a programming project, a person who does little programming, but keeps the other members coordinated by integrating the work and giving feedback, may be the most important team member.
You can ask questions about the team's artifact on an exam. Those that have done the most work will likely be the most familiar.
jjj
Peer Grading can be used to let the students themselves decide who has contributed most and have it reflected in the grade.
Fair Project Grading has suggestions for repetitive grading based on components in order to be fair, if the main artifact is a project.
If team members really disagree about a grade distribution, you can ask them to suggest one that they and you can all agree on. Ideally, let it be written down and signed by each member of the team.
Joe Bergin remembers one student in particular (though there have been others) with modest technical skill who would, he predicted, soon be the employer of the hotshot programmers in the class because she repeatedly asked the right question: Why are we doing this?
This pattern was written by Joseph Bergin.
You teach a course in which the students interact, either in teams or generally. Students in the course produce artifacts of various kinds and some of them are complex. Some of the artifacts depend on others. Poor quality in one part affects overall quality.
jjj
You want to teach your students how to evaluate quality and how to negotiate for it. You want to get them to accept evaluation by peers and to make this comfortable.
jjj
Therefore, make it possible for students to provide part of the grade for other students. The portion of the overall grade provided by peers should be small and objectively assigned.
One possibility is to design it in such a way that students are rewarded for good work, rather than just punished for bad. In other words, the points can be bonus points, perhaps. Another is to design the system so that all must volunteer in some way to earn the points. For example, a team with points to “pay” might need an artifact produced and would advertise the need. Other students or teams could then contract to perform the service for an agreed upon number of points needed for their own grades. Their actions result in earning points, not an abstract evaluation by others on unknown criteria.
This is especially useful in team projects where team members can evaluate each other. It can also be used in situations in which teams or individuals can provide services (artifacts) to other teams.
jjj
One way to do this is to give each student “points” that they must give to other students for services rendered on contract -- producing a subroutine, for example, or a service class in an object-oriented program. The contract can specify the number of points to be awarded, as well as due dates and quality constraints. You can provide special forms to facilitate this.
Students are reluctant to do this, of course. One way to help make it more acceptable is "musical chairs" (a game with one less chair than people). In a team of five, give each student two special forms that they fill in with their own name, the name of a student on the team and a short explanation of the special contribution of the other student. These are given to the student awarded the bonus and turned in with the work. Each award can affect the grade positively by a fixed amount. Even simpler is to ask each student in a team which two students contributed the most, with one vote possible for themself.
Another example of use is to give each team "point dollars" that they spend in an open market place. For example, in a software engineering course, a team must contract with other students to build its design. It pays with point dollars. Contracts, with possible penalties, are negotiated by the students. Students volunteer to build for other teams and thus earn points for their own grade. You may need to set up a contract resolution service if you use this, however [Stev01].
This pattern was written and revised by Joseph Bergin [JB2].
You give large projects to your students. Often the size of these implies that the evaluation of the project will have a large effect on the student's grade. You want grading to be both fair and based on overall effort and accomplishment.
jjj
Large projects are difficult to grade fairly. It will likely take you a long time to look at all of them and your grade on later ones should be consistent with the ones you grade early, though you may be fatigued at the end. If your grading scheme is too monolithic, some student or team may suffer from a particular error you were looking for in an otherwise good piece of work.
jjj
Therefore divide up the evaluation into different components, each of which will be given an independent grade. Determine the weight of each component prior to giving the assignment and publish the grading rubric. Preferably use more than seven components. This makes it less likely that your personal preferences or subconscious biases will affect anyone's grade more than a small amount. If you have too few categories you can possibly remember too much from one student’s work that carries over to the grading of the next.
Create a checklist of the individual components and their weights with room to write down the grade for each component. Leave room on this sheet for feedback that you can later give to the student.
Sometimes as you grade you learn things about your students that affects how you grade and you realize that some things have more importance than you gave them when you looked at and marked the first few papers. You want to try to minimize the effect of this for fairness, unless you are willing to make two passes over the work. The checklist helps minimize the effect of this. You can record on a master copy the decisions you made about grading early ones. This especially helps if you must grade over more than one session.
One of the most important characteristics of the successful professor is perceived fairness. Sometimes it is harder to convince yourself that you have been fair to each student than it is to convince him or her. In all or nothing grading schemes it is very difficult to maintain fairness as fatigue and even emotion often get in the way.
jjj
Joe Bergin teaches a course in which virtually the entire grade is based on a team project. To make the grading fair, he requires that the teams present their work at least twice and grades the individual presentations as well as the written work. The written work is graded more than once (midterm and final) and on different criteria for different components. Each piece is assigned a number of points up to a maximum known in advance. There is a total of 1000 points possible, divided into about seven categories. Each grading task is relatively small, and even if a group is penalized for some act or omission in one part, it will have only a small overall effect, thus guaranteeing that the overall grade depends most on the overall effort. In essence, to get a bad grade, requires consistently poor work even though the grading is all based on one project.
Another course has 70% of the course grade in one project and by the nature of the project it can be graded only at the end of the course. Therefore, individual grades are given for different parts of the project, with the breakdown known to the students in advance. A check sheet is used to compile the overall grade on each project with individual marks for each component. This has about 15 different parts. Most of the points are given for understanding the key ideas, not necessarily perfect completion. See Key Ideas Dominate Grading.
The number seven was mentioned above based on Miller’s “The Magical Number Seven, Plus or Minus Two”[GM]. You want enough different, independent, parts so that you don’t carry all of the information in your head as you go. It helps keep the grading of the parts independent.
This pattern is based on Participant’s Feedback, written by Astrid Fricke and Markus Voelter [VF] and revised by Jutta Eckstein.
You want to improve your way of teaching, both in terms of style and contents.
jjj
You believe you use a teaching style that enables learning. However you have just a one-sided view on your teaching style and you can never be sure how well this style is received by the students and how well this supports their requirements of a good learning environment. Successful exams and exercises are one way of receiving feedback if your teaching style is efficient, but still students might have only passed, because they asked for outside help.
jjj
Therefore, invite the participants to provide feedback on your teaching style.
You can establish an open discussion about your teaching style and efficiency, however the possibility of Anonymous Feedback leads often to more honest results. To get a comparable result, be sure to provide enough guidance for the participants by asking specific questions on your teaching style and supply some criteria so they can rate you on a given scale.
jjj
A very common technique is the feedback form that each participant can fill in. It keeps the participants anonymous if they want to be.
In order to not lose valuable feedback during a course and to enable course corrections along the way, ask the participants to provide feedback not only at the end of the course, but also during the course.
This is a rewriting and extension by Joseph Bergin of Anonymous Mailbox [VF].
You are teaching a course and you value the opinions of your students. You want your course to be as good as possible and to improve over time.
jjj
Often your students know things about your course that you do not. Sometimes they have definite opinions about the things you do, some positive and some not. If you don’t learn these things you can’t respond to them. In particular you cannot dispel misconceptions.
jjj
Therefore, provide an Anonymous Feedback channel with which your students can communicate with you. Encourage them to say whatever is on their mind.
The best Anonymous Feedback channel is a public one. This seems counterintuitive, but it actually works to your benefit when some students are dissatisfied. Students can and will defend you from the occasional attack that is not justified.
There is web technology that can be used to provide this. A form on a web page can be used to send you arbitrary information. An even better solution is a specialized web server called a wiki in which every page can be edited by every visitor. Here a student can post a comment, praise or complaint, at any time, and have it answered by others. Some online chat facilities can provide anonymous feedback.
A simpler technique is just to request anonymous messages from students in your regular mail, or provide a special box in which to place anonymous messages.
If students point out problems in your methodology it is, perhaps, best not to respond in words, but in actions, changing your techniques and materials. In particular, don’t adopt a defensive attitude in responding to suggestions. It will probably not help your image. However, if students ask you to do something that you know you can and should not do (make their lives easier, for example) you can devilishly point them to your Pedagogical Patterns, which explain in detail why it is good that you work them so hard.
Of course, most universities and many companies have a form in which the students can comment in a structured way on the course. Encourage written comments on these forms that go beyond the standard questions.
jjj
If you get a lot of negative feedback this way, you should rethink your presentation style. Perhaps you are not open enough to your students. Perhaps you need to improve your pedagogy. It can be humbling, but it can also be a powerful way to improve your teaching.
Joe Bergin uses this technique in each course (a wiki). He also uses a list server in which participants are not anonymous, however, a student can comment to the list from an unknown email address. One very dissatisfied student once posted a scathing attack on the course and on the professor’s teaching techniques. He didn’t have to respond to this as many other students came immediately to his defense, suggesting that the student had ignored clear instructions.
The original wiki is the cyberspace home of the patterns community. http://c2.com/cgi/wiki. It is driven by a set of Perl scripts. Others use Java or other languages to implement the web server.
The following patterns are not part of this language, but they are referred to by one or more patterns above. All of them can be found in [EMWM].
Abstraction Gravity [EMWM]
Concepts that must be understood at two levels of abstraction require time for a Spiral [BEMW] approach to learning. However this can be time consuming.
Therefore, introduce a concept at its highest level of abstraction and use reflection on the concept to link the higher-level abstraction to the lower one.
Build and Maintain Confidence [EMWM]
How do you challenge students to develop their own solutions? Students expect the one and only right solution to a problem from the instructor. But often there is no single answer, but many equally correct answers.
Therefore, present a problem taken, as in Solution Before Abstraction [EMWM] from the domain of the students. Provide some hints via questions that have to be answered and that may lead to a solution.
Explore for Yourself [BEMW]
A person’s success is based mainly on her ability to learn new concepts efficiently and to act as a team player by sharing knowledge and insights. You want to give your students the ability to learn in the future and to communicate their wisdom, but students are often afraid of taking responsibility for their own learning.
Therefore, assign topics to the students that they have to learn on their own and ask them to present the topic afterwards. It is helpful toprovide hints for resources related to the topic.
Different Approaches [BEMW]
Communication always takes place between a sender and a receiver, and the effectiveness of communication isn’t measured by what the sender says, but by what the receiver understands. Every person obtains information differently, using different sensory modalities. Some people, the visuals, learn most effective by watching; the auditory learners, by listening; and the kinesthetics learners, through action. Be aware: Not every student uses the same sensory modality as you!
Therefore, provide different approaches to the same topic. Accept different learning styles by addressing various sensory modalities. It might be difficult to provide different approaches for every single topic, but make sure to at least change the approach when you change the topic.
One Concept Several Implementations [EMWM]
An abstract concept is hard to understand without a concrete implementation or realization. However, teaching a theory using a concrete implementation might blur the concept itself, because the concrete implementation might not follow exactly the abstract model.
Therefore, use several different implementations of the concept as examples while teaching the abstract concept.
Reduce Risk [JB2]
You realize that students come to you to be educated, not examined. You realize that the purpose of a course is not to have the students prove to you that they don't need that course. You want to enable the students to take risks, but not to feel at risk for doing so.
Therefore take effective action to reduce your student's risk of course failure. For example, exams can count for a smaller part of their grade.
Round Robin [EMWM]
One of the most difficult aspects of teamwork is getting everyone in the room to work on equal footing. However, you want to get everyone's participation and input and you especially want to encourage the quieter members to take a more active role.
Therefore, use a round robin technique to solicit suggestions.
Solution Before Abstraction [EMWM]
An abstract concept can become the basis for a large number of applications. However, it is hardly considered useful unless it is related to concrete experience.
Therefore, give the students an example of the problem in a setting that they are comfortable with.
Spiral [BEMW]
You want to enable students to solve meaningful problems as early in the course as possible. Students learn best when they are doing things, and meaningful problems motivate them to work harder.
Therefore, organize the course to introduce topics to students without covering them completely at first viewing so that a number of topics can be introduced early and then used. The instructor can then return to each topic in turn, perhaps repeatedly, giving more of the information needed to master them.
Team Mixup [TBP]
When left to choose teams themselves, students will tend to stick with people who are similar to them, in terms of gender, age, ethnic background, skill level etc. It is more advantageous and more realistic to have a mixture of people in one team.
Therefore, you choose the teams. For class work sessions the group assignments must be done on the spot, and counting off numbers or pulling names from a hat will work.
We thank all those who have contributed to the Pedagogical Patterns project, especially the authors of the patterns we have revised and incorporated into this paper. We also thank our EuroPLoP 2002 shepherd, Michael Kircher, for helping us to improve the paper.
|
Joseph Bergin, Jutta Eckstein, Mary Lynn Manns, Eugene Wallingford. Patterns for Gaining Different Perspectives, Proceedings of PLoP 2001. |
|
|
Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., Stal, M. (1996). Pattern-Oriented Software Architecture: A System of Patterns. Chichester, England: John Wiley & Sons. |
|
|
Christopher Alexander et.al., A Pattern Language: Towns Buildings Construction. Oxford University Press 1977 |
|
|
Christoph Steindl, Self Test, Proceedings of EuroPLoP 2000, UKV Konstanz, 2001. |
|
|
Jutta Eckstein, Mary Lynn Manns, Eugene Wallingford, Klaus Marquardt. Patterns for Experiential Learning, Proceedings of EuroPLoP 2001. |
|
|
Fernando Brieto e Abreu, Peer Review and Corrective Maintenance, http://sol.info.unlp.edu.ar/ppp/pp14.htm |
|
|
Fowler, Martin (1997). Analysis Patterns. Reusable Object Models. Reading, MA: Addison-Wesley Longman, Inc. |
|
|
Joseph Bergin, Introvert Extrovert, http://csis.pace.edu/~bergin/patterns/introvertExtrovert.html |
|
|
Joseph Bergin, Student Online Portfolio, Fair Project Grading, Key Ideas Dominate Grading, http://csis.pace.edu/~bergin/patterns/coursepatternlanguage.html |
|
|
Joseph Bergin, Gold Star, http://csis.pace.edu/~bergin/PedPat1.3.html |
|
|
Jutta Eckstein. Learning to Teach and Learning to Learn, Proceedings of EuroPLoP 2000, UKV Konstanz, 2001. |
|
|
George Miller. http://www.well.com/user/smalin/miller.html |
|
|
Kevin Morris Marler, In-Line Exercises, http://sol.info.unlp.edu.ar/ppp/pp42.htm |
|
|
Martin L. Barrett, Lecture-Examples-Activity-Student Presentation-Evaluation, http://sol.info.unlp.edu.ar/ppp/pp3.htm |
|
|
K. Todd Stevens, Experiences Teaching Software Engineering for the First Time, ITiCSE 2001 Proceedings. Canterbury, England, June 2001, pg. 77 |
|
|
TBP |
To be published. |
|
Markus Voelter, Astrid Fricke, SEMINARS, http://www.voelter.de/seminars |
|
|
Webster’s New Collegiate Dictionary. (1959). G & C Merriam Co. |
Last Updated: May 7, 2002