Effective Quiz Practices in Moodle - Part Three: Tips for Creating Questions

This is the 3rd article in a series related to using the Moodle Quiz Tool.

Tips for Creating Effective Quiz Questions

Assessments are critical to teaching because they allow instructors to evaluate if students have met the course objectives, and to identify areas in the course where improvements need to be made. The Moodle quiz activity is an essential tool for assessment. This article lists tips for writing effective quiz questions.

General Suggestions

• Follow the three-step process for creating quizzes: First, create or upload from a textbook quiz bank all of the questions to the question bank. Second, add the quiz activity to your course with the settings for timing, feedback, randomization of answers, etc. Last, add the questions from the question bank into the quiz you just created. Creating the quizzes in this manner will make it easier to reuse questions in the future.
• Make sure your questions align with the level of Bloom's taxonomy set in your course objectives. This will help avoid making questions that are too hard or too easy.
• Organize your questions in the question bank into meaningful categories. You might group them by chapter, topic, or learning objective depending on the types of quizzes you give.
• Naming questions: Be consistent with your question names so that it will be easier to locate questions later. The question bank organizes questions first by type and then alphabetically. Only you see the question name so be descriptive enough that you can tell what the question is about without having to preview it.
• Shuffle the order that answers appear (this is the Moodle default setting). You may have a tendency to place correct answers in the same position, but you can allow Moodle to automatically reorder these for you. You can also enable this in the question behavior setting for the quiz in the "shuffle within questions" setting.
• Moodle allows you to add images into both questions and answers. The trick is to first save the image to your computer and then upload it into your question and/or answer with the text editor tool.
• Weight your questions in the Quiz. You don't need to match the total points to the maximum grade of the Quiz - you can let Moodle scale it for you. For example, multiple-choice questions might be weighted as one point, and the weight for matching questions would equal the number of matching items (i.e. 5 items to match, a weight of 5 for the question).
• Avoid tricky questions. You don't want to confuse your students. If students are consistently missing a question, then evaluate it and find out why.

Feedback Tips

• General feedback: If you are going to use a question for a low-stakes test you might want to provide information on where a student would find the correct answer. When the student reviews the quiz, they will be given this information or if you turn on the option for deferred feedback in the quiz settings, this information will available during the quiz.
• Feedback for incorrect responses: If you are entering feedback for incorrect responses, provide specific reasons for why an answer is incorrect.

Matching Questions

• Avoid having too many items to match in one question. This can be overwhelming for students when presented with too many items at once in the drop-down answer menu. It can also cause unnecessary scrolling, which can affect usability. A general guideline would be about 4 to 6 matching items per question.
• Avoid having long answers within matching questions because that is what is placed into the drop-down answer choice menu. This can make it hard for students to read when trying to match the terms. In Moodle, the correct answer and distracters should go into the Answer area and the matching item should go into the Question area. You might consider reversing the two for readability. For example, if you want students to match terms to their definitions, then it would be best to write the definitions in the Question area and the terms in the Answer area.

Multiple-Choice Questions

•  For multiple-choice questions with multiple correct answers, make sure that you give distracters negative point values so that students are penalized when selecting an incorrect response. If you do not, then students could select all answers and receive full credit even though they selected an incorrect response.
• You can use the multiple-choice question type for fill-in-the-blank questions, but with choices instead of requiring the entry of a short answer (which creates problems for automatic grading). If you decide to create a fill-in-the-blank question, always use a standard number of underscores to indicate the blank so that the length of the line does not give any indication of how long the answer should be. You can also use two fill-in-the-blanks in a sentence, but avoid using a blank at the beginning of the sentence. Instead, have the question stem appear first.
• Avoid using the option "All of the above" when using randomization. 
• Make the length of distracters similar to that of the correct answer. The correct answer is typically longer.
• Avoid creating question distracters that are obviously incorrect. Well-written distracters should be plausible - this is one of the most challenging parts of question writing.

True/False Questions

• Avoid using the words "only", "never", and "always" within questions - especially True/False statements.
• Avoid using too many True/False questions. Make sure that you are assessing the intended level of learning.

Essay Questions

• Remember that essay questions require manual grading. If the answer will require more than a few sentences, you may want to evaluate the question in an Assignment instead using the Online text submission type.

Coming next: Effective Quiz Practices in Moodle - Part Four: Reports

This information is from the Moodlerooms.com resources blog: Best Practices: 30 Tips for Creating Quiz Questions which can be viewed at: http://www.moodlerooms.com/resources/blog/best-practices-30-tips-creating-quiz-questions-0

Effective Quiz Practices in Moodle - Part Two: Quiz Security and Cheating

This is the 2nd article in a series related to using the Moodle Quiz Tool.

Quiz Security and Cheating

Online testing presents opportunities for students to be creative in their attempts to "game the system" when it comes to taking quizzes. Most online quizzes are meant to be taken outside of class and for that reason the instructor has limited control over the students' conduct during a quiz. Students can download the questions and print them out, take pictures of questions with their mobile device and send them to other students, have other students take the test for them, use their notes and textbooks during the quiz, and the list goes on and on!

The anonymity of the online environment may open up new avenues for cheaters, but it's not really much different than your face-to-face classes. A few people will go to great lengths to cheat, but most students will be honest as long as it's not too easy to get away with cheating. A few creative strategies can be employed to eliminate most of the easy cheats and to make cheating more trouble than it's worth for students. Read on for a few strategies for countering cheating schemes.

Printing and Sharing Questions

• If you have the quiz enabled to display feedback and correct answers, students can print the results page from their web browser or take a picture of it and share it with others. Or students can print the quiz questions directly from the quiz while it is in progress (by using the print feature in the web browser or screen capture). According to Moodle, the key to discouraging this behavior is to randomize the question order and the answer order. This makes printouts a lot less useful because students will have to look through the printout for the corresponding questions and pay attention to what the correct answers are instead of just the letters corresponding to the answers.
• Another strategy is to create larger question banks and use subsets of questions and let Moodle randomly choose questions from each subset. With this randomizing it is less likely that students will be delivered the same test and it will discourage them from attempting to share tests.
• In the quiz settings, under layout choose to have every question delivered on a new page. When this setting is enabled, the student would have to print a separate page for each quiz question which will use up their time limit on the quiz and also deplete the balance on their student print account.

Using the Textbook During the Quiz

• During an unsupervised quiz, students have the opportunity to look up the answers in the textbook or their notes. There are ways to make the textbook and notes less directly useful. Timed quizzes are the single most effective tool for eliminating the temptation to use the textbook. A timed quiz requires students to complete the quiz in a certain amount of time. If you add enough question to the quiz and make the time short enough, students won't have time to look up every answer. Moodle.org recommends about 30 seconds per multiple-choice question.
• Designing questions that require students to synthesize and apply information will also discourage them from trying to look up answers. These types of questions require that students understand the material and be able to apply it creatively to answer the question. So while students may still take the time to look at the book, they will need to understand what they have read to successfully answer the question.

Working with Friends

If your students are on the same campus, they may get together in a lab and try to take the quiz together. Random question order, random answer order, and questions randomly pulled from subsets of the test bank will discourage this behavior. If one student's screen doesn't look like the other person's it is harder for them to quickly compare and answer the questions together. A timed quiz also makes it hard for two people to cheat if they have different questions and only a short amount of time to answer.

Having Someone Else Take the Test

It is sad to say, but students will sometimes pay their classmates or others who have already taken the course in the past, to take online quizzes for them. There are two ways to counter this cheating strategy:

1. Have an occasional proctored exam where students are required take the test in a lab or testing center and show their student ID in order to take the quiz. In the quiz settings, require a password (in the Extra restrictions on attempts section) so that students cannot enter the quiz without the proctor entering the password into Moodle. If students have not taken other quizzes or done the work until then, they will do poorly on the proctored exam.
2. To eliminate current classmates from taking each others quizzes, only make them available for a short time. You could require everyone take the test within a 2 or 4 hour block. If the test is properly randomized, it will be very difficult to take it more than once during the open test period. The test taker will worry about their own grade first, then about the other student's grade.

Coming next: Effective Quiz Practices in Moodle - Part Three: Tips for Creating Quiz Questions

This information is from the Moodle.org document: Effective Quiz Practices which can be found at: https://docs.moodle.org/27/en/Effective_quiz_practices.

Effective Quiz Practices in Moodle - Part One: Introduction

This article will be the first in a series related to using the Moodle Quiz Tool.

Introduction

The Moodle quiz engine is a powerful and flexible tool for assessing certain types of outcomes in a course. Using this tool effectively can boost the effectiveness of your teaching practices and promote student performance.

In this article you will learn about computer-scored quizzes and how to incorporate good strategies into their design and use.

Quiz Strategies

Using the Moodle quiz engine effectively takes some work and practice. The first step is to use question design strategies in order to ask good questions that will result in an assessment of your students' understanding of the material. Below are a couple of tips:

• Tie each question to a course objective or learning goal.
• Try to ask multiple questions about each important idea in the material. This will give you more data points about student understanding.
• When writing a multiple-choice question, be sure each wrong answer represents a common misconception. This will help you diagnose student thinking and eliminate easy guessing.
• Write questions that require your students to think at different levels. Include some recall questions, some comprehension questions and some application and analysis questions. This will help you to determine where students are having difficulty with the material, for example can they recall the material, but not apply it?
• Test your questions. As you develop your question bank and incorporate questions into exams, use the system reports to determine which questions are useful and which aren't.
• Once you have a few well-written test banks, be sure to use the quiz reports and statistics to monitor your students' performance. The detailed reports and statistics available to you are valuable tools for assessing student comprehension of the material. (Watch for more on this in a future blog.)

Creative Quiz Uses

The Moodle quiz engine makes it easier to utilize educationally sound assessment strategies that may have been more difficult to implement with paper and pencil. Instead of thinking of tests as a high-stakes activity - like a midterm or final, a better strategy is to incorporate frequent, low-stakes assessments so that your students are guided through the material throughout the semester. Creating a series of smaller quizzes gives you a flexible system for gauging performance and keeping students engaged in the class. Below are a few ideas for quick quizzes that you can use as part of a larger assessment strategy.

Chapter Checks

• As instructors, we know that reading the assigned materials is critical to the understanding of the course content and crucial to success in class, but getting students motivated to complete the reading can be a challenge. Creating a short test for each reading assignment encourages students to do the reading so that they can score well on the quiz, but it also gives students feedback on how well they understood the material. The instructor is provided with information about what aspects of the reading that students found confusing so that you can focus your class lectures on those topics.
• For a reading mini-test, set the time restrictions and only allow students to take the quiz once. Because it is a low-stakes activity and you want students to use it for self-assessment, enable the settings to display the feedback and correct answers  once the quiz is closed. If you are concerned about students sharing answers after they have taken the quiz, randomize the question and answer order. If you have a test bank with extra questions, make some of the questions random as well. 
• As an additional assignment, you could have your students view their test attempt and write down one question they have about a quiz question they missed.

Test Practice

• Many students have anxiety about taking tests - especially high-stakes tests. This is often caused by not knowing what to expect on the test. You can help alleviate some test anxiety by creating a practice test that students can take in order to get used to the format of the test, the types of questions that might be asked, and how detailed the questions will be. These tests are usually based on old questions similar to the current test questions such as last year's final exam.
• To set up a practice exam, create a zero point test with questions from the year before in random order with random answers. Allow students to take the test as many times as they would like so that they can test themselves as much as they need. Enable the settings to display feedback but not correct answers so that it presents more of a challenge. In the question feedback, give the students an indication of where they might find the correct answer (page number of book, lecture, etc.)

Data-Gathering Quiz

• A data-gathering quiz is similar to a chapter check, but it takes place after a class meeting or lecture. Your goal is to quickly obtain some feedback on your students' understanding of the material that you presented. This will help you to gauge what concepts your students found difficult and what they may have found so easy that they were bored in class.
• Set up the quiz to open for a limited time, such as opening an hour after class meets and closing an hour before the next scheduled class meeting. Allow students to take the quiz once and display feedback and correct answers after the quiz closes.

Coming next: Effective Quiz Practices in Moodle - Part Two: Quiz security and cheating

This information is from the Moodle.org document: Effective Quiz Practices which can be found at: https://docs.moodle.org/27/en/Effective_quiz_practices.

Building Rapport with Students in an Online Course

Creating an online community and building rapport with your students is essential to an effective learning environment. The concept of social presence is a psychological sense of being "real" and connection with others via technology. We want our students to feel that we value them as a real person in order for them to be satisfied and successful in our courses. This article provides suggestions for building rapport with our online students.

• Introduce yourself to the class by providing the following information:
• Your teaching philosophy
• Teaching experience
• Personal information such as interests, travel, etc.
• Office hours and location (real or virtual)
• Contact information (email, Skype ID, phone number, etc.)
• Photograph so students can put a face to the name and/or voice
• Provide an ice-breaker activity during the first week of class in which students can introduce themselves. Your students need a way to get to know you and each other. A good student introduction helps to create a supportive learning environment and a sense of community. Students who are new to online learning may be anxious about this method of delivery and will appreciate the chance to get to know their classmates and settle in to the routine by posting and responding to discussion type activities early on in the course. Ideas for student introductions:
• Have students find a digital image that represents who they are and upload it to the discussion and explain why it represents them
• Have students write 3 things about themselves - two are true and one is false. The class tries to guess which thing is false.
• Post a quote and ask students to comment whether they agree or disagree. Or require them to find a similar or contradictory quote and post it.
• Send a video chat inside an email to each student introducing yourself and welcoming them to the course. Eyejot.com is a program for creating video emails that offers a free trial version. GoAnimate.com is a program for creating animated messages that can be emailed to students or placed directly into a course (there is a charge for this site).
• Model the behavior you want from your students. If you want them to log into the course 4 times a week - you should log in 4 times a week.
• Set up a discussion or wiki in which students can ask general questions not related to the subject matter. You can call it the Question & Answer Room, the Watercooler, etc. Give clear directions to the students that this is the location for conversations unrelated to the course materials. Encourage students to answer each others' questions.
• Front-load your course with "low stakes" assignments or activities such as a syllabus quiz or introduction forum so that students can get used to the technology.
• Think outside the box when grading assignments - use a technology like Screencast-O-Matic (a free online screen and audio capturing software) or Camtasia to record yourself talking as you grade a student's assignment. They will receive visual as well as audio cues and it makes the feedback more personal.
• Communicate with your students often through announcements, tweets, emails, etc. so that they know you have a presence in the course and the expectations for participation are clear.

Tips for Flipping Your Class

What does it mean to "Flip Your Class"? 

The flipped classroom inverts traditional teaching methods so that face-to-face class time is used to create, collaborate, discuss and make connections as a result of students being introduced to content and the review of concepts outside of class through online content such as video, game simulations and other forms of content delivery.

The flipped learning model shifts the classroom from being teacher-centered where the instructor is the sole content expert delivering information to students, to a student-centered approach, where in-class time is meant for exploring the subject in greater depth through the creation of rich, interactive learning experiences (Hamdan, McKnight, McKnight, & Arfstrom).

"As flipped classroom pioneer Jon Bergmann says, 'The flipped classroom helps teachers break the habit of lecture.' Flipping provides a mechanism to transition toward deeper learning, opening up avenues for exploration and experimentation by freeing up class time. Beyond supporting teachers in their transition from "sage on the stage" to "guide by their side," what really happens in a flipped classroom is that the students become empowered learners with a host of tools to demonstrate their understanding. If we see flipped as an opportunity to break the habit of lecture, then a whole new set of learning opportunities begins to emerge" (Holland, 2013).

Another aspect of the flipped classroom is called "Just-in-Time Teaching" (JiTT) and is a technique for getting students to prepare before coming to class. JiTT uses formative assessment to determine student's understanding of course material so that class time can be planned and/or modified accordingly (Schaffhauser, 2014). For more information on JiTT, read Schaffhauser's article: 2 Great Techniques for the Flipped Classroom.

Research supports flipped learning

A recent literature review, "A Review of Flipped Learning" which is based on teacher reports, course completion rates, and supported methodology research indicates that flipped learning is more than just a fad for teachers and students--it's improving student achievement in classrooms across the country (Stansbury, 2013). According to the review, active learning has been shown to improve the academic performance of students in the areas of engagement, critical thinking and attitude (Stansbury, 2013). "A Review of Flipped Learning" - includes a review of how the model serves diverse student populations, the role of technology, and the research base that the flipped learning model is built upon. It also provides an analysis of implementations and results in higher education (Stansbury, 2013).

Evidence indicates that the type of active learning that occurs through the flipped learning environment improves academic performance, increases engagement and critical thinking, and improves the attitude of students (Hamdan, et al.).

Tips to Flip Your Class

1.  It is not an "all or nothing" deal. Instructors do not have to present all of their content online. Holland and Morra (2013) recommend that you start small and build a library of resources for your students, choosing carefully when it is appropriate and reasonable to have your students learn independently. Once you get started, momentum will build and it will be come easier as you locate resources and add them to your teaching material.
2. Recording Lectures:
1. Smile - even if your face is not on screen, people can hear you smile, your video will have more energy, and your students will hear your passion and excitement for the subject.
2. If you aren't enjoying the process of creating the video, then your students probably won't enjoy listening to it. Seek out a better or different way to present the material.
3. Test your video once you have placed it into your course to make sure it will play.
3. Use technology such as mobile devices to allow students to respond and give feedback during class. Socrative.com is a free tool that can be used to turn any mobile device into a personal response system.
4. Have a plan for how you will use the in-class time with structured activities and objectives. For an explanation of the Flipped Classroom Model, see Gerstein's article, "The Flipped Classroom Model: A Full Picture".
5. Create opportunities for peer instruction. The instructor might start with a short lecture that introduces or reviews the topic and then let students take turns leading class discussions. This encourages students to read the materials before coming to class and in order to have their topic prepared. 
6. For more great ideas on how to flip your class read Beth Holland's blog, The Flipped Mobile Classroom: Learning "Upside Down".

Resources:

Flipped Learning Network. A website containing case studies, white papers, resources, events, and support for flipped learning. www.flippedlearning.org

Gerstein, J. (2011, June 13).  The flipped classroom model: A full picture. http://usergeneratededucation.wordpress.com/2011/06/13/the-flipped-classroom-model-a-full-picture/

Hamdan, N., McKnight, P., McKnight, K., & Arfstrom, K. (2013). A review of flipped learning. Flipped Learning Network.

Holland, B. (2013, October 30). The Flipped Mobile Classroom: Learning "Upside Down". edutopia.org. Retrieved from: http://www.edutopia.org/blog/flipped-classroom-learning-upside-down-beth-holland

Holland, B., & Morra, S. (2013, August 14). 5 Flipped Classroom Issues (And Solutions) For Teachers. Edudemic. (This article focuses on using video lectures to present material outside the classroom. It lists free apps for the iPad that can be used to create videos and other types of lecture materials.)

Infographic on "The Flipped Classroom"

Schaffhauser, D. (2104, August 13). 2 Great techniques for the flipped classroom. Campus Technology. http://campustechnology.com/Articles/2014/08/13/2-Great-Techniques-for-the-Flipped-Classroom.aspx?Page=1

Stansbury, M. (2013, September). Does research support flipped learning? eSchool News, 16(8). p. 6. Retrieved from: http://www.eSchoolNews.com

Stansbury, M. (2013, October). Creating videos for flipped learning. eSchool News, 16(9). p. 26. Retrieved from: http://viewer.zmags.com/publication/b124b13d?page=36#/b124b13d/36 

http://21centuryedtech.wordpress.com/2012/07/18/flipping-the-classroom-a-goldmine-of-research-and-resources-to-keep-you-on-your-feet/

What Should be in your Course Syllabus?

Every academic course has a syllabus that lists at a minimum the course objectives, schedule, textbook requirements, and how to contact the instructor outside of class. In an online course, a detailed course syllabus serves as an essential tool for the communication of expectations and requirements to the students enrolled in the course. In addition, the course and/or institutional policies with which the student is expected to comply should be clearly stated in the syllabus. Below are a list of suggestions for an effective course syllabus:

• Instructor Information - how can students contact you? What are your office hours?

• Textbook Requirements - provide a link to the bookstore or a website for renting or purchasing the book. Is the book available as an e-book?
• Course Objectives - what will the student learn in this course?
• Communication - how should students communicate with you and when will you be available? How should students communicate with each other? When are students required to log into the course to view announcements and updated course information? How long will it take for you to respond to an email message or question posted to a forum?
• Attendance - how will this be tracked? How often do students need to log into the course?
• Participation - clearly state the time commitment with detailed course pacing, due dates and requirements for participating in activities.
• Technology - what skills are necessary to be successful in the course? What is the policy if their computer fails or their Internet goes down? What software will they be required to use outside of the LMS? Who do students contact for technical support with the LMS?
• Policies - what is the institution's policy and what is your policy on things such as academic honesty, late work, absences, etc. Provide a link to the institutional student handbook or webpage.
• Student Services - provide specific information on how to find student resources for disabilities, counseling, tutoring, etc. Provide phone numbers, locations, and a link to the website for each service.
• Student Conduct - explain to students how you want them to behave in the course and what the consequences are of any unacceptable behavior. Provide information on student netiquette - don't assume that your students know how to communicate properly with technology.

The Importance of Policies in E-Learning Instruction by Waterhouse and Rogers provides excellent examples of the types of student policies that should be covered in an online course; a student code of conduct contract; and examples of intellectual property rights policies.

Need Help Developing Your Syllabus?

The Instructional Technology Resource Center at Idaho State University is available to assist you with utilizing tools within Moodle such as the Book or special blocks to deliver the syllabus information in an effective and efficient manner.

Additional Resources for Examples:

Online & Hybrid Course Syllabus Example provided by Pasadena city College

Netiquette Guidelines - Santa Barbara City College

Technology Support - Walden University

Looking Back at Previous Courses to Improve Your Next Online Course

The countdown is on for the start of the Fall semester, and with that comes the work of getting our courses ready to go live. This article is about looking back at previous semesters and finding information that will help us improve our future courses. Below are tips for where to look for clues on how we can improve:

• Student Evaluations: read them and take note of the good, the bad, and the ugly. We are often so involved with our teaching and trying to get all the information covered before the end of the term that we overlook items that are important to the students. Evaluations can often reveal the missing pieces.

• Student comments and emails: Look back at student emails and forum posts for additional insight into what needs clarification. For example, if numerous questions were asked about assignments, grading policies, etc., this would indicate that further explanation is needed in the course. Also take note of positive comments from students that indicate what activities and resources they especially liked and use this as a guidepost for designing future ones.

• Activity reports: Look at the activity reports within Moodle to see how often students viewed specific resources or utilized tools that were included in the course. This will indicate resources that could be eliminated or improved and also emphasize the types of resources that students found most useful.

• Take note of the "unexpecteds" that occurred during the previous semester. Were there activities that did not go as you planned, or assignments that took too long to grade, or quiz questions that seemed to stump the entire class? In the future, start a file or document where you can collect these items and your responses to them so that the next time one pops up you will be prepared and you can spend less time and energy dealing with it. This will also serve as a reminder of what to go back and fix the next time the course is offered.

• Check your syllabus for items that need to be added, clarified, or updated. If students had questions about your late work policy, add this information to the syllabus. If you received numerous poorly written emails, include a section on netiquette.

• Conduct an honest self-evaluation and look over and learn from your mistakes, oversights, and inconsistencies. Look for activities that you did not like or could have run smoother and improve them.

•  Do a thorough check of your course for any broken links, layout problems, or inaccessibility issues. Be sure to test all url's to make sure they still work - Internet materials can be there one day and gone the next!

• Keep a file of positive comments, new ideas you come across and student interactions that went especially well. The comments will serve as a reminder of your positive efforts and how they were appreciated by the students. The documentation of new ideas will help you to remember what you wanted to add in the future. The transcript of student interactions can be used as guide for future students who may need assistance with the project.

• Set aside enough time to look over your course before it goes live to update due dates, textbook information and lectures. 

Remember that you may have taught this course dozens of times, but "to each of your students this may be a first experience having you as an instructor and thus having a course that works well in all aspects--layout, technical, explanation of assignments, correct dates, working links, and an engaged and concerned instructor--is what that student expects..and deserves. And only you can make that happen" (Sull, 2010).

Sull, E. C. (2010, January). Use end of an online course to better your next online course! Online Cl@ssroom.