Monday, November 15, 2010

Improving Questions: Sapling’s Quality Control Process

Sapling Learning is committed to creating the highest quality content, but we also devote a considerable effort towards maintaining and improving the efficacy of the existing questions. This post covers what happens to a question after it is "live" (available for use in assignments).

There are two opportunities for question revision during a question's lifetime: revision prompted by instructor or student comments, and revision through periodic, statistical reviews. The types of question flaws caught in each instance tend to be very different in nature.

Occasionally a Tech TA or Sapling Support will receive an email or call from an instructor or student alerting us to a potential error with a question. The question may have a wording issue, the tolerances on the question may be too tight, or, rarely, the correct answer may be incorrectly coded. Regardless, the issue with the question is often specific and the action is immediate: we may remove the question from the assignment, or replace the question with a corrected version.

There can be other issues with a question that are more subtle, and that is why we perform periodic, statistical reviews. During these reviews, Sapling analyzes statistical data for the questions that we have written. The question statistics include information about how many students have viewed the question, the average number of attempts per student on the question, the average number of points each student got on the question, and the difficulty rating that the original question author projected the question to have.* For each difficulty rating (easy, medium, and hard) within a discipline, we look for outliers—that is, questions for which students have taken significantly more or less attempts or have received more or less points than average for that difficulty rating. Those questions are flagged, and we examine the questions to see if we can find anything that might cause the unusual behavior. Sometimes the incorrect feedback could be improved, sometimes an issue only occurs for some values of a randomly generated variable, and sometimes question tolerances are too tight. In the instances where we cannot find a flaw in the question, we consider giving the question a new difficulty rating.

Quickly addressing potential question flaws, regardless of how the flaw is brought to our attention is one way that we continuously improve our question bank.  However, there are always opportunities to make this process better. As an instructor, what would you like to see us do to improve our content? Are there any statistics, like average time spent on a question, that you feel we should analyze when determining what questions should be flagged for review?

* The number of attempts on a question and the number of points received for the question are not necessarily giving you the same information if a question has more than one part.

44 comments:

  1. Useful details like this one must be kept and managed so I will put this one on my save list! Thanks for this amazing publish and expecting to publish more of this

    Digital Marketing Company

    ReplyDelete
  2. Your blog is looking good and its having the useful information, so most of the peoples are getting good knowledge to read your article. thnx for your creation.

    Medical Tourism to India

    ReplyDelete
  3. very useful details Thank you for your occupation…

    Seo company bangalore

    ReplyDelete
  4. I just want to tell you that I’m all new to blogging and site-building and really loved your web page. Very likely I’m want to bookmark your site . You amazingly have remarkable posts. Thanks a bunch for sharing your blog.
    Regards - www.office.com/setup

    ReplyDelete
  5. I am convinced that your website is convinced that weblog is very new. Probably I am going to bookmark your site. There are certainly impressive stories. Passion for sharing your website with us.
    Regards - www.norton.com/setup

    ReplyDelete

  6. Hey Thanks for sharing this valuable information with us. I will come back to your site and keep sharing this information with us.
    Best Regards - office.com/setup
    office.com/setup
    office.com/setup
    office.com/setup
    office.com/setup

    ReplyDelete
  7. power testo blast
    scurrent health and fitness and wellness. Once you have your ideas set on what you want to achieve it's a lot easier to create to the getting and system that will be expected for you to lessen fat. Use both inner and
    https://newsletterforhealth.com/power-testo-blast/

    ReplyDelete
  8. cerisea medica and flatulence. Also the ingredient Ephedrine in many organic health and fitness and health items causes enough harm. This is why FDA or American Food and Drug Association has banned this particular ingredient. To sum up , I would say that it is not wise to compartmentalize organic items as 'safe' or take
    https://newsletterforhealth.com/cerisea-medica/

    ReplyDelete
  9. Beta keto
    aout needs because the kids take up so much of your energy and attempt but you can find cut out small bits of your energy and attempt depending on how many bodyweight you want to lessen and them that you need to do to get to know your goals. One of the many activities you can do to start with is to watch
    https://newsletterforhealth.com/beta-keto/

    ReplyDelete
  10. Keto fast normally expect to wait a few a few several weeks before locks start to build. While the locks is locking, it will need to be washed. Here is where washing should be extended for a while if possible, so that the locks can be allowed to lock for Two several weeks to about a 30 days without manipulation. When you into
    https://newsletterforhealth/keto-fast/

    ReplyDelete
  11. reviva brain
    You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site. 2 Responses to “Male Hair Loss - Tom’s Story.” 1. George Standing Says: May 25th, 2007 at 11:34 pm I think many people have to remember thinning locks is an ingredient
    https://newsletterforhealth.com/reviva-brain/

    ReplyDelete
  12. rapid slim
    determine they want. The acai berry berry fruits berry slim are found in the type of tablets or items and juices. So you can get any one of them and using it. Body weight Decrease Program with The acai berry berry fruits Berries Slim can help you get rid of 5lbs of bodyweight in just one 7 times which indicates
    https://newsletterforhealth.com/rapid-slim/

    ReplyDelete
  13. maxx power libido enhance the metabolic process and thus help in dropping of the undesirable flab in your physique system. The metabolic process increases due to excellent concentration of catechin polyphenols. This intensifies the amount of fat oxidation and thermogenensis. thermogenic properties. Thermogenic drinks as well as
    https://newsletterforhealth.com/maxx-power-libido/

    ReplyDelete
  14. keto blast
    , , particularly if you have injuries or other limitation, see a professional!* If you think ok starting up something on your own, let me put out a simple plug for one of my favorite online workout websites, Fitness Blender.com. For approachable, achievable and FUN workouts, this site is the best. Run by a
    https://topwellnessblog.com/keto-blast/

    ReplyDelete
  15. Thanks for sharing this marvelous post. I m very pleased to read this article. mcafee.com/activate |norton.com/setup|mcafee.com/activate |office.com/setup

    ReplyDelete
  16. provexum these problems should avoid man-made elements and stick to natural elements such as cotton, especially when it comes to undergarments. • 3. Soap and laundry detergent– While cleansers and cleansers can keep a clean, clean scent, they are often manufactured with dehydrating ingredients that leech the sebum from your epidermis part, making it sensation complicated, tender and itches.r
    https://newsletterforhealth.com/provexum/

    ReplyDelete
  17. Well i am a regular reader of new blogs and i found this one is also such a nice one... keep it up
    mcafee.com/activate | mcafee.com/activate | www.norton.com/setup

    ReplyDelete
  18. You cannot imagine simply how much time I had spent for this information! Thanks!
    office.com/setup | norton.com/setup | www.mcafee.com/activate

    ReplyDelete
  19. After visiting office.com/setup login to microsoft account to buy contract of office setup, Any misery in www.office.com/setup our sting will assert going on for office install.

    ReplyDelete
  20. What a superb post! I have no words to describe this post because everthing is clear with your wonderfull words. I really feel out for world reading your post, it is full of fresh and usefull. I really appreciate, keep the work continue.

    www.norton.com/setup ! norton.com/setup ! office.com/setup ! office.com/setup ! mcafee.com/activate ! mcafee.com/activate

    ReplyDelete
  21. I certainly needed to post a note in order to express profound gratitude to you for those astounding exhortation you are composing at this site. norton.com/setup | norton.com/setup | norton.com/setup | office.com/setup

    ReplyDelete
  22. Mcafee Activate has recognized to be an effective way of keeping up your devices and your data safe from getting infected by any hackers and virus attackers. For more details visit mcafee activate today.

    ReplyDelete
  23. With Mcafee.com/activate , you can prevent devices and network against threats like malware, worms, spyware and runs in the background and does not cause any problem nor slows down the computer.

    ReplyDelete
  24. To activate Hulu Plus on your device, either use the on-screen keyboard to enter your Hulu log in information or go to hulu activate and enter the device activation code. Either of these methods will allow you to use Hulu on any Hulu-supported device.

    ReplyDelete
  25. Hulu supports a wide variety of devices that you can use to stream with us. You can activate any of them at any time, as well as remove them from your account. click hulu.com/activate today.

    ReplyDelete
  26. Install mcafee antivirus in your computer with high class professionals and best tech team. Just ring us and we are ready to assist you till the last minute of installation - www.mcafee.com/activate today.

    ReplyDelete
  27. Mcafee offers you the McAfee activate option to give best Home network protection, Multi-device protection, Web Protection, and Identity Theft. visit Mcafee.com/activate today.

    ReplyDelete
  28. This comment has been removed by the author.

    ReplyDelete
  29. mcafee antivirus is widely used aantivirus helps to detect and neutralize computer virus, the mail worms,the trojan programs,and also helps your system free of virus and other malware is quite a daily challenge. for more details visit: mcafee activate today.

    ReplyDelete
  30. roku link this is least difficult approach to stream stimulation to your TV. On your terms. With a great many accessible channels to look over for more details.visit: roku link today.

    ReplyDelete
  31. www.OFFICE.COM/SETUP DOWNLOAD, INSTALLATION AND ACTIVATION FOR OFFICE SETUP. Microsoft Office suite contains reliably used ventures and organizations. The most used tasks of Microsoft Office are Word, a word taking care of program, Excel, a spreadsheet program, PowerPoint, a presentation maker, and Access, a database application

    ReplyDelete
  32. Consistent and clear marking is one of the keys to fast, stress-free movement. Some people use the numbering system, while others create detailed color coding systems. No matter which tagging method you use, consider the following:
    شركة نقل عفش بالرياض

    ReplyDelete
  33. Thanks for sharing this with us......this is really appreciating.
    office.com/setup
    mcafee.com/activate
    norton.com.setup

    ReplyDelete
  34. I'd really love to be a part of online community where I can get opinions from other knowledgeable individuals that share the same interest. List of Retail Companies in Singapore

    ReplyDelete
  35. Fantastic site you have here but I was curious if you knew of any user discussion forums that cover the same topics discussed in this article? I'd really like to be a part of online community where I can get advice from other knowledgeable individuals that share the same interest. executor of will singapore

    ReplyDelete
  36. Read about best smartphone news reviews, price and specifications and much more features
    Done
    Done
    Done
    Done

    ReplyDelete
  37. We have identified the list of available devices for Amazon Prime Video; we can also check out the step by step to activate Primevideo.com/mytv on your smart TV. We only need a prime video activation code for the activation method. You can create the activation code on your device with the occasional help of Prime Video application. With this information handy, you can get a unique code and initiate their devices in less than one minute.
    Read more…

    ReplyDelete
  38. Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with extra information? It is extremely helpful for me. venturi scrubber

    ReplyDelete
  39. To be able to watch Prime Video on a Smart TV, the TV must have the application pre-installed in your device. Otherwise, you must have a similar

    platform to the App Store or Play Store on Google, so download the application from the website. Devices Compatible with Prime Video. Exclusive

    content and Amazon Original series Enjoy exclusive movies and TV shows on your device.
    primevideo.com/mytv

    ReplyDelete
  40. Www.youtube.com/activate - look at the state website site and input digit confirmation code during activation procedure. User can trigger YouTube the subsequent apparatus like smart Television, appletv, roku, play station, x box and other buffering media players. After you register into youtube program on SMART-TV or alternative apparatus, your device exhibits 5 digit code. This really is actually the youtube activation code that's usedto trigger youtube.
    youtube.com/activate
    www.youtube.com/activate

    ReplyDelete
  41. Steps to Configure Tubi Television on Your Own Apparatus: Inch Download the Tubi Television program in your own Television 2 Open the program and click on the house 3 Out Of house, click Sign in 4 The program on Television will display an activation code 5 Type the activation code properly 6 Tubi Television will trigger with No Issue
    tubi.tv/activate | tubi.tv/activate enter code | tubi.tv/activate

    ReplyDelete

  42. One can not Convert quicken file to QuickBooks for Windows file. We must either update our Quicken file to a newer version of the software or start a new company file. However, there are few steps to Quicken to QuickBooks. Firstly you have to Open QuickBooks and choose the file menu, select utilities and then directly convert it to the quicken option.

    ReplyDelete
  43. This comment has been removed by the author.

    ReplyDelete