Instructions for Graders

Setting up

1. Grading is done via command line scripts in python on a unix machine
2. You can use the terminal window on a Mac or install a virtual machine on Windows.
3. Your first step is to go to
   https://bitbucket.org/danielmai/code-check-homework-grading
4. You need to clone the repository. You should also do a git pull each time before you grade to get the latest scripts
5. For complete instructions on setting up your computer to grade and on grading itself, go to
   http://canvas-codecheck-grading.readthedocs.org/

Before you start grading an assignment

1. Read the assignment in Canvas
2. git pull the latest script from the repository
3. You can find instructions for using the scripts at
   http://canvas-codecheck-grading.readthedocs.org/
   

What the students submit

1. There are two assignments due every week: a draft and a final.
2. There are three problems in each
3. Students will submit three files for both the draft and the final
4. Each file is the output from Codecheck, an automated grading program
5. The files have a .signed.zip extension

Do the following for both the draft and the final

1. Follow the directions for running the grading script
2. The first time grading, you will need a Canvas access token. Follow the readthedocs.
3. You need to edit config.json
   1. If you have graded before, you will need to change the CourseID in config.json
4. Look at the aggregate_report.html for each student. There is a handy script that will open all the aggregate reports, or some subset of them, at one time. I open all the reports for students whose names start with 'A' and then with 'B' etc.
5. Just because the code passed Codecheck does not always mean the student gets full credit
6. You need to look at each problem in the report. You may need to adjust the scores in grades.txt for certain errors
   1. Does it follow the directions? For example, does it use a loop if a loop is specified?
   2. Are instance variables private?
   3. Will the code get the right answer with different data. Be sure they did not hard code the answer or always return the same value which is occasionally correct
   4. I will give you things to look for, but I do not think of everything a student might do wrong. If you see something that looks strange, ask if you should deduct points.
7. Make a comment in grades.txt for at least one of the three problems. For example:
   1. Do not use if/if/if for mutually exclusive cases
   2. Doing ... was very clever.
8. You can adjust scores in the _comment area in grades.txt.
   • See the examples at http://canvas-codecheck-grading.readthedocs.org/
   • Put score adjustments in _comments
     • State which problem you are discussing
     • Write a comment that makes sense
     • Indicate how many points are being added or deducted
     • Things in _comment are displayed to students when you upload. Things in _notes are not.
   • This student would get 17 points

      _dir: programmer--joe_1234567
     _name: programmer--joe
     _canvas_id: 1234567
     _total_score: 18/18
     _comment:

     
     -1 BoxManager needs private instance variable
     
     

9. When you are done, send your professor the grades.txt file for approval
10. When grades.txt is approved, run the script to upload the scores to Canvas.
11. I will give you a rubric for the assignment
    • It will tell you some things to watch for
    • If I forget to send it, ask me for it
    • If you see anything strange, ask me about it.
12. You can leave me notes in the _notes section.
13. If there is cheating, call it to my attention in the email

Responsibilities

1. This is not an activity to be done when convenient
2. This is a job even though you set your hours
3. We are depending on you to return the grades to the students quickly (within 3 days of the due date). Schedule your time accordingly
4. You need to do the grading even during your own midterms.