The Marmoset Project

Marmoset Open source project

The Marmoset source code repository. Marmoset is open source under the Apache 2.0 license. At the moment, it is undergoing rapid evolution and stabilization; we are hoping to find several other universities willing to try using it for the Spring 2012 semester.

Survey on collection and grading of student programming projects

A survey about to assess current practices in the collection and grading of student programming projects. Results of the survey will be made available from this web page.

Marmoset Publications

Links to Marmoset papers and bibtex entries.

What is Marmoset?

Marmoset is described as part of a talk given by Professor William Pugh on Innovation in teaching software development skills Dec 2nd, 2011 at SUNY Stony Brook.

Look for more information and updates on this web page shortly.

Some of the information below is rather dated, but provided until we can provide more up to date information.

Goals for Marmoset

1. To create a better experience for students learning to program.

   Learning to program is difficult! Students in introductory programming courses are asked to solve difficult problems using unfamiliar concepts in an unforgiving environment. We want to provide more feedback and better feedback.

2. To better understand how students learn to program.

   Have you ever asked questions like "What the heck are students doing when they pull marathon all-nighters?" or "It looks like many students are making the same mistake; can I find these errors automatically so my students don't waste their time?".

   We've found that the data Marmoset collects allows us to answer these questions, and many others.

Collecting Fine-grained Snapshots

The resulting fine-grained revision history provides a unique perspective into the development process for students. And because the plugin works automatically and transparently, we don't burden introductory students with version control--yet another piece of software with an unintuitive interface.

Automated Testing

Of course we don't show the students all the results from the unit tests. It's pretty clearly unproductive to show students all the test results, for fear they'll tweak their code in a seemingly random fashion until it passes enough test cases. We call this programming by "Brownian motion".

The typical compromise is to have public test cases that the students can see and secret test cases that the students can't see. The problem here is that the secret tests cases are usually not executed (and often not even written!) until after the project has been submitted.

So a student submits something that passes all the public test cases, and two weeks later finds out she got 50/100 because she failed all the secret tests. This feedback is two weeks late to be useful in fixing the program!

Release Testing

• Public tests are distributed to students directly. They can run these tests and see the results at any time.

• Secret tests are only used for grading purposes. They are not distributed to students and the results of individual tests are not revealed to students before the deadline. After the deadline, secret test results can be optionally revealed to students.

• Release tests are not distributed to students. However, students may request the results of running them on a particular project submission. For a release test request to be granted, students must have a release token available. Students are given a limited number of tokens, each of which regenerates in a fixed period of time, e.g. 3 tokens every 24 hours. Viewing release results allows the student to see the number of release tests passed and failed as well as the names of the first two tests failed. For example, for a project requiring the evaluation of various poker hands, a student may discover that her submission has passed 6 out of 12 release tests and that the first two tests failed were testFullHouse and testThreeOfAKind.

Data Collected so far...