The Marmoset Project

Marmoset Demo Server	The Marmoset Demo Server allows you to register an account with our demo server and then sample our software.
Marmoset Publications	Links to Marmoset papers and bibtex entries.

What is Marmoset?

Goals for Marmoset

1. To create a better experience for students learning to program.

   Learning to program is difficult! Students in introductory programming courses are asked to solve difficult problems using unfamiliar concepts in an unforgiving environment. We want to provide more feedback and better feedback.

2. To better understand how students learn to program.

   Have you ever asked questions like "What the heck are students doing when they pull marathon all-nighters?" or "It looks like many students are making the same mistake; can I find these errors automatically so my students don't waste their time?".

   We've found that the data Marmoset collects allows us to answer these questions, and many others.

Collecting Fine-grained Snapshots

The resulting fine-grained revision history provides a unique perspective into the development process for students. And because the plugin works automatically and transparently, we don't burden introductory students with version control--yet another piece of software with an unintuitive interface.

Automated Testing

Of course we don't show the students all the results from the unit tests. It's pretty clearly unproductive to show students all the test results, for fear they'll tweak their code in a seemingly random fashion until it passes enough test cases. We call this programming by "Brownian motion".

The typical compromise is to have public test cases that the students can see and secret test cases that the students can't see. The problem here is that the secret tests cases are usually not executed (and often not even written!) until after the project has been submitted.

So a student submits something that passes all the public test cases, and two weeks later finds out she got 50/100 because she failed all the secret tests. This feedback is two weeks late to be useful in fixing the program!

Release Testing

• Public tests are distributed to students directly. They can run these tests and see the results at any time.

• Secret tests are only used for grading purposes. They are not distributed to students and the results of individual tests are not revealed to students before the deadline. After the deadline, secret test results can be optionally revealed to students.

• Release tests are not distributed to students. However, students may request the results of running them on a particular project submission. For a release test request to be granted, students must have a release token available. Students are given a limited number of tokens, each of which regenerates in a fixed period of time, e.g. 3 tokens every 24 hours. Viewing release results allows the student to see the number of release tests passed and failed as well as the names of the first two tests failed. For example, for a project requiring the evaluation of various poker hands, a student may discover that her submission has passed 6 out of 12 release tests and that the first two tests failed were testFullHouse and testThreeOfAKind.

Data Collected so far...