Online Experiments for Language Scientists

Academic year 2025-2026

Postgraduate (LASC11167) Assignment brief, 2025/2026

Sole assessment: coding project plus report, due 4th December, worth 100% of course mark

The assessment has two parts:

A functioning experiment running on jspsychlearning.ppls.ed.ac.uk
A report explaining the motivation behind that experiment: i.e. the research question it is intended to answer (including briefly reviewing relevant literature and justifying that research question in terms of broader questions in the field), explanation/justification for any complex or questionable design decisions you took in designing the experiment (there’s no need to justify low-level stuff), an appraisal of weakness of your experiment or possible ways it could be improved/extended. The maximum length for the report is 1000 words.

The experiment has to be relevant to using online experimental methods to study language or language-relevant phenomena, but beyond that there are no constraints on what you tackle - please have a short conversation with me (Jenny) if you are at all unsure about what you have in mind is appropriate.

We will assess these final projects based on two components: the technical ambition and implementation of the experiment, and the quality of the accompanying report and explanation. For technical ambition / implementation we will give high marks to challenging coding problems (e.g. going beyond the template experiments we provide for practicals), and experiments that work well and look good. For the report we will give high marks to projects that are well motivated by the literature, answer interesting well-explained research questions, and demonstrate interesting critical insights on your own work (e.g. interesting thoughts on design decisions or methodological weaknesses, even if you weren’t able to resolve those in the code).

You can choose how you weight your effort across these two components, e.g. if you tackle a demanding coding project we will be satisfied with a lower-effort report (e.g. with a less thorough literature review or less extensive evaluative remarks); if you are less ambitious on the technical side (e.g. largely re-using code we provide and ‘just’ plugging in different stimuli, trial lists etc) then we would expect a more ambitious report (e.g. with a more careful and detailed literature review situating your work in the literature in a thoughtful way). Again, please have a short conversation with Kenny if you want some guidance on how you should balance up these two components or how you should make sure you are doing something at the appropriate level of ambition.

Important points on providing a working URL for your experiment:

Please include the URL of your experiment (hosted on the jspsychlearning server) near the top of your report so we can access it. This will be of the form https://jspsychlearning.ppls.ed.ac.uk/~UUN/path where UUN is your student number (e.g. s24…) and path is the path to your experiment’s html file. If you include a link to the local files on your computer rather than a version of the experiment on the jspsychlearning server, we won’t be able to access your code and you will fail. If you include the wrong URL (e.g. using the “Open URL” option in cyberduck, which generates a URL with an error in it) then we may have difficulty accessing your experiment, and you may fail. We show you how to put experiments on the server and access them via a URL in the labs, so make sure you understand this!
We will run through everyone’s experiment, so if there is anything we need to know (e.g. if you have to give us a password to progress past a certain stage, if we need to run in two browsers simultaneously, etc) please include this info at the top of your report.

Additional notes:

This assignment is intentionally open-ended - I want everyone to engage with both aspects of the course (engaging with the literature, building actual web experiments), but I also want to be flexible enough to allow you to focus on what you are most interested in and/or what you feel will best illustrate your engagement with the course, and to allow flexibility in terms of how technically challenging your project has to be - not everyone is going to become a coding whizz in 1 semester, you are all starting from different levels of coding experience, but the intention is that this assignment is flexible enough to allow everyone to get a good mark.
Because it’s open-ended, we are relying on you to not overcommit on this one assignment - while nothing in the format prevents you from handing in a challenging programming exercise and a publication-quality report, this is not necessary to get a good mark and you will eventually face diminishing returns.
It’s up to you whether you want to build an experiment with a novel element (e.g. a tweak to an existing experiment, or something entirely new) or one which closely replicates an existing study in the literature. Either is fine, but if you are focussing on replicating an existing experiment then you need to justify why that replication is necessary in your report (e.g. do you have reasons to be sceptical about the original result, is it particularly important that its robustness is checked, does going online allow you to access some theoretically-important new population, …).

Course main page

XXProject maintained by jennifer-culbertson Hosted on GitHub Pages — Theme by mattgraham