Investigating Cognitive Training: Lumosity's "Crossword" Study

Lumosity draws on research that has developed over decades, but we are at the forefront of a new and rapidly innovating field, which means there is a lot we don't yet know. In recent years, there’s been a plethora of interesting -- and often promising -- research about different cognitive training programs and their uses. Lumosity is proud to be a part of these research efforts: through the Human Cognition Project, we aim to help accelerate the pace of research on cognition and cognitive training by providing independent researchers with access to our online tools, and we also have an in-house science team that designs and conducts studies of the Lumosity program.

In September 2015, our team published one such study in the peer-reviewed journal PLOS One: “Enhancing Cognitive Abilities with Comprehensive Training: A Large, Online, Randomized, Active-Controlled Trial.” Below is a high-level summary of that publication (and for any data lovers out there, we also open-sourced the study data).

What We Did

Our scientists recruited 4,715 new Lumosity members to complete the study. At the beginning of the study, every participant completed Lumosity’s NeuroCognitive Performance Test (NCPT), our online battery of cognitive assessments. Over the next 10 weeks, participants trained five days per week for fifteen minutes, but were split between two groups: one that trained with Lumosity and one that did Lumosity-branded online crossword puzzles to control for placebo effects. At the end of the 10 weeks, the participants retook the NCPT.

In addition to the NCPT, the study participants also completed an online survey before and after the 10 weeks of training. The survey asked the participants to report their personal assessment of their cognition and emotional status.

Why Crosswords?

We chose to use crossword puzzles as the active control because they are commonly believed to be a cognitively stimulating activity. Some research on placebo effects suggests that cognitive training studies may be confounded by participants’ belief that the training program will help them, and that any observed improvements might just be the result of a placebo effect. To mitigate placebo effects, then, it was important to the researchers that the control group participants view their games or puzzles with some equivalency. Lumosity-branded crosswords seemed to fit the bill.

What We Found

The researchers analyzed the data and found that after the 10 weeks, the Lumosity group improved more than the crosswords group on the NCPT. Specifically, the researchers calculated a “Grand Index” on the NCPT, which reflected the aggregate score across seven subtests. The Lumosity group showed statistically significant improvements on the Grand Index and on five of the seven subtests: a visual short-term memory assessment, a working memory assessment, a problem solving and fluid reasoning assessment, a response inhibition and processing speed assessment, and an arithmetic reasoning assessment. Crosswords players showed more improvement on the assessment of grammatical reasoning, while there was no statistically significant difference between the two groups on the measure of divided visual attention.

Analyzing the two groups’ responses to the self-reported survey before and after the study, the researchers also found that, overall, the Lumosity training was more effective than the crosswords training for improving self-assessed cognition and emotional status. However, the reliability of this self-reported survey is up for debate in the scientific community, and these results should be interpreted accordingly.

Interpreting the Results and Next Questions

If you’ve ever read a news article reporting on new research, you’ve probably noticed that oftentimes researchers will caveat their findings with a disclosure that more research needs to be done to fully understand the implications of the study. This is the nature of scientific inquiry: there is always more to learn, more to question, and every study can be improved on. The same is true with our crosswords study: we believe that the results are promising, and we’re excited to build upon them.

In particular, we need to do more research to determine the connection between improved assessment scores and everyday tasks in participants' lives. In any follow-up studies, we also plan to think about refining aspects of study design.

Future research should also address limitations of this study, which include the risk of inadvertent experimenter bias and the risk of attrition bias, as both the Lumosity and crossword arms had approximately 50% attrition rate. As with all scientific research, there is also a risk of publication bias.