A case study in iterative development for crowdsourcing biomedical data analysis (Beta Catchers)
This blog post recaps one of Pietro Michelucci’s talks from the November Citizen Science for Health 2025 Conference in Zurich, Switzerland. Pietro was the main presenter, but coauthors include Lisa Gusman, Laura Onac, Paz Santander, Margaret Lane, Ravish Dussaruth, Gretė Vaičaitytė, Jerry Lou, Shino Magaki, Michael J. Keiser, and Brittany N. Dugger.
In his second presentation at the conference, “A Case Study in Iterative Development for Crowdsourcing Biomedical Data Analysis,” Pietro shared new results from Beta Catchers, HCI’s upcoming platform that turns the public into research partners by having volunteers examine whole-slide images (extremely large, high-resolution scans of human brain tissue) to answer questions about Alzheimer’s disease.
The recording is below.
Summary
In Beta Catchers, volunteers complete a two-step workflow: first, they review large image regions called megatiles (big sections of the tissue) to spot and label Alzheimer’s-related lesions; then they move to minitiles (zoomed-in, smaller regions) to trace the outlines of those lesions. This design — developed with input from communities disproportionately affected by Alzheimer’s disease — shows strong “wisdom of crowds,” meaning many volunteers together produced results as accurate as experts. As Pietro added, “citizen scientists do a fantastic job of that part.”
The challenge comes in the megatile step. In the pilot study, algorithms struggled to know when two volunteer clicks referred to the same lesion.
With only a few volunteers, the system could misinterpret these clicks — “the algorithm would get it completely wrong,” Pietro noted — and achieving accuracy required 30–50 people per image, which isn’t scalable.
The team’s new solution is a lightweight human-in-the-loop step, where a volunteer simply answers a quick microtask (“same or different?”) to guide the merge.
This small human judgment, easy for people but hard for machines, dramatically reduces error and makes the pipeline viable.
Work with us!
Want to learn more about our experience at the conference? Read our conference recap blog post, and also check out another blog post recapping the keynote talk Pietro gave at the conference about AI and health.
If you were part of the CS4Health community — or if you’re just curious about human computation — we’d love to continue the conversation. Email us at info@humancomputation.org to say hello and discuss collaborations. Maybe we can come to a city near you – invite us to your next conference!