Data Quality in Citizen Science: How Projects Make Volunteer Data Trustworthy

Citizen science & data · By the Common Inquiry editorial team · Published 2026-10-26

A volunteer in an orange vest looks through a spotting scope on a tripod across a mountain valley — A volunteer scans cliffs for mountain goats during a citizen science survey in Glacier National Park, Montana. US government photo (National Park Service), public domain.

The most common question asked about citizen science is whether data collected by volunteers can be trusted. Two decades of research give a nuanced answer: volunteer data can be as reliable as professional data for many tasks, but only when the project is designed for quality from the start. This article explains how.

Start with fitness for purpose

Professional scientists do not produce error-free data either. Every dataset has bias, noise and gaps. The useful question is whether the data are good enough for a specific use. Monitoring whether a stream is visibly polluted after storms, documenting that a problem exists, estimating long-term trends in bird populations and providing legal evidence in a regulatory case all demand different levels of precision and documentation.

The US EPA's quality assurance handbook for citizen science projects, published in 2019, asks projects to decide early how their data will be used and to plan quality measures accordingly, using a simplified version of the quality assurance project plan that EPA requires of its own studies. Its guidance distinguishes uses ranging from education and awareness to screening, to decision-making and regulatory purposes.

Before collection: design for quality

Clear, tested protocols

A protocol tells volunteers exactly what to record, how, where and when. Good protocols are short, use plain language, include pictures, and are tested with volunteers before launch. Many quality problems come from ambiguity: does "near the river" mean five metres or fifty? Should a bird heard but not seen be recorded?

Training and testing

Training can be a short video, an in-person workshop or a quiz that volunteers must pass before submitting data. Some projects certify volunteers for specific tasks, such as identifying difficult species, and weight their records accordingly.

Instruments that suit the task

Where measurements are taken with kits or sensors, projects need to know the instrument's accuracy, detection limits and known failure modes, and to calibrate or check it regularly. Low-cost air sensors are a good example, discussed in our article on community air quality monitoring.

During collection: structure and checks

Structured forms and apps limit free-text entries, record time and location automatically and prompt for required fields.
Automatic filters flag unlikely records, such as a species far outside its known range or season, or a reading outside a physically plausible range, for review rather than rejecting them outright.
Photographs and audio attached to records allow later verification.
Effort data (how long the volunteer looked, how far they walked, whether they reported everything they saw or only highlights) make it possible to correct for differences between observers. eBird, for example, asks birders whether a checklist is "complete", meaning that it lists every species they could identify.

After collection: validation and analysis

Expert review

Many biodiversity platforms route flagged records to experienced reviewers. This works well for rare events but does not scale to millions of routine records.

Consensus

Online projects often show the same image to several volunteers and combine their answers. Galaxy Zoo and later projects on the Zooniverse platform aggregate multiple classifications and can weight volunteers by their track record. Agreement among independent volunteers has been shown in published studies to match expert classifications for many tasks.

Statistical modelling

Ecologists use models that account for differences in observer skill, effort, time of day and habitat. These methods allow trends to be estimated from data that were not collected in a strict sampling design.

Comparison with reference data

Running a subset of samples in parallel with professional measurements (for example, sending some water samples to an accredited laboratory, or placing a sensor next to a regulatory monitor) gives an estimate of error that can be reported alongside results.

What the research says

A widely cited review, "Assessing data quality in citizen science" by Margaret Kosmala, Andrea Wiggins, Alexandra Swanson and Brooke Simmons, published in Frontiers in Ecology and the Environment in 2016, concluded that data quality depends heavily on project design, and that volunteers can produce accurate data across a range of tasks when projects use appropriate protocols, training and analysis. The authors noted that the same quality problems, such as observer variability, affect professional data too, and that good projects measure and report data quality rather than assuming it.

The lesson is that "citizen science data" is not a single category. A well-designed project with transparent methods can produce data used in peer-reviewed research and public policy; a poorly designed one cannot, regardless of who collects the data.

Document everything: metadata

Data that others cannot understand cannot be reused. Projects should record and publish:

the protocol version used for each record;
the instruments and their calibration history;
how records were validated, and which were changed or excluded;
known biases, such as more records near roads or in wealthier neighbourhoods.

Biodiversity projects commonly use the Darwin Core standard to share records through the Global Biodiversity Information Facility (GBIF). Metadata standards for describing citizen science projects themselves, such as PPSR Core, have also been developed by the citizen science community.

Quality and participation

Strict quality rules can conflict with the goals of participatory projects, for example if complex protocols exclude people with less time or formal education. Co-created projects often resolve this by involving volunteers in designing the protocol, which tends to improve both clarity and commitment. Feedback also matters: volunteers who see how their data are used, and receive individual feedback on errors, tend to improve. Our introduction to citizen science describes the different project models, and our guide to open and community data covers publishing results responsibly.

Sources

US Environmental Protection Agency, Quality Assurance Handbook and Guidance Documents for Citizen Science Projects, 2019.
Margaret Kosmala, Andrea Wiggins, Alexandra Swanson and Brooke Simmons, "Assessing data quality in citizen science", Frontiers in Ecology and the Environment, vol. 14, no. 10, 2016, pp. 551-560.
Brian L. Sullivan et al., "eBird: a citizen-based bird observation network in the biological sciences", Biological Conservation, vol. 142, no. 10, 2009, pp. 2282-2292.
Biodiversity Information Standards (TDWG), Darwin Core standard.