OOI Data Quality Control procedures were designed with the goal of meeting IOOS Quality Assurance of Real Time Ocean Data (QARTOD) standards. In addition to daily human-in-the-loop QC tests, as data streams are collected, data products are run through up to six automated QC algorithms. QC reports are created on a biweekly or monthly basis.
Specific details of QA/QC methods for OOI data and data products and physical samples can be found within the Protocols and Procedures for OOI Data Products document. This includes calibration and field verification procedures.
Automated QC Algorithms
Data products are run through six automated QC algorithms. QC reports are created on a biweekly or monthly basis. Automated QC algorithms were coded based on specifications created by OOI Project Scientists and derived from other observatory experiences. The six algorithms currently implemented are:
|OOI Test||OOI Description||QARTOD Equivalent||QARTOD Recommendation (from manuals)||Notes|
|Global Range Test (pdf)||Data are flagged unless they fall within valid world ocean ranges or instrument limits (whichever is more restrictive)||Gross Range||Only considers manufacturer-defined sensor and calibration limits||Different tests, different names.
OOI test is currently operational.
|Spike Test (pdf)||Deviation from mean compared to 2*N neighboring points||Spike||N=1, default threshold is based on the rate of change distribution from previous data sets||Roughly identical, same nomenclature.
OOI test is currently operational.
|Stuck Value Test (pdf)||If 2 neighboring values differ by less than the resolution of the sensor for more than N repetitions, data are flagged||Stuck Sensor||Manual suggests 3 consecutive points for a stuck sensor suspect flag and 5 for a fail flag.||QARTOD manual suggestion may be too low for well-mixed portions of the water column.
We are evaluating the results from the OOI lookup values.
|Local Range Test (pdf)||Data are flagged unless they fall within locally valid site-specific or depth ranges. Interpolates thresholds between depth and season intervals||Local Range||Starts with constant limits for each depth/season interval||Roughly identical, same nomenclature.
OOI Local ranges are still being established and entered based on first year of operations.
|Gradient Test (pdf)||If d(data)/d(t) between two points is greater than a set threshold, all following points fail until one falls within an absolute limit (TOLDAT). First data point is assumed good unless a “good” starting data (STARTDAT) point is defined.||Rate of Change||QARTOD recommends two neighboring points and does not incorporate TOLDAT or STARTDAT values.||Different tests, different names.
OOI Gradient test is under review and not currently operational.
|Trend Test (pdf)||Data are flagged as having a trend if the SD of the residuals to a polynomial curve < original data, multiplied by some factor. Designed to test for sensor drift.||N/A||No QARTOD equivalent||OOI only.
This test is not currently operational, and is being reviewed for efficacy.
A seventh (Density Inversion) test will hopefully also be implemented, based on its utility in other observatory projects (e.g. Argo). This test generates a flag if data at increasing depths do not also increase in density (or vice versa)
QC algorithms are only run on science data products, not on auxiliary or engineering parameters. The automated QC algorithms do not screen out or delete any data, or prevent it from being downloaded. They only flag “suspect” data points in the plotting tools and deliver those flags as additional attributes in downloaded data.
The algorithm code used to process the QC algorithms in the OOI Cyberinfrastructure system can be found in the ion-functions GitHub repository.
QC lookup tables that describe these limits can be in the ooi-integration GitHub repository.
Manual QC Tests
After automated QC flags are generated by the algorithms and saved, daily human-in-the-loop QC tests are performed. Manual QC tests are conducted by a team of four data evaluators, led by the data manager. Tests include both Quick Look tests (a first pass by evaluators using automated tools) as well as Deep Dives (a closer look at data flagged as suspect, drawing in Subject Matter Experts). The data team will clearly annotate any data stream that triggers QC-related alerts, as well as any data that are flagged as suspect during manual inspection.
If a user identifies an issue with an OOI data product or has a question about QC procedures, please contact the Data Team through the OOI Helpdesk.
Quality Control Goals
OOI instrument deployment and data quality control procedures were designed with the goal of meeting QARTOD quality control standards:
- Every real-time observation must be accompanied by a quality descriptor
- All observations should be subject to automated real-time quality tests
- Quality flags and test descriptions must be described in the metadata
- Observers should verify / calibrate sensors before deployment
- Observers should describe methods / calibration in real-time metadata
- Observers should quantify level of calibration accuracy and expected error
- Manual checks on automated procedures, real-time data collected, and status of observing system must be provided on an appropriate timescale