Home :: Data Tools ::Need Help? ::FAQs (Knowledge Base) ::Interpreting QC variables and results

As part of the OOI’s QA/QC procedures, most scientific data products provided by the Data Portal are run through a standard set of quality control tests. You can find information on the tests and the algorithms used on the Quality Control page. Here we will focus on how to interpret the results from the QC tests.

First, it’s important to note that the QC checks are still under active development. As noted on the QC page, not all tests are currently operational.

In addition, data are not removed based on the results of the QC tests. While some data points may fail one or more QC tests, all data points are still included in the file. It is up to you to decide whether data points should be omitted based on these tests or others you may run. As noted on the QC page…

The automated QC algorithms do not screen out or delete any data, or prevent it from being downloaded. They only flag “suspect” data points in the plotting tools and deliver those flags as additional attributes in downloaded data.

When you open up a NetCDF file, you will notice that each variable (aka parameter or data product) produced by the OOI data system will have a corresponding *_qc_executed and *_qc_results variable in the outputted file. For most time-series measurements, the dimensions of this variable will correspond in length to the number of data points (that is, time measurements) in the file. So, generally, it will be a 1D array like the corresponding data variable.

Determining which QC Tests were run

The qc_executed value denotes which QC checks were run and included in the results for that time point. For example, let’s say a variable in a METBK (Bulk Meteorology instrument) data file has a constant qc_executed value of 29. In Matlab, you can convert this value to a binary value using…

dec2bin(29) = 11101.

Taking these values in reverse order means that the 1st, 3rd, 4th and 5th QC checks were run. We can pad the number with leading zeros (e.g. 00011101) and assume any tests greater than the 5th one were not run in this case.

According to the following list, this means that the Global Range, Spike, Trend and Stuck Value tests were run.

QC Test Order

  1. global_range_test
  2. dataqc_localrangetest
  3. dataqc_spiketest
  4. dataqc_polytrendtest
  5. dataqc_stuckvaluetest
  6. dataqc_gradienttest

As noted, you can find more information on how these tests operate on the QC information page.

In this case, the Local Range test was not run by the system because those values are still being defined and have not been entered into the system yet. One of the major upcoming tasks for the OOI Data Evaluation Team is to define local ranges for all parameters, now that we have 1-2 years of data from many of our instruments.

Likewise, we are still improving the Spike, Trend, Stuck Value, and Gradient tests, so the results may not be applicable to every dataset at this time. In particular, the Trend test may not be working as designed (to check if a significant fraction of the variability in a time series can be explained by a drift, assumed to be a polynomial of specified order), because it requires the system to compare data prior to the user request date, and because there are many reasons other than sensor drift why the data might fit a polynomial curve. The Spike test is currently very simple, and needs tweaking to avoid false positives/negatives (especially in biological data) and to work with certain data types. The lookup table values for the Stuck Value test need to be reviewed to ensure that they accurately reflect the instrument sensitivity, as many of them were filled in prior to instrument procurement. The Gradient test (designed to determine whether the gradient between successive points fits within a specified range) is complicated for the system to apply, because it requires a 2D dataset. So the results for these tests, even when they are run, may not be ideal for all user purposes.

Determining which QC Tests passed

Next, you can interpret the qc_results value to figure out which checks passed. For example, a METBK file may have values of 21 or 20. A value of 20 equates to 10100 in binary, which means that the Spike and Stuck Value tests passed, but Global Range and Trend tests failed. And since the Local Range test wasn’t run, by default it was marked as failed as well. A value of 21 (which translates to 10101) means the Global Range test also passed.

Generating a list of explicitly failed tests

As you may have noticed, the qc_results will include 0’s for all tests that failed or were not run. If you are working in Python, you can use the following snippet to figure out which tests failed, or whether all tests that were run passed.

( 0b1111111111111111 XOR qc_executed ) OR qc_results

This generates a list of the “effective good” flags. This first part generates a list of “passes” for all non-executed functions. Then the results of the executed functions are passed in with “OR”, leaving the explicitly failed executed functions as the only zeros.

For more guidance, please check out this iPython notebook that walks through a similar process for deciphering the QC fields in Python.

— Last revised on June 28, 2017 —