Changes Affecting Data
This page contains a list of changes that may affect previously downloaded data. These changes are not specified in data descriptions for they apply more broadly than to a specific OOI dataset.
The list below includes changes made to specific metadata, algorithms used to calculate some datasets, corrections to data of discovered errors, and tweaking of instrument calibrations. Details are provided about any actions taken, how data may have been affected, and the time frames involved.
Ongoing Corrections
UPDATED 2020-02-18: As part of our ongoing verification of OOI data products, the data teams have undertaken a thorough, program-wide review of all critical metadata in the system, including instrument calibration coefficients, instrument deployment assignments, and deployment dates.
Learn more about the combined human-in-the-loop (HITL) and automated auditing process. Users can review the report to determine if any metadata changes have affected previously downloaded data. If a subsequent change has been made that impacts data, users should re-download their data to ensure that they have the corrected values. This initial review of calibration metadata was completed at the end of December 2019. Moving forward, all metadata verification will be consistent with the process established during this review.
This metadata widget is a tool you can use to search for any metadata changes that may affect your previously downloaded data. You can search by the original download date, array, platform, and instrument. An option is given to download any metadata changes found as a CSV file.
2022
Dissolved oxygen data for several early OOI glider deployments were incorrect due to an unexpected setting on the oxygen optode. This issue, which impacted both recovered and telemetered L2 data, has now been corrected. The issue arose from changes to an internal oxygen sensor parameter used to compensate the instrument’s dissolved oxygen concentration for a constant salinity value. In many instances, this internal parameter was inadvertently set to 35 ppt, typically following routine service and recalibration. The OOI salinity and compensation algorithm in OOINET assumed that this parameter would always be set to zero.
The mathematics of the salinity compensation algorithm can normally account for any constant salinity value by computing the difference from the true salinity values obtained from the CTD. However, the existing OOINet ion-function algorithm assumed that the salinity preset value was always zero. As a result of this oversight, any deployments with preset salinity different from zero will have a potential error in the L2 salinity and pressure compensated dissolved oxygen. Our analysis shows that this error approaches 65 µmol/kg or as much as ~18% of typical surface ocean oxygen concentrations.
This problem mainly affected deployments from 2013-2018. The OOI Data Team has now examined all glider deployment records and data and cataloged the affected deployments. Annotations were added to relevant data streams, indicating which deployments had the incorrect preset salinity value of 35 ppt. All current and future deployments from approximately 2018 onward are routinely checked following service intervals and the preset salinity value is reset to the correct zero value.
The salinity and pressure compensation algorithm has now been updated to incorporate an optional preset salinity reference value input. The asset-management calibration and configuration metadata system handles storing the salinity reference value for deployment configurations and provides the input. The algorithm can be found on Github.
Note that the L1 data (in which salinity and pressure compensation are not yet applied) for the affected deployments will retain the original, incorrect, constant salinity value of 35 in the resulting data products. Therefore, OOI discourages use of the L1 data in any subsequent analyses. OOI encourages that users preferentially use the L2 data product. This will have the true CTD salinity and pressure compensation applied and incorporates the new correction for the incorrect preset salinity values. These corrections were also applied to the datasets submitted to the IOOS GDAC and OOI Data Explorer.
Deployments with the affected oxygen data were added to the OOI Critical Metadata review and are available via the OOI Metadata Widget.
We are in the process of changing the values of data stream parameters to be more Climate and Forecast (CF) compliant and to make the data you are looking for easier to find. For example, the parameter “pressure_depth” has been renamed to simply “pressure.” This next round of changes are documented in file format (excel, .csv) and accessible below. A list of variable name changes is also provided below. The updates are scheduled to be released on August 25th at 1:00pm ET. Should you have any questions about this process or others, please don’t hesitate to reach out to our HelpDesk.
2021
In 2021, the OOI Data Team fixed all of the Coastal Surface Piercing Profiler (CSPP) data. The CSPP control can was putting incorrect time stamps and corresponding depths on data. Code is now applied to profiler output that corrects data before they are sent to the cyberinfrastructure (CI) and made available for use. All data on the CI are now fixed, including raw data. Coastal Endurance Array Research Associate Russ Desiderio wrote the code that fixes these data.
An example of CSPP corrected data
2020
UPDATED 2020-02-04: Blank values (e.g. in the data stream: optical absorbance ratios in stream pco2w_b_sami_data_record_cal) are used to calculate the data product “pCO2 Seawater (µatm)” at a specific timestamp.
A fix was applied on 2020-02-04 to ensure that any pCO2 data request will use a linearly interpolated value from the closest blanks if no blank value is found within the requested time range. This prevents the system from returning NaNs or empty datasets in those cases. Restrictions were also put in place to prevent interpolation across deployments, which could pull blanks from different instrument serial numbers, potentially creating bad data. These fixes apply to all pCO2 data, so all derived Level 1 pCO2 data products generated prior to this notification date should be re-requested to ensure utilization of the correct interpolation code.
2019
UPDATED 2019-01-15: After careful review, the original data product algorithm (pco2_pco2wat) for Sunburst sensors was found to contain an error that resulted in calculation of incorrect derived L1 pCO2 values. In consultation with the vendor, the algorithm has been corrected, and is in the process of being verified. Once complete, this notification will be updated, and all derived L1 pCO2 data products generated prior to the notification date should be re-requested in order to ensure utilization of the correct algorithm. See the updated algorithm on Github.
2018
2015-02-18 16:06:00 to Present: The telemetered FDCHP data contains an onboard processing error that makes the motion-corrected wind speed and flux data unusable. The instrument subject matter expert is being consulted, and the algorithms are under review.
Only the processed, recovered data should be used. This impacts instruments on the Endurance Oregon Shelf Surface Mooring (CE02SHSM), the Pioneer Central Surface Mooring (CP01CNSM), the Global Irminger Sea Surface Mooring (GI01SUMO), and the Global Southern Ocean Surface Mooring (GS01SUMO). Note: New units were deployed at various sites from August 2019-January 2020 and end dates for each platform can be found in the annotations.
2016
A data anomaly regarding some of the OOI oxygen data (DOSTA) was discovered. Many of the currently deployed Aanderaa optode oxygen sensors (see specific list) had an error during the manufacturer’s calibration of the instruments. Incorrect offset values were causing the OOI algorithm to report processed O2 values an order of magnitude higher than expected. (Due to the source of the error, the raw data reported from these sensors was affected).
Correct O2 values can likely still be calculated from raw phase and temperature data, and this error should not ultimately lead to a gap in overall O2 data. The OOI Data Team informed users of the issue, and removed the erroneous calibration values to keep the system from continuing to produce incorrectly derived O2 products. In the medium term, the Data Team used a back-calculation from the phase data to replace the calibration values. Moving forward, the data team kept a careful record of what was done and added that information to data annotations and to the website.
- The Observatory
- About OOI
- Arrays
- Global Irminger Sea Array
- Global Station Papa Array
- Coastal Endurance Array
- Coastal Pioneer Mid-Atlantic Bight Array
- Coastal Pioneer New England Shelf Array
- Regional Cabled Array
- Cabled Continental Margin Array
- Cabled Axial Seamount Array
- Cabled Endurance Array
- Global Argentine Basin
- Global Southern Ocean Array
- Additional Observing Assets in the Region
- Platforms and Nodes
- Infrastructure
- Instruments
- Science
- Participate in OOI
- Community Engagement
- OOI Data
- Resources