The OOI Data Team continues to listen to data users’ feedback to refine and improve Data Explorer. Many of those improvements are reflected in the latest release of Data Explorer, version 1.2, which is now operational. Data Explorer was originally released in September 2020, and this latest version is the second round of improvements made by Axiom Data Science, working with the OOI Data Team.
This version makes more OOI data accessible online and brings new features for gliders and profilers. It is also now possible to search for cruise data in the tabular search interface. Once there, you can select specific cruises, see their data profiles, and have a three-dimensional view of where the samples were taken. Glider data are also now available online and searchable by time and location. Once you’ve identified a glider of interest, it is possible to map or plot the glider’s route and compare data collected with data from other sensors. With another click, you can compare sensor data with profiler information, then change the parameters on the screen to learn more.
All data can be downloaded in csv, GeoJSON, KML, and ShapeFile formats for future use.
Additionally, discrete sample data (chemical analyses of seawater collected during shipboard verification sampling) have been added. Water samples are collected during OOI cruises at multiple depths, and analyzed for oxygen (Winkler), chlorophyll-a fluorescence and pigment distribution, nitrate/nitrite, and potentially a full nutrient suite, total DIC (dissolved inorganic carbon) and total alkalinity, pH, and salinity. These data can be used to compare to in situ instrument data or CTD casts in order to ensure OOI data quality. It is now possible to use Data Explorer 1.2 to convert discrete dissolved oxygen sample data from milliliters to micromoles and create standard_name mapping for discrete sample data.
In response to users’ feedback, many defects found in version 1.1 have been fixed. A summary of all the new features and bug fixes is available in the release notes.
“Data Explorer is a tool that allows users to access, manipulate, and understand OOI data for use in their research and classroom,” said Jeff Glatstein, OOI Data Delivery Lead and Senior Manager of Cyberinfrastructure. “Users’ feedback has been—and will continue to be— extremely useful in refining Data Explorer to ensure it meets users’ need and expectations. We are holding regular open meetings as one way to ensure that we receive timely feedback and work with our users to meet their needs.”
An open OOI Town Hall previewing some of the new and special glider-related features was held on 24 August 2021, where user input was welcomed. A recording of the session is available here.Read More
It’s official. As of Friday 30 July 2021, OOI data are now being stored and served on their new Cyberinfrastructure housed at the OOl Data Center at Oregon State University (OSU). The transfer of data from Rutgers, the State University of New Jersey, has been in the works since last October, when OSU was awarded the contract for the center.
“The transfer of 105 billion rows of data was nearly seamless, which attests to the collaboration between the OOI technical and data teams, said Jeffrey Glatstein, OOI Data Delivery Lead and Senior Manager of Cyberinfrastructure. “It literally took a village and we are grateful to the marine implementing organizations at the University of Washington, Woods Hole Oceanographic Institution, and Oregon State University for their part in making this transfer happen while data continue to be collected 24/7.”
OSU Project Manager Craig Risien added, “July 30, 2021 was the culmination of about ten months of working with our Dell EMC partners, the OOI Marine Implementing Organizations, and the OOI Cyberinfrastructure team to deliver a state-of-the-art and highly extensible data center to meet OOI’s present and future data handling needs. We are very pleased with how the data center migration project has proceeded, thus far.”[feature]
[caption id="attachment_21888" align="aligncenter" width="519"] The primary repository of OOI raw data is the Dell EMC Isilon cluster, a scale out network-attached storage platform for storing high-volume, unstructured data. Photo: Craig Risien, OSU.[/caption]
OSU’s Data Center was designed to handle telemetered, recovered, and streaming data for OOI’s five arrays that include more than 800 instruments. Telemetered data are delivered to the Data Center from moorings and gliders using remote access such as satellites. Recovered data are complete datasets that are retrieved and uploaded once an ocean observing platform is recovered from the field. Streaming data are delivered in real time directly from instruments deployed on the cabled array.
“The OSU Data Center includes modern storage solutions, Palo Alto next-generation firewalls that ensure system security, and a hyperconverged ‘virtual machine’ infrastructure that makes the OOI software and system easier to manage and more responsive to internal and external data delivery demands, “ explained OSU Principal Investigator Anthony Koppers. “With this equipment now operational at OSU, we are well positioned to seamlessly serve the more than 1 PB of critically important data collected by the OOI to the wide-ranging communities doing marine research and education with ample space to grow well into the future.”Read More
Recognizing that freely accessible ocean observatory data has the potential to democratize interdisciplinary science for early career researchers, Levine et al. (2020) set out to demonstrate this capability using the Ocean Observatories Initiative. Publicly available data from the OOI Pioneer Array moorings were used, and members of the OOI Early Career Scientist Community of Practice (OOI-ECS) collaborated in the study.
A case study was constructed to evaluate the impact of strong surface forcing events on surface and subsurface oceanographic conditions over the New England Shelf. Data from meteorological sensors on the Pioneer surface moorings, along with data from interdisciplinary sensors on the Pioneer profiler moorings, were used. Strong surface forcing was defined by anomalously low sea level pressure – less than three times the standard deviation of data from May 2015 – August 2018. Twenty-eight events were identified in the full record. Eight events in 2018 were selected for further analysis, and two of those were reported in the study (Figure 24).[media-caption path="https://oceanobservatories.org/wp-content/uploads/2021/07/CGSN-Highlight.png" link="#"]Figure 24. Two surface forcing events (16 and 27 November) identified from the time series of surface forcing at the Pioneer Central surface mooring. Vertical lines indicate the peak of the anomalous low-pressure events (gray), as well as times 48 h before (red) and after (blue). (A) sea level pressure, (B) wind speed, (C) air temperature, (D) latent (solid) and sensible (dashed) heat fluxes, (E) sea surface temperature, and (F) surface current speed and direction. [/media-caption]
The impact of surface forcing on subsurface conditions was evaluated using profile data near local noon on the day of the event, as well as 48 hr before and after (Figure 24). Subsurface data revealed a shallow (40-60 m) salinity intrusion prior to the 16 November event, which dissipated during the event, presumably by vertical mixing and concurrent with increases in dissolved oxygen and decreases in colored dissolved organic matter (CDOM). At the onset of the 27 November event, nearly constant temperature, salinity, dissolved oxygen and CDOM to depths of 60 m were seen, suggesting strong vertical mixing. Data from multiple moorings allowed the investigators to determine that the response to the first event was spatially variable, with indications of slope water of Gulf Stream origin impinging on the shelf. The response to the second event was more spatially-uniform, and was influenced by the advection of colder, fresher and more oxygenated water from the north.
The authors note that the case study shows the potential to address various interdisciplinary oceanographic processes, including across- and along- shelf dynamics, biochemical interactions, and air-sea interactions resulting from strong storms. They also note that long-term coastal datasets with multidisciplinary observations are relatively few, so that the Pioneer Array data allows hypothesis-driven research into topics such as the climatology of the shelfbreak region, seasonal variability of Gulf Stream meanders and warm-core rings, the influence of extreme events on shelf biogeochemical response, and the influence of a warming climate on shelf exchange.
In the context of the OOI-ECS, the authors note that the study was successfully completed using open-source data across institutional and geographic boundaries, within a resource-limited environment. Interpretation of results required multiple subject matter experts in different disciplines, and the OOI-ECS was seen as well-suited to “team science” using an integrative, collaborative and interdisciplinary approach.
Levine, RM, KE Fogaren, JE Rudzin, CJ Russoniello, DC Soule, and JM Whitaker (2020) Open Data, Collaborative Working Platforms, and Interdisciplinary Collaboration: Building an Early Career Scientist Community of Practice to Leverage Ocean Observatories Initiative Data to Address Critical Questions in Marine Science. Front. Mar. Sci. 7:593512. doi: 10.3389/fmars.2020.593512.Read More
Hydrographer Leah McRaven (PO WHOI) from the US OSNAP team provided the following CTD resources to help researchers and others better how she and the Irminger Sea Array team are working with the near real-time data being provided by CTD sampling from the R/V Neil Armstrong:
There are four main sources considered in this list:
- Seabird Electronics is one of the most commonly used manufacturers of shipboard CTD systems. Their CTDs allow for integration of instruments from several other manufactures.
- The Global Ocean Ship-Based Hydrographic Investigations Program (GO-SHIP) provides decadal resolution of the changes in inventories of heat, freshwater, carbon, oxygen, nutrients and transient tracers, with global measurements of the highest required accuracy to detect these changes. Their program has documented several methods and practices that are critical to high-accuracy hydrography, which are relevant to many CTD data users.
- The California Cooperative Oceanic Fisheries Investigations (CalCOFI) are a unique partnership of the California Department of Fish & Wildlife, NOAA Fisheries Service and Scripps Institution of Oceanography. CalCOFI conducts quarterly cruises off southern & central California, collecting a suite of hydrographic and biological data on station and underway. CalCOFI has made great effort to document methods that are helpful to those collecting hydrographic measurements near coastal regions.
- University-National Oceanographic Laboratory System (UNOLS) is an organization of 58 academic institutions and National Laboratories involved in oceanographic research and joined for the purpose of coordinating oceanographic ships’ schedules and research facilities.
Instrument care and use
Seabird training module on how sensor care and calibrations impact data: https://www.seabird.com/cms-portals/seabird_com/cms/documents/training/Module9_GettingHighestAccuracyData.pdf
Data acquisition and processing
Notes on CTD/O2 Data Acquisition and Processing Using Seabird Hardware and Software: https://www.go-ship.org/Manual/McTaggart_et_al_CTD.pdf
CalCOFI Seabird processing: https://calcofi.org/about-calcofi/methods/119-ctd-methods/330-ctd-data-processing-protocol.html
Seabird CTD processing training material: https://www.seabird.com/training-materials-download
Within this material, discussion on dynamic errors and how to address them in data processing: https://www.seabird.com/cms-portals/seabird_com/cms/documents/training/Module11_AdvancedDataProcessing.pdf
General overview documents and resources
GOSHIP hydrography manual: https://www.go-ship.org/HydroMan.html
CalCOFI CTD general practices: https://www.calcofi.org/references/methods/64-ctd-general-practices.html
OOI uses the SAMI2-pH sensor from Sunburst Sensors, LLC to measure seawater pH throughout the different arrays. Assessing the data quality from this instrument is an involved process as there are multiple parameters produced by the instrument that are then used to calculate the seawater pH. These measurements are subject to different sources of error, and those errors can propagate through the calculations to create an erroneous seawater pH value. Based upon the vendor documentation and MATLAB code Sunburst provides to convert the raw measurements, OOI data team members have created a set of rules from those different measurements to flag the pH data as either pass, suspect or fail.
The resulting flags can be used to remove failed data from further analysis. They can also be used to help generate annotations for further Human in the Loop (HITL) QC checks of the data to help refine quality metrics for the data. OOI team member, Chris Wingard (OSU), has written up the QC process as a Python Jupyter notebook. This notebook and other example notebooks are freely available to the scientific community via the OOI GitHub site (within the OOI Data Team Python toolbox accessed from https://oceanobservatories.org/community-tools/ ).
In this notebook, Wingard shows how the quality rules can be used to remove bad pH data from a time series, and how they can be used to then create annotations. The impact of using these flags is shown with a set of before and after plots of the seawater pH as a function of temperature. The quality controlled data can then be used to estimate the seasonal cycle of pH to set climatological quality control flags.
Here an example is shown using data from a pH sensor on the Oregon Inshore Surface Mooring (CE01ISSM) near surface instrument frame (NSIF), deployed at 7 m depth (site depth is 25 m).[media-caption path="https://oceanobservatories.org/wp-content/uploads/2021/07/EA-Highlight.png" link="#"]Figure 25: pH data from the Oregon Inshore Surface Mooring (CE01ISSM) near surface instrument frame (NSIF). Good data are shown in black, failed data in red. Note that simple range tests on the final calculated pH are often not enough to distinguish good from failed data. The automated QC processing examines intermediate measurements and fails data if intermediate measurements are outside acceptable ranges and propagated to final measurements.[/media-caption] [media-caption path="https://oceanobservatories.org/wp-content/uploads/2021/07/EA-highlight-2.png" link="#"]Figure 26: Good data together with annual cycles (red) constructed with available good data from initial deployment through 2021. Data which falls outside three standard deviations of the climatology is flagged as suspect. The climatological tests are used to flag suspect data. Simple range tests for suspect (cyan) and failed (magenta) data are also shown. The annual cycle at this site is strongly influenced by annual summer upwelling and winter storms and river plumes. The summer decrease in pH is consistent with cold, relatively acidic upwelled water high in CO2 (see e.g., Evans et al., 2011)[/media-caption]
Evans, W., B. Hales, and P. G. Strutton (2011), Seasonal cycle of surface ocean pCO2on the Oregon shelf,J. Geophys. Res., 116, C05012, doi:10.1029/2010JC006625.Read More
The OOI’s Coastal and Global Array teams regularly use Teledyne-Webb Slocum Gliders to collect ocean observations within and around the array moorings. The gliders fly up and down the water column from the surface down to a maximum depth of 1000 meters, collecting data such as dissolved oxygen concentrations, temperature, salinity, and other physical parameters to measure ocean conditions.
OOI shares its glider data with the Integrated Ocean Observing System (IOOS) Glider Data Assembly Center (DAC). IOOS serves as a national repository for glider data sets, serving as a centralized location for wide distribution and use. It allows researchers to access and analyze glider data sets using common tools regardless of the glider type or organization that deployed the glider.
OOI serves data to these repositories in two ways. When the gliders are in the water, data are telemetered, providing near real-time data to these platforms. Once the gliders are recovered, data are downloaded, metadata provided, and data are resubmitted to the Glider DAC as a permanent record.
The behind-the-scene process transmitting this huge amount of data is quite complex. OOI Data Team members, Collin Dobson of the Coastal and Global Scale Nodes at Woods Hole Oceanographic Institution (WHOI) and Stuart Pearce of the Coastal Endurance Array at Oregon State University (OSU) teamed up to streamline the process and catch up on a backlog of submission of recovered data.
Pearce took the lead in getting the OOI data into the DAC. In 2018, he began writing code for a system to transmit near real-time and recovered data. Once the scripts (processing code) were operational by about mid-2019, Pearce implemented them to streamline the flow of Endurance Array glider data into the DAC. Dobson then adopted the code and applied it to the transmission of glider data from the Pioneer, Station Papa, and Irminger Sea Arrays into the repository.
As it turned out, timing was optimum. “ I finished my code at the same time that the Glider DAC allowed higher resolution recovered datasets to be uploaded,” said Pearce. “So I was able to adjust my code to accommodate the upload of any scientific variable as long as it had a CF compliant standard name to go with it.” This opened up a whole range of data that could be transmitted in a consistent fashion to the DAC. CF refers to the “Climate and Forecast” metadata conventions that provide community accepted guidance for metadata variables and sets standards for designating time ranges and locations of data collection. Dobson gave an example of the name convention for density: Sea_water_density.
“Being CF compliant ensures your data have the required metadata and makes the data so much more usable across the board,” added Dobson. “If I wanted to include oxygen as a variable, for example, I have to make sure to use the CF standard name for dissolved oxygen and report the results in CF standard units.”
The Endurance Array team was the first group to add any of the non-CTD variables into the Glider DAC. This important step forward was recognized by the glider community, and was announced at a May 2019 workshop at Rutgers with 150 conveyors of glider data in attendance. One of Pearce’s gliders was used as the example of how and what could be achieved with the new code.
To help expedite the transfer of all gliders into the DAC, Pearce made his code open access. The additional metadata will help advance the work of storm forecasters, researchers, and others interested in improving understanding ocean processes.
Since its inaugural launch in October 2020, OOI has been working with users of Data Explorer to learn what features worked for them, which could be improved, and what could be added to optimize users’ experiences. This input has been put into practice and is now available for further testing on Data Explorer v1.1.
Improvements made to this version include the addition of five new instrument data types: Wire-following, Surface-piercing, Cabled Deep and Shallow Profilers, and Cabled Single Point Velocity Meters. Changes were made to improve the display and use of ERDDAP data. Now it is possible to print custom configuration of time-series and data comparison charts.
A global search capability was added to allow users to use search terms to discover data sets in the Data Explorer. The search and navigation functions were tweaked to also find the data sets across all instruments and times. Other behind-the-scenes fixes were implemented to improve the site’s overall operability and functionality for users. The release notes can be viewed here.
“This version of Data Explorer incorporates suggestions from its growing community of users. We’re pleased to have received feedback that is serving to make Data Explorer a tool that meets users’ needs, which is our ultimate goal.” said Jeffrey Glatstein, OOI Data Delivery Lead and Senior Manager of Cyberinfrastructure.
A preview of the new features of Data Explorer v1.1 was held on 9 April 2021 and can be viewed below[embed]https://youtu.be/WhXgQ5qe78E[/embed]
OOI shares data with partner repositories and institutions that host similar data but have different user bases. These partnerships expand the data available for forecasting models, help provide insight into current ocean conditions, and serve as important resources for many ranging from fishers and other maritime users to land-based researchers and students.
With the exception of the Station Papa Array, the OOI Coastal and Global Arrays maintain surface buoys. Instruments deployed on these buoys measure meteorological variables such as air temperature, barometric pressure, northward and eastward wind velocities, precipitation, solar radiation, and surface water properties of sea surface temperature and salinity. Other instruments on the moorings collect wave data, such as significant wave height, period, and direction. These data are then consumed by national and regional networks to improve accuracy of weather forecasting models.
The Regional Cabled Array (RCA) consists of fiber-optic cables off the Oregon coast that provide power, bandwidth, and communication to seafloor instrumentation and moorings with instrumented profiling capabilities. A diverse array of geophysical, chemical, and biological sensors, a high-definition camera, and digital still cameras on the seafloor and mooring platforms, provide real-time information on processes operating on and below the seafloor and throughout the water column, including recording of seafloor eruptions, methane plume emissions and climate change. These data are available for community use. Since 2015, the RCA has fed data into Incorporated Research Institutions for Seismology (IRIS), the primary source for data related to earthquakes and other seismic activity. In addition, data including zooplankton sonar data, are being utilized within the Pangeo ecosystem for community visualization and access and pressure data are incorporated into NOAA’s operational tsunami forecasting system.
Helping Improve Models and Forecasting
One of the recipients of OOI data is the National Data Buoy Center (NDBC), part of the National Oceanic and Atmospheric Administration’s (NOAA) National Weather Service. NDBC maintains a data repository and website, offering a range of standardized real-time and near real-time meteorological data. Data such as wind speed and direction, air and surface water temperature, and wave height and direction are made available to the broader oceanographic and meteorological community.
“Many researchers go to NDBC for their data, “said Craig Risien, a research associate with OOI’s Endurance Array and Cyberinfrastructure Teams, who helps researchers gain access to and use OOI data. “NBDC is a huge repository of data and it’s easy to access. So there’s a low barrier for researchers and students who are looking for information about wind speed, water temperature and a slew of other data. OOI contributing to this national repository significantly increases its data reach, allowing OOI data to be used by as many people as possible. “
OOI sea surface temperature data also make their way into the operational Global Real-Time Ocean Forecast System (RTOFS) at the National Centers for Environmental Prediction (NCEP), another part of NOAA’s National Weather Service. RTOFS ingests sea surface temperature and salinity data from all available buoys into the Global Telecommunications System (GTS). OOI glider data also are pushed in near real-time to the US Integrated Ocean Observing System Glider Data Assembly Center (DAC). From there, the data goes to the GTS where it can be used by the operational modeling centers such as NCEP and the European Centre for Medium-Range Weather Forecasts.
The GTS is like a giant vacuum sucking up near real-time observations from all sorts of different platforms deployed all over the world. On a typical day, the GTS ingests more than 7,600 data points from fixed buoys alone. As a result of this vast input, researchers can go to the GTS, pull available data, and assimilate that information into any model to improve its prediction accuracy.
Advancing Forecasting of Submarine Eruptions
As the first U.S. ocean observatory to span a tectonic plate, RCA’s data are an invaluable contributor to IRIS’s collection. Since 2015, the user community has downloaded >20 Terabytes of RCA seismometer data from the IRIS repository. Fourteen different sampling locations include key sites at Axial Seamount on the Juan de Fuca mid-ocean ridge spreading center, near the toe of the Cascadia Margin and Southern Hydrate Ridge. RCA data are catalogued and available on the IRIS site, using the identifier “OO.”[caption id="attachment_21046" align="alignleft" width="300"] Data from short period seismometers installed at RCA’s Axial Seamount and Southern Hydrate Ridge sites are streamed live to IRIS. Credit: UW/NSF-OOI/Canadian Scientific Submersible Facility, V13.[/caption]
“RCA is a critical community resource for seismic data. Axial Seamount, for example, which erupted in 1998, April 2011, was the site of more than 8,000 earthquakes over a 24-hour period April 24, 2015 marking the start of large eruption,” explained Deb Kelley, PI of the RCA. “Being able to witness and measure seismic activity in real time is providing scientists with invaluable insights into eruption process, which along with co-registered pressure measurements is making forecasting possible of when the next eruption may occur. We are pleased to share data from this volcanically and hydrothermally active seamount so researchers the world over can use it to better understand processes happening at mid ocean ridges and advance forecasting capabilities for the first time of when a submarine eruption may occur.”
Providing Data with Regional Implications[caption id="attachment_21047" align="alignright" width="203"] Data from Endurance Array buoy 46100 are fed into WCOFS, where they are accessible to maritime users. Credit: OSU[/caption]
OOI also provides data to regional ocean observing partners. Data from two Endurance Array buoys (46099 and 46100), for example, are fed into a four-dimensional U.S. West Coast Operational Forecast System (WCOFS), which serves the maritime user community. WCOFS generates water level, current, temperature and salinity nowcast and forecast fields four times per day. The Coastal Pioneer Array is within the future Northeastern Coast Operational Forecast System (NECOFS). Once operational, Pioneer’s observations will potentially be used for WCOFS data assimilation scenario experiments.
Coastal Endurance Array data are shared with the Northwest Association of Networked Ocean Observing Systems (NANOOS), which is part of IOOS, and the Global Ocean Acidification Observing Network (GOA-ON). Endurance data are ingested by the NANOOS Visualization System, which provides easy access to observations, forecasts, and data visualizations. Likewise, for GOA-ON, the Endurance Array provides observations useful for measuring ocean acidification.
Data from three of the Pioneer Array buoys also are part of the Mariners’ Dashboard, a new ocean information interface at the Northeastern Regional Association of Coastal Ocean Observing Systems (NERACOOS). Visitors can use the Dashboard to explore the latest conditions and forecasts from the Pioneer Inshore (44075), Central (44076), and Offshore (44077) mooring platforms, in addition to 30+ other observing platforms throughout the Northeast.
“We are working hard to distribute the OOI data widely through engagement with multiple partners, which together are helping inform science, improve weather and climate forecasts, and increase understanding of the ocean,” added Al Plueddemann, PI of the Coastal and Global Scale Nodes, which include the Pioneer, Station Papa, and Irminger Sea Arrays.
Data from three of the Ocean Observing Initiative’s (OOI) Pioneer Array buoys are now part of the Mariners’ Dashboard, a new ocean information interface launched by our partners at NERACOOS (Northeastern Regional Association of Coastal Ocean Observing Systems). Visitors can use the Dashboard to explore the latest conditions and forecasts from the Pioneer Inshore, Central, and Offshore mooring platforms, in addition to 30+ other observing platforms throughout the Northeast.
The Mariner Dashboard delivers high-quality, timely data from a growing network of buoys and sensors into the hands of mariners heading to sea. The data provided by the Pioneer moorings are particularly valuable because there are few other observing platforms in the highly traveled and productive shelf break region.
Observations provided range from air pressure and temperature, sea surface temperature, wave height, direction, velocity, and duration to salinity. Check out the Pioneer Array’s contributions to the wealth of information on the new Mariners’ Dashboard here:[caption id="attachment_20963" align="alignleft" width="113"] Components used to collect and transmit ocean observations to shore.[/caption]
Pioneer offshore https://mariners.neracoos.org/platform/44077
Pioneer central: https://mariners.neracoos.org/platform/44076
Pioneer inshore: https://mariners.neracoos.org/platform/44075
“We are pleased to see the Pioneer Array data being made more widely available through the Mariners’ Dashboard to help provide information about current ocean conditions,” said Al Plueddemann, head of the Ocean Observatories Initiative Coastal and Global Surface Nodes team, which includes the Pioneer Array. “This is just one example of how OOI data, which are freely available to anyone with an Internet connection, are being put to good use.”
Data Explorer continues to be refined in response to data users’ suggestions. Upcoming updates and refinements are previewed in this 55-minute video. The new, improved site will launch 5 May 2021. Look for it!
[embed]https://youtu.be/WhXgQ5qe78E[/embed] Read More