Top Menu

Renaming Data Stream Parameters

We are in the process of changing the names of data stream parameters to be more user-friendly and make the data you are looking for easier to find.  For example, the parameter “pressure_depth” has been renamed to simply “pressure.”

This renaming process has started with pressure_depth, and will be ongoing in our attempt to improve access to OOI data for all users.  We will post data stream name changes as they occur on both the Data Portal and the Data Updates page.  Should you have any questions, please don’t hesitate to reach out to our helpdesk.  Note that the renaming process only applies to data requests, and for now, only applies to “pressure_depth. “

The mechanism for renaming parameters is explained below:

  • We’ve added a new “netcdf_name” column in the preload database “Parameter” table.

The corresponding CSV file in the preload-database code repository is ParamterDefs.csv. When no netcdf_name is specified in the CSV file, the netcdf_name will be populated in the database using the value of the “name” column from the CSV file. The effect of this is that netcdf_name will exactly match the original name unless the netcdf_name has been changed in the CSV file. We chose to use the new netcdf_name column rather than the existing name column because the name column, in many cases, is coupled to the table name for the data in Cassandra, the OOI database system. Having the Cassandra table name and the preload-database name column out of sync would affect the ability to match data to fulfill data requests.

  • For those seeking more details, here’s how the renaming is accomplished:

The Stream Engine computes a dataset in the same way that was done before with one additional step.  The parameters of the data request and its name will appear in the netcdf_name column (i.e. parameter.netcdf_name from the database). Once this renaming is complete, the Stream Engine will write the output data files.

This renaming will apply to all data requests, both synchronous and asynchronous, including plotting and all output formats— CSV, JSON, and NetCDF. Plotting in the UI  has been updated to including the renaming of parameters.

The plotting parameter dropdown menu displays parameter.displayname. But, a new NetCDF attribute, “alternative_parameter_name” has been added to help users recognize when parameter renaming has occurred. In addition to renaming the NetCDF variables themselves, Stream Engine also renames parameters in the annotation JSON file (if requested), in the ‘ancillary_variables’ attribute, and other attributes computed when writing the NetCDF file, as well as renaming quality control variables.  So, for example, “pressure_depth_qc_executed” is now “pressure_qc_executed”. This same renaming strategy applies to QARTOD variables.

The renaming of parameters only applies to the results of data requests. References to parameters outside of data request results should still use either the parameter identifier (e.g. PD7) or the old name (i.e. parameter.name). For example, quality control records and annotation records should NOT use parameter.netcdf_name.

How this change will affect end users:

  • All data request output (async and sync, NetCDF, CSV, JSON, plotting), where data variables have been named using the parameter.name attribute, now will be named using the parameter.netcdf_name attribute.
    • Scripts that used parameter.name should now use parameter.netcdf_name. The preload web service has been updated to provide this information in its JSON responses. Alternatively, the information is available in the preload-database file ParameterDefs.csv.
    • QARTOD and legacy QC test variables have also been renamed.
  • In data request annotations JSON files, parameter names in the JSON will be replaced with parameter.netcdf_name.
  • In NetCDF variable attributes, parameter names will be replaced with parameter.netcdf_name, specifically in the case of ‘ancillary_variables’, but potentially elsewhere, as well.
  • A new “alternative_parameter_name” attribute will be included for parameters in NetCDF files to inform users of the original parameter name.
  • As of 8 June 2020, this new renaming process only applies to a single variable, “pressure_depth” which the netcdf_name column overrides to its original name. In all other cases, parameter.name = parameter.netcdf_name. More variables will override their original names as needed in the future, but the default case is to use the original name.

 

 

 

 

[MOU1]Can we spell out this acronym please?