Posts Tagged ‘Software’
Remotely Fixing and Preventing Mooring Issues
Alex Franks’s job is a big one. He is charged with fixing various issues that occur on OOI moorings, while they are hundreds and sometimes even thousands of miles away in the ocean. As an Engineer II at Woods Hole Oceanographic Institution (WHOI), Franks is intimately familiar with the mooring system controller software, which allows him to troubleshoot and fix instrumentation problems on OOI moorings, regardless of their location.
Franks has been working with electronics for over a decade and solving OOI mooring-related challenges since 2015. Many examples exist of his innovative solutions. In 2020, for example, the satellite Internet service that was being used to send data from OOI moorings to WHOI servers was no longer a viable solution. The WHOI team faced the task of either finding a replacement system, or working with the then-current system. One easily implementable solution was to move to transmitting data through OOI’s Iridium radio antennas full time. There were downsides to this solution, however. It would allow no margin of error, would consume more power, and still not be able to send data from all the instruments.
Franks figured out a better solution that would both keep costs manageable and continue to meet timely data transmission goals by modifying the Iridium file transfer portion of the mooring software to accommodate a new data transfer scheme. The new scheme used a feature of the computer program rsync, a fast and versatile file copying tool, called “diff”. Instead of using rsync to communicate with shore servers and determine the “delta” or change between the new instrument data on the mooring and the instrument data files on the WHOI server, he used one of the mooring’s onboard computers as an intermediary server to generate “diff” files against (delineating old from new data). These files were then generated and stored, and sent over the Iridium connection. Using this new configuration, Franks succeeded in sending the entire dataset of all instruments on the mooring, except one that was sent at a reduced sample rate. While transmission times can vary with weather conditions, this newly configured system sends data to the server every 20 minutes every other hour, reducing transmission times from 1440 minutes per/day to about 240 minutes per day.
[media-caption path="/wp-content/uploads/2022/04/DSC_0639-copy.jpg" link="#"]Waves in the North Atlantic can get pretty large, which makes it hard to conduct research at sea, especially in winter. The waves and wind in the Irminger Sea also create challenges for ocean observing equipment in the water there year-round. Credit: ©WHOI.[/media-caption]
Franks also found ways to remotely manage mooring issues caused by weather and sea state by modifying software that controls wind turbines. Wind turbines play a critical role on OOI moorings, providing power to recharge the main system batteries. At the Irminger Sea Array, where the sun is absent for months at a time (the moorings also utilize solar panels), these wind turbines are critical. Prior to Franks’ software fix, human input was required to disable the turbines to prevent them from spinning while wave heights were too great. Franks modified the software used to control the spinning of the turbines to read environmental data from the buoy itself and make automated decisions in real-time that previously had to be done manually. The system now changes its configuration based on a variety of sensor inputs, which make for more immediate decisions to ensure the continued safe operations of the turbines. The software modifications not only help mitigate heavy sea damage to the turbines but saves power, as well. The software detects when the air temperature is above freezing and turns off the precipitation sensor heaters, conserving energy when possible. The software also has fail-safes in place for high or low voltage and to determine hydrogen concentration levels inside the electronics. An illustration of this software configuration is provided below.
[media-caption path="/wp-content/uploads/2022/04/Mooring-system-software-upgrade.png" link="#"] This new software configuration detects when the air temperature is above freezing and turns off the precipitation sensor heaters, and has fail-safes in place for high or low voltage and for hydrogen concentration levels inside the electronics .Credit: ©WHOI.[/media-caption]
Franks has also developed software improvements to the power system controller inside the OOI surface moorings. His work ran the gamut from disabling operational bugs in the system to reducing power consumption to fixing software errors to increase reliability. During a year-long deployment in the Irminger Sea, part of the power system controller board failed. Franks installed a software patch remotely that was able to limit the level of charge coming from wind turbines and wrote a fail-safe feature for the system to disconnect all charging sources if the voltage approached dangerous levels.
The challenges are what keeps Franks enthusiastic about his job, “I just love trying to figure out a solution and it’s particularly rewarding to be able to remotely resolve issues with equipment deployed in the open ocean.”
Read More
Improving Remote System Response in Increasingly Hostile Oceans
Wind and Waves and Hydrogen, Oh My!
Improving remote system response in increasingly hostile oceans
This article is a continuation of a series about OOI Surface Moorings. In this article, OOI Integration Engineer Alexander Franks discusses the mooring software and details some of the challenges the buoy system controller code has been written to overcome.
Components of the OOI buoys working in concert make up a system that is designed for deployment in some of the most challenging areas of our world’s oceans. These systems collect valuable scientific data and send it back to Wood Hole Oceanographic Institution (WHOI) servers in near real time. Mechanical riser pieces (wire rope, and/or stretch hoses) moor the buoy to the bottom of the ocean. Foam flotation keeps the buoy above water in even the worst 100-year storm, while its masthead supports instrumentation and satellite radios that make possible the continuous relaying of data. The software controlling the system is just as important as the physical aspects that keep the system operating.
The system software has a variety of responsibilities, including setting instrument configurations and logging data, executing power schedules for instruments and parts of the mooring electronics, controlling the telemetry system, interfacing with lower-level systems including the power system controller, and distributing GPS and timing. The telemetry system is a two-way communication path, so the software controls data delivery from the buoy, but also provides operators with the ability to perform remote command and control.
[caption id="attachment_22938" align="alignnone" width="745"]
Software flow diagram created by OOI Integration Engineer Alex Franks[/caption]
The unforgiving environment and long duration deployments of OOI moorings lead to occasional system issues that require intervention. Huge storms, for example, can build waves so high that they threaten wind turbines on the moorings. At the Irminger Sea Array, ice can accumulate so much as to drastically increase the weight of the masthead, and with subsequent buoy motion, risk dunking the masthead and instruments. Other mooring functions require constant attention. The charging system must be monitored to ensure system voltages stay at safe levels and hydrogen generation within the buoy itself is kept within safe limits. Two-way satellite communication allows operators to handle decision making from shore using the most up-to-date information from the buoy.
“Since starting in 2015 and following multiple mooring builds and deployments, I’ve realized that issues can rapidly arise at any time of the day or night. I started thinking about what the buoys can do for themselves, using the data being collected onboard,” Franks said.
One of the game-changing upgrades implemented by Franks was to read environmental data and make automated buoy safety decisions in real-time that were previously performed by the team manually. For example, previously, the team would need to monitor weather forecasts and decide preemptively whether changes to buoy operations were advisable. With recent software changes, the system can now change its configuration based on a variety of sensor inputs. These variables include system voltage, ambient temperature, hydrogen levels inside the buoy well, wind speed, and buoy motion (for sea state approximation). In addition to the software updates, the engineering team redesigned the power system controller. They added charge control circuits and the ability to stop the wind turbines from spinning. The software and electrical upgrades now provide redundant automated safeguards against overcharging situations, hydrogen generation, and turbine damage, maximizing buoy operability in harsh environments.
[caption id="attachment_22946" align="alignleft" width="650"]
Onshore engineers are able to keep track of Irminger Sea buoys and instrumentation on this new new dashboard.[/caption]
With a largely independent system, operators also needed a way to easily monitor status of the buoys and instrumentation. The software team created a new shoreside dashboard that allows operators to set up custom alerts and alarms based on variables being collected and telemetered by the buoy. While the buoy systems can now operate autonomously, alerts and alarms maintain a human-in-the-loop component to ensure quality control.
As operations and management of the moorings have progressed, the operations team has found opportunities to fine tune how operators and the system handle edge cases of how the system responds to hardware failures and extreme weather. In the past, sometimes conditions changed faster than the data being transmitted back to shore. This new sophisticated software automates some of the buoy’s responses to changing conditions in real time, which helps to ensure their continued operation even under challenging conditions. The decreased response time to environmental and system events using an automated system, coupled with the ability to monitor and interact remotely, has increased the reliability and survivability of OOI moorings.
Read More