RCA Technical Wizards: The Saga of Repairing an RCA Primary Node
Chuck McGuire and Larry Nielson were able to take a cabled power and communications substation that sat inoperable on the dark, cold seafloor of the Pacific for two years and make it work again. The odds were against them. There’s usually less than a 50-50 chance to make a complex technology operational again after this length of inactivity – inactivity compounded by the extreme pressure, cold temperature, and biofouling that occurs on the bottom of the Pacific Ocean. But these two engineering geniuses succeeded in repairing, rebuilding, and ultimately reconnecting Primary Node 1B (PN1B), which provides power to the Southern Hydrate Ridge portion of OOI’s Regional Cabled Array (RCA).
It was an arduous, complicated journey, fraught with delays outside of the engineers’ control. The recovery, repair, and reinstallation took two years to complete. We share some of the highlights below for they illustrate the many challenges of operating seafloor cable infrastructure on an active tectonic plate in the Pacific Ocean.
Chuck McGuire, a senior principal engineer, is the chief engineer for the RCA and principal investigator for the engineering portion of the RCA, which is conducted at the University of Washington’s Applied Physics Laboratory (APL). Larry Nielson is a field engineer three at APL. They have been working on the RCA project since its inception in 2009 (Construction and deployment began in 2013).
When PN1B failed in 2020, the duo was stymied at first. They had to call in a variety of people with different skills to recover, rebuild, and redeploy it.
“Restoration of the PN1B is really a testament to the engineering that went into these nodes, which are good and work well under very harsh conditions, “explained McGuire. The Primary Nodes were designed and built by L3 Maripro with oversight by the APL Engineering team.
A Cable Fault Causes Power Loss
The saga began during RCA’s annual maintenance expedition in summer 2020. Everything was smooth sailing until suddenly the science team aboard the R/V Thomas G. Thompson learned of a cable fault on the southern cable line, which had shorted out. McGuire and team instantly went into troubleshooting mode and determined that the problem was in PN1B, which powers the Southern Hydrate Ridge section of the 900 kilometers of cable that comprise the RCA.
Luckily, they were able to divert the ship, the R/V Thomas G. Thompson, to the PN1B location to investigate. McGuire was on shore at the Shore Station in Oregon where he could troubleshoot the power system. McGuire worked with a team onboard to use the Remotely Operated Vehicle (ROV) Jason, operated by the National Deep Submergence Laboratory of the Woods Hole Oceanographic Institution, to unplug and plug back in the science interface assembly (SIA) that provides power and communication to secondary infrastructure on the array (e.g. extension cables, junction boxes, instruments, and moorings). The SIA was attached to and powered by PN1B. The SIA wasn’t the cause of the short, so the next step was a visual inspection of the cable with the ROV, tracing the cables back from the node to see if there was any external aggression or damage to that cable. None was seen. The next step was to determine where the power was lost. By controlling the power from the Shore Station and sending “tones” (an oscillating frequency) down the cable, they were able to isolate the affected area. They determined that the PN1B was no longer transmitting any power to the southern cable line running east of the node.
McGuire and Nielson set to figuring out how to replace the damaged node with a spare. A specialized cable ship would be required. A request for proposal was released and the cable ship IT Integrity was secured.
With a cable ship lined up, the engineering team set about reconfiguring the Primary Node and a replacement SIA, which would be plugged into PN1B when both were back on the seafloor. The assembly was complicated and required reconfiguration of all the optics in the SIA’s interior. “Larry was able to get the SIA unassembled and rigged up in a way that made it possible for me to go in and remove and replace the components that needed to be replaced,” explained McGuire. “All of this was unexpected work, which took time, and there was much aggravation. But Larry had some amazing solutions to some interesting rigging problems.”
The two engineers had to rely on their experience to tear down the SIA and then reconstruct it. Their challenge included having to create specialized tools to get into extremely tight spaces. McGuire likened the work to repairing a car engine, where one slowly takes things off, working through the engine until finding the problem. The difference between the Primary Node and a car, however, was that spare parts to fix any issues found weren’t readily available and would have to be manufactured if needed. Here, again, the skill, experience, and tenacity of this duo were called into play. After taking the node apart and putting it back together again, they are now experts.
Their work was driven by a tight schedule needed to get the node and SIA ready for the next RCA recovery and deployment cruise in July 2021. The plan was to have the cable ship work simultaneously with the ROV Jason that was aboard the Thompson. During the 2021 expedition, Jason successfully unplugged and recovered the SIA. The cable ship Integrity then dropped a recovery line next to the node and Jason hooked the recovery line to the node. The cable ship recovered the node, returned it to the dock in Newport, where it picked up the replacement node. All seemed to be going well and on schedule.
While the transit and offloading were taking place, the team had tied the cables to a surface buoy to keep them accessible for reconnection to the new node. When back at the buoy, engineers on the Integrity began the process of jointing—essentially splicing—the tied-off cables and attaching them to new cables connected to the replacement node. When all was connected, the next step was to use a cable winch to lower the cables down to the bottom. Unfortunately, during deployment of the replacement PN1B, the system was damaged which resulted in a shunt fault. Tests indicated the damage was most likely in or very close to the cable termination assembly (CTA) making a repair with the node, at that time, not possible. The replacement PN1B was recovered to deck.
Rather than admit defeat, McGuire came up with the idea to bypass the replacement node and connect the cable downslope from the west to the eastern cable together to provide power and communication to Primary Nodes PN1C and PN1D. McGuire figured that if optical power was too high, communications would fail since PN1B would not be in the system to attenuate the signal. After a few quick calculations, McGuire determined what the attenuation needed to be in the line so that the power wasn’t too high to prevent communication down the cable. The required attenuators were driven down to Newport while the ship headed to port to retrieve the critical parts. The cables were then jointed with the attenuators in the optical circuit, restoring about 65 percent of the power and communications lost to the cable system initially and allowing RCA instruments and moorings at the Oregon Offshore and Oregon Shelf sites to become operational again. The maneuver was innovative and an engineering feat.
Upon returning to the lab, the team faced other challenges. They had to reverse engineer a CTA. Because many of the parts could not be seen without destroying other critical parts, a CT scanner was used to scan its many parts, and point clouds (discrete set of data points in space) were developed. These cloud points were then fed into Solidworks, a software program, to create models of each part that needed to be machined or refurbished.
“CTAs are a very complex component,“ explained McGuire. “They terminate a cable into an object that must retain its watertight integrity at over 3000 meters water depth, while withstanding 35,000 pounds of pulling force and protecting the optical fibers in the cable. Also, there’s not a rock solid on-land way to test its water tight integrity once it’s been built with the cable, ready for deployment.”
In getting ready for the next deployment attempt, the engineering duo encountered other obstacles from COVID-related delivery delays to manufacturers who no longer manufactured needed components. They persevered. The team tore down the replacement node a second time and rebuilt it, with the same deliberate process given the first replacement node. Once completed, they took the replacement node through a complete battery of tests, making sure that it was capable of being powered up, could communicate with the SIA, and provide power to the cabled system. They then shipped the node to Portland to be loaded on a larger cable ship, the C.S. IT Intrepid.
In June 2022, the replacement node was again ready for deployment. McGuire boarded the Intrepid with field engineer Paul Aguilar to oversee deployment. After two days of bad weather, the ship finally was on site. The deployment went without incident except that the node landed 400 meters short of where it was to connect to the cable. To reposition the node, PN1B would have to be lifted back on deck and four kilometers of cable would have to be “peeled” off the seafloor. This would greatly increase the risk to PN1B and the cable, so the team came up with the idea of essentially creating an extension cord to make the cables meet.
“The node was working, the whole system was working except we couldn’t plug anything into it yet, because it was too far away from where it needed to be,” said McGuire. “That was in the middle of June. We had until the end of July, (for the 2022 RCA recovery and deployment expedition) to acquire the parts, build an extension cable and come up with a plan to connect the two cable segments. “
As they do, McGuire and Nielson came up with a plan. By the July departure date, McGuire was aboard the Thompson to oversee the connection. Nielson was onshore to support logistics and attend to any contingent plans that might be needed. The deployment included coming up with a new rigging configuration that required hanging 500 meters of cable off the bottom of the Remotely Operated Platform for Ocean Sciences (ROPOS), a 40 hp Science/Work Class ROV operated by the Canadian Scientific Submersible Facility, using Yale grips every 100 meters or so to allow for proper positioning once the cable was on the sea floor. Using ROPOS, the team guided the end of the cable lit with a transponder and put the end right next to the Primary Node. They then “walked” the ship towards where the cable had to be plugged into the node, essentially laying out a long string in the water to the correct spot.
“And it worked on the first shot, which still amazes me today,” concluded McGuire. “I cried and went to bed.”
