Autonomous Systems as a Force Multiplier for Maritime Domain
The following article was published in the March issue of Sea Technology Magazine:
As the third-largest department of the U.S. Government, the Department of Homeland Security (DHS) is a massive and critical division including border protection, the U.S. Coast Guard (USCG) and others. Using land, air and marine resources, DHS is ultimately responsible for counterterrorism; border, port and maritime security; protection of critical infrastructure; protection against chemical, biological and nuclear threats to the homeland; response to disasters and more. The challenges for DHS as it ensures our nation’s security are great – including terrorism, criminal activities and natural disasters – but are amplified when done so on the water.
According to the National Institute for Occupational Safety and Health, marine workers face a higher risk of fatality and injury than the average American worker. The reason is obvious – on top of some of the hazards seen in other industries, including fatigue and use of heavy equipment, our industry faces additional risks of vessel collisions, allisions, groundings, severe weather and more. Since even before DHS’ inception in 2002, marine-industry challenges have simply become widely accepted as risks, inherent to the job. But that is now changing with the introduction of advanced perception and autonomous command vessel technologies that mitigate many of these risks while empowering mariners to conduct missions more productively, predictably and safely.
Marine Autonomy Explained
Autonomous control of a vessel is a highly practical technology that aids the navigation of vessels and improves the productivity and safety of mariners on the water today.
Lloyd’s Register defines autonomy for commercial marine operations across six categories, ranging from #1 Low Automation (all actions taken by human operators, but decision support tool can present options or influence decisions making, with data provided by systems on board) through #6 Unmanned Operations (unsupervised operation where decisions are entirely made and actioned by a system). Today’s commercially available systems, such as those offered by Boston-based Sea Machines Robotics, fall in the middle, between #3 Conditional Automation (decisions and actions are performed autonomously with human supervision and authorization) and #4 High Automation (decisions and actions are performed autonomously with human supervision, high-impact decisions allow human operators to intervene and override).
By this definition, understand that Sea Machines is designed as a human-on-the-loop system. Most current use cases involve supervised autonomy for locally controlled or remotely operated missions, with unmanned configurations primarily being conducted in controlled domains. The technology enables operators to work with less risk and greater redundancy while leveraging manpower.
Using Modern Technology to Solve Age-Old Challenges
Autonomous-command and remote-helm control systems offer solutions to marine operators in several ways. For on-water incidents caused by fatigue, they offer obstacle detection and collision avoidance capabilities. Commercially available Sea Machines systems use continuous data from sensors such as computer vision, radar, AIS and IMU, and offer 24/7 watch redundancy. When fused, these technologies are more accurate than the human eye, especially in times of low light or in poor sea conditions. This can reduce operator fatigue and risk of operational incidents.
Another risk includes crew exposure, either to sea states, toxic conditions or direct threats to life. For those performing patrol, disaster-response or search-and-rescue (SAR), missions may be conducted in extreme environments. Autonomy systems increase safety by reducing or eliminating the need for humans to be on board vessels during missions. Today’s tech also uniquely executes with human-like behavior, intelligently factoring environmental and sea conditions (including wave height, pitch, heave and roll) and makes controlled speed changes between waypoints for added crew comfort.
Another hurdle operators face is manual operations. Across all industries, autonomy automates tedious, redundant and dangerous tasks, allowing workers to focus on higher-level operations. Mariners now can program vessels to autonomously deploy pre-established routes and can remotely command workboats to follow paths in unmanned or autonomous modes. Offering greater predictability and higher performance, these autonomous missions can be saved and reused for future efficiency.
Yet another challenge solved by today’s autonomous marine systems is the limited shoreside visibility humans have into at-sea vessel operations. Sea Machines enables crews to remotely monitor operations of working vessels in real-time from a shoreside location or second vessel. Remote management and watch redundancy can prevent operational incidents and keep operations safer. For DHS, remote vessel monitoring capabilities could also foster inter-agencies sharing of pooled autonomous assets for even greater efficiency.
Another significant challenge the marine industry faces is vision obstruction due to vessel structure or cargo. In traditional operations, crews are confined to a wheelhouse to operate the vessel and on-board payloads. This fixed location doesn’t always offer the best vantage point for operators, and in some cases, requires the use of signals to be relayed from another mariner to the wheelhouse. Replacing this conventional system are wireless and remote-helm control technologies that free crew from the wheelhouse to conduct operations from any location that offers the greatest visibility and safety. Systems like Sea Machines’ SM200 enable wireless helm and propulsion control, as well as remote control of auxiliaries and payload equipment. Last year the USCG and American Bureau of Shipping (ABS) approved the SM200 for installation aboard a class of U.S.-flag tugboats that support articulated tug-barge sets.
An example DHS application for shore-based remote operations is in port security, where patrol vessels are used for surveillance and monitoring of vessel traffic, as well as searching for potential malicious or suspicious behaviors. Using autonomy, port security authorities can remotely operate and monitor navigation and remotely access a vessel’s situational awareness data via sensors and cameras. This empowers shore-based crew to be virtually present with the vessel and make informed decisions about any needed action or vessel response.
Beyond the ports are remote regions, like the Arctic, where DHS must have a greater focus in the future. Drivers for this include fisheries, tourism and commercial shipping, which are all seeing increased activity in remote areas. DHS groups can leverage marine autonomy to patrol and surveil remote areas with reduced-crew or unmanned surface vessels (USVs) that collaborate to create a force-multiplier effect with less resources required. A single shoreside operator can command multiple autonomous boats with full situational awareness. With their long endurance, USVs can provide persistent domain awareness in remote regions.
An often overlooked but critical challenge of our industry involves the massive size of our oceans and waterways contrasted with our limited resources available for managing them. Whether the missions be border patrol or security for ports and other critical infrastructure, autonomous vessels can follow grid patterns and perform other coordinated tasks along the same paths at set distances apart. These capabilities can be particularly useful in protecting seafood fraud and preventing Illegal, Unreported, and Unregulated (IUU) fishing. This is a high-priority focus for DHS, as it is currently placing authorities at the border to inspect and seize fraudulent or illegally imported seafood.
USCG Is Now Trialing Sea Machines Systems
In October 2020, the USCG’s Research and Development Center (RDC) kicked off an evaluation of USVs and their ability to provide persistent maritime domain awareness in remote areas. Supporting these efforts, Sea Machines partnered with Metal Shark Boats to supply an autonomous Sharktech 29 Defiant vessel offering a full range of advanced capabilities.
The USCG said the purpose of the testing was to examine “the operational utility of the USV, including feasibility, costs and benefits. While potentially applicable to many Coast Guard missions, there is potential these technologies will help enable the Coast Guard to better protect critical natural living marine resources from IUU fishing and other illicit activities.”
During the event, the RDC operated Sea Machines’ autonomy system in real time. Planned exercises included transit and grid autonomy missions, use of the wireless beltpack from the vessel and shore, and an observance of the system’s obstacle detection and collision avoidance feature. Operators saw how the technology enabled optionally manned or autonomous-assist (reduced crew) modes that can reduce mission delays and maximize effort — a critical feature for time-sensitive operations, such as on-water SAR and other expeditious missions.
“It’s clear that autonomous technology is a growing industry, and has great potential to enhance USCG operations,” Cmdr. Blair Sweigart, the RDC demonstration’s director, said in MY CG, the USCG official overview site. “Combined with AI algorithms, unmanned systems could be a game changer.”
“The first application that was immediately apparent was the utilization of the USV as a force multiplier from a cutter or shore unit. Having another asset on scene to assist as a show of force or presence is invaluable to the CG’s law enforcement missions,” said BM1 Houston Riley, also in MY CG. “During an event like the Fourth of July, you can leave it unmanned or have someone operate the vessel remotely, to keep people at bay or create an atmosphere of safety. It has video cameras on it, so the operator or someone back on shore could have visual input. What I really found interesting was using the USV as a manned vessel for SAR ops. As the USV ‘self-operates’ on a search pattern, the operators are freed to keep a lookout for a person in the water or vessel.”
The Sharktech vessel is now home at the RDC’s New London, CT, facility, where it is being used for additional testing and operations.
First Autonomous Search-and-Rescue Vessel is Coming
The use of autonomy for searching large domains is an ideal use case and the technology is quickly being put to work. An example is Sea Machines’ partnership with Hike Metal, a manufacturer of workboats in Ontario. In what will mark the world’s first autonomous and unmanned SAR vessel, Hike Metal will integrate Sea Machines’ autonomous vessel control system aboard a patrol boat to develop and demonstrate the capabilities of autonomous marine technology in increasing the productivity and safety of SAR operations. Demonstrations will be conducted alongside the Canadian Coast Guard in mid-2021.
“We see the need to increase response capabilities and reduce the risk to first responders. We feel this technology and platform will be a valuable tool to all Coast Guard Societies around the world,” said Hike Metal’s Roger Stanton, senior project manager.
Autonomy in Beyond Line-of-Sight Applications
Sea Machines has partnered with global communications company Viasat to make beyond Line-of-Sight (LOS) autonomous operations not only available, but attainable for vessel operators. Leveraging Viasat’s highly secure, high-speed maritime broadband services on expansive satellite networks, operators can now conduct autonomous missions across North America, Central America, the Caribbean and Europe without signal disruption. This capability is ideal for government operators executing global, offshore and long-distance missions that may utilize reduced-crew or unmanned daughter craft, with full situational awareness provided to operators located aboard a crewed mothership or on land.
What’s Next: Computer Vision
The next wave of progress will include A.I.-powered perception systems (computer vision) for autonomous vessels – including ships, tankers, cruise ships and ferries – that will provide advanced situational awareness for piloting. Computer vision allows for increased situational awareness beyond capabilities of radar through identification, tracking, geo-locating and classifying targets of interest via proprietary Sea Machines’ software and AI-models built from the companies’ industry-leading catalog of digitized data from operational vessels. This library includes routes, positions, attitudes, domain imagery and machinery conditions in varying seasons, locations, atmospheric and oceanic states.
Sea Machines is releasing this capability to work with its flagship autonomous control system, the SM300. With computer vision as an added sensor, the autonomous capability for vessels to track targets avoid collisions is greatly increased.
The company is also developing a product that uses computer visions for traditionally controlled ships as a crew- assist tool, which will display these data feeds in a user-friendly way on wide-angle panoramic screens located in the wheelhouse and other areas of the ship. The system is “always on watch,” and supports navigation 24/7, even in poor visibility and challenging weather conditions.
Sea Machines is now trialing a computer vision system aboard an A.P. Moeller-Maersk ice-class container ship in Denmark. The product will be commercially available this year.
The main advantages of advanced perception and situational awareness technologies are the reduced risk of uncontrolled incidents, accidents and delays that impact cargo schedules and reduce operators’ bottom lines. These incidents are traditionally caused by limitations in conventional shipboard instruments and the perception limitations of human operators.
The Future Is Now
Autonomous technology is available now and being adopted faster than other industries. Many early adopters in the marine industry have already begun leveraging autonomous and remote-helm control technologies to increase their capability and improve operational vessel safety, productivity and predictability. The result will be the start of an unprecedented era and the introduction of a myriad of new skill sets for mariners. DHS and other operators must lean into modern technologies to keep our nation and economy strong and protected.