News Steering the Future with Autonomous Control and Intelligent Perception Systems
January 29, 2020
The following feature article appeared in the January issue of The Society of Naval Architects and Marine Engineers (SNAME)’s Marine Technology magazine. Click here to access the full issue (log-in required).
It’s hard to scan marine industry news without coming across a handful of stories each week dedicated to the subject of marine autonomy. While much of the industry chatter is still focused on technologies that are clearly in the development stage – like fully unmanned containerships – there are also stories circulating now about the practical use cases for today’s commercially available autonomous systems.
Autonomous command of a vessel is a highly practical technology that aids the navigation of vessels and improves the productivity and safety of mariners on the water today. Though some autonomous marine technology developers promote concepts that involve building entirely new unmanned vessels, companies such as ours, Boston-based Sea Machines, are offering commercially available systems that require no new vessel construction. More affordable retrofit options like these have made autonomous technologies more accessible to marine operators who aren’t ready or able to add new vessels to their fleets.
Sea Machines’ SM300 autonomous-command and remote-control product was released to the market last year and is now commercially available for installation aboard workboats and small-to-medium sized vessels. Operators can also capitalize on Sea Machines’ SM200 system, which singularly provides wireless remote-helm operation of vessels and on-board equipment by way of an industrial-grade beltpack. Since their release, both products have been installed aboard existing and new-build commercial vessels to provide immediate new capabilities across a variety of vessel types – ranging from marine spill response, survey, fire, search-and-rescue, patrol, aquaculture, dredging, offshore oil and gas, windfarm support and more.
As an example of what is available to commercial operators today, consider what these autonomous-command and remote-control systems offer in terms of added operational capabilities:
Autonomous Control
- Autonomous command and waypoint following
An operator using the SM300 can command and control a commercial vessel from anywhere in the world that has a network connection. Using the system’s TALOS technology, the vessel operator can plan ENC-based paths, track waypoint following and record voyage data. Real-time situational awareness is provided and a human operator can adjust or override vessel controls at any time. - Autonomous collaborative following
Two vessels can autonomously collaborate with exact matched speeds and courses, creating a force-multiplier effect over large surface areas. - Autonomous obstacle detection and avoidance
SM300-enabled autonomous vessels come equipped with obstacle detection and collision avoidance capabilities. Again using ENC, AIS, GPS, radar, computer vision and more, Sea Machines not only provides intelligence about objects in a vessel’s path, but will autonomously course-correct the vessel to avoid hazards. Once the obstacle is safely out of the vessel’s path, the SM300 will autonomously re-route the vessel to the planned track line. - Voyage archiving
The value of today’s autonomous technology doesn’t stop when the mission ends. When operations cease, mariners have access to archived mission data. This data can inform operators on ways to improve work on the water or can be repurposed as plug-and-play inputs for future projects. Remote-Helm Control Technology like our industrial-grade beltpack allows operators to remotely command a vessel from the shore or a secondary vessel from a distance of up to 1KM. This functionality is available in both the SM200 and SM300 systems. Remote Payload Control Both the SM200 and SM300 systems enable operators to remotely control on-board payloads, such as skimmers, fire monitors, sensors, cameras and other tools, directly from the beltpack. This ability eliminates the need for a mariner to physically engage on-board equipment from within the wheelhouse or even on the vessel.
Remote-Helm Control
Technology like our industrial-grade beltpack allows operators to remotely command a vessel from the shore or a secondary vessel from a distance of up to 1KM. This functionality is available in both the SM200 and SM300 systems.
Remote Payload Control
Both the SM200 and SM300 systems enable operators to remotely control on-board payloads, such as skimmers, fire monitors, sensors, cameras and other tools, directly from the beltpack. This ability eliminates the need for a mariner to physically engage on-board equipment from within the wheelhouse or even on the vessel.
Evidence that Autonomous Systems Work
The technology described above may sound futuristic, but the reality is that it’s here and already at work aboard commercial vessels. The most recent example of this comes from an August headline story out of Portland, Maine. There, Sea Machines deployed the first autonomous spill response skimmer. For the event, which garnered mass industry attention, Sea Machines demonstrated its SM300 in action aboard a manned Marine Spill Response Corporation (MSRC) skimming vessel as it executed oil-spill recovery exercises in the harbor for the U.S. Department of Transportation Maritime Administration (MARAD). Though no actual oil was released into the harbor for the events, back-to-back demonstrations proved the technology’s ability to increase the safety, response time and productivity of marine spill-response and recovery operations.
The SM300 demos were executed in front of an audience of government, naval, international, environmental and industry representatives. During the event, these attendees witnessed:
Shoreside autonomy
Via laptop computer, a land-based operator commanded the skimmer to execute back-and-forth grid patterns on the water, as if it were collecting spilled product from the water’s surface. In addition to providing value during spill clean-ups, MSRC pointed out the value of autonomous control in the early phases of a response, when site surveys and air and water quality tests are conducted. An unmanned, autonomous boat outfitted with cameras, sniffers, sensors and other equipment can send data back to a shore-side operator without exposing humans to unknown conditions.
Remote control
From the dock, attendees had the opportunity to wear the beltpack and steer the skimmer around the harbor via joystick control. One MSRC spill responder who was observing recalled a hazardous marine spill response that he had managed several years ago in the U.S. Gulf of Mexico. With temperatures reaching 110 degrees most days, he and his colleagues donned heavy, bulky protective equipment and respirators to manually skim the water’s surface for hours at a time from cramped wheelhouses. After sharing his experience, the responder said that had he been able to remotely operate a skimmer from an air-conditioned mothership, he could have avoided unnecessary exposure, challenging shift changes and operator fatigue.
Remote payload control
From both the laptop computer and the remote-control beltpack, Sea Machines engaged the on-board boom arm and skimmer belt. The operator’s ability to control such critical equipment with the push of a button from a remote location reinforced the benefits of removing mariners from hazardous operations.
Collaborative operations
Not shown, but equally important were collaborative operations. During a spill event, a mothership and unmanned daughtercraft could remotely or autonomously collaborate to dually tow boom, a capability that removes mariners from dangerous environments and increases productivity.
Challenges
The success of the demonstrations wasn’t without challenge, however. As mentioned above, these capabilities were conducted in Portland with mariners on board. This was done in part to satisfy current U.S. Coast Guard regulations and in part to ensure safety throughout the event. But having people on board the vessel showcased another very real and common challenge marine operators regularly face: The weather. Throughout the demo day, near-constant thunderstorms (as shown above) disrupted operations. Each time lightning was detected, crewmembers aboard the MSRC skimmer were required to cease operations and come ashore for safety. The series of stop-work periods added large amounts of time to the mock responses. If the demos had been real events, these human-safety related delays could have contributed towards additional uncontrolled spread of products in the environment and potentially increased damage.
By contrast, unmanned vessels could have continued working safely throughout the storms from an office or nearby covered area. Whether the challenges present as extreme temperatures, hazardous fumes, dangerous fires, extreme sea conditions or something else entirely, autonomous systems ensure that marine operations go on uninterrupted and with reduced negative impact to humans.
Other challenges unrelated to the MARAD event in Portland do exist for autonomous marine system developers. Because companies like Sea Machines are building technologies that are available for retrofit aboard existing or new-build vessels, its developers had to build them “interface agnostic” so they could integrate with the myriad of products that exist in the market. Much like how Windows products can be used across PCs, Apple computers and other third-party interfaces, installable autonomous marine systems must so integrate with whichever interface is already on a customer’s vessel. This challenge required countless hours of custom programming by the Sea Machines team to ensure compatibility with the most commonly used interfaces – including propulsion and steering systems, instruments, and hydraulic payloads. These efforts have resulted in today’s commercially available products that can be installed aboard most commercial vessels.
So far, none of these challenges has been insurmountable. In fact, most of them are viewed as opportunities by developers, who want to continue refining their systems. Much like any new technology that has radically shifted the way industrial work is done, challenges of autonomous marine technologies will continue present as they are used in real operations. As more marine operators adopt the systems, developers will have new opportunities to make the systems more intuitive, intelligent and indispensable.
Diverse Applications
While a good portion of this article has been dedicated to autonomous marine technology in use aboard spill response operations, these systems have many more applications aboard a wide variety of workboats and commercial vessels. Across all marine operations, autonomy automates tedious, redundant and dangerous tasks, allowing an on-board crew to focus on higher-level operations. On-water incidents can be prevented with obstacle detection and collision avoidance capabilities that Sea Machines built using computer vision, radar, AIS and GPS data. Sea Machines also helps to reduce operator fatigue, a major casualty factor in marine incidents during nighttime operations, long-distance transfers and challenging sea states. And autonomous missions can be saved and reused for future efficiency.
A handful of additional use cases for autonomous-command and remote-control technologies are as follows:
Hydrographic survey and marine patrol operations benefit from multiple autonomous workboats operating collaboratively along pre-planned routes and repetitive paths. These coordinated efforts create a force-multiplier effect that can cover large surface areas more safely and productively.
Similarly, autonomous ice-breaking tugboats can reliably zig-zag through harbors and near-shore waterways during freezing conditions to help keep shipping lanes open.
Autonomous security boats can match the speed and course of larger ships, making escorting of vessels carrying high-value cargo safer and more cost-effective.
Broad coverage areas or long transits to an offshore site from a mainland aboard offshore commercial boats can be executed autonomously, using dynamic waypoint following capabilities. Pairing manned mother vessels with unmanned daughter craft – ideal for offshore surveillance and monitoring, surveying, seismic operations and spill responses – reduces crew expenses and can increase operational periods due to the reduction in stop-work periods related to shift changes.
High-bollard pull tugboats towing out loaded barges can be programmed to operate in collaborative following modes. Such capabilities allow tugboats in complex formations to maintain an exact course and speed from the point of departure to the offshore project site, eliminating fatigue and increasing operational predictability.
Operators can program autonomous aquaculture workboats to execute predictable routes to sites, such as those to deep-sea fish farming sites, or to maintain station-keeping. Autonomous support boats can more efficiently haul feed, monitor operation sites, clean nets and dredge the sea beds beneath farms than traditionally operated boats or can be commanded in unmanned configurations or remotely.
Tugboats involved in the fleeting, shifting and moving barges can be remote-controlled from the shoreline, a second vessel or location outside of the wheelhouse for increased safety. Oftentimes during these operations, visibility from a tugboat’s wheelhouse can be impaired. With remote-control operability, a mariner can safely and confidently control the vessel and load, as well as any connected payloads or auxiliary equipment, from anywhere on board, without relying on a remote spotter.
The same concept applies during dredging operations, whereby mariners can remotely operate tugboats handling dredge barges from locations with better visibility.
In marine emergency response scenarios, stationed vessels – such as those used for security or fires – can be remotely deployed immediately, without waiting for a full crew to arrive. Because responses aren’t slowed down waiting for responders to travel in, incidents can be attended to faster and often before they escalate into large-scale situations.
In the case of marine fire responses, two unmanned boats can autonomously collaborate in highly aggressive sweeping patterns that put out flames faster than more conservative, manned boats could.
Operators can very quickly deploy unmanned or minimally manned search-and-rescue vessels, helping to locate missing people at sea faster. These vessels can be outfitted with thermal and night-vision cameras, as well as other sensors, to provide real-time situational awareness to remote vessel operators. Off-boat operators can remotely control these on-board payloads.
For government operators, unmanned vessels can be stationed long-term at sea to serve as the vital communication link between aerial and subsea assets. These vessels can also serve as a “floating battery,” providing power to connect stand-off vessels to SATCOMs.
For special forces ops, minimally manned and unmanned marine operations allow for removal of military personnel from potential hostage situations. Unmanned drone boats can also serve as diversions, allowing crewed boats to complete missions safely.
Autonomous marine assets can support marine rescue operations, expeditionary logistics and humanitarian relief efforts because they can deliver cargo, ammunition and personnel faster and more cost-effectively. Minimally manned vessels can also serve as efficient “floating hospitals,” allowing more room for medical staff and patients. Following disasters near coastal areas, an unmanned vessel stationed near shore can provide a signal to restore communications and connectivity.
Again, these are just some of the many use cases for today’s available autonomous marine technology. Every day, more and more applications arise, each born out of the need to improve operational safety, productivity and predictability.
What’s Next: Technology for Larger Vessels
The marine industry is on the cusp of even larger changes due to this surge of technological innovation. The next wave of progress will include Artificial Intelligence (A.I.)-powered perception systems for bigger vessels – including ships, tankers, cruise ships and ferries – that will provide advanced situational awareness for piloting. Such technology will provide mariners aboard with a full picture of ship’s surrounding domain, traffic and obstacles using data from conventional marine sensors (like radar and AIS) fused with new technologies, such as real-time image recognition for vessel detection and tracking and Light Detection and Ranging (LiDAR).
The version of this technology under development by Sea Machines will display these data feeds in a user-friendly way on wide-angle RGB and thermal panoramic screens located in the wheelhouse and other areas of the ship. Sea Machines reports that its system is “always on watch,” and supports navigation 24/7, even in poor visibility and challenging weather conditions.
Along with serving as an advanced situational awareness system, Sea Machines’ system will also act as a hub and conduit for shipboard digital data. This system will collect, display, record and transmit operational telemetry and data, such as navigation and traffic information, videos of the operating domain, environmental information and the condition of on-board machinery.
The main advantages of advanced perception and situational awareness technologies is the reduced risk of uncontrolled incidents, accidents and delays that impact cargo schedules and reduce operators’ bottom lines. These incidents are traditionally caused by limitations in conventional shipboard instruments and the perception limitations of human operators.
Sea Machines is now trialing its A.I.-powered perception and situational awareness technology aboard an A.P. Moeller-Maersk’s new-build VISTULA-class ice-classed container ships in Denmark. The project has been significant not only to Sea Machines and Maersk, but also to the larger maritime industry as the installation marked first time computer vision, LiDAR and perception software have been utilized aboard a container vessel to augment and upgrade transit operations. This system is expected to become commercially available to maritime operators and naval architecture and marine engineering firms in 2020.
Advanced Technology is a Differentiator for Naval Architects
Autonomous control and intelligent perception systems are establishing themselves as differentiators for naval architecture and marine engineering firms who offer them. As operator demand for more modern marine technology grows, the firms that offer these solutions to customers will define themselves as forward-thinking, relevant and highly competitive. An uptick in technological innovation demand on the naval and marine government side is already being seen, with the commercial market quickly following.
While some naval architects are designing new, purpose-built autonomous vessels from the hull up, firms can also consider more flexible, installable autonomous marine systems as an option for customers seeking innovative, new capabilities. The latter is often a more realistic and cost-effective option for marine operators, since installation doesn’t always require the commitment of a new-build vessel.
As an example, Sea Machines’ retrofit-ready systems require only 10 components for installation and can be added to new or existing vessels in less than a week – a tremendous value-add for design firms. With Sea Machines’ return on investment typically seen within a year for commercial operators, naval architects have little reason not to include it as an option for buyers. Even for marine construction projects happening now in shipyards, Sea Machines’ interface-agnostic systems can be added on typically without significant impacts to current delivery timelines.
Naval architecture firms have the unique opportunity to now develop their reputations as innovative solution providers both through the development of custom autonomous vessel designs and retrofit autonomy options. Each option is viable and can be rationalized in today’s marketplace. No matter the approach taken, naval architecture and marine engineering firms have an important role in autonomy’s adoption, which is rapidly changing the way marine business is conducted.
A Safer, More Efficient and More Competitive Industry
In summary, autonomous-command and remote-helm control systems generally offer marine operators the following advantages:
- Autonomous command for more productive, predictable and safer marine operations;
- Remote-helm control for faster responses and reduced operational costs; and
- Remote payload control for on-board equipment cameras, sensors and more;
- Collaborative operations for a force-multiplier effect;
- Crew support to reduce incidents related to fatigue, poor visibility and challenging environments;
- Unmanned operations for increased safety and reduced stop-work periods; and
- Overall reduced manual effort that allows human operators to focus on higher-level tasks.
During the MASS Conference 2019, the U.S. Department of Transportation Maritime Administration’s Richard Balzano, deputy administrator (shown above, operating Sea Machines’ remote helm control), said:
“The way we see automation at the Maritime Administration is as a potential enabler to a safer, more efficient and more competitive mode of water transportation that provides more and better jobs for America’s highly trained and dedicated mariners.”
As the marine industry workforce ages, modern technology will play a role in drawing in younger recruits. In this day and age of smart phones and TVs and self-parking cars, the next generation of mariners will not only appreciate and respect the capabilities of modern “smart ships,” but will also expect it.
It’s up to all of us in the industry – whether we are naval architects, marine engineers, vessel owners, mariners or enthusiasts – to recognize the value modern technology brings to operations by way of increased productivity, predictability and safety. Those who capitalize on today’s available technology will reap the greatest benefits as others in the industry play catch-up in the coming months and years.