Deep learning may help to prevent salmon escapes in rough seas

Artificial intelligence can be of great benefit underwater and SINTEF, in collaboration with the research center SFI Exposed, is developing systems that will help to boost fish farm safety and security under harsh sea conditions.

You may already have heard about deep learning. It is a key process in the field of machine learning, enabling computers to expand their capabilities in response to information provided to them. By exposing a computer brain to concepts that it is required to understand, it can be taught to recognize contextual information. In a data structure modeled on the human brain, devices are linked to form what we call a neural network. Such networks are built up of many layers that interact to enable what we call deep learning.

We have utilized this concept to develop and improve systems that can identify irregularities in salmon cage nets on fish farms. We do this by feeding a digital neural network with images showing the building blocks of an intact net. These images then enable the program to react when it observes something out of the ordinary, such as a hole. 

Escaped fish are a major problem

Frequent inspections of sea cage nets, suspended underwater, are intended to help prevent fish from escaping. Escaped farmed salmon may carry disease and can also migrate upstream along rivers with wild fish, disturbing and participating in the spawning process. These factors only serve to weaken wild salmon populations, so it is in everyone’s interest to prevent escapes from sea cages.

The Norwegian government recently gave notice that it intends to introduce stricter technical requirements for the escape-proof design and operation of fish farms. These new regulations leave us in no doubt that we have to think innovatively, and may offer us opportunities to apply or further develop some of the concepts that we are currently working on at the SFI Exposed research center. 

Current standard operations involve the use of a camera-mounted, remotely operated vehicle (ROV) that is launched into a cage and manipulated by an operator who then inspects the images transmitted back. It is difficult for an operator to concentrate for hours on end looking at a monotonous set of video images of underwater nets. A computer brain, on the other hand, never gets tired or bored, so such operations are ideal for the application of autonomous vehicles using image recognition.  

When a computer brain is better than a human brain

Regardless of whether the images will be assessed by humans or machines, we still need to take video films of the sea cage netting. Navigating an ROV along a sea cage net is a technically demanding task. The vehicle’s camera has to approach close enough to get clear images without colliding with the net wall. 

Our research colleagues who work with autonomous systems and technologies for use in remote areas have been looking into the types of sensor technology that can enable an ROV to determine its spatial position within a sea cage. Such information is key to identifying which part of the net is being inspected at any given time.  

It is essential that an ROV knows exactly where it is for all types of autonomous operations. It may have to hold a stable position in the face of strong currents and high seas, or track across the net wall at a predetermined fixed distance.

The result of all this research is a laser camera system for so-called net-relative navigation. Information about the ROV’s distance and angular orientation to the net is obtained with the help of two parallel laser beams that illuminate the net wall.  Such measurements are essential if the ROV is to keep an appropriate distance from the net wall without an operator having to make steering adjustments every time the net moves due to currents or wave motion. A lot of exciting research and development is being carried out into autonomous robotic operations, both in research centers and industrial settings. At SINTEF, we see this as the beginning of a journey of system development that will enhance operations in the aquaculture sector.

Hazardous close contact

SFI Exposed is a center for research-based innovation that works in many fields with a variety of perspectives on the challenges facing offshore aquaculture operations in exposed settings. Our colleagues researching into safety aspects have revealed that there may be some overlap between incidents that increase the risk of fish escapes and those that threaten the safety of facility employees. 

We believe that when it comes to safety, it is important to consider the big picture, and that the most effective approach is to take all aspects into account as early as in the product development stage. If employee safety is built into the technology such that it becomes impossible to cause hazard by incorrect use, we will effectively have reduced the risk of incidents that result in both fish escapes and injuries to employees.

But since we are talking about risk. Clearly, an ROV sets no HSE requirements. However, HSE considerations are essential for those personnel whose job it is to release ROVs into fish cages and recover them again. This task sounds simple enough, but it requires that a crane-mounted vessel be brought alongside and moored against the cage. In many situations, personnel have to be deployed along the float ring in order to the release the ROV—a job that involves a lot of difficult lifting.

Movement between the vessel and the cage is a risky undertaking at the best of times, and it is in precisely such situations that many sector-related accidents occur. In locations subject to large movements and accelerations due to strong currents, high seas and winds, and where the vessels and facility infrastructure are larger than in sheltered sites, it is vital to have systems in place that make it safe to move between a cage and a moored vessel.  

A robot arm can do the job

We first studied the ways in which current crane operations are carried out in order to evaluate the associated risks and challenges. We then went on to see how new technologies might improve the situation. One example involves our concept studies for ROV sea cage “Launch and Recovery.” Another is a concept that involves the use of an advanced robot arm that enables operations to be carried out without any form of contact between the vessel and the cage.  

Imagine that the vessel on which the robot arm is mounted is moving up and down on the waves, and that the sea cage is doing the same, out of step and at some distance away. A robot arm that can stretch across from the vessel to the cage in order to carry out precision work simply has to be an impressive piece of high-tech kit. But it is in fact entirely possible. 

We have used model experiments to demonstrate how a robot arm can compensate for the relative movements of a vessel and a sea cage. We have also made assessments of the operations that can be carried out and of any changes that have to be made to existing facilities to enable this technology to be brought into use.

Better decision support

The risk of accidents and fish escapes may increase when sea conditions are poorer than anticipated, such as in cases of unexpectedly strong currents or rough seas with powerful waves. Currently, our decisions about whether or not to continue operations under such conditions are often made on the basis of discretion and experience. Better defined operational limits in the form of guidelines as to when operations can be performed or not, may make a valuable contribution towards enhancing safety. 

At SINTEF Ocean we have been building an infrastructure for data capture from buoys, offshore vessels and facilities, and have collected meteorological data, as well as measurements of water movement and quality, as a means of constructing a robust statistical database. The more precise the information we have about wave, current and weather conditions, the easier it will be to make a decision about whether or not an operation can be carried out.  When we link these statistics to physical measurements and digital twins, we obtain an excellent foundation for operational planning.