Sharper images, more sensors – littoral surveillance is adding new angles to its field of view.
Electro-optic and infra-red (EO/IR) imaging systems are an essential part of any surveillance system designed to monitor the littoral regions of the world. EO/IR equipment forms one part of the wider intelligence, surveillance and reconnaissance (ISR) capabilities in coastal areas and islands that includes radar, satellite imaging and other sensing devices.
EO/IR sensors are important as they detect reflected light radiated in the electromagnetic spectrum, including bands beyond the visible spectrum, producing high-resolution images in real-time.
Compared to radar and satellite imagery, EO/IR systems are shorter in range but provide a much more detailed image of an object. This can give the level of detail necessary to support decision-making and provide enough data for precise target identification and classification.
Furthermore, EO/IR is passive when detecting radiation giving a stealthy option for surveillance. Meanwhile, whilst radar offers longer-range detection it is often active, emitting a radio frequency signal and detecting the returns. This means radar signals can be detected by the target. Other sensors such as optical imaging satellite cannot see through clouds which means they are limited in bad weather.
EO/IR systems gather information using an advanced set of cameras to produce high resolution imagery either in day or night. These cameras are usually set inside a rotating mechanical gimbal or a pan/tilt system that has windows through which the cameras can see. The gimbal or pan/tilt system can spin 360 degrees to allow the camera to scan its field of regard and can be directed to focus on specific areas of interest. The gimbals also provide stability and protection for the cameras.
Stability is important as gimbals can be mounted on aircraft that are moving at speed through the air or naval vessels on the surface of the water. Stabilisation ensures that the gimbal can remain directed on target and provide clear images that are not blurry despite any juddering or movement of the platform. Protection is needed to ensure the camera is not damaged by the conditions of the maritime environment with salt and water ingress.
Coastwatch
Using EO/IR systems in coastal areas presents challenges relating to atmospheric scattering and absorption that can produce blurred images of targets due to reflected radiation and the atmosphere absorbing some energy whilst allowing other wavelengths to pass through. In the IR spectrum there is a need to ensure a thermal contrast between the target and the background. As with all IR imaging systems the temperature emissions from the target, such as from the engines and other systems are decisive for target signature generation.
Alexander Ogger, communications manager at Hensoldt, a German sensor solutions company told AMR: “The most critical challenge for EO/IR systems under these conditions are range performance based on sufficient resolution and line-of-sight stability based on EO/IR system stabilisation performance, compensating all system platform motions.”
He said that Hensoldt is addressing these challenges by integrating state-of-the-art detector and focal plane array (FPA) technologies into its EO/IR systems.
“This includes the integration of multispectral imaging from the ultraviolet (UV) over the visible (VIS) to the thermal infrared (IR) spectral range, advanced signal processing and stabilisation mechanisms to improve the performance of their systems when operating on land, at sea or in the air in coastal environments,” he said, “In addition, the aim today is already to merge the information generated from the image data as quickly as possible and to show the operator appropriate options for action. Software is therefore the key to utilising the already excellent hardware even better and more efficiently.”
The size of an EO/IR system will dictate what additional cameras and capabilities can be fitted inside. The larger the gimbal the more sophisticated it can be and offer longer ranges with better resolution.
Mark Garland, director of quality at Teledyne-FLIR, a US sensor systems company, told AMR: “The larger the gimbal allows either larger optics and imagers or more systems can be placed in the gimbal.”
He explained that an EO/IR system as a minimum will usually have a visible colour and low light camera and the thermal camera to provide day and night capabilities. As the size of the gimbal increases the optics and focal plane arrays of these cameras can be made larger to increase the pixel resolution, or additional cameras or laser range finders can be included.
However, not all airborne or surface vessels can host large heavy EO/IR systems, which will require more size, weight and power (SWaP) to operate and become more expensive to buy.
“The value of EO/IR systems in the littoral environment starts with stand-off distance. The ability to observe, surveil and interrogate an object or potential threat at distance,” Garland said, “Then it becomes about the size of the boat and how a big a system can be fitted and what system is affordable for the vessel.”
For example, a Rigid Hull Inflatable Boat (RHIB) or other small craft can only support a limited payload weight, especially if positioned on a fixed or telescopic mast high above the deck. Therefore a trade-off is required to try and get the maximum capability into smaller gimbals in the right price window that can be fitted onto these smaller platforms. It means companies usually offer a range of EO/IR systems that best complement the platform and the role of the vessel.
This is reflected in the Teledyne-FLIR’s offering of scalable EO/IR products. Its SeaFLIR is available in different sizes starting with the 230, and 240, then 280 and higher. With each iteration the size of the gimbal increases and additional capabilities can be added. In the SeaFLIR 280 an additional camera called a ‘spotter’ has been fitted that can be a shortwave IR camera that complements to mid-wave. Larger gimbals like the 380 will have a fourth imager such as a long-wave or second type of mid-wave; or a mid-wave, shortwave and longwave.
But there is a physical limit to how large an EO/IR systems can be. It all comes down the size of the FPA as the pixel density can’t get any more compact and is close to saturation. The optical glass element has also reached its limit in scientific terms. An optic and FPA can only be so large inside a gimbal, but there are other technological improvements in the mechanical system and with miniaturisation of parts and reduced weight of new materials that can offer some capability improvements.
Garland said that the development of the SeaFire 240 system started in 2019 and it has been on the market since 2022. At 11inches in diameter the gimbal is slightly larger than the nine inches diameter of the preceding SeaFire 230 gimbal. But technology enhancements mean the capability of the 11-inch ball can now be packaged in the nine-inch offering operator the option of a system with smaller size, weight and power requirements for smaller boats but with the capability of a larger system.
However, other improvements in EO/IR capability will come from the faster processing of the data and the way the EO/IR system can integrate with other ISR sensors to speed up decision-making.
Automatic classification means that the EO/IR sensor can separate out a buoy, RHIB, boat, sailing ship and tanker into different sets as well as detect humans on the coast or uncrewed aerial vehicles (UAVs) in the air. It can also gather more dynamic data including trajectory, size and shape, speed, a vessel’s AIS data and flag up whether a target is friend or foe.
AI and ML Interpretation
Cooperation between sensors is also important. A radar can tell the EO/IR system that it has a contact at a certain degree and direct the EO/IR gimbal to point in that direction to get a full visual of the object. Radar is becoming more advanced with aided target recognition (ATR) capabilities that can analyse the signatures received and identify an object as a boat. In the future Artificial Intelligence (AI) and Machine Learning (ML) may even help the ATR to classify the signatures, if they are strong enough, to classify the type of boat.
But AI and ML are also used to enhance EO/IR systems as a way of giving operators more information about a target on screen faster well before the human eye can identify it. This will help manage the amounts of data, improve the fidelity and accuracy, improve resolutions as well as speed up processing and integration with other systems.
Detection, recognition, identification (DRI) was the focus of the technological efforts to mathematically model and enhance EO/IR systems so that they could detect a vessel of a specific size at a specific distance under a set of atmospheric conditions.
This allows operators to fit the right product into their operational requirements. For example, if a naval vessel does not want a potential threat to be any closer than 2.7 nautical miles (5km) then it will need a certain sized EO/IR gimbal that can help to conduct DRI at distances in excess of this and avoid risk to the platform.
“Today, we are taking DRI into the AI realm of imaging processing and doing processing at the edge in real-time building that capability directly into the gimbal,” Garland said, “It means you don’t have to take the video, feed that into a secondary computer and wait for the processing – this might only be second latency but any latency is critical in the theatre of war.”
This would require more CPU and GPU with associate power and cooling systems in the gimbal, adding to the complexities of the trade-off for space inside the system and is only really possible in larger gimbals. However with technological development it is only a matter of time before this capability will trickle down and the processing systems fitted in larger gimbals can be expected to be fitted into smaller ones soon enough. This will provide reduced latency in all EO/IR systems.
If targets can be detected from further away it gives more time for operators to react and respond. The operator with the ability to do DRI at the longest-range detection capability has the strategic advantage. Being able to do this passively offers additional protection if the seeing element is not detectable by an adversary. It is no longer about being able to steam faster with a boat or fly faster with the aircraft – capability overmatch is delivered by the onboard systems.
With sensors working together the ATR capability can also queu the operator to take a look at a specific target of concern and have a target handoff and slewing capability for weapons that reduces the operator workload and reduces response times.
“But it should be appreciated that as much as ATR and AI/ML is going to transform the industry, it has yet to gain significant trust,” Garland said, “Generals and admirals recognise the value but they still have to fully understand the accuracy and probability that is right. It’s trusting the machine versus trusting the human and that is going to take some time.”
These kinds of capabilities are of particular importance in contested areas such as the South China Sea where there is an acute need for more persistent ISR capabilities and faster reaction times to incursions across maritime boundaries. But the ability to conduct ISR and provide accurate data on objects and shipping is becoming more important across the wider Indo-Pacific region, where drug trafficking and Illegal Unreported and Unregulated (IUU) fishing are growing problems.
Meanwhile more uncrewed systems are entering service both in the air and on the surface with platforms of different sizes. These will use EO/IR systems as part of a sensor package and depending on the size of the platform it will have SWaP requirements for a specific payload.
Other options in the future include teams of Uncrewed Surface Vessels (USVs) that can be deployed to spread out in a spider formation. Each will have an EO/IR system with the ability to communicate with each other. During the observation of their sector the USVs can automatically que each other to bring in additional imaging capabilities from the other platforms if needed. This capability to mesh and make more strategic decisions about how best to assess a target at distance is something that is expected to develop rapidly in the near future.
As more USV and UAVs enter service it will mean an expansion in the use of EO/IR systems as they will all need networked EOIR capabilities as part of their sensor solution fit.
Meanwhile use of satellite optical imagery for littoral surveillance is also increasing in volume as the market sees new entrants in addition to the traditional satellite operators.
Writing in the Naval War College Review in 2021, Gregory Poling, Senior Fellow and Director, Southeast Asia Program and Asia Maritime Transparency Initiative at the Center for Strategic and International Studies stated in his article ‘From Orbit to Ocean—Fixing Southeast Asia’s Remote-Sensing Blind’ that over the past 15 year that falling prices for satellite construction and launch have driven down the cost of imagery collection.
He cited two categories of imagery providers specialising in maritime domain awareness. The first includes Airbus, Maxar, and ImageSat, that have relatively small constellations of large, expensive satellites with long lifetimes.
“They provide the highest-resolution imagery on the market—in some cases down to 3cm,” Poling explained, but which are reliant on government contracts. “Their imagery can provide a high degree of detail about a vessel, making them particularly helpful in the identification part of MDA,” he added, “But their relatively small constellations also make their imagery expensive and available only infrequently.”
The second group is being led by PlanetLabs, which rely on larger constellation of cheaper satellites with shorter lifetimes operating in low earth orbit (LEO).
“Their maximum resolutions are generally lower, making identification of specific vessels, or even vessel types, difficult, but their larger constellations allow much more persistent coverage of an area at a lower cost,” Poling wrote, “And as their satellites improve they are closing the resolution gap rapidly, suggesting that soon it will be possible to get both persistent coverage and very high resolution at low cost.”
But he added: “However, that will not fix the greatest limitation of electro-optical imagery for MDA: the weather. Satellites cannot take pictures through cloud cover, and in Southeast Asia that can mean long stretches of blindness owing to storms.”
As more uncrewed systems become an integral part of the maritime domain awareness mission and the number of satellites in LEO expands in the coming years, EO/IR systems will become increasingly important as a way of providing precise real-time imagery about targets and objects of interest improving littoral domain awareness.
by Tim Fish
