The Top 10 Questions about Airborne Hyperspectral Imaging

1 – What is the purpose of airborne hyperspectral imaging?

Any material that is detectable either directly or indirectly based on its spectral features, can be mapped with an airborne hyperspectral camera. The point of airborne hyperspectral imaging is in creating a material map of the study area, land or water surface. Taking the sensor in the air gives you a vantage point to search these materials, plant species etc. from the much larger area than what is immediately visible on the ground – easily hundreds of square kilometers at the time, only limited by altitude and time spent flying. While airborne sensors usually create detailed geometric models (LiDAR) or imagery for human interpretation (multispectral cameras, SAR), hyperspectral sensors create data which is analysed into a thematic map of material features. Imagine that – you can be kilometers up in the air, speeding hundreds of kilometers per hour, and still create an exact map of materials, minerals or plant species at your survey area. No other passive imaging technology can do that!

Imagine you are in a boat on this river, tasked to assess the existence of particular plant species of interest in the area (red). How accurate situational awareness of the target plant quantity would you expect to create, compared to the real situation apparent from the above image? (Image courtesy of SpecTIR LLC)

2 – How does it differ from multispectral airborne imaging?

“Multispectral” is one of the most confused and misused umbrella terms used in remote sensing. Multispectral imagers typically have 3-5 broad bands with gaps in between, depending on which applications the multispectral imager is built for. Let’s remember that even normal digital camera found from every smartphone, is multispectral imager with 3 spectral bands. Thus, multispectral can mean almost anything from general consumer camera to an application specific imager.

By definition, hyperspectral imaging collects hundreds of contiguous, narrow spectral bands. This means that there are no “gaps” between the bands. Hyperspectral means far more, more narrow bands than multispectral imaging.

Resulting differences for a user are threefold. With hyperspectral imager, you can differentiate material (minerals, plants etc) of much smaller spectral difference, thanks to much higher spectral fidelity. To put it in layman’s terms, with multispectral imager you can – for example – tell if the area has vegetation or not. With hyperspectral imager, you can tell which species the vegetation consists of, and moreover, if those plants are suffering a stress, and at best what is the cause of the stress. Multispectral imaging may be enough to tell asphalt apart from gravel or concrete, but in order to tell how old the asphalt is and what is its material composition, you need more and narrower bands provided by the hyperspectral imager. Only hyperspectral imager can tell apart minerals that are important for geologists, but which exhibit so minute spectral differences that multispectral imager is unable to tell them apart. There are many of such minerals!

Land classification map with roof/road separation task left, RGB multispectral image right. (Analysis work courtesy of Dr. Kati Laakso)

Multispectral imager is always built for a particular application, whereas hyperspectral imager is application agnostic; it collects all the spectral information from the target and so doing serves all possible applications, only limited by the skill of the analyst. This means you can come back to the collected data and turn it into an application – or several applications – you were not tasked with originally. As long as the data is collected well, it can be turned into dozens of different applications. If organization is collecting hyperspectral data for a client, it is wise to price full rights reflecting the future potential of the dataset! Remember that every time you collect a snapshot of the world at that moment, you constitute a starting point for a temporal study.

Multispectral image is typically a photo that is interpreted by the human eye. Hyperspectral image quantifiable set of data instead of just an image. While the end result may still be an image like a thematic map or a detection alert in operators screen, it is an end result of an algorithm based on characterised data. And it depicts phenomenon that would not be detectable with less or broader spectral bands. Hyperspectral data is more empirical “big data”. Whether it should be considered sparse or dense, depends on the viewpoint.

3 – Are pixels true-orthorectified?

Yes. When data is georeferenced using DSM (see glossary in the end of the text), Specim´s CaliGeoPRO preprocessing software is computing the exact distance between the sensor –the nodal point of central projection – and every image pixel. The end result is an image where parallax errors due to elevation differences are removed, i.e. pixels are shifted to their correct locations so that resulting image can be used as a map where directions, distances and geolocations at each pixel are correct. If DSM is not available, DEM/DTM can be used to remove the parallax errors of the ground surface. This would of course still leave parallax errors to everything on top of the surface, but it is still better than using flat earth model only, especially over rolling terrain.

Let’s have an example of the importance of DSM compared to giving just single elevation value to the whole image. 115m tall cooling towers of a coal powerplant were collected using SPECIM AisaFENIX. Cooling towers were symmetrical, meaning both roots and the mouths of the towers were perfectly round. When the distance was given to the roots of the towers, the roots were round but mouths were incorrectly oval, meaning they have parallax error and thus also wrong geocoordinates. When the distance was given to the mouth of the towers, they were round but the roots of the towers were oval in turn, meaning parallax error had moved there (see image below). This is simple geometry and explains georeferencing errors when using only flat earth model. True-orthorectification is required and DSM is the key because through DSM the exact distance to each point in the image can be calculated, whether they are ground features, trees or man-made structures like the cooling towers used in the example.

Left: Roots correctly round – mouths incorrectly oval Right: Mouths correctly round – roots incorrectly oval. Also note
the difference in swath widths of the two different georectification results of this same dataset

115m high cooling towers seen from the air.

4 – What should I know of airborne sensor mountings?

The main difference to the ground based use of hyperspectral sensors is the need for geolocating image pixels, most typically by using a GNSS/IMU. One of the first things to do is to decide the required minimum georeferencing accuracy against the purchase cost of the corresponding positioning system. As a general rule, GIS-accuracy is meters, survey accuracy is centimeters and higher accuracy simply costs more.

GNSS receiver obviously needs a suitable antenna or several, and this antenna has to be suitable for airborne use. IMU installation must be rigid with the sensor, there must be no damping elements or flexible structures between IMU sensor and the camera for obvious reasons. IMU update rate should be chosen with expected vibration profile. As a rule, helicopter type vibrations require update rates of 200Hz or better. If update rate is too low, platform vibration will cause waviness to the image.

Examples of “heli-wobble” from two different helicopter installations. What we see in these images is a georeferencing error caused by unchecked platform vibration. It happens when IMU measurement rate is slower than the frequency at which the sensor is vibrating. Lacking high rate IMU, mounting should be at least vibration damped and if possible also gyrostabilized, to reduce the sensor vibration to the minimum. Perfectly damped and stabilised sensor does not necessarily need IMU at all.

You also need an airborne mounting system. There are several different options to consider, mostly depending on the aerial vehicle available. Although often overlooked, the mounting system is one of the key drivers of aerial survey success, because unsatisfactory mounting reduces data quality. Bad mounting has a capability to render sensor inoperative for example through extreme cold, or in extreme cases even damage the sensor through high shock loadings or water seepage. Mounting should also prevent oil or water from finding its way to sensor optics. Engine and/or hydraulic oils are often found flowing in small quantities along the fuselage bottom if not checked regularly. There are further complications connected to the pressurised windows like fogging, coating, ageing and calibration.

Hard mounting

This is the most common and works well when the platform is stable and does not vibrate too much compared to the sensor weight and lever arms between the fixtures. Note that Specim sensors are largely insensitive to vibration because they have a minimum amount of moving parts, but like with any imaging sensors, vibration does reduce the image quality.

Hard mounted SPECIM AisaFENIX.

Vibration damped mounting

Normally based on either silicone or spring damping elements around the centre of gravity of the sensor. Used to improve image quality and protect the sensor against landing shocks. Needs correct dimensioning across the platform vibration envelope, in order to avoid resonance and worse results than without damping. There are large variations in vibration profile depending on the aerial vehicle used.

SPECIM AisaOWL thermal hyperspectral sensor installed to a helicopter using silicone vibration damping elements. Note how the combined weight of two sensors are used to assist in passive damping, how damping elements are spread over a large area using “damping floor” arrangement, and how extra lead weights are used to correct the center of gravity of the damped weight.

Gyrostabilized mounting

The sensor is held pointing nadir and direction of travel despite platform rotations. May or may not include vibration damping. Used for improved image quality and survey efficiency, especially when aerial vehicle is unstable (again like all RPAS and rotary wing craft), or flies at low altitudes where the air is more turbulent. It is advisable to use mounting that is collecting the instantaneous rotation angles for correcting the antenna excentricity. Also, it is beneficial to use gyrostabilized mounting model that supports the used GNSS/IMU and FMS systems. GNSS/IMU support means accepting levelling assistance from the IMU, and FMS support means accepting drift correction and mode switching commands in various phases of the survey flight.

AisaFENIX mounted on a gyrostabilized mount.

Comparison between non-gyrostabilized(left) and gyrostabilized AisaFENIX imagery. Both datasets are non-georectified, but in the right image a high-end gyrostabilizing cameramount removes waviness caused by aircraft roll, pitch and yaw movements.

Turret mounting

For non-nadir standoff-type scanning, increased FOV through scanning motion, gyrostabilization or them all. Sometimes the STC of an air vehicle is designed for a particular type of turret, making it the preferred mounting option.

Pod mounting

Used for environmental protection of the sensor when payload must be hung outside the airframe for the lack of suitable fuselage apertures. The pod is a certified enclosure with STC-accepted mounting. It is typically rigidly mounted to a hardpoint, so possible vibration damping or gyrostabilization must be built inside the pod.

Nosepod of a DA42MPP with AisaFENIX mounted on gyrostabilized cameramount.

Power supply

Power supply of the air vehicle should, of course, be sufficient for the sensor system. The load may vary depending on the ambient temperature, and typically hot conditions require most power. This is because AISA sensors are temperature stabilized, i.e. cooled or heated according to ambient temperature. Note that small GA aircraft or RPAS often do not have sufficient generator supply as standard, requiring separate battery supply for the sensor.

5 – What are the most difficult things I should give a thought on when planning hyperspectral aerial survey project?

Weather is the number one obstacle in any aerial survey using passive daylight sensors. While the capacity to cover large areas in short period of time is huge, the window of weather opportunity allowing data collection is usually limited or very limited and if missed, next one may be weeks or months away. Therefore surveys must be designed around the priority of utilizing every single weather opportunity at short notice, during the weekend or public holiday if necessary. Above all, enough days should be reserved as weather forecasts maintain reasonable accuracy only about 3 days ahead.

Main weather constraint is the clouds. While it is possible to process hyperspectral data collected under the cloud, SNR of such data is usually unsufficient.

Best aerial survey operation is the one with most knowledge of the diurnal cumulus/fog/advection cloud patterns, approaching frontal and pressure systems and the ability to react to those quickly and flexibly, being ready to change the plan even during the flight based on weather observations. In short, aerial survey success depends on the quality of weather tactics employed.

SNR

Unlike lab spectroscopy, airborne remote sensing does not have the luxury of artificial illumination of unlimited intensity. Moreover, airborne platforms usually have some minimum flying speeds setting certain limits to how long exposures are available.

Differentiating materials, minerals and plant species of sometimes minute spectral variations requires good SNR. This simply means that the required spectral features must emerge above the noise. Good SNR requires exposure times long enough for the reflective properties of the target and illumination conditions. Although the exposure selection is freely selectable in Specim sensors within the frame period, the maximum available exposure time is limited by the speed at which pushbroom line images are collected. This speed, called Frame Rate, is dictated by GSD and groundspeed of the aerial vehicle.

Quite according to normal photography, a dark target like sea surface requires a long exposure, and bright target like desert can do with short exposure, faster frame rates and thus higher ground speeds. At Specim, we consider 300:1 as a minimum for “good” SNR. The main thing to remember is that SNR is not fixed, but apart from sensor quality, greatly affected by flight planning and operation. As a rule, the slower you can fly, the better the SNR.

Difference between good SNR (black) and poor SNR (red).

Atmospheric correction

In order for the material detection to be possible, effects introduced to the data by sensor, illumination and atmosphere must be removed. Sensor effects are removed by radiometric processing using radiometric calibration.

Unlike in lab spectroscopy where white reference normalization can be done, in airborne remote sensing, the effects of atmosphere and illumination are best removed by algorithm-based atmospheric correction routines like FLAASH(-IR), QUAC, ACORN or ATCOR-4. However, atmospheric correction may not always be required. If the aim is only to compare in-scene spectral variations without a need to apply spectral libraries or temporal comparisons, atmospheric correction may not be required at all.

If required, atmospheric correction method should be chosen before the survey, and required support data like sensor characterization and Digital Surface Model reserved ready. Even the best atmospheric correction algorithm is only as good as the quality and characterization of the input data.

The difference between the graphs is due to residual atmospheric influence after reflectance processing. In this example, the atmospheric correction methodology should be reassessed.

Navigation

Airborne surveys are conducted along straight flight “lines” that must be flown accurately in both horizontal and vertical dimensions. Moreover, survey line should be reached with a minimum of the valuable flight time spent during the turns between the lines. Accuracy requirement is normally much higher than what standard aircraft navigation instruments are able to provide because these are made to find locations of the size of airports. Purpose-built aerial survey flight management system, FMS, should be applied. This is often forgotten when main attention is concentrated around sensors and their operation. If the problem of precise survey navigation is left to the pilot to manage without proper FMS nor navigator with navsight, the survey may prove totally impossible, and in best case poor in quality. Even if there is a means to navigate along the lines using aircraft instruments with some degree of accuracy, changeovers from line to line are typically very inefficient so that up to 80-90% of the flight time may be wasted flying just turns between the lines. A good turn lasts only 2 minutes. If there are 60 lines to fly, one extra minute spent during each turn means one extra flying hour. Aerial Survey FMS systems have been developed to minimize the time spent in the turns and to keep the aircraft with its sensor flying accurately according to the plan.

6 – What are the weather requirements for hyperspectral imaging?

There must be no clouds or their shadows on the imaging area. Note that as especially the Cumulus clouds often follow solar angles, choosing the minimum solar angle usually calls for a compromise if any data is to be collected at all. If a too high solar angle is required, the weather tends to be more cloudy up to being unflyable during the whole daily period allocated for the survey.

Special case: thermal (LWIR) hyperspectral imaging using AisaOWL. No clouds below the survey altitude, but clouds above the survey altitude may be either beneficial or detrimental depending on the type of survey. Warm surface temperatures improve SNR.

7 – How does the direction of the sun affect airborne hyperspectral collection?

Sun should be above 30-45 degrees above horizon. This in order to enable atmospheric correction routines, reduce the amount of shadow in the scene and ensure good illumination intensity. There are sun angle calculators available for the task.

Sun should also be as close to/from flying direction as possible to reduce BRDF effects. Generally, north/south directions should be used apart from corridor mapping projects. Flight lines perpendicular to the sun should be avoided.

Sun angles above 90deg minus FOV/2 will create hot spots or contrast less and potentially saturated spots in the data. When operating close to the equator, the period of time sun spends above this angle may have to be avoided depending on the image quality requirements. Apart from hot spots, high solar angles also cause specular reflections on water surfaces near to the edges of the swath, if the flying direction is too close perpendicular to the sun. Can be considered to be a flight planning error with respect to alignment of flight line compared to the sun, as BRDF effects will also be severe in general if specular reflections exist in the data.

Again special case: thermal (LWIR) hyperspectral imaging using AisaOWL. Does not need illumination and can be flown either day or night, often both to assist atmospheric correction. Still, in daylight LWIR surveys the direction of sun creates asymmetric warming in the various sides of scene objects, visible as temperature “contrast”.

8 – What parameters should be defined when doing flight planning for hyperspectral aerial survey?

All aerial survey planning starts from clearly defined area delimitation. After that, collection resolution has to be decided and minimum airspeed of the used aerial vehicle clarified. With these start values, following parameters can be determined:

FPS

Or Frames Per Second setting used during flight. Calculated based on height over the ground and expected groundspeed, in order to create square instead of stretched pixels on the ground. Groundspeed value should be minimum practical airspeed reported by the pilot, plus the expected wind speed, usually +5m/s works as a good average.

The initial assumption is always maximum exposure time for best possible SNR. Therefore exposure time does not have to be calculated in advance, only reduced in flight in case saturation is observed by the operator, and then only saturation on targets of importance considering the goals of the project.

Exact flying height over the ground to satisfy the ground sampling distance requirement

Note that in mountainous terrain minimum and maximum GSD tolerances must be discussed and planned accordingly, cutting flight lines as required and adjusting their individual flying altitudes in order to keep flying height correct on the average. Flying height is passed to FMS planner for the creation of FMS navigation files.

Flight line spacing

Swath width–dependent distance at which flight lines should be placed from each other in order to satisfy lateral overlap requirement – typically 30% with fixed sensor mounting and 20-25% with gyrostabilized sensor mounting. The less lateral overlap, the fewer flight lines to cover an area and thus less flight time required, but with increased risk of gaps between flight lines. Gaps are especially possible in rolling terrain. Use of DEM/DSM is highly recommended in support of flight planning. Flight line spacing, like height over ground, is also passed to FMS for the creation of FMS navigation files, consisting typically of start and end coordinates of each flight line, including the ground elevation and flight altitude at that line.

9 – What kind of calibration does an airborne hyperspectral sensor need?

Compared to lab sensors, airborne hyperspectral sensors have some unique needs what comes to calibration.

Radiometric calibration

Because airborne hyperspectral sensor cannot rely on having a full-swath white reference for normalizing the DN values directly to reflectances, a two-step process is required. The first step is removing sensor effects from the data and converting DN values to physical at-sensor radiance measurement values. This is based on calibrating the sensor using calibrated light source, exposing sensor with a precisely known amount of light. Each pixel receives a calibration coefficient used for converting DN values to radiances.

Extended spectral calibration

Basically, this is no different for airborne sensors compared to lab sensors, but atmospheric correction routines additionally require precise measurement of spectral resolution in terms of Full-Width Half Maximum for each central wavelength. The shape of spectral response curve can also be measured for highest accuracies.

Geometric calibration

Required for defining exact FOV for flight planning, for removing geometric distortions for the benefit of georeferencing and also for creating scan angle file for BRDF removal.

Boresight calibration

Attitude difference between the camera and IMU. The aim is to be able to read camera attitude from IMU readings. As it is not possible to mount IMU so that it would have same “zero” attitude as principal axis of the sensor, boresight correction angles must be worked out based on calibration flight. Name “boresight” comes from the method by which the sights of artillery pieces used to be adjusted by looking to the target down the barrel and then checking where the sights were pointing. Boresight calibration of an IMU sensor follows the same principle. Provided that there have been no excessive shocks, boresight calibration remains unchanged as long as IMU remains connected with the camera.

10 – What kind of aerial vehicles can be used for hyperspectral survey?

Any vehicle capable of lifting the system weight can be used, but generally the slower the vehicle the better. Some very general pros and cons of the various air vehicles.

Fixed-wing manned aircraft

Pros:

– Fast with long endurance, equating with high mission efficiency especially in large projects or when several projects are spread over a large geographic area.

– High maximum altitude

– More stable and vibrating less than rotary wing aircraft.

– Plentiful everywhere in the world, with well-established procedures and support facilities.

– Superior level of technical safety compared to other airborne platforms, with the exception of LTA´s.

– Cheaper per flying hour than comparable manned rotary wing aircraft.

– Access to practically any airspace in the world above local minimums, by employing the established airspace usage routines.

Cons:

– There is a minimum speed, the higher the faster aircraft. This limits the smallest achievable ground sampling distance(GSD) and signal to noise ratio (SNR).

– Needs airfields for takeoff and landing.

Rotary wing manned aircraft (helicopters)

Pros:

– Lower minimum speeds at altitude compared to fixed wing aircraft, with corresponding benefits for minimum achievable GSD and SNR

– Does not need fixed airports for takeoff and landing

Cons:

– Unstable around all axises with complex high amplitude vibration profiles

– More expensive per flight hour than comparable fixed wing due to high mechanical complexity (excluding autogyros)

– Slow with short operational radius

– Despite being able to fly slow, they still can’t go below certain speed at altitude

– Low maximum altitude

VTOL RPAS (Vertical Take Off and Landing Remotely Piloted Airborne Systems)

Pros:

– Flexibility regarding take off and landing site.

– They do not have minimum speed, which equates to not having minimum for the GSD

Cons:

– Low endurance

– Vibration

– Poor flying safety record due to inherent technical complexity coupled with often low-tech components

Fixed wing RPAS

Pros:

– Long endurance

– High performance regarding speed and altitude

– Better safety than with VTOL RPAS

Cons:

– They require runway or launch/recovery arrangements with added complexity and restrictions

– Vibration when using IC engines

Manned/ RPAS LTA (Lighter Than Air: airships, balloons)

Pros:

– Very long endurance measured in days, making them suitable for long duration surveillance

– Slow and able to hover at any altitude

Cons:

– Slow from the point of view of covering project areas and ferrying between them within the given daily weather opportunity

– Severe wind restrictions

– Complex and costly filling, storage and transport

Glossary

BRDF = Bidirectional Reflectance Distribution Function. Change of apparent illumination depending on viewing surface angle.

DEM = Digital Elevation Model = model of the ground surface without trees and houses etc. In LiDAR terms, the last pulse surface.

DSM = Digital Surface Model = model of ground surface including treetops and man-made objects like houses, bridges etc. In LiDAR terms, the first pulse surface.

DTM = Digital Terrain Model . Can be considered to be manually corrected DEM where among other things, linear features in otherwise bare earth model are highlighted.

LiDAR-derived DSM-model where elevations are presented as grey values.

FOG = Fibre Optic Gyro, a gyroscope where sensing of movement is based on light travelling a long distance in fibre optical coil. Extremely precise, but heavier and much more expensive than MEMS gyros.

FOV = Field Of View. The opening angle of a front objective.

FMS = Flight Management System. Here, aerial survey navigation system. Used to navigate the aerial vehicle to a preplanned position.

FPS = Frames Per Second. Collection speed of the hyperspectral sensor

GIS = Geographic Information System. Typically a database containing spatial information for various environmental research or urban planning uses.

GNSS = Global Navigation Satellite System. A system capable of calculating positioning solutions based on not only GPS but also GLONASS, Galileo and/or BeiDou. Compared to GPS only-system, additional satellites like those from GLONASS, Galileo and BeiDou constellations improve accuracy.

GSD = Ground Sampling Distance, also known as ground or spatial resolution.

IMU = Inertial measurement Unit. A device that senses rotation and movement. A gyroscope.

LiDAR = Light Detection and Ranging. A system that uses Laser light to measure distances and create a 3D map of the object.

LWIR = Long Wave Infrared, thermal infrared spectral region

MEMS gyro = gyroscope that is based on vibrating element, typically piezoelectric one. Cheaper and smaller than FOG´s, but also less accurate.

NRT = Near Real Time hyperspectral sensor system, able to process the data right after collections in a time frame from a fraction of a second to few minutes.

RPAS = Remotely Piloted Aerial System. Loosely synonymous but more accurate definition to what we have been used to call as UAV:s, for the fact that UAV is never really “unmanned” except when fully autonomous, which is still so rare that it is a virtually non-existent concept. Currently, there is always human involvement in the operation, even if he would stay on the ground monitoring operation. RPAS is an aircraft where pilot stays on the ground, while a fully autonomous vehicle is a robot that requires no human intervention. This is an important distinction from the point of view of, especially regulatory authorities.

SNR = Signal to Noise Ratio

STC = Supplemental Type Certificate. Authorized method of modifying an airborne vehicle.

VTOL = Vertical Take Off and Landing. Typically, a helicopter or multirotor.