Approximately 66% of all global surfaces are covered by clouds [Wilson & Jetz 2016]. While some areas are very rarely covered by clouds, others present cloud occurrences above 75%, which reduces the opportunity of optical sensors to measure a clear surface signal. Thus, an accurate cloud screening algorithm is needed for most downstream applications. To provide reliable and consistent cloud masking algorithm results, an independent validation source that fulfills a set of requirements is needed. While a few studies have quantitatively inter-compared some of the state-of-the-art cloud detection methods using reference datasets, until most recently, no study had compared the used reference datasets themselves.
There are only a few available reference datasets that can be used for cloud mask validation. Before the Cloud Masking Intercomparison eXercise (CMIX), conducted within the Committee on Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV), no independent analysis on the quality and usability of Sentinel-2 and Landsat 8 reference datasets (Baetens & Hagolle 2018, Hollstein et al. 2016, Paperin et al. 2021a, Paperin at al.2021b, Skakun et al. 2020, U.S. Geological Survey 2016) had been made, nor had they been compared. Results from CMIX revealed that all datasets have shortcomings. One major shortcoming of all reference datasets is the manual - and thus to a certain extent subjective - interaction needed to generate the cloud masks, which potentially introduces a bias and leads to temporarily very limited reference datasets.
In 2020, Skakun et al. showed the usefulness of sky images from ground-based cameras for satellite-based cloud mask validation and presented an inexpensive approach for the generation of such data using a Raspberry-PI based system. While those approaches still relied on manual interaction, the work presented here aims to develop automated procedures for the generation of reference datasets. Within the ESA Quality assurance framework for Earth Observation (QA4EO), and in cooperation with University of Maryland/NASA, a stereo pair of Raspberry-PI based sky cameras were installed at La Sapienza University in Rome. In addition, a new ceilometer, called RAP (Raymetrics Aerosol Profiler) was installed in the QA4EO framework, to validate the sky camera-based cloud heights. Together with an additional pair of cameras placed at the Goddard Space Flight Center in Greenbelt, MD, USA, the Rome site is used as a testbed to develop algorithms for automated reference dataset generation. The goal of the work presented here was to analyze the general requirements for a reference dataset for validation of satellite-based cloud masks, and to evaluate the suitability of the sky camera approach and, if necessary, to propose modifications for the improvement of the measurement setup.
The strength of the tested sky camera approach is a nearly continuous measurement in very short time intervals, allowing the data to be used as validation source for a great number of different optical satellite sensors. However, some challenges are still present in the measurement setup, such as the comparison of clouds observed from different viewing points and the appropriate matching of satellite and camera images which considers the effects of lens distortion. While the approach is under development, if proven robust, the quite inexpensive setup would allow for an expansion towards a global network of sky cameras, potentially providing a unique multi-temporal, near real-time validation source.
[HOLLSTEIN ET AL. 2016] HOLLSTEIN, ANDRÉ, KARL SEGL, LUIS GUANTER, MAXIMILIAN BRELL, AND MARTA ENESCO. 2016. "READY-TO-USE METHODS FOR THE DETECTION OF CLOUDS, CIRRUS, SNOW, SHADOW, WATER AND CLEAR SKY PIXELS IN SENTINEL-2 MSI IMAGES" REMOTE SENSING 8, NO. 8: 666. https://doi.org/10.3390/rs8080666
[BAETENS & HAGOLLE 2018] LOUIS BAETENS, & OLIVIER HAGOLLE. (2018). SENTINEL-2 REFERENCE CLOUD MASKS GENERATED BY AN ACTIVE LEARNING METHOD [DATA SET]. ZENODO. HTTPS://DOI.ORG/10.5281/ZENODO.1460961
[PAPERIN ET AL. 2021A] PAPERIN, MICHAEL, STELZER, KERSTIN, LEBRETON, CAROLE, BROCKMANN, CARSTEN, & WEVERS, JAN. (2021). PIXBOX LANDSAT 8 PIXEL COLLECTION FOR CMIX (VERSION 1.0) [DATA SET]. ZENODO. https://doi.org/10.5281/zenodo.5040271
[PAPERIN ET AL. 2021B] PAPERIN, MICHAEL, WEVERS, JAN, STELZER, KERSTIN, & BROCKMANN, CARSTEN. (2021). PIXBOX SENTINEL-2 PIXEL COLLECTION FOR CMIX (VERSION 1.0) [DATA SET]. ZENODO. https://doi.org/10.5281/zenodo.5036991
[SKAKUN ET AL. 2020] SKAKUN, SERGII; VERMOTE, ERIC; SANTAMARIA ARTIGAS, ANDRES EDUARDO; ROUNTREE, WILLIAM; ROGER, JEAN-CLAUDE (2020), “DATA FOR: AN EXPERIMENTAL SKY-IMAGE-DERIVED CLOUD VALIDATION DATASET FOR SENTINEL-2 AND LANDSAT 8 SATELLITES OVER NASA GSFC”, MENDELEY DATA, V1, DOI: 10.17632/R7TNVX7D9G.1
[SKAKUN ET AL. 2021] SKAKUN, S., VERMOTE, E.F., ARTIGAS, A.E.S., ROUNTREE, W.H., ROGER, J.-C., 2021. AN EXPERIMENTAL SKY-IMAGE-DERIVED CLOUD VALIDATION DATASET FOR SENTINEL-2 AND LANDSAT 8 SATELLITES OVER NASA GSFC. INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION 95, 102253.
[U.S. GEOLOGICAL SURVEY 2016] U.S. GEOLOGICAL SURVEY, 2016. L8 BIOME CLOUD VALIDATION MASKS. U.S. GEOLOGICAL SURVEY, DATA RELEASE. DOI:10.5066/F7251GDH.
[WILSON & JETZ, 2016] WILSON AM, JETZ W (2016) REMOTELY SENSED HIGH-RESOLUTION GLOBAL CLOUD DYNAMICS FOR PREDICTING ECOSYSTEM AND BIODIVERSITY DISTRIBUTIONS. PLOS BIOL 14(3): E1002415. DOI:10.1371/JOURNAL. PBIO.1002415” DATA AVAILABLE ON-LINE AT HTTP://WWW.EARTHENV.ORG/.
Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masks have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. Here, we summarize results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10-30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), were evaluated within the CMIX. Those algorithms varied in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs were evaluated against existing reference cloud mask datasets. Those datasets varied in sampling methods, geographical distribution, sample unit (points, polygons, or full image labels), and generation approach (experts annotations, machine learning, or sky images).
Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in cloud definitions used when producing the reference datasets. Average overall accuracy (across algorithms) varied 80.0±5.3% to 89.4±2.4% for Sentinel-2, and 79.8±7.1% to 97.6±0.8% for Landsat 8, depending on the reference dataset. An overall accuracy of 90% yields half the errors than an overall accuracy of 80%. The study identified algorithms that provided a balance between commission and omission errors, as well as algorithms, which are cloud conservative (high user’s accuracy) and non-cloud (clear) conservative (high producer’s accuracy). With repetitive observations like those of Sentinel-2, it seems reasonable to favor non-cloud conservative approaches, with maybe the exception of very cloudy regions where every cloud free observation is critical. When thin/semi-transparent clouds were not considered in the reference datasets algorithms’ performance generally improved: overall accuracy values increased by +1.5% to 7.4%. It should be noted though that these clouds are commonly occurring and are often present in optical imagery.
Within CMIX, we also developed recommendations for further activities, which include provision of a quantitative definition for clouds (targeting moderate spatial resolution imagery by Landsat 8 and Sentinel-2), generation of new reference datasets, and expansion of the analysis framework (for example, multi-temporal analysis and application-driven validation). Such intercomparison studies will hopefully help the community to improve the algorithms and move towards standardization of cloud masking. Given the importance of cloud masking in optical satellite imagery we encourage CEOS to continue the CMIX activities.
Fiducial Reference Measurements (FRM) are a suite of independent, fully characterised, and metrologically traceable ground measurements that follow the guidelines outlined by the Quality Assurance Framework for Earth Observation (QA4EO) of the Committee on Earth Observation Satellites (CEOS). These FRM provide the maximum Return On Investment (ROI) for Copernicus satellite missions by delivering, to users, the required confidence in data products, in the form of independent validation results and satellite measurement uncertainty estimation, over the entire duration of a mission. Within this context, the European Space Agency (ESA) has initiated a series of projects over the years targeting the validation of satellite data products (atmosphere, land, and ocean) and set up the framework, standards, and protocols for future satellite validation efforts. In 2016 – 2019, ESA funded the first phase of the FRM4SOC (Fiducial Reference Measurements for Satellite Ocean Colour) project to improve ocean colour validation through a series of proof-of-concept tasks. These included developing measurement protocols and organising laboratory and field inter-comparisons. Studies of the FRM4SOC confirm that traceability of in situ measurements to the international system of units (SI) with related uncertainty evaluation is crucial in producing reliable remote sensing data for Ocean Colour (OC) studies.
The FRM4SOC Phase 2 was launched by EUMETSAT in April 2021 and is funded by the European Commission. It builds on the achievements of the first FRM4SOC study to expand the Copernicus FRM capabilities further and ensure the adoption of FRM principles across the ocean colour community. A network of radiometric measurements with confirmed compliance to FRM quality requirements will be developed for that purpose. The activities in Phase 2 involve completing the essential laboratory and field activities started in Phase 1 and developing the tools, protocols, procedures and datasets needed to set up a network of FRM-certified OC instruments, including the calibration, characterisation and deployment history of such instruments (FRMOCnet) and demonstrate its operation. FRMOCnet will start with the most common hand-deployed or stationary field hyperspectral radiometer types (TriOS and HyperOCR), as identified in the FRM4SOC Phase 1 study. The network will be based on advanced calibration and characterisation (cal/char) guidelines to establish the metrological traceability of measurement results to the units of SI with related uncertainty evaluation for these radiometer types. The radiometers and their operators in this network will be tagged according to their “FRM status”. The measurement protocols will be enhanced; a community processor will be developed to process the radiometric data in a standardised way and to include detailed FRM uncertainty propagation and SI traceability. The developed tools, protocols and uncertainty budgets will be tested and validated in dedicated laboratory and field intercomparison exercises. A Copernicus Database for in situ Ocean Colour measurements (OCDB, available to the community at ocdb.eumetsat.int since 2019) to collect, host, and ensure long-term stewardship of FRM quality ocean colour related in situ measurement data will be under continuous upgrading and will be maintained as a part of the FRM4SOC Phase 2.
Nowadays, ocean colour satellite observations represents very well adapted tool to study and minotor water quality parameters in nearshore waters such as estuaries, bays and coastal lagoons. However, significant efforts are still required to improve the processing of satellite data over such optically complex environments, notably in terms of atmospheric, glint and a
adjanecy effects corrections, so as detection of bottom contamination, in order to accurately retrieve the water reflectance signal used to derive biogeochemical products such as water turbidity, concentrations of algal and non-algal particles.
This is one of the reasons why the international HYPERNETS network (www.hypernets.eu) is developed. Each station part of this water network includes (i) a hyperspectral ‘low-cost’ radiometer used to measure the water reflectance concomitant with satellite data acquisition, (ii) a sensor just below the air-water interface continuously recording the water turbidity and chlorophyll-a (Chla-a) fluorescence. These field autonomous measurements, regularly calibrated using water samples collected on site and anayzed in laboratory, are used to validate : (i) the atmospheric corrections applied to satellite data, then (ii) the concentrations of suspended particulate matter (SPM) and Chla (proxy of the phytoplankton biomass), i.e. used to quantify the uncertainties associated to these parameters locally derived from satellite data.
In France, a first HYPERNETS is operational in the Berre coastal lagoon since February 2021. This site, highly impacted by human activities (e.g., industries, se is caracterized by frequent and intense algal blooms of sometimes toxic species, and receive massive inputs of turbid freshwater from an electrical power plant. High (20 m) and medium (300 m) spatial resolution satellite data recorded by the Landsat8-OLI, Sentinel2-MSI and Sentinel3-OLCI are used to monitor various water quality parameters in this lagoon after local calibration and validation of satellite products using field fata provided by the HYPERNETS station and GIPREB company (https://etangdeberre.org).
Since November 2021, a second site is in operation at the mouth of the highly turbid Gironde Estuary, by the MAGEST-Verdon autonomous monitoring station, https://magest.oasu.u-bordeaux.fr) in order to calibrate and validate multi-sensor satellite data used to estimate the solid fluxes discharged from the estuary to the adjacent coastal ocean.
A detailed analysis of HYPERNETS field data recorded in these two French sites is presented together with the validation of multi-sensor satellite data corrected for atmospheric effects using up-to-dates algorithms designed for turbid coastal waters.
Current satellite sensors support water quality observations at unprecedented spatial and temporal scales. Inland and coastal waters exhibit widely ranging optical behaviours, with the concentration of key biogeochemical components ranging independently over multiple orders of magnitude. Atmospheric correction of optical imagery is still considered the most prominent source of product uncertainty. There is thus a clear requirement for globally representative reference data records in inland and coastal waters to identify and address prominent sources of uncertainty. This requirement has not been met for any Earth-observing optical satellite sensor deployed to date. There is a further need for the in situ observation capability to span multiple spatial (from meters to kilometres) and temporal (hourly to weekly) scales, to address the sensitivity of satellite observation to sub-pixel horizontal and vertical variability of water and atmospheric columns, and to relate what is observable with satellites to the information needed for effective management of optically complex water bodies.
The H2020 MONOCLE project (Multiscale Observation Networks for Optical monitoring of Coastal waters, Lakes and Estuaries) has developed a range of solutions to address current observation gaps. These range from increased autonomy in hyperspectral reflectance (WISP-M by Water Insight and So-Rad, the Solar-Tracking Radiometry Platform by PML) and atmospheric transmission (HSP-1, the Hyperspectral Pyranometer by Peak Design) to data processing workflows designed to deliver high quality observations from Unpiloted Aerial Vehicles (‘drones’) equipped with multispectral imaging payloads (MapEO water by VITO), and to low-cost devices designed to be suitable for non-experts to capture transparency (mini-Secchi disk, KduPro, KduStick and KduMod), water colour and atmospheric properties (iSPEX 2 smartphone spectropolarimeter), supported by established citizen science kits for nutrients and turbidity measurements (FreshWater Watch).
This presentation will detail the achieved technological readiness, realised cost-savings and availability of the new sensing technologies, including examples of their use in the demonstration phase of the project, where we deploy individual sensors and study synergies in the simultaneous retrieval of water and atmospheric properties.
All MONOCLE sensor and platform developments are aimed to realise FAIR data properties and policies so that their outputs can be accessed by their operators and users alike through OGC compliant data services. Most processing tools and software, and several hardware designs, are available open-source for the purposes of research and monitoring, to support expansion of observation networks and further development.
H2020-MONOCLE started in Feb 2018 and runs until July 2022. For more information and to discuss applications please visit https://monocle-h2020.eu or follow the project on Twitter at @MONOCLE_H2020.
Satellite observations of water reflectance are used to derive water quality products to match key indicators in inland and marine aquatic ecosystem monitoring. To validate water reflectance derived from optical sensors such as Sentinel-2 MSI and Sentinel-3 OLCI, and assess atmospheric correction schemes, in situ reference measurements are needed. Atmospheric correction of optical satellite imagery is particularly difficult to perform for coastal and inland waters due to the impact of adjacent land and highly variable atmospheric properties. Furthermore, these waters are optically complex which means that in situ validation needs to be performed across a range of target sites that capture a wide optical diversity. As a result, autonomous sensor systems that perform high-frequency and accurate retrieval of in situ water reflectance (or enable atmospheric characterization that aids in this task), while also covering transects across optical gradients, are highly desirable.
We present results from a low-cost solar-tracking radiometry sensor platform (So-Rad, see monocle-h2020.eu), specifically designed for in situ reflectance retrieval from moving vessels or buoys. The So-Rad adapts the azimuth viewing angle of the sky- and water-pointing radiance sensors to avoid sun glint and platform shading. Remote-sensing reflectance is derived from these observations in a fully automated processing chain, with quality control that efficiently removes suspect observations, such as those affected by variable illumination conditions, shading, or rain. Examples that illustrate the high data throughput of So-Rad are shown from recent deployments at Lake Balaton, the Western Channel, the Danube Delta, and the Tagus Estuary. We present an across-algorithm assessment of the 3C algorithm (where reflectance is derived from inversion of atmospheric and water optical models) and the Fingerprint algorithm (where reflectance is derived from minimization of atmospheric absorption features). Recommendations are then given for algorithm usage across a range of atmospheric conditions, including the clearer skies required for satellite validation, as well as water reflectance monitoring in cloudier conditions.
We further present results from the synergistic use of the So-Rad reflectance measurements with a hyperspectral pyranometer, termed HSP-1. Our assumption is that this can improve reflectance processing via direct characterization of the atmosphere; specifically, measurement of the direct-diffuse partition of downwelling irradiance. We illustrate how HSP-1 measurements can be incorporated within 3C reflectance processing, thereby testing if the previous (cloudless) model implementation experiences estimation bias due to cloudier conditions. Another compelling feature of HSP-1 is that it enables the estimation of atmospheric optical thickness. We explore how this atmospheric information can be used within satellite water reflectance validation to assess the sensitivity of different atmospheric correction schemes to atmospheric disturbances.