Agricultural field boundaries information is an enabler for numerous agricultural applications. For example, it produces relevant input for crop monitoring and yield forecasting applications; it provides a foundation in countries without cadastral data; it is applicable in precision agriculture. Additionally, an up-to-date Integrated Administration and Control System (IACS) including parcel geometric delineations with an accuracy better than 1:5000 is a prerequisite to monitor the EU’s Common Agricultural Policy (CAP) subsidy expenditures. Therefore, improving IACS by exploiting the potential of digital technologies, eventually reducing administrative burden for farmers, paying agencies and other stakeholders is needed. An automatic system for parcel boundary delineation based on satellite or aerial imagery can aid the farmers to speed up the declaration process and the paying agencies to better monitor changes in agricultural use. Automating the process of mapping these boundaries using remote sensing imagery would permit a more agile and rapid update, especially when large swaths of land are in scope. However, this is a complex task due to the limited number of available data sets, the lack of detection methodology benchmark, and the high imbalance between classes. Besides, the field boundaries are frequently irregular with lower spectral contrast among the edges, and the field area and rapid seasonal changes may further complicate the detection process.

This contribution aims to advance and facilitate development and comparison for field boundary detection. Specifically, we want to benchmark several deep learning techniques such as the UNet, DeepLabv3 and the Vision Transformer. Furthermore, as the data set is extremely unbalanced (0.027% boundary vs 99.973% interior pixels), we will also explore the combination of models, optimisers, loss functions, and accuracy metrics that yield the best results. Our initial results have shown that segmentation boundary loss and triangular cycling learning rate outperform baseline losses such as binary cross-entropy (BCE), intersection over union (IoU) or Dice because they do not penalize boundary misalignment adequately.

To create a strong benchmark, we will use an open AI-ready data set, which will be presented in another session, that consists of a single-date data set (labels + images) based on a 1-m aerial orthophoto for regional-scale analyses.

To facilitate benchmarking with other methods as well as uptake from end-users, we will publish the full comparison results, model weights and source code. Such results and open source code would highly be beneficial for national administrations to update their parcel mapping.

Agriculture and rural areas are central to the European Green Deal and the new Common Agricultural Policy (CAP) will be instrumental in delivering on this high European Green ambition by managing the transition towards a sustainable food system.

For the period 2023-27, the new CAP will be built around nine key objectives, among which the climate change action, the environmental care, the landscape and biodiversity preservation and the food and health quality protection. Each European country will have to design a national CAP strategic plan to contribute to these nine objectives, combining funding for income support, rural development and market measures. Eco-schemes are a new instrument in the CAP to be included in the national strategic plans. They need to receive at least 25% of the budget for direct payments, providing stronger incentives for climate- and environment-friendly farming practices and approaches (such as organic farming, agro-ecology, carbon farming, etc.) as well as animal welfare improvements.

This new CAP legislation requires new tools for deciding the payments allocation and assessing the policy impact. This is what is called the “Monitoring and Evaluation Approach”, which moves from the process of controlling the agricultural practices compliance with the regulations over 5% of the arable land in each country to the process of monitoring the agricultural practices over the whole countries and during the whole year. The monitoring approach has the twofold objective to accompany the farmers towards more sustainable practices and to simplify the Integrated Administration and Control System (IACS - which is the European system to administer CAP payments) and make it more efficient. Within this reform, new technologies and satellite Earth Observation (EO) is seen to take an increasing role.

In 2017, ESA launched the Sentinels for Common Agricultural Practices (Sen4CAP) project which was aimed at demonstrating how Sentinel derived information can support this new “CAP monitoring approach” through use case studies. The Sen4CAP project was conducted with the guidance of a Steering Committee made of DG-Agri, DG-JRC and DG-Grow. Czech Republic, France, Italy, Lithuania, Netherlands, Spain and Romania act as pilot countries.

The project delivered an open-source Sen4CAP system which automatically ingests and processes Sentinel-1 and Sentinel-2 (and possibly Landsat 8) time series in a seamless way to compute more than 120 markers every week for each parcel and store them in a massive database which is growing over time. The usefulness and relevance of these markers were demonstrated in the framework of three case studies selected from an in-depth consolidation of Users Requirements: the crop diversification, the monitoring of permanent grassland and of Ecological Focus Area (EFA). These three schemes are the actions that farmers have to put in place to receive the Greening Payment.

Based on the Sen4CAP markers database, four EO products were generated in 2019 and 2020: (i) crop type maps, (ii) grassland mowing detection, (iii) harvest detection and (iv) monitoring of catch crops, nitrogen-fixing crops and land-lying fallows. The products accuracy was assessed based on the subsidy applications layer provided by the Paying Agencies, farmer’s interviews and Planet data. This demonstration was run on the cloud at national scale in the seven pilot countries (more than 15 million of parcels monitored over 635.000 km²). It was conducted hand in hand with the Paying Agencies, ensuring to evaluate the markers usefulness to support the payment decision and enabling a continuous adjustment of the developed methodologies to their real needs.

The overall accuracy of the national crop type maps (focusing on more than 100 crop types by country) ranged between 74% and 97%. The recall and precision values of the grassland mowing detection ranged from 72% to 88% and from 51% to 77%, respectively. In the EFA context, the harvest of the main crop was first detected: most of the harvest events were detected within a week (values ranging from 53% to 75%) or within two weeks (values ranging from 69% to 89%). The markers used to assess the compliancy of the parcels regarding the practices’ regulations were shown to correspond very well, with a few exceptions, to the validation data.

The first version of the open-source Sen4CAP system was released in November 2019, with a dedicated hands-on training organized in January 2020 with 44 participants coming from 20 different countries. Regular webinars have been organized, with more than hundred participants each time, and a very active forum is now in place. In total, more than 40 Paying Agencies have been engaged with the project, having either used or installed the system or participated to trainings. So far, the Sen4CAP system has more than 370 downloads and the version 3 is now available, showing the very active users’ community behind. Indeed, the markers database which was demonstrated in the limited context of the selected Greening use cases, can be accessed through an API interface and therefore used in a very flexible way according to each Paying Agency and new application.

From 2021, the Sen4CAP system in general - and more specifically, its makers database - is evolving to support new use cases more related to the sustainable agriculture practices and the policy performance measurements. So far, this evolution relied on users’ feedback and now, a formal selection of new use cases coming from Paying Agencies is taking place. They might be for instance the length of the period where bare soil is present, the detection of land cover change, the presence of irrigation, the computation of an agri-environmental indicator for “High Nature Value” farmland. These new use cases will be developed and assessed with Paying Agencies during the first half of 2022, and then implemented into new open-source modules in the Sen4CAP system.

Leading the way to digital CAP monitoring; a fully operational national-scale monitoring system in Denmark

Satellite remote sensing provides extraordinary potential for timely and effective CAP monitoring. Denmark has been one of the leading countries in Europe to embrace Earth Observation (EO) data for CAP monitoring as a means to substitute costly on-the-spot checks. In this presentation we will share our experiences from the last years cooperation between DHI and the Danish Agricultural Agency that has now led to a fully operational national-scale system used by the authorities in Denmark. In 2021 we just finalized the second year of a fully operational season in Denmark monitoring of approximately 600.000 agricultural parcels for various requirements.

The system includes three core components to provide a streamlined end-to-end monitoring system (VeriCAP) 1) a powerful cloud-based back-end used for image processing and database creation, 2) an analysis toolbox used for deriving key information and statistics for individual parcels, and 3) an intuitive front-end web viewer to visualize and interpret results. Once results are received by the Danish Agricultural Agency, they are shared with the farmers in an online traffic light GIS system ensuring transparence and allowing the farmers to follow the status of the remote sensing monitoring results during the season. Via an App the farmers can send in geo-tagged photos as proof of activity at the end of the season if their parcels were flagged as possibly non-compliant during the automatic monitoring.

Our work so far has focused on use Sentinel-1, Sentinel-2, and Landsat imagery for providing near-real time information at the national scale. We present insights into using this rich time series of imagery for parcel boundary validation, crop classification, monitoring grassland activity, harvest detection, as well as catch crop monitoring. Furthermore, methods have been developed and implemented to deliver results not only on the parcels level but on the sub-parcel level to allowing the paying agency to follow activities on the sub-parcel level.

We also discuss some key technical challenges related to agricultural monitoring including training data collection and dealing with imbalanced classes. Overall, we show a practical ‘real world’ implementation of a CAP monitoring system that has been well received by public authorities and the farming community. In 2021 the solution has received two Danish awards: the Danish public sector innovation prize in September and in November at the annual Danish geospatial professionals gathering the solution won the jury prize.

Grassland mowing event detection from multi-sensor time series to evaluate agri-environmental measures – a model comparison exercise


Grassland areas are an essential part of the agricultural landscapes across Europe and provide key ecological functions and services in a multifunctional agriculture. The conservation and sustainable use of grassland areas directly contributes to policies, strategies and measures at European and national level that aim at a transition towards more sustainable agriculture within the scope of the European Green Deal.

Grassland areas are managed intensively or extensively for the provision of food, fodder or raw biomass for energy production. Due to their high relevance for the preservation of biological diversity, they are subject to agricultural policy measures as well as nature protection. Besides overall conversion embargos, those measures are also targeted towards a more extensive use of grassland, which is known to have positive impact on biodiversity. On the other hand, the management intensity in grasslands can also be an indicator for the evaluation of climate protection measures (e.g., in peatlands). For meadows, the management intensity can be described by proxies such as the mowing frequency where a higher number of cuts indicates higher intensities. Besides mowing frequency, the date of mowing (e.g., date of first cut) is also a relevant parameter for the evaluation of agri-environmental measures under the Common Agricultural Policy (CAP), e.g., for fallow land.

However, as information on grassland management is usually not reported there is a lack of knowledge on the spatial distribution of grassland use intensity. Lately, it has been shown that the availability of dense time series of remote sensing data enables to fill this gap. Various approaches have been published that make use of radar data, optical data or a combination of data from both domains. Two recently published studies of the authors underlined the overall potential of dense time series from combined Sentinel- and Landsat-data (Lobert et al. 2021; Schwieder, et al. 2021). In another recent publication, de Vroey et al. (2021) evaluated the rule-based approach of the Sen4Cap processor for the detection of mowing events for Belgium, which was developed in the Sentinels for Common Agricultural Policy framework, and makes use of Sentinel-1 and Sentinel-2 time series. All studies highlight the potential as well as limitations of the proposed data and methods for selected regions and provide accuracy measures for the evaluation of their results. However, the accuracies among the different studies are hardly comparable due to different settings and measures of the validation framework.

Against this background we conducted a model comparison study to predict mowing date and mowing frequency in grassland and fallow land from dense time series of Sentinel-data and third-party missions. We here present the concept and result of the comparison study where we estimated mowing events and dates for the federal state of Brandenburg using i) a convolutional neural network approach with time series of Sentinel-1, Sentinel-2 and Landsat 8 data (Lobert et al. 2021), ii) a rule-based algorithm using time series of Sentinel-2 and Landsat 8 data (Schwieder et al. 2021), iii) a rule-based algorithm based on Sentinel-1, 2 and Landsat 8 data (Schwieder et al. 2019) and iv) the Sen4Cap processor (de Vroey et al. 2021). The validation concept builds on parcel boundaries as an input, for which we used the Land Parcel Identification System (LPIS), which enabled to differentiate fallow land, temporally used and permanent grassland. However, as these data are not always widely available and do usually not allow to differentiate management variations within grassland parcels (e.g., partly mown and/or pasture use), we additionally derived parcel boundaries for Brandenburg based on a processor for the fully automatic parcel delineation from S1/S2 data as proposed by Tetteh et al. 2021, and compared the output from both data sets, including an independent validation from field management information and PlanetScope data.

We will present the results of the comparison exercise as well as a feasibility use case, in which we use the estimated grassland management information for the assessment of measures in nature protection (e.g., FFH areas) and agricultural policy (e.g., ecoschemes for the next CAP period). The output of this feasibility study will be summarized as an outlook for the future application of Sentinel-based indicators of grassland management within the monitoring and evaluation framework of the CAP.


References:

De Vroey, M.D., Radoux, J., Zavagli, M., Vendictis, L.D., Heymans, D., Bontemps, S., & Defourny, P. (2021). Performance Assessment of the Sen4CAP Mowing Detection Algorithm on a Large Reference Data Set of Managed Grasslands. In, 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS (pp. 743-746).

Lobert, F., Holtgrave, A.-K., Schwieder, M., Pause, M., Vogt, J., Gocht, A., & Erasmi, S. (2021). Mowing event detection in permanent grasslands: Systematic evaluation of input features from Sentinel-1, Sentinel-2, and Landsat 8 time series. Remote Sensing of Environment, 267, 112751.

Schwieder, M., Frantz, D., Loibl, D., Griffiths, P., Pfoch, K., Lilienthal, H., Hostert, P. (2019). Grassland Use Intensity Metrics from Sentinel-1 and Sentinel-2 Data. In, ESA Living Planet Symposium Milan, Italy.

Schwieder, M., Wesemeyer, M., Frantz, D., Pfoch, K., Erasmi, S., Pickert, J., Nendel, C., & Hostert, P. (2021). Mapping grassland mowing events across Germany based on combined Sentinel-2 and Landsat 8 time series. Remote Sensing of Environment, 112795. https://doi.org/10.1016/J.RSE.2021.112795

Tetteh, G.O., Gocht, A., Erasmi, S., Schwieder, M., & Conrad, C. (2021). Evaluation of Sentinel-1 and Sentinel-2 Feature Sets for Delineating Agricultural Fields in Heterogeneous Landscapes. IEEE Access, 9, 116702-116719.

Sentinel-based agricultural subsidy monitoring is applied by an increasing number of national paying agencies. The aim of our study is to test the application of Sentinel data analysis for preliminary checking of claims and monitoring SAPS (Single Area Payment Scheme) and greening subsidies at national scale in Hungary. Preliminary checks involve identifying false claims by crop type and parcel boundary before the final deadline of claim correction (early June). SAPS eligibility is checked based on the presence of a cultivated crop, the absence of weeds, mowing and avoidance of overgrazing. The greening criteria checked include crop diversification at farm level, maintaining sensitive grasslands, fallow land, nitrogen-fixing crops and catch crops. The pilot project was checked against the requirement of reducing the necessary on-the-spot checks from the currently required 5% of the parcels.
Sentinel-2 images were pre-processed in the open-source FORCE framework to the level of bottom of atmosphere (Level-2A) images. The first step was to develop a random forest based crop classification scheme at the level of individual parcels. For this, we used Red Edge 1, Red Edge 2, Shortwave Infrared 1 and Shortwave Infrared 2 bands, calculated NDVI, Bare Soil Index, Enhanced Vegetation Index, Structure Insensitive Pigment Index and developed our own Yellow Crop Index. Parcel boundaries were extracted from all claims and the mean values of these indices extracted within each parcel and for each image date. These time series of the individual parcels were categorized with a random forest classifier, and probabilities of individual classes were output for all parcels. This allowed calculation of categorization certainty using the probability surplus index, enabling the paying agency to inform those farmers whose claims were non-compliant with the highest certainty. For the preliminary checks, a system of 9 crop classes was developed and gradually upgraded to a scheme of 22 classes for the monitoring pilot. Parcel boundaries were identified by raster analysis using directional kernel edge detection. Cultivation events such as mowing, overgrazing, sowing and harvesting were identified from NDVI timelines using expert-generated rulesets.
For the initial check, Sentinel-2 based crop classification of the time series between January and the end of May provided an overall accuracy of 76%. For the time series of the whole year, the 22 classes reached an accuracy of 88%, with the most widespread crops (winter cereals, maize, sunflower, rapeseed , grapes) reaching accuracies above 90% (89% for grasslands). For both cases, ranking the parcels based on probability surplus enabled selection of the cases where erroneous categorization was least probable. Detection of cultivation events was possible with an overall accuracy of 86%, as checked against visually interpreted parcels. These processing steps were combined in a decision tree ruleset for checking the individual subsidy criteria.
Basic cultivation was checked based on the crop classification together with fuzzy probability of the "weed" class. Additionally, the presence of at least one mowing event and the absence of overgrazing was checked for grasslands. For crop diversification, the only parameter available for remote sensing was whether the crop identified from the imagery matched the claimed crop. Maintenance of sensitive grasslands was also checked using the classification results. Identifying fallow land ecological focus areas required a combination identifying set-aside parcels with classification and checking that no mowing has taken place. For nitrogen-fixing crops, rules include a fixed period of cultivation which was checked using the NDVI time series together with the crop class. Finally, for catch crops the same cultivation period check was performed. For this set of monitoring criteria, the number of uncertain cases identified was slightly below 5%, with most of these cases contributed by uncertain classification of rare crops or fallow land.
All in all, the monitoring pilot demonstrated how Sentinel-2 can be reliably used for monitoring these regulations. Fuzzy classification and the evaluation of individual probabilities proved crucial for separating erroneous claims from likely image misclassifications. However, there was one major limitation: the size and shape of the monitored parcels was constrained by the spatial resolution: parcels of less than 40 meters width were considered too narrow. Hungarian agriculture is characterized by relatively small parcels, therefore close to 40% of the parcels (by count) were below this threshold in 2020. By merging adjacent small parcels that were covered by similar crops, 25% of the parcels were successfully added back to the database and analysed, but the rest had to be excluded.
Additional improvements for the 2021 season included integration of Sentinel-1 SAR polarimetry data into the crop classification, deep learning-based parcel boundary detection, and a more inclusive parcel size rule. By adding Sentinel-1, the accuracy of the 2020 preliminary check (which had only 9 classes) was reached for 22 classes in 2021. Deep learning based parcel edge delineation provided a basis for quantitative evaluation of boundary cardinality and correctness. Reducing the parcel size rule to a minimum of 15 pixel centroids within the parcel outline after a negative buffer of 5 meters reduced the number of excluded parcels to only 3% of the total count.
The responses of farmers contacted during preliminary checks were overwhelmingly positive: many of the claims were successfully modified, avoiding penalty administration or unjustified payment. The monitoring pilot also allowed detection and follow-up of many cases of non-compliance that would otherwise have been undetected. In the future, deep learning based classification and the inclusion of SAR polarimetry to cultivation detection is expected to further improve accuracy, and a smartphone app will also be introduced in order to facilitate communication with farmers.

Mountain agriculture is a cornerstone of alpine identity having tremendous impacts not only on socio-economical systems but also on climate change adaptation and mitigation, biodiversity, and risk reduction. It is also because of these positive externalities that mountain agriculture is supported by the Common Agricultural Policy (CAP) and its national and regional implementations. In the last years, the unprecedented opportunities represented by EO-based approaches led to the development of many initiatives aiming at monitoring agricultural practices from space. The most relevant one is the Sen4CAP project that provides validated algorithms, products, and workflows tailored to the modernization and simplification of the CAP and to the improvement of the Integrated Administration and Control System (IACS). In this presentation, we will discuss the first results from the application of Sen4CAP in an alpine region (Aosta Valley, approximately 3000 Km2, Western Alps). We will focus in particular on the assessment of the accuracy of the mowing detection product by comparing the results with field observations (i.e. phenocam network). The seasonal course of canopy greenness computed from phenocam data can indeed provide relevant opportunities to automatically detect mowing and grazing practices and thus analyze the accuracy of Sen4CAP results: number of use and date of use. Secondly, we compare Sen4CAP outputs with the results of a methodology that we developed based only on Sentinel-2 data for the same purposes. Our approach relies on the fitting of seasonal ndvi trajectories and on the unsupervised detection of mowing and grazing events based on the differences between actual trajectories and theoretically unmanaged ones. One of the main differences with the Sen4CAP approach is that the outputs of our methodology are provided either as raster and as vector layers allowing a more sophisticated evaluation of accuracy and spatial representativity of the results. We will discuss the pros and cons of the two approaches with some suggestions of possible further developments.