Information and Communication Technologies (ICT) and Earth Observation (EO) are increasingly evolving as a result of the growing data availability, technological sophistication and integration capacity, while in parallel benefit from the increasing adoption of innovation by the markets. Whereas previously an EO application would be specifically scoped and developed on-demand by specialized EO information providers, with low automatization and significant delivery times, today there is a shift to the provision of an EO capacity embedded directly within the workflows of target users or made available in easy-accessible platforms, to be serviced rapidly and automatically, with integration of ICT analytics and other added-value information.

This is the case of the ESA’s EOLAW project that was developed by a consortium led by GMV, which aimed at demonstrating the benefits of the combined use of EO, ICT analytics and non-EO data, for the provision of services through a web-platform, in support of the law-enforcement sector.

For this effect, the consortium developed a series of use cases in several relevant law enforcement domains that covered different topics. These domains were:

• Environmental crimes – Illegal Timber Extraction and Trafficking. Illegal logging is a very relevant topic as it is very difficult to estimate with precision how much wood is moved in this market. The UN Environment Programme reports that illegal logging accounts for between 15% and 30% of global timber trade, and rises to 50% to 90% of the trade from tropical countries. The EU’s figures are slightly higher. They estimate that between 20% and 40% of the global timber trade comes from illegal sources. Illegal logging activities often take place in remote and isolated areas, making it difficult for traditional law enforcement approaches to cope with this type of problem.
For this domain, three services have been developed and implemented in strict articulation with engaged users, in order to allow the provision of de facto useful added value data that could help them on their daily activities. These services were:
o Detection of forest change (clear cuts) – that aimed at identifying decreases of vegetation cover in satellite images as a result of illegal logging activities. This was done analysing multispectral data from bi-temporal consecutive pairs of EO images;
o Detection of logging support infrastructures – aiming to detect camp areas and related structures, machinery and vehicles that could potentially be used for trafficking of timber activities; and
o Detection of routes for movement of timber – aiming to detect routes that could potentially be used for trafficking of timber.

• Environmental crimes - IUU Fishing Services. The depletion of marine resources due to overfishing and illegal, unreported, and unregulated fishing and also human activity (pollution) is an ever-growing problem.
In this sense, the combined usage of Earth Observation (SAR and Optical), data fusion (AIS, VMS) and human analytics is a powerful solution for the sustainable management of marine resources.
In the scope of EO Law, the concept of operations (CONOPS) for satellite maritime services took advantage of the synergies between cooperative tracking systems and non-cooperative observations by radar satellite to deliver vessel detection surveillance services in order to evaluate vessel activities and detect illegal activities at sea.
Within the project, three services were provided to the IUU Fishing end-users that were involved in the activities:
o Area Surveillance – using analysis of historical AIS (Automatic Identification System) data in order to identify areas where it was likely to occur IUU fishing activities and subsequent active monitoring of those areas;
o Fast Imagery Analysis Delivery – designed to provide the users with short notice analysis of suspicious illegal activities over given areas of interest; and
o Large-scale systematic Analysis – aiming to provide a large-scale analysis of IUU fishing activities over very large maritime areas.

• Crimes against humanity – anomaly detection and correlation of evidence. As crimes against humanity are still occurring on a regular basis, they are in contrary often not “visible” to the wider public due to the fact that hostile actions typically take place in remote and difficult to reach areas, where accessibility is limited either due to e.g. armed conflicts or governmental restrictions. Simultaneously, as the amount of in-sight information is rapidly increasing during the last years based on e.g. social media, the confidence level of the information overload decreases making it hard to distinguish between “wrong” or “right”.
In this sense EO-data and EO-derived information can be seen as an independent information source suitable to verify on-sight information and to gain a more sound evidence of violent actions taking place.
Within EO Law, for the Crimes against Humanity domain, the goal was to define services as generic as possible and not solely stakeholder-tailored in order to gain a cross-stakeholder applicability, even though that the thematic input for the single services was stakeholder-driven.
The following services were developed and implemented:
o Mass graves site suitability modelling – providing a predictive multi-criteria approach to narrow down potential locations of mass grave sites based on expert knowledge and input data information layer;
o Fire detection in settlements – aiming to detect burned areas in settlements and surroundings comparing satellite images of two timestamps and suitable indices for fire activity detection; and
o Settlements developments and change detection – focused on the detection of urban sprawl and dynamics applying AI methods to gain multi-temporal settlement layers and change analysis.

• Terrorism and organized crime - training camp characterization, anomaly detection and activity monitoring. Societies today constantly face terrorist and organize crime actions, which require new methods for modelling and analysis, inherited from various sectors and technological domains. In fact, Law enforcement organizations, analysts and field operators fighting terrorism and organized crime need front-line integrated technologies to support their cooperative work.
Typically, terrorists engage in a variety of non-terrorist criminal conduct prior to the commission of any terrorist act and these behaviours ultimately culminate in acts of terrorism.
By examining these preparatory behaviours, routinized patterns of activity potentially can be identified and linked to activities related to training facilities, movement of materials and assets and anomalous activities or assets/targets of interest.
It was considering these aspects that three services, with their corresponding data analytics processes have been implemented in the scope of EO Law:
o Data analytics for the comprehensive and contextual imagery intelligence analysis combining EO data and media sources (OSINT) - aiming to combine information derived from Media Mining Analysis with abnormal activity monitoring using EO data;
o Hotspot detection layer with potential training camps service – following the previous service, the aim was to provide the users with potential locations of training camps related to terrorism and organized crime; and
o Hotspot detection layer with potential abnormal activities related to terrorism and organized crime service – also following the first service, the aim was to detect abnormal changes that could be associated with movement of materials, persons, and / or assets related to terrorism and organized crime.

In the end of the project, the users and stakeholders were provided with valuable information that underlines the benefits of the combined use of EO, ICT analytics and non-EO data in alternative to business as usual approaches.

All the services were developed through a virtual web platform that was used for data processing and information dissemination. This platform is built with a set of software components integrated together and deployed on a data-rich online cloud infrastructure. This way the required EO data can be accessed locally without the need to transfer it from external sources and the provision of outputs is made via a dedicated online OGC compliant services catalogue.

Since the launch of the first satellite (Sentinel-1A, April 1st 2014) The Copernicus Program has generated and provided an unprecedented amount of free and open data to monitor the Earth system as whole. Among the various satellites composing the space component, Sentinel-2 supports a wide range of applications, from agriculture to security passing through natural disasters, providing multispectral optical imagery at 10m resolution at global scale every five days. Unfortunately for such domains as security or for humanitarian response, the spatial resolution proposed remains too coarse for precise analysis and commercial very high resolution (VHR) satellites still have a financial barrier for many users. In the context of the European Space Agency (ESA) project «Earth Observation for Yemen» (EO4Yemen), artificial intelligence solutions were proposed to increase the operability Sentinel-2 and to support the humanitarian aids of specific areas over the country. Indeed the region is facing a catastrophic situation since 2015 because of an ongoing civil war, some areas are targets of airstrikes and military conflicts forcing the population to constantly move from areas to find safer places. In this context of war, satellites provide data over time to monitor the situation. A workflow was proposed to perform change detection over the country, in one way to identify the impact of war over cities but also to monitor the displacement of the refugees by using a proxy as the apparition of new settlements. This work aims at proposing innovations in remote sensing and deep learning fields but also to provide as much as possible precise information over the country by using Copernicus and open source datasets.
The full data analysis workflows foresees three steps:
1. Super Resolution : With the rise of AI applications, enhancement of satellite images is now feasible «artificially», in this context we proposed a solution to perform a super resolution over several Sentinel-2 MSIL1C bands (near-infrared, red, green and blue). For this purpose Worldview-2 images (2m) were used to create a reference dataset and increase the spatial resolution of the Copernicus sensor from 10m to 5m (Figure 1). The enhancement permits to answer multiple questions, indeed with a better geometrical definition of the buildings over the created images, smaller objects can be identified and be used as a proxy for population displacements.
2. Building classification : The generated super resolution images are then integrated into an AI classification to automatically identified buildings over areas of interest. The model was trained with open source building dataset over Africa regions with similar landscape than Yemen to be able to transpose it to the country we are interested in. This step consists in providing a preliminary analysis of an area by identifying automatically all the buildings on a Sentinel-2 image with a resolution of 5m (Figure 2). Small settlements can now be identified and with a temporal analysis evolution of the urban area is highlighting.
3. Object based change detection : Finally the preliminary analysis made over the buildings is refined by a change detection algorithm, focusing directly on objects. This algorithm performed a detection of changes as apparition of new settlements (Figure 3), but also disappearance of buildings identified in the past. This information can be used as proxy to monitor population displacement over several regions but also the impact of the conflict on the urban tissue. The method proposed is based on parameters directly linked to the classification and can be adapted with the type of landscape seen on the images (high urban density / isolated buildings ...).
The entire workflow, is in on one hand improving the use of the Open Source dataset provided through the Copernicus program, indeed this work is integrated into an innovation process with the use of artificial intelligence solution for earth observation analysis. On another hand it gives a meaningful support for humanitarian aids over specific areas where it is difficult to operate directly on the ground.

Satellite SAR systems can provide MTI (Moving Target Indication) capabilities that currently are not fully exploited in the Maritime Awareness Domain (MAD). Such capabilities are of great potential interest for surveillance in both military and civil protection applications. Due to the SAR imaging mechanism, moving targets within SAR images appear defocused. Therefore, the possibility to correctly focus the target and to estimate its moving parameters represents a clear improvement for target detectability, tracking and recognition.
MTI capabilities are usually provided increasing the system complexity including multiple receiving (rx) channels. Nevertheless, also single receiving channel systems (such as the Cosmo-SkyMed Constellation) can theoretically provide reasonable performance if proper signal processing techniques are used.
In the framework of the ESA project “Maritime Awareness Pre-Operational Demonstrations”, ruled by Contract 000134655/21/I-NB, different algorithms based on Inverse SAR (ISAR) principles are considered to refocus maritime moving targets and to estimate their motion parameters. The experimental validation is based on SLC SAR data of COSMO-SkyMed Constellation acquired in Stripmap HIMAGE and Spotlight-2 modes.
Particularly, the correction of the defocusing effects on the target due to its movement, which is unknown, is the main objective of ISAR techniques. In this work, different techniques, SAR and ISAR, are experimented for maritime targets in different operational situations. The compensation of translation motion components of the vessels is performed through the application of fast SAR refocusing techniques based on Chirp Scaling (CS) and/or on Phase Gradient Algorithm (PGA) and/or aiming to optimize contrast metrics such as image entropy, contrast and spectral power density. The residual and most complex movements (non-linear motions), such as rotations induced by the sea waves or due to vessel manoeuvring, are taken into account including in the processing automatic sub-aperture analysis and cross-range scaling algorithms. Such hybrid SAR/ISAR techniques allow also the retrieval of an estimation of the target movement parameters.
The quality of the output products clearly depends on the specific contingent situations such as the sea state, the complexity of the vessel manoeuvres, mutual geometry between the SAR satellite configuration and the target cruise. For these reasons, the algorithms are experimented and tested on different scenarios, built on a collection of COSMO-SkyMed data, in order to encompass different operational conditions and evaluate applied ISAR techniques performances.

The global and diverse nature of threats to our security composes a complex environment that is extremely difficult to understand and manage. Border control, search operations and planning of preventive actions to ensure public order at national or even regional levels are key areas for the use of earth observation (EO) technologies. High-resolution satellite images, which provide information over large and difficult-to-reach areas, have been a source of information for critical issues of national security and provide a unique perspective in understanding this environment. In addition, the increasing availability of “big data” from online sources offers new perspectives for capturing information related with security issues. In fact, security agencies can integrate large volumes of Open-Source INTelligence (OSINT) data from social media sources with EO IMage INTelligence (IMINT) in order to facilitate their related operations. Planetek Hellas has the capability to leverage these data and contribute to the production of high-level insightful information, considering the successful implementation of a number of security related case studies and our interaction with various national and internationals Law Enforcement agencies. Specifically, our case study implementations concern the investigation of a wide range of security applications. Our IMINT and OSINT integration tools mainly accomplished to monitor border crossing activities, in different areas of interest. These tools have also contributed to the successful analysis of pre and post event activities regarding illegal actions. The aforementioned studies include several methods to achieve abnormal activity detection, such as localization of activities with media mining techniques (word cloud, hashtag cloud and relations graph), satellite imagery photointerpretation and advanced processing (e.g., change detection in time series of SAR and Optical images, object detection and multi temporal coherence analysis), and geospatial analysis (e.g., spacetime cube for emerging hotspot analysis and least cost path analysis). An important aspect of our inventory is the presentation of the core results in the form of tables, graphs and maps, included in well-structured and comprehensive reports. Combining quality information from OSINT and IMINT, we delineate with the best representation the results in a broad-spectrum of visualizations for a wide range of end users.
Besides the increasing volume of remote sensing data, due to the dynamic nature of security related issues, the availability of near real-time satellite data is very important in order to accurately provide enriched information. However, due to satellite tasking limitations data acquisition plan cannot be operational over certain regions. Furthermore, big OSINT and IMINT data require high computational and storage resources, which can increase the operation cost, significantly.
Investigating the relationships between information from EO data and OSINT analysis can considerably assist to address the security challenges. Developing new methods based on big data from online sources and earth observation can provide a fresh perspective at this manner. Artificial intelligence (AI) and in particular Deep Learning (DL) architectures have contributed to a breakthrough in remote sensing imagery processing. Developing and employing relevant models could significantly enhance our proposed workflows in future case studies, in order to make the most out of the available data. Each particular approach should be considered as an important and valuable tool for supporting and corroborating current end-user practices.