The National Aeronautics and Space Administration (NASA) and U.S. Geological Survey (USGS) jointly led Landsat program is the world’s longest-running Earth-observing satellite program. In July 2022, the Landsat program will reach its 50th anniversary. From the launch of Landsat 1 in 1972, through the launch of Landsat 9 in 2021, and into the future, the Landsat archive represents an unbroken record of Earth observations. Through continual calibration, product improvements, and collection-based reprocessing, the Landsat archive is kept consistent, enabling accurate time-series analyses, while usability is improved by providing higher level products in more analysis ready frameworks. With the release of Landsat Collection 2 in December 2020, Level 2 global surface reflectance and surface temperature products have been available for the user community. To ensure the accuracy of these higher-level products, efforts have increased to collect in-situ validation reflectance and temperature measurements over a large range of surface and atmospheric types at the same time as Landsat satellite overpasses. Multiple geometric and radiometric improvements were also incorporated into the products making them more accurate and precise and consistent with Copernicus Sentinel 2 products by adopting the Global Reference Image (GRI), derived by the European Space Agency (ESA) from Sentinel 2 data, as the basis for georeferencing the entire Landsat archive. The radiometric basis of the Landsat archive is the National Institute of Standards and Technology (NIST) tracible Landsat 8, and by virtue of the continuity of the Landsat program, cross-calibration from Landsat 8 to 7 to 5 to 4 to 3 to 2 to 1 is possible using overlapping time frames over pseudo-invariant calibration sites (PICS). After its recent launch, Landsat 9 data was added to the USGS’s archives and, using coincident imagery with Landsat 8 and known calibration sites, was placed on the same radiometric scale and geometric reference as the previous missions, continuing the Landsat legacy.

The Earth Observing System Data and Information System (EOSDIS) was developed in the early 1990s and was considered operational in 1994 when the Version O of the basic Information Management System was made publicly available. Many of the system architectural decisions made in the 1990s have continued to this day, one of the reasons for the systems longevity. Policies that were developed, such as the Earth Science Open Data Policy, have helped inform all subsequent changes. Many years later, the EOSDIS continues to thrive having evolved and adapted new technologies along the way. Today EOSDIS is used to archive and disseminate many other NASA Earth Science mission beyond the designated on-orbit EOS missions. These missions include instruments on the International Space Station, airborne missions, many field campaigns, and even international missions.
The EOSDIS involves partnerships among NASA Centers, other US agencies and academia that process and disseminate remote sensing and in situ Earth science data. One of the components of EOSDIS is the set of 12 discipline-based Distributed Active Archive Centers (DAACs). Because of their active role in NASA mission science and with the science community, the DAACs perform many tasks beyond basic data stewardship, representing a distinct departure from typical data archives. Managed by NASA’s Earth Science Data and Information System (ESDIS) Project at the Goddard Space Flight Center and geographically distributed across the US, these DAACs, serve nearly 4 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as the Science Investigator-led Processing systems (SIPS), common metadata and metrics management systems, specialized network systems, standards management, and centralized support discovery and access capabilities.
NASA’s Earth Science Data System (ESDS) Program recognizes that new strategies are required in order to meet the data needs of the research community in this modern age. ESDS also strives to encourage the use of Earth observation data by a broader user community. The Interagency Implementation and Advanced Concepts Team (IMPACT) program addresses these data needs through its focus on improving data acquisition, management, analysis, and exchange.
By 2025, several new high-data-volume missions will be launched, requiring the EOSDIS archive to grow from the nearly 60PBs today to over 150PBs over the next several years. With the impending arrival of these new missions, the need to effectively archive and process significantly larger data volumes will require new data management technologies and architectures. This was the catalyst for NASA to begin planning for the EOSDIS’s next evolutionary step.

To meet these needs, NASA has adopted a strategic vision to develop and operate multiple components of the EOSDIS in a commercial cloud environment (Earthdata Cloud). Moving Earth science data and computations into the cloud will enable new science and application of large-scale analytics. The Earthdata Cloud will create opportunities for innovation around new services, such as sequencing data to support machine learning and artificial intelligence. The Earthdata Cloud will also improve the efficiency of data systems operations, increase user autonomy, maximize flexibility, and offer shared services and controls.
When data scientists are able to effectively combine two or more observational data streams into a single data set, the value increases even more. In order to address a scientific need for more frequent global surface reflectance observations for land monitoring applications, observations from instruments on multiple platforms can be used to improve temporal coverage. To combine data from similar instruments and platforms, the data must be harmonized, or made more consistent. To provide the frequent observations needed, the Harmonized Landsat 8 Sentinel-2 (HLS) data products were developed by NASA. The IMPACT team has expanded the implementation of the HLS algorithm from the initial 120 sites to a global scale, generating data products in near real-time along with a full archive of the HLS data products. Historically, the production of planetary-scale Earth observation products has been managed by high performance compute clusters and on-premise data production systems. However, products such as HLS can require a massive scaling of resources to support archival processing for historical data or re-processing due to algorithm updates. The team migrated HLS processing to Amazon Web Services (AWS) to streamline development, facilitate dynamic scaling for archival processing, and optimize data transfer to NASA’s Land Processes Distributed Active Archive Center (LP DAAC) cloud distribution system
NASA is making a long-term commitment to building an inclusive open science community over the next decade. EOSDIS has adopted NASA’s open-source science commitment to the open sharing of software, data, and knowledge (algorithms, papers, documents, ancillary information) as early as possible in the scientific process.

In this presentation, we will discuss the current status and the lessons learned encountered during the evolution of EOSDIS including the migration to a commercial cloud architecture. We will also discuss our adoption of the open-source science principles as well as the tools and techniques to make Earth science data readily available and more accessible to our user community.

Satellite missions during the 1960s and 1970s generated significant amounts of Earth observation data. The majority of these data are not currently exploited in climate reanalysis, despite their potential value for constraining the evolution of global weather. This paper highlights work from a C3S satellite data rescue project, aiming to recover, assess, improve and prepare a selection of early satellite data records that will help to improve the ECMWF’s next centennial climate reanalysis, ERA6.

The early datasets addressed cover mostly the period 1964 to 1979. They were measured by a range of infrared radiometers, sounders and imagers, flown mainly on the Nimbus series of satellites. These sensors include the single-channel High Resolution Infrared Radiometer (HRIR) with data available during parts of 1964 (Nimbus 1), 1966 (Nimbus 2) and 1969-70 (Nimbus 3), the five-channel Medium Resolution Infrared Radiometer (MRIR) on Nimbus 2 and 3 (1966, 1969-70) and the two-channel Temperature-Humidity Infrared Radiometer (THIR) on Nimbus 4-7 (1970-1985). They also include four other sensors with sounding capabilities: the Satellite Infrared Spectrometer (SIRS) on Nimbus 3 and 4 (1969-71), the Infrared Interferometer Spectrometer (IRIS) on Nimbus 4 (1970-71), the Pressure Modulated Radiometer (PMR) on Nimbus 6 (1975-78) and the Vertical Temperature Profile Radiometer (VTPR) on the NOAA 2-5 satellites (1972-79).

We present examples of improvements made to data quality such as revised timing and geolocation, reducing errors typical of these heritage datasets. For example, for VTPR, geolocation errors of up to 400 km can be observed in some instances. We have performed extensive quality flagging to enhance the overall quality and usability of each dataset, based on detailed expert investigation of data relationships. We present comparisons of the observations to ERA5-based simulations (O-A), which also provides a means of assessing the data quality and characterising biases in the data, as well as information about the quality of the reanalysis. From O-A analysis for three of the sensors (IRIS, SIRS and VTPR), we find evidence for a potential warm bias in ERA5 in the upper stratosphere in south polar winter. This is an example where the assimilation of these early satellite observations can result in a direct improvement in climate reanalysis.

Where possible, the analysis of uncertainties in the satellite datasets is carried out via a metrological approach. This includes obtaining estimates of the random component of the uncertainty (noise) as well as estimating systematic uncertainties arising from instrumental issues. We will also discuss methods for reducing systematic errors in the data such as corrections to the geolocation. The final set of data will then be used to help build a more robust picture of weather and climate during the 1960s and 1970s.

MIPAS is a Fourier Transform spectrometer that measured the atmospheric limb emission spectra in the middle infrared on board the ENVISAT satellite. These measurements allowed the global monitoring of the three-dimensional (latitude, longitude and altitude) distribution of temperature and of the concentration of many species, during both day and night, for 10 years, from July 2002 to April 2012.
MIPAS measurements allowed to study the atmosphere from the upper troposphere to the stratosphere and above, up to the thermosphere.
The interest in these measurements goes beyond the end of the mission, as they can be used in long time series of data to determine changes in atmospheric composition and in our planet’s climate. Furthermore, if the Changing-Atmosphere Infra-Red Tomography Explorer (CAIRT) mission, one of four candidates for Earth Explorer 11, will be selected, MIPAS data will constitute a benchmark for these measurements. CAIRT exploits indeed the heritage of MIPAS on ENVISAT, but allows to measure the composition of the atmosphere with unprecedented three-dimensional resolution being the first imaging Fourier Transform spectrometer sounding the limb of the atmosphere from space.
For the last reanalysis of the whole MIPAS mission, a significant effort was made by the MIPAS Quality Working Group, supported by ESA, to improve both L1 [1] and L2 processors, as well as spectroscopy and Level 2 Initial Guess profiles [2], with the objectives of obtaining L2 products with increased accuracy, better temporal stability, and a larger number of retrieved species. The main improvements of L1 processor were related to the radiometric calibration and pointing. With these new processors a MIPAS full mission reprocessing has been recently performed ([1] and [3]). The quality of this final operational data set has been assessed with comprehensive validation studies including comparisons to ground-based in-situ and balloon-borne measurements. The dataset containing the new version 8 of both L1 and L2 products and covering the entire MIPAS operational lifetime period (2002-2012) is available at ESA Earth Online web site.
This paper will focus on the lessons learnt, on the quality of the reprocessed data, on the remaining problems, and on further improvements that could improve the quality of both MIPAS L1 and L2 datasets.

[1] Kleinert et al. Kleinert, A., Birk, M., Perron, G., and Wagner, G.: Level 1b error budget for MIPAS on ENVISAT, Atmos. Meas. Tech., 11, 5657–5672,https://doi.org/10.5194/amt-11-5657-2018, 2018
[2] Raspollini, P., Arnone, E., Barbara, F., Bianchini, M., Carli, B., Ceccherini, S., Chipperfield, M. P., Dehn, A., Della Fera, S., Dinelli, B. M., Dudhia, A., Flaud, J.-M., Gai, M., Kiefer, M., López-Puertas, M., Moore, D. P., Piro, A., Remedios, J. J., Ridolfi, M., Sembhi, H., Sgheri, L., and Zoppetti, N.: Level 2 processor and auxiliary data for ESA Version 8 final full mission analysis of MIPAS measurements on ENVISAT, Atmos. Meas. Tech. Discuss. [preprint], https://doi.org/10.5194/amt-2021-235, in review, 2021.
[3] Dinelli, B. M., Raspollini, P., Gai, M., Sgheri, L., Ridolfi, M., Ceccherini, S., Barbara, F., Zoppetti, N., Castelli, E., Papandrea, E., Pettinari, P., Dehn, A., Dudhia, A., Kiefer, M., Piro, A., Flaud, J.-M., Lopez-Puertas, M., Moore, D., Remedios, J., and Bianchini, M.: The ESA MIPAS/ENVISAT Level2-v8 dataset: 10 years of measurements retrieved with ORM v8.22, Atmos. Meas. Tech. Discuss. [preprint], https://doi.org/10.5194/amt-2021-215, accepted, 2021.

In recent decades, passive microwave remote sensing at lower frequencies (1-10 GHz) has become a primary means to routinely monitor soil moisture on a global scale. Despite the success of a number of L- and C/X-band radiometers independently developed and launched by various government agencies over the last two decades, there has not been a concerted effort to leverage the combined brightness temperature (TB) observations from these instruments to derive an integrated soil moisture data record within a consistent geophysical inversion framework. The availability of such a consistent data record would provide critical insights into the dynamics of surface hydrological processes, including anomaly detection, interannual variability, and monitoring of the onset and evolution of long-term spatial and temporal variability due to natural or anthropogenic changes in land surface conditions.

Recent advances in TB intercalibration on current and historical satellites have resulted in the availability of consistent TB observations that extend from years to decades. For passive microwave remote sensing of soil moisture, satellite intercalibration undertaken by the Global Precipitation Measurement (GPM) mission [1-2] has resulted in a decadal repository of intercalibrated TB observations at X-band (10.7 GHz) frequencies from GMI (2014-present), AMSR2 (2012-present), WindSat (2003-present), TMI (1997-2015) and AMSR-E (2002-2011). Likewise, recent studies on relative calibration by SMOS (2009-present) and SMAP (2015-present) teams have also enabled the production of a similar repository of intercalibrated TB observations for soil moisture estimation at L-band (1.41 GHz) frequencies [3]. When used as inputs to a common geophysical inversion model, these TB observations can be used for soil moisture estimation. Because consistency has been reinforced at the level of TB observations among satellites, the resulting record of soil moisture retrieval is expected to exhibit the same internal consistency. Together, therefore, these TB repositories provide the foundation for the development of current and historical consistent soil moisture data records with more frequent and wider coverage than any single satellite can achieve alone.

In this presentation, we will describe a NASA-funded initiative [4] (MEaSUREs: Making Earth System Data Records for use in Research Environments) to create a consistent soil moisture decadal data record from multiple satellites for terrestrial hydrological applications. Preliminary retrieval performance results on an integrated long-term L-band SMOS/SMAP soil moisture record will be discussed in this presentation.