Data Warehouse Requirements - Version 2.0
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
EUROPEAN COMMISSION
ENTERPRISE AND INDUSTRY DIRECTORATE-GENERAL
Aerospace, Maritime, Security and Defence Industries
Copernicus : Infrastructures
Data Warehouse Requirements – Version 2.0
Copernicus Data Access
Specifications of the space-based Earth Observation needs for
the period 2014-2020
European Commission, B-1049 Brussels - Belgium. Telephone: (32-2) 299 11 11.
APPROVAL
Title:
Data Warehouse Requirements – Version 2.0
Copernicus Data Access
Specifications of the space-based Earth Observation needs for the period 2014-2020
Issue 2 Revision number: 0
Author: D. Quintart (ENTR G/3) Date: 5/3/2014
Approved by: R. Schulte-Braucks (ENTR G/3) Date: 5/3/2014
ii
CHANGE LOG
Reason for change Issue Revision Date
iii
CHANGE RECORD
Issue Revision
Reason for change Date Pages Paragraph
iv
TABLE OF CONTENTS
DATA WAREHOUSE REQUIREMENTS – VERSION 2.0 ............................................. 1
1. INTRODUCTION ....................................................................................................... 4
1.1. Background........................................................................................................ 4
1.2. Scope and objective ........................................................................................... 4
1.3. Reference documents ......................................................................................... 5
2. GENERAL DEFINITIONS ......................................................................................... 7
3. NOTIONS AND CONVENTIONS............................................................................. 9
3.1. User categories and use purposes ...................................................................... 9
3.1.1. Copernicus Services [Copernicus Services] ........................................ 9
3.1.2. Institutions and bodies of the Union [Union_Inst] .............................. 9
3.1.3. Participants to a research project financed under the Union
research programmes [Union_Research_Projects] .............................. 9
3.1.4. Public authority [Public_Auth] .......................................................... 10
3.1.5. International Organisations and international NGOs
[INT_ORG_NGO] ............................................................................. 10
3.1.6. Public [Public] ................................................................................... 10
3.2. Usage types (services) ..................................................................................... 11
3.2.1. DISCOVERY Service ....................................................................... 11
3.2.2. VIEW Service .................................................................................... 11
3.2.3. DOWNLOAD Service ....................................................................... 11
3.3. Usage rights ..................................................................................................... 12
3.4. Access to datasets for public tasks and research projects ................................ 12
4. GENERAL OVERVIEW OF THE REQUIREMENTS............................................ 13
4.1. CORE datasets ................................................................................................. 13
4.2. ADDITIONAL datasets with predefined quotas ............................................. 13
4.3. Access to Contributing Mission data provided under previous phases ........... 14
4.4. Licensing conditions ........................................................................................ 14
5. DATASET REQUIREMENTS ................................................................................. 16
5.1. Overall dataset requirements ........................................................................... 16
5.2. Overall requirements for CORE datasets ........................................................ 16
1
5.3. Dataset requirements for Copernicus Land service over Europe .................... 17
5.3.2. Optical HR Pan Europe coverage (HR_IMAGE_2015) .................... 19
5.3.3. Optical VHR multispectral and panchromatic coverage over
Europe (VHR_IMAGE_2015)........................................................... 25
5.3.4. European optical MR1 composites (MR_IMAGE_2015) ................. 30
5.4. Dataset Requirements for Copernicus Global land ......................................... 34
5.4.1. Optical worldwide HR1 coverage (HR1
_OPTICAL_GLOBAL) ..................................................................... 34
5.4.2. Optical HR2 worldwide coverage
(HR2_OPTICAL_GLOBAL) ............................................................ 35
5.4.3. Optical MR2 worldwide coverage
(MR_OPTICAL_GLOBAL).............................................................. 36
5.4.4. Optical LR worldwide coverage (LR_OPTICAL_GLOBAL)........... 37
5.4.5. SAR MR1 Worldwide coverage (MR1_SAR_GLOBAL) ................ 39
5.4.6. SAR LR Worldwide coverage (LR_SAR_GLOBAL) ...................... 40
5.4.7. SAR Altimetry MR2 Worldwide coverage
(MR2_ALTIMETRY) ....................................................................... 41
5.5. Fixed / CORE Dataset requirements for marine, atmosphere, climate
change applications.......................................................................................... 42
5.5.1. Sea Ice Monitoring MR1 SAR (MR1_SAR_SEA_ICE) ................... 43
5.5.2. Global/Regional Systematic Ocean Colour data (CORE_012) ......... 44
5.5.3. Systematic Global and Regional sea surface temperature data
(CORE_013) ...................................................................................... 44
5.5.4. Systematic Global and Regional Altimeter / Sea Level data
(CORE_014) ...................................................................................... 44
5.5.5. Data for aerosol monitoring and forecasting (CORE_017) ............... 44
5.5.6. Data for sulphur dioxide (SO2) atmospheric composition
monitoring and forecasting (CORE_018).......................................... 45
5.5.7. Data for formaldehyde (HCHO) atmospheric composition
monitoring and forecasting (CORE_019).......................................... 45
5.5.8. Data for Ozone (O3) atmospheric composition monitoring
and forecasting (CORE_020) ............................................................ 45
5.5.9. Data for Carbon Monoxide (CO) atmospheric composition
monitoring and forecasting (CORE_021).......................................... 45
5.5.10. Data for Carbon Dioxide (CO2) atmospheric composition
monitoring and forecasting (CORE_022).......................................... 45
5.5.11. Data for Methane (CH4) atmospheric composition monitoring
and forecasting (CORE_023) ............................................................ 45
5.5.12. Data for Nitrogen Dioxide (NO2) atmospheric composition
monitoring and forecasting (CORE_024).......................................... 46
2
5.6. Archived CORE Datasets ................................................................................ 46
5.7. Flexible / ADDITIONAL datasets requirements ............................................. 47
5.7.1. Overall requirements are: .................................................................. 48
5.7.2. Archive rush retrieval ........................................................................ 49
5.7.3. New acquisitions in rush mode.......................................................... 49
5.7.4. Archive standard retrieval ................................................................. 50
5.7.5. New acquisitions in standard mode ................................................... 50
5.7.6. Quota management ............................................................................ 52
5.8. Already acquired ADDITIONAL datasets ...................................................... 53
6. FUTURE REQUIREMENTS .................................................................................... 53
6.1. Datasets to be considered for future requirements .......................................... 53
6.1.1. Video datasets .................................................................................... 53
ANNEX 1. ADDITIONAL DATASET SPECIFICATIONS ............................... 55
ANNEX 2. FILE FORMAT CONVENTIONS .................................................... 57
ANNEX 3. MAP OF "LARGE REGIONS" FOR HR IMAGE 2015 AND
VHR IMAGE 2015.................................................................................................... 58
ANNEX 4. MAP OF "ZONES WITH DIFFERENT ACQUISITION
APPROACHES" FOR HR IMAGE 2015 ................................................................. 59
ANNEX 5. HR IMAGE 2015 AND VHR IMAGE 2015: ACQUISITION
WINDOWS 61
ANNEX 6. NATIONAL PROJECTIONS TABLE .............................................. 64
ANNEX 7. DATA QUALITY LAYER ............................................................... 65
ANNEX 8. QUALITY BASED COVERAGE PRICING .................................... 66
ANNEX 9. EXAMPLES OF TYPICAL CLOUD COVER AND HAZE
SITUATIONS ........................................................................................................... 67
3
1. INTRODUCTION
1.1. Background
For an initial period of 2007-2011, data access management was funded through a
data access grant between the EC and ESA.
As from October 2010 the next step of the EC-ESA cooperation, the Delegation
Agreement [RD1], ensured continuity of the data provision. That new phase was
intended to widen the scope of the data access to a broader community. It introduced
the concept of a Data Warehouse, capturing the requirements of the different user
communities. The Data Warehouse is based on two types of data: (i) a fixed part,
called 'CORE datasets', which are typically well defined large datasets covering the
needs of FP7 project participants and other users and (ii) a flexible part called
'ADDITIONAL' datasets. This approach has proven to be flexible enough to
accommodate further additional requirements or specific requirements which were
not covered by the CORE datasets and not known in advance.
This Data Warehouse v2.0 document builds on the previous experience and contains
the new Copernicus data requirements for the period 2014-2020 during which time a
fully operational Copernicus programme will be established. This document also
addresses access to observation datasets already provided under the previous phases.
1.2. Scope and objective
This document presents the Copernicus requirements for space-based Earth
Observation data for the period covered by the Copernicus Regulation. It contains
the requirements collected from Copernicus services and other users requesting
Earth Observation data, whether financed by the Union or related to policies of the
Union, like Union-financed research projects and the activities of Union agencies
(EEA, EMSA, SatCen (previously EUSC), etc.).
This document has a strong focus on the requirements for Contributing Mission data
to be procured, it being understood that priority should be given to the use of
dedicated Copernicus (Sentinel) data wherever feasible. Thus, it may progressively
evolve towards the consolidation of all satellite data requirements for Copernicus
services independently of whether licences for the data need to be procured or not. It
should also be noted that some users are exclusively interested in Earth Observation
data openly disseminated to ensure that any interested parties questioning the results
of their analysis may put them to the test using the same Earth Observation data as
inputs.
Where datasets need to be procured, licensing conditions are negotiated with
contributing mission data providers and will depart from conditions of use of
datasets available under an open data policy. The procurement of datasets requires
the establishment of a contractual framework between the Copernicus Contributing
Mission (CCM) Entities, ESA and the users, which should not be confused with the
Copernicus Data and Information Policy which enshrines free, full and open access.
4
Technical specifications should be extracted from the requirements expressed in this
document to ensure the provision or procurement of Earth Observation data from
Copernicus Contributing Missions. ESA plays a key role in the provision of these
Earth Observation data from Copernicus Contributing Missions as captured in the
Delegation Agreement between the Commission and ESA [RD4]. EUMETSAT also
plays an important role in providing Contributing Mission data in accordance with
the Delegation Agreement between the Commission and EUMETSAT [RD5].
The approach suggested in this document aims at deepening, as much as possible,
users' rights along the following major principles:
(1) A strong emphasis on providing the necessary datasets to ensure the optimal
functioning of the Copernicus services while mutualising as much as
possible the needs of different user communities, mainly institutional and
related to the Space theme of Horizon 2020 research programme;
(2) The licensed rights must ensure the fulfilment of the Copernicus services'
operational tasks, the goal is to avoid any restrictions on the dissemination of
the information extracted from the CCM data by the Copernicus services;
(3) Predefinition of CORE datasets when the specifications can be provided in
advance with sufficient detail;
(4) Bulk agreement for ADDITIONAL datasets with flexible specifications, e.g.
geographical area, under the establishment of predefined quotas;
(5) Access to datasets procured in the previous phases of the provision of
Contributing Mission datasets.
The CORE Datasets are defined in this document. By definition, the ADDITIONAL
datasets are flexible and are not known in advance, therefore technical details cannot
be provided at this stage, but only general requirements and principles on the quota
management mechanism.
As some ADDITIONAL dataset requirements are more precisely defined than
others, it is interesting to provide the available information on these requirements
while, for the time being, leaving them in the ADDITIONAL datasets category.
The approach for the procurement of the two types of datasets will consequently
also be different: CORE datasets can be procured on the basis of pre-defined
specifications, ADDITIONAL datasets through a quota mechanism and bulk
agreements with data providers for the provision of data within a financial envelope.
1.3. Reference documents
RD[1]: Commission Delegation Agreement with ESA (2009)
RD[2] Data Warehouse Document version 1.9
RD[3] GMES Space Component Data Access Portfolio: Data Warehouse
2011-2014, GMES-PMAN-EOPG-TN-11-0006, version 2.7.1
RD[4]: Commission Delegation Agreement with ESA (in preparation)
5
RD[5] Commission Delegation Agreement with EUMETSAT (in
preparation)
62. GENERAL DEFINITIONS
(1) Primary Dataset: Any spatial dataset originating from a Copernicus Contributing
Mission Entity (CCME) and provided by ESA/EUMETSAT/CCME to the
beneficiary. A Primary Data set is understood to include spacecraft and instrument
data and is comprised of single-mission Level 1 or Level 2 data (with exceptional
provision of higher-level products such as single-mission Level 3 data on a case-
by-case basis).
(2) Altered Primary Dataset: A dataset derived from a Primary Dataset retaining
enough information to allow the retrieval of the Primary Dataset (e.g. sensor pixel
information) and therefore to allow the replication of the Primary Dataset as such.
Examples: histogram stretched images, ortho-rectified images, resampled and
rescaled images and mosaics.
(3) Derived Information, Derived Product or Secondary Dataset: A Dataset or
information derived from a Primary Dataset or Altered Primary Dataset which
does not allow the retrieval of the Primary Dataset (e.g. sensor pixel information)
and therefore does not allow the replication of the Primary Dataset as such.
Examples: land cover classifications, vegetation indexes.
(4) Metadata: Information describing spatial data sets and spatial data services and
making it possible to discover them, populate an inventory with them and use
them. In the case of spatial datasets, metadata may include a preview of the full
dataset.
(5) Union Public tasks: The development, implementation and monitoring of
policies and related activities as defined by the EC Treaty and subsequent
Community legislation.
(6) Spatial Datasets, Spatial Data and Spatial Data Services: As defined in
Directive 2007/2/EC of 14/3/2007.
(7) Composite: Multi-scene or multi-image aggregation over the same scene/image.
Usually used to remove clouds for optical sensor datasets. Can be applied to
different sensors, but is preferably based on the same sensor. A composite is
multi-temporal (made of several scenes/images acquired at different dates/times).
(8) Mosaic: Multi-scene or multi-image aggregation to cover a larger area than a
given scene/image. Can be applied to different sensors. A mosaic is generally
mono temporal (or made of scenes/images collected within a given time period
that can range from a few days to a season).
(9) Bundled Data Product: Refers to a commercial offer providing all datasets
acquired simultaneously by the same satellite mission and simultaneously
covering more than one range of spectral resolutions (e.g. multispectral and
panchromatic), which are grouped into a single product, but which are still
comprised of each of the separate spectral bands.
7(10) Ground sampling distance (GSD): Specifies the extent of a single pixel of the
sensor(s) that makes the dataset. Pixels are assumed to be square. Unless
otherwise specified the values are:
(a) VHR1: 1m – 4m – 10m – 30m – 100m – 300m
83. NOTIONS AND CONVENTIONS
Many requirements contained in this Data Warehouse document will be used to
draft the technical specifications for the public procurement of EO remote sensing
datasets. In this context a number of notions and conventions must be provided and
are part of the requirements.
3.1. User categories and use purposes
User types are defined below. The sections below also define the usage allowed by
each user category. User categories are referred to in the specifications by their
abbreviation in square brackets. These user categories shall only be applied for
datasets provided under restrictive dissemination conditions by the CCME.
Wherever contributing mission data providers make available datasets under an
open data policy, the user indications under licensing requirements are meaningless
most of the time as all users including the general public may have access to the data
from the contributing mission data providers.
3.1.1. Copernicus Services [Copernicus Services]
Copernicus Services first established under the EU Regulation 911/2010 and
then under the subsequent EU Regulation establishing the Copernicus
programme.
This category is provided for clarification. Legally speaking, Copernicus
Services providers are either Institutions and bodies of the Union (see below)
or their sub-contractors and therefore do not need an extension of licensing
rights beyond these provided to the [Union_Inst], in particular for already
acquired licensing rights on datasets from previous Data Warehouse
exercises.
3.1.2. Institutions and bodies of the Union [Union_Inst]
The institutions and bodies of the Union, as well as their contractors, may
use the Primary Datasets and Altered Primary Datasets for activities whose
purpose is within the Union Public tasks (development, implementation and
monitoring of policies and related activities as defined by the Union Treaties
and subsequent Union legal acts).
It includes:
(1) The European Institutions set up under the Union Treaties;
(2) The European Union Agencies.
3.1.3. Participants to a research project financed under the Union research
programmes [Union_Research_Projects]
This includes any natural or legal person officially registered as participant
of a project funded under the FP7 or Horizon 2020 Space themes.
Participants in such projects may use the Primary Datasets for activities
within the project.
9As an option, participants to a project financed under Horizon 2020 outside
the Space themes may use for activities within the project, already acquired
Primary Datasets.
3.1.4. Public authority [Public_Auth]
Public authorities include:
(1) Any government or other public administration of States participating
in the Copernicus programme, including public advisory bodies, at
national, regional or local level.;
(2) Any natural or legal person performing public administrative
functions under national law, including specific duties, activities or
services in relation to a Union policy;
(3) Any natural or legal person having public responsibilities or
functions, or providing public services relating to a Union policy
under the control of a body or person falling within (1) or (2), such as
a contractor of a public authority;
(4) Any research and academic organisation.
Public authorities, as well as their contractors, may use the Primary Datasets
and Altered Primary Datasets for activities whose purpose is within the
Union Public tasks.
3.1.5. International Organisations and international NGOs
[INT_ORG_NGO]
International Organisations are defined as: Any International Governmental
Organisation created by an international treaty which can be looked up in the
UN online database of treaties. Specialised agencies of the UN are included;
International Organisations, as well as their contractors, may use the Primary
Datasets for activities whose purpose is within the Union Public tasks.
The international NGOs are defined as: Any International Non-
Governmental Organisation specialised in humanitarian, development or
environmental activities.
Non-Governmental Organisations, as well as their contractors, may use the
Primary Datasets for activities whose purpose is within the Union Public
tasks.
3.1.6. Public [Public]
Any natural or legal person.
In the case that the Download service (see below) is authorised for the
Public, the use of the Primary Datasets may be limited to non-commercial
activities.
103.2. Usage types (services)
The usage types have been defined adhering to the INSPIRE nomenclature (Article
11(1) (a), (b) and (c) of Directive 2007/2/EC). For each dataset the usage type per
user category should be specified.
3.2.1. DISCOVERY Service
The DISCOVERY service shall make it possible to search for spatial
datasets and services on the basis of the content of the corresponding
metadata and to display the content of the metadata.
All users can access such services.
3.2.2. VIEW Service
The VIEW service shall make it possible, as a minimum, to display,
navigate, zoom in/out, pan or overlay viewable spatial datasets and to display
legend information and any relevant content of metadata.
3.2.3. DOWNLOAD Service
The DOWNLOAD service shall enable the beneficiary:
(1) to make an unlimited number of copies of the Primary Dataset as
needed (archiving and backup purposes included);
(2) to install on as many individual computers as needed, including
internal computer network;
(3) to alter or modify the Primary Dataset by invoking a computer
application to produce Altered Primary Dataset and Secondary
Dataset;
(4) to post Metadata of the Primary Dataset or its Altered Primary
Dataset on an Internet website with the display of a credit in the
following form: “includes material (c) Mission name (year of
acquisition), all rights reserved”;
(5) to make hard copies or to display on an Internet web site a
Representation of any extract of the Primary Dataset or its Altered
Primary Dataset at any resolution, with the display of a credit in the
following form: “includes material (c) Mission name (year of
acquisition), all rights reserved”; the data received by client
applications, through the above Internet posting, should be such that
it is not possible to recreate the Primary Dataset or Altered Primary
Dataset (unless that right is provided);
(6) to use the Primary Dataset or its Altered Primary Dataset for internal
or external demonstration purposes;
(7) to retain all Intellectual Property Rights associated with any
Secondary Dataset developed on the basis of the Primary Dataset.
113.3. Usage rights
(a) The licensed usage rights must ensure the fulfilment of the
Copernicus services' operational tasks as defined under the
Copernicus Regulation. In particular the licensed rights shall not
prevent the generation and dissemination of Copernicus service
information (derived information from CCM primary datasets used as
inputs) under a free, full and open dissemination regime in
application of the Commission Delegated Regulation No 1159/2013
and the Copernicus Regulation.
(b) In the case of the Copernicus Emergency Management Service and
the Copernicus Security Service, [Public_Auth] shall have access to
Primary Datasets.
(c) When the CCME provides download access to CCM data with the
authorisation to set up a viewing service for these data, the
[Union_Inst] will have the right to set up such viewing service.
3.4. Access to datasets for public tasks and research projects
The operational nature of Copernicus implies operational access conditions
to contributing missions. However, some Copernicus users may not need
access to remote sensing data in operational mode and therefore should also
enquire about other access schemes to these data.
– Researchers may benefit from access schemes to EO data designed for
their needs;
– Public authorities also benefit from specific access schemes for data
needs in the accomplishment of their public tasks.
124. GENERAL OVERVIEW OF THE REQUIREMENTS
4.1. CORE datasets
The CORE datasets aim at consolidating predefined needs collected from
Copernicus services and other activities requesting Earth Observation data
whether financed by the Union or related to policies of the Union, like Union
financed research projects and the activities of Union agencies (EEA,
EMSA, SatCen, etc.). This approach allows a more robust and cost-effective
access mechanism, which will be offered to a broad range of users and
activities. In particular:
(1) For the fulfilment of the data needs for the Copernicus Land
Monitoring service, it should cover:
(a) Pan-European (EEA39) High Resolution (HR) cloud free
image coverage for the requirements of land cover/land cover
change activities (Corine: CLC and CLCC), and 5 High
Resolution Layers (HRLs) on land cover characteristics
(imperviousness, forestry, agriculture (grasslands), permanent
wetlands and small water-bodies. It shall include access to
archives.
(b) Full European Very High Resolution (VHR) coverage over
Europe (EEA39) matching the requirements of applications at
European level (Urban Atlas, Land cover on riparian zones for
the purpose of biodiversity monitoring, monitoring of coastal
areas, risk areas, protected areas (Natura 2000 sites), Land
Parcel Identification and at national level.
(c) For Dynamic Land monitoring: daily Low Resolution (LR)
and Medium Resolution (MR) full globe coverage for the
production of biogeophysical parameters in the global
component of Copernicus Land Monitoring service.
(d) For seasonal vegetation monitoring: monthly to 15-day cloud
free composites of Medium Resolution (MR) full European
(EEA39) coverage during the vegetation period March-
October.
(2) For the fulfilment of the data needs of the Copernicus Marine
Environment Monitoring, Atmosphere Monitoring and Climate
Change services.
4.2. ADDITIONAL datasets with predefined quotas
ADDITIONAL datasets are needed to complement the CORE datasets, as
many data characteristics are not known in advance (e.g. satellite tasking for
rapid mapping or security applications, selected areas for biodiversity
monitoring).
13As for the CORE datasets, these ADDITIONAL datasets are expected to
fulfil the needs collected from Copernicus services and other activities
requesting Earth Observation data, whether financed by the Union or related
to policies of the Union, like Union financed research projects and the
activities of Union agencies (EEA, EMSA, SatCen, etc.).
Some assumptions have been made for the specification of requirements:
The envelope estimates should be made on the basis of a
combination of statistics and requirements from operational
Copernicus services, current European research projects, previous
projects and other related activities;
The needs of Copernicus services in support to crisis management
are considered with the highest priority;
ADDITIONAL datasets covering 'non-crisis' activities (e.g. specific
requirements for Copernicus Land Monitoring service not covered
by the CORE datasets) will be provided with medium priority;
In addition some specific requirements from EU Institutions and
agencies (e.g. EMSA, SatCen) can be satisfied through the provision
of ADDITIONAL datasets;
Horizon 2020 project needs may be covered by ADDITIONAL
datasets.
4.3. Access to Contributing Mission data provided under previous phases
Access to CORE datasets and ADDITIONAL datasets from previous
acquisition campaigns should be ensured wherever feasible. Where the
licensing conditions made for the use of these datasets were limited in time,
the extension of the licence shall be made when the datasets are used in
recurring monitoring activities and reference. As a principle the licence shall
not limit in time the use of the datasets by Copernicus.
4.4. Licensing conditions
The overall principle should be to have a broad access to CORE datasets,
whereas access to ADDITIONAL data managed through quotas could be
more limited.
Regarding the pan-Europe and Europe CORE datasets (wall-to-wall HR pan-
Europe coverage, VHR Europe coverage, monthly 15-day HR composites)
and the Global MR and LR coverage, the licences should include
unrestricted access to the Primary Dataset for the Copernicus Services,
Union Institutions, Public Authorities and Union financed research projects.
This should be the case also for the archived full sub-Saharan HR coverage,
which also includes access for International organisations and NGOs
involved in land monitoring activities in these African countries.
14Regarding the ADDITIONAL dataset for emergency service activities
included in Union Public Tasks, the licences should include unrestricted
access to the Primary Dataset for Copernicus services, Union Institutions,
and Union financed research projects, as well as for Member States
authorities willing to benefit from unrestricted access to the Primary Dataset
to ensure an efficient management of emergencies.
155. DATASET REQUIREMENTS
This section provides the definition of required datasets. The objective of this
section is to present the Copernicus requirements, identifying the criteria, which
will have a significant impact on the cost of data. This should give sufficient
details to derive technical specifications as input for the data/licence procurement.
5.1. Overall dataset requirements
REQ-5.1.1-1 Use of Sentinel data
Wherever Sentinel data can be used to fulfil the requirements expressed in
this document, they should be considered with priority. In view of
budgetary limitations, some requirements may need a progressive phase-in
with the availability of Sentinel data fulfilling these needs.
REQ-5.1.1-2 Data archiving and access to archives
When the licences on satellite datasets need to be procured, the datasets
must remain accessible as archives for future use by users benefiting from
the rights attached to the DWH licence.
5.2. Overall requirements for CORE datasets
REQ-5.2.1-1 Sensor transitions
The provision of datasets shall support a seamless transition when satellite
missions supporting the data provision are replaced by new ones, as far as
technically possible. The data provision scheme shall thus support the
parallel provision of data from the old and the new missions for the
duration necessary to allow cross calibration and comparability of datasets
(as far as the implication on the budget distribution is reasonable).
REQ-5.2.1-2 Consistency
During each acquisition campaign the dataset characteristics of procured
data shall not be modified unilaterally by the data providers.
REQ-5.2.1-3 Formats, INSPIRE metadata and File naming
CCME data shall be delivered in standard formats; with complete and
validated INSPIRE compliant metadata. The datasets shall be identified
according to a standard file naming convention.
A single, common harmonised format should be considered for the
delivery of individual files used to provide wall-to-wall coverage of an
AOI from different missions.
165.3. Dataset requirements for Copernicus Land service over Europe
The requirements for Copernicus Land service over Europe have been
drafted within the following boundary conditions:
Requirements are based on the knowledge as of today of the pan-
European and local Land service configuration for the reference year
2015 +/- 1 year. Continuity of a series of existing services for the
reference year 2012 +/- 1 year are taken as a baseline, extended with
requirements for the 2015 +/-1 reference year.
Subject to adaptations as soon as the Sentinel data start to become
available: this is expected to considerably improve the acquisition
possibilities, and therefore needs a distinction between the periods
before and after Sentinel data become available in nominal mode.
The land service portfolio for the 2018 +/- 1 reference year is subject
to partial continuity, partial changes, following the outcome of a
(continuous) process of Land service evolution. The sensor offer will
have substantially changed by then. Both conditions entail the need for
a revision before starting the 2018 (+/- 1 y) reference year acquisitions.
REQ-5.3.1-1 Preserving spectral information
Each optical CORE dataset shall be provided as a set of individual images
(scenes) with separated spectral bands at pixel value (bundled with other
products if requested or offered) to allow for the maximum level of
flexibility in automated classification and visual interpretations, including
future developments.
REQ-5.3.1-2 Standard Formats
The delivery of each individual file in a dataset shall be provided in a
single, common harmonised format for level 2 and 3 products. From the
start of the operational delivery of Sentinel-2 data, the delivery of CCME
level 2 and 3 products shall meet a standard file format and structure with
complete and validated INSPIRE compliant metadata, and according to the
standard file naming convention (see Annex 2).
The general aim of the harmonisation of file structure, metadata and
naming shall be to facilitate automation of the retrieval and processing of
the datasets, as much as possible on a common basis.
Furthermore, from the start of the operational delivery of Sentinel-2 data,
each dataset shall be accompanied by a data quality layer, encoded
according to the specifications in Annex 7.
REQ-5.3.1-3 Reference Year and Repetitiveness
The acquisition campaigns shall be repeated every 3 years taking the year
2012 as starting reference year (n-3). To ensure the full coverage of the
area of interest (AOI), the acquisition campaign shall start the year before
(n-1). The overall objective shall be to acquire full coverage of the AOI
17within the first 2 years. The year after (n+1) may be used for remaining
gap-filling. Adjustment must be made to make optimal use of available
SWIR sensors in particular in 2014.
REQ-5.3.1-4 Harmonised Cloud Cover delineation
For procured optical imagery, a common and consistent way of delineating
and encoding Cloud Coverage and Cloud shadows shall be put in place,
taking into account the cloud fringe areas and treat separately the
indication of cloud shadow areas and the presence of haze in order to
allow automatic processing of the data. Guidelines on how to identify
typical Cloud Coverage issues are made available in Annex 9.
REQ-5.3.1-5 Unique and single workflow monitoring tool
To ensure an adequate management of each step in the full workflow of
acquisition campaign, pre-processing and information production over
large territories and involving multiple stakeholder groups, it is mandatory
to maintain the status of all key steps in the full cycle in a single master
shape-file, constantly updated during the full cycle of any acquisition
campaign (yearly basis) with key attributes, allowing all to have access to
the same comprehensive status overview.
The acquired scenes shall be referred to in a master shape-file of the
EEA39 countries in LAEA ETRS89 (EPSG code 3035) as background
layer, and comprising layers on a per-sensor and per-year basis. Each layer
shall contain ALL scene footprints taken into consideration for
programming the acquisitions (each scene being the smallest spatial
object).
Separate layers shall be added for gap-filling through archive retrieval
and/or extensions to other sensors.
Each footprint shall have as minimum the following list of attributes:
Identification of the scene;
Date of acquisition;
Status of pre-processing;
Encoding of cloud cover;
Encoding of haze;
Encoding of technical quality of image;
Availability in the data warehouse;
Acceptation/rejection.
18Attributes shall be encoded in a semantically standardised structure, thus
allowing easy search, selection, filtering and combination queries in order to
facilitate management of the actual status of acquisitions.
The master shape-file shall be managed by ESA, and constantly updated
during the acquisition campaign by the CCMEs. The master-file shall be
accessible for all stakeholders in the process through a point of access
facility ensuring that all partners at any point of time are dealing with the
same information content.
REQ-5.3.1-6 Success criteria and remuneration
For all wall-to-wall optical CORE dataset acquisition campaigns, in order to
maximise the chances of successful coverage matching the acquisition
requirements, a new remuneration mechanism shall be applied. Simple per-
scene remuneration has shown not to be appropriate. Therefore a scheme
shall be put in place, which, while encouraging data providers to acquire and
deliver as much images as possible, will reward them in the end according to
the quality of the best possible coverage accomplished.
At the end of the acquisition campaign for any year, the data providers
(CCMEs) shall propose a mosaic composed of all scenes they consider
matching the acquisition specifications, for assessing the quality parameters,
which will determine the final price.
A total of five quality indicators will be evaluated based on the final mosaic:
Total coverage achieved;
Number of scenes required;
Average size of all areas composed of contiguously connected scenes;
Max size of the largest data gap (incl. clouds and snow, excl.
perennial snow and glaciers);
RMS distance (in days) of acquisitions date from target date;
Max difference of acquisition date of adjacent pixels.
An example of a quality based pricing scheme is provided in Annex 8.
5.3.2. Optical HR Pan Europe coverage (HR_IMAGE_2015)
Applications
Main purposes: continuation of pan-European land services, including
Corine Land Cover (CLC) and the production of High Resolution Layers
(HRL) on land cover characteristics by the Union, EEA and Member States.
These requirements could also partially meet EFFIS requirements.
REQ-5.3.2-1 Sensor acquisition characteristics
19Spatial resolution: Sensors between 5m and maximum 30m GSD (ground
sampling distance) shall be considered. (Remark: the acquisition pixel size is
linked with the final data product resolution, see Data Product requirements
below).
Spectral resolution: Optical multispectral sensors used for all
HR_IMAGE_2015 acquisitions must have at least one independent spectral
band centred in each of the following spectral ranges to allow optimal
analysis of vegetation:
Green [0.5-0.6μm], Red [0.6-0.68μm], NIR [0.7-0.9μm], SWIR1 [1.5-
1.8μm]
Radiometric resolution: minimum bit depth of 7-bits signal dynamic range is
acceptable, higher bit depth is preferred (e.g. 16-bits).
REQ-5.3.2-2 Areas of Interest (AOI)
The overall AOI of interest comprises the EEA39 national territories to a
total of ~6 M km². Within this Area a number of 'large regions' have been
delineated as acquisition subunits as displayed in Annex 3.
The overall AOI shall be extended by a 2 nautical miles buffer for the 2014-
2015 acquisition campaign. From the operational availability of Sentinel-2
onwards, the standard territorial water of 12 nautical miles into the sea/ocean
shall be taken as the buffer.
REQ-5.3.2-3 Tasking and Acquisition – Objectives
The objective of the wall-to-wall CORE HR image acquisition is to cover
once or twice the entire AOI (depending of the Zone) with:
The least possible number of scenes;
Coming from the least possible sensors;
Leaving only the smallest possible gaps and
Being acquired as close as possible to the respective reference date.
Data acquisition shall start the year before the reference year. The bulk of
acquisitions shall be within the reference year. The reference year +1 is only
used for gap-filling, in which case those scenes would receive an additional
7 days “off date” quality bias. In that case the increase in quality and
coverage will determine the price.
REQ-5.3.2-4 Acquisition zones
For the acquisition campaign, separate zones are identified in Annex 4
covering a range of areas indicatively corresponding with risk levels to
acquire complete coverage. It comprises a Zone A for the cloudiest areas,
Zone B for cloud-prone areas, Zone C for the areas with statistically limited
cloud cover frequencies and Zone D for French DOM situated under the
20tropics. The requirements for Zone C apply to Zone B and D if not specified
otherwise.
To account for differences with respect to phenological effects and
observation conditions (cloud probability, sun angles) and the resulting
difficulties to meet the quality requirements, each AOI subunit is assigned to
one of the following zones:
Zone A: a minimum of one cloud-free coverage (VNIR + SWIR).
Areas at northern latitudes and/or with very narrow phenological windows
and or very difficult observation conditions.
Zone B: two cloud-free coverages (one VNIR + SWIR; second: preferably
VNIR + SWIR, but VNIR only acceptable as second choice).
Areas at mid-latitudes with a high probability of suitable acquisition
windows during the observation periods.
Two target dates will be specified, however the completion of the second
coverage is considered a best effort.
Zone C: two cloud-free coverages with preferably homogenous sensor
VNIR + SWIR acquisitions.
Areas at lower latitudes with extended phenological observation periods and
predominantly clear sky conditions during those.
Two target dates will be specified and accordingly two coverages are
expected.
Zone D: one cloud-free coverage (VNIR + SWIR).
Overseas areas (e.g. French DOMs).
REQ-5.3.2-5 Acquisition windows
The acquisition windows are defined in Annex 5 with extended and narrow
windows. The starting and ending dates of these extended and narrow
windows can be subject to modifications with a ten day advance notice to
allow for extension of acquisition with an earlier start and/or later end.
REQ-5.3.2-6 Acquisition strategy
Zone A: One VNIR + SWIR coverage in the narrow windows of acquisition;
multi-mission systematic tasking over the same areas to ensure adequate
coverage of the cloudiest regions.
Purchase criteria for Zone A: As a minimum, any part of acquisition (grid)
with less than the high cloud coverage defined in 'Cloud Cover and Haze'
requirements as long as it contributes to reaching the goal of one cloud free
coverage of Zone A.
21Zone B: A first and most important coverage including VNIR + SWIR shall
be acquired during the narrow window of acquisition, preferably through
systematic tasking of one sensor, complemented with speculative tasking
from other sensors and/or outside the narrow window but still inside the
extended window.
The second coverage shall be acquired with ideally 6 weeks, but minimally
with a 4 week time difference to the first coverage. The second coverage can
be either upfront or after the first coverage, but within the extended
acquisition window. Preference is given to VNIR + SWIR. VNIR only is
acceptable for the second coverage if SWIR is not available (second
priority).
Purchase criteria for Zone B: As a minimum, any part of the acquisition grid
with less than the cloud coverage defined in 'Cloud Cover and Haze'
requirements as long as it contributes to reaching the goal of completing
Zone B.
Zone C: A first and most important coverage including VNIR + SWIR shall
be acquired during the narrow window of acquisition.
The second coverage shall be acquired with ideally 6 weeks, but minimally
with a 4 week time difference to the first coverage. The second coverage can
be either upfront or after the first coverage, but within the extended
acquisition window. Preference is given to VNIR + SWIR. VNIR only is
acceptable for the second coverage if SWIR is not available (second
priority).
Purchase criteria for Zone C: As a minimum, any part of the acquisition grid
with less than the cloud coverage defined in 'Cloud Cover and Haze'
requirements as long as it contributes to reaching the goal of completing
Zone C.
Zone D: One coverage including VNIR + SWIR shall be acquired during the
extended window of acquisition.
Purchase criteria for Zone D: As a minimum, any part of the acquisition grid
with less than the cloud coverage defined in 'Cloud Cover and Haze'
requirements as long as it contributes to reaching the goal of one cloud free
coverage of Zone D.
REQ-5.3.2-7 Cloud Cover and Haze
The maximum cloud coverage allowed is defined as per a grid of 25x25 km
and varies per Zone A, B. C or D (see Annex 4). It is also a function of the
spatial distribution of the clouds on the scene (the presence of cloud spread
all over the scene is not acceptable). This restriction aims at reducing the
number of scenes to be processed and ensuring processing homogeneity. See
Guidelines in Annex 9.
22The overall AOI comprising zones A, B, C, D will be subdivided in a
uniform grid of 25 km cell side. Grid cells will, according to zone
dependency, have a maximum allowed Cloud Coverage threshold as follows:
Zone A (cloudiest): CC 20% max;
Zone B (cloud prone): CC 10% max;
Zones C/D (limited cloud): CC 5% max.
Scenes comprising haze shall be flagged accordingly for further
consideration, and are subject to rejection by the Copernicus service
coordinator, in consultation with the service providers implementing the land
services.
Presence of perennial snow and glaciers is allowed. However, temporary
snow, saturated pixels and data gaps have to be included for the above listed
calculation of the max. allowed percentages.
REQ-5.3.2-8 Illumination/Radiometry
The sun elevation angle shall be higher than 20° to ensure sufficient
illumination and to minimize the effect of shadows.
The off-nadir viewing angle shall be less than 25°.
Saturated pixels, snow (excl. perennial), clouds and areas with a signal to
noise ratio of less than 50:1 (at 35% ground reflectance) will count as data
gaps.
REQ-5.3.2-9 Selection and rejection of scenes
The data provider must assess the quality of the datasets.
It is up to the service coordinator, in consultation with the service providers
to review the scenes proposed by the CCMEs at the end of the acquisition
campaign on their appropriateness for further processing.
REQ-5.3.2-10 Processing Levels
The scenes shall be provided with two different levels of processing:
Level2 (ortho-rectified to ETRS89/ETRS-LAEA)):
Ortho-rectified at-sensor radiance values (resampled using cubic
convolution method);
GCP file with respect to reference data set;
INSPIRE compliant metadata.
Level1 (system correction):
23 System corrected and at-sensor radiometric and geometric calibrated
data;
Ortho layer, (i.e. x, y coordinates (ETRS89)) INSPIRE compliant
metadata.
REQ-5.3.2-11 Radiometric and atmospheric correction
A cascade approach may be considered for radiometric and atmospheric
corrections in decreasing order of priority as follows:
(a) Atmospheric corrected Surface Reflectance (or Bottom of Atmosphere
(BOA)), 16-bit data. However this option requires a well-established
method. Atmospheric bands in Sentinel-2 will allow for that, entailing
that this option only be considered from Sentinel-2 onwards.
(b) Top of Atmosphere (TOA) planetary reflectances, 16-bit data. Sensor
specific in band solar irradiance values (ESUN) shall be provided.
(c) ESUN corrected but not solar elevation corrected.
(d) Radiometric calibrated data with calibration parameters enabling the
conversion of image digital numbers (DN) into spectral radiance
(W/m2/sr/um). Solar spectral irradiances within each spectral band
and the solar elevation for each scene are required to convert the
image DNs into TOA (Top Of Atmosphere) reflectance values.
REQ-5.3.2-12 Ortho-rectification
Ortho ready processing shall include basic corrections such as
rectilinear geometry, radiometric distortion (i.e. source data).
Ortho-rectification has to be full-parametric or making use of RPC
functions and shall be based on the EU-DEM. In case a different DEM
is used it must be supplied with the data.
European projection is required for all coverage ETRS89, LAEA
(EPSG code 3035). National projection is optional and may be
requested on a case by case basis.
Grid alignment: GeoTIFF key: The GeoTIFF key
"GTRasterTypeGeoKey" shall have the value 1, corresponding to
"RasterPixelIsArea".
Pixel alignment: The values of the coordinates for the upper-left
corner (x and y) of the upper-left pixel of an image dataset shall be an
integer multiple of the pixel size (resolution) in the corresponding
direction (x and y).
Ortho-rectification accuracy:
o Relative accuracy (2014-2015): Reference less than 1 pixel
difference with HR_IMAGE_2012.
24o Absolute accuracy (from S2 onwards): Ortho-rectification must
be accurate to less than 0.75 pixel size equivalent RMSE and a
max displacement of less than 1.5 pixels in no more than 0.1%
of any 50x50sqkm area within the AOI with respect to the
reference data set.
REQ-5.3.2-13 Data Products
Optical multispectral HR data products with a 20 m pixel resolution. GSD of
the sensor shall be no worse than 1.5 times the product raster width.
From the operational availability of Sentinel-2 onwards, HR products will be
delivered at 10 m pixel resolution.
Level 2 products shall be provided according to standard and harmonised
format specifications (see Annex 2).
REQ-5.3.2-14 Delivery
Zone A: After selection, delivery will be made no later than 60 days after
selection.
Zone B: To the extent that systematic tasking has been applied for a sensor
over Zone B, the same delivery requirement shall be maintained as for zone
A. For non-systematic acquisitions delivery shall be made on a continuous
basis, but no later than 60 days after the end of the extended acquisition
windows.
Zones C: Delivery shall be made on a continuous basis, but no later than 60
days after the end of the extended acquisition windows.
For all zones, delivery shall be made per large scale regions (acquisition
subunits defined in Annex 3).
REQ-5.3.2-15 Licensing (for datasets with restrictive licensing conditions)
Multi-user DOWNLOAD and VIEW licences shall be acquired to cover as a
baseline all relevant activities of users in [Copernicus Services],
[Union_Inst], [Union_Research_Projects] and [Public_Authorities].
[INT_ORG_NGO] and [Public] shall have access in DISCOVERY mode.
VIEW mode should however be considered an option towards
[INT_ORG_NGO] and will be used if affordable.
5.3.3. Optical VHR multispectral and panchromatic coverage over Europe
(VHR_IMAGE_2015)
Applications
Land local component applications for hotspots, i.e. specific areas of interest
at pan-European level (EEA39) (land cover over riparian zones for the
purpose of biodiversity monitoring, monitoring of coastal areas, risk areas,
25protected areas (Natura 2000 sites), Land Parcel Identification…) and at
national level.
Land local component applications over hotspots at pan-European level
(Urban Atlas, Land Parcel Identification…) and at national level.
REQ-5.3.3-1 Sensor acquisition characteristics
Spatial resolution: pixel sizes up to maximum 2m shall be considered.
(Remark: the acquisition pixel size is linked with the final data product
resolution, (see. Data Product requirements below)).
Spectral resolution: Optical multispectral sensors used for all
VHR_IMAGE_2015 acquisitions must have at least one independent
spectral band centred in each of the following spectral ranges to allow
optimal analysis of vegetation:
Green [0.5-0.6μm], Red [0.6-0.68μm], NIR [0.7-0.9μm].
Panchromatic [0.45-0.85μm].
SWIR1 [1.5-1.8μm] is considered optional. The availability of sensor
capacities allowing for systematic inclusion without jeopardising other
requirements shall trigger an investigation by ESA on the feasibility of
revision of specifications to include SWIR.
Radiometric resolution: minimum bit depth of 7-bit signal dynamic range is
acceptable, higher bit depth is preferred, e.g. 16-bit depth.
REQ-5.3.3-2 Areas of Interest (AOI)
The overall AOI of interest comprises the EEA39 national territories to a
total of ~6 M km².
The overall AOI shall be extended by a 2 nautical miles buffer.
REQ-5.3.3-3 Tasking and Acquisition
Objectives:
The objective of the wall-to-wall CORE VHR image acquisition is to cover
once the entire AOI with:
The least possible number of scenes;
Coming from the least possible sensors;
Leaving only the smallest possible gaps and
Being acquired as close as possible to the respective reference date.
Data acquisition shall start the year before the reference year. The bulk of
acquisitions shall be within the reference year. The reference year +1 is only
used for gap-filling, in which case those scenes will receive an additional 7
26days “off date” quality bias. In that case the increase in quality and coverage
will determine the price.
REQ-5.3.3-4 Acquisition windows
The acquisition windows are defined in Annex 5 with extended and narrow
windows. The starting and ending dates of these extended and narrow
windows can be subject to modifications with a ten day advance notice to
allow for extension of acquisition with an earlier start and/or later end.
REQ-5.3.3-5 Acquisition Strategy
The acquisition strategy shall take into account the knock-on effects of
scattered footprint patterns (i.e. different sensors, different years, different
seasons), having a multiplicative effect on the coherence and size of working
units as a collection of images with sufficiently stable characteristics to
automate classification processes on single selection of areas. This entails
optimisation of series of acquisitions with regard to the areas as defined in
Annex 4.
REQ-5.3.3-6 Cloud Cover and Haze
The maximum cloud coverage allowed per scene is function of the swath of
the sensor and the percentage of cloud defined per Zone A, B, C or D (see.
Annex 4). The maximum cloud coverage allowed per scene is also function
of the spatial distribution of the clouds on the scene (the presence of cloud
spread all over the scene is not acceptable). This restriction aims at reducing
the number of scenes to be processed and ensuring processing homogeneity.
The overall AOI comprising zones A, B, C, D will be subdivided in a
uniform grid of side 25 km. Grid cells will, according to zone dependency,
have a maximum allowed Cloud Coverage (CC) threshold as follows:
Zone A (cloudiest): CC 20% max;
Zone B (cloud prone): CC 10% max;
Zones C/D (limited cloud): CC 5% max.
Scenes comprising haze shall be flagged accordingly for further
consideration, and are subject to rejection by the Copernicus service
coordinator, in consultation with the service providers implementing the land
services.
Presence of perennial snow and glaciers is allowed. However, temporary
snow, saturated pixels and data gaps have to be included for the above listed
calculation of the maximum allowed percentages.
REQ-5.3.3-7 Illumination/Radiometry
The sun elevation angle shall be higher than 20° (preferable higher than 23°)
to ensure sufficient illumination and to minimize the effect of shadows.
27You can also read
