Data access

The mandatory GetCapabilities operation allows CSW clients to retrieve service metadata from a server. The response to a GetCapabilities request shall be an XML document containing service metadata about the server. This subclause specifies the XML document that a CSW server shall return to describe its capabilities.

Examples of EMODnet CSW GetCapabilities:

https://www.emodnet.eu/geonetwork/emodnet/eng/csw?service=CSW&request=GetCapabilities&version=2.0.2 

 https://sextant.ifremer.fr/geonetwork/srv/eng/csw?service=CSW&request=GetCapabilities&version=2.0.2 

EMODnet central portal GetRecords example:

https://www.emodnet.eu/geonetwork/emodnet/eng/csw?REQUEST=GetRecords&SERVICE=CSW&VERSION=2.0.2&ELEMENTSETNAME=summary&OUTPUTSCHEMA=http://www.opengis.net/cat/csw/2.0.2&CONSTRAINTLANGUAGE=FILTER&CONSTRAINT_LANGUAGE_VERSION=1.1.0&RESULTTYPE=results&TYPENAMES=csw:Record&CONSTRAINT=%3Cogc:Filter%20xmlns:ogc=%22http://www.opengis.net/ogc%22%3E%3Cogc:PropertyIsEqualTo%3E%3Cogc:PropertyName%3Edc:type%3C/ogc:PropertyName%3E%3Cogc:Literal%3Edataset%3C/ogc:Literal%3E%3C/ogc:PropertyIsEqualTo%3E%3C/ogc:Filter%3E&maxRecords=1000 

EMODnet Physics GetRecords example:

http://catalog.emodnet-physics.eu/geonetwork/srv/eng/csw?REQUEST=GetRecords&SERVICE=CSW&VERSION=2.0.2&ELEMENTSETNAME=summary&OUTPUTSCHEMA=http://www.opengis.net/cat/csw/2.0.2&CONSTRAINTLANGUAGE=FILTER&CONSTRAINT_LANGUAGE_VERSION=1.1.0&RESULTTYPE=results&TYPENAMES=csw:Record&CONSTRAINT=%3Cogc:Filter%20xmlns:ogc=%22http://www.opengis.net/ogc%22%3E%3Cogc:PropertyIsEqualTo%3E%3Cogc:PropertyName%3Edc:type%3C/ogc:PropertyName%3E%3Cogc:Literal%3Edataset%3C/ogc:Literal%3E%3C/ogc:PropertyIsEqualTo%3E%3C/ogc:Filter%3E&maxRecords=1000

GetRecordById requests retrieves catalogue metadata records using their identifier.

EMODnet central portal GetRecordById example

https://www.emodnet.eu/geonetwork/emodnet/eng/csw?request=GetRecordById&service=CSW&version=2.0.2&elementSetName=full&id=5d4696dc-bac4-4dd4-82c5-258700553cb4 

The mandatory GetCapabilities operation allows WMS clients to retrieve service metadata from a server. The response to a GetCapabilities request shall be an XML document containing metadata of the service (proposed layers, associated projections, author ...).

The standard to construct a WMS GetCapabilities request, depending on the version:

(wms endpoint url)?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities or (wms endpoint url)?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetCapabilities

Example of a GetCapabilities request addressed to the EMODnet Physics view service:

https://geoserver.emodnet-physics.eu/geoserver/emodnet/wms?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities 

Using the information given in the GetCapabilities request, the GetMap request returns a raster map containing the requested data layer selected from all the available layers as defined in the XML document. The elements such as the data layer, region, projection, size of the returned image, image format, etc. are defined in the form of arguments.

Example of a GetMap request that returns an image of the EMODnet Bathymetry Mean depth (DTM) based on source resolution of 1/8 arc minute (~250 meter) in multi colour style: https://ows.emodnet-bathymetry.eu/wms?service=WMS&service=WMS&request=GetMap&version=1.1.1&layers=emodnet%3Amean_multicolour&styles=&format=image%2Fpng&transparent=true&info_format=text%2Fhtml&tiled=false&width=400&height=628&srs=EPSG%3A3857&bbox=-2669794.954627626%2C2250306.11271559%2C4800533.188102208%2C14538934.276066802 

Returns:

WFS defines a standard for exchanging vector data by querying both the data structure and the source data. The basic operations are GetCapabilities, DescribeFeatureType, and GetFeature. WFS supports a variety of WFS output formats (Ex: GML, shapefile, json, geojson, CSV,...). The full list of output formats supported can be found by performing a WFS GetCapabilities request.

The EMODnet WFS services are accessible from following endpoints:

A GetCapabilities request generates a metadata document (xml) describing a WFS service provided by server as well as valid WFS operations and parameters.

Example of a EMODnet human activities GetCapabilities request:

https://ows.emodnet-humanactivities.eu/wfs?SERVICE=WFS&VERSION=1.1.0&request=GetCapabilities

A DescribeFeature request returns a description of feature types supported by a WFS service.

Example of a EMODnet Biology DescribeFeature request: http://geo.vliz.be/geoserver/Dataportal/wfs?service=wfs&version=2.0.0&request=DescribeFeatureType&typeName=Dataportal:eurobis&outputFormat=application/json

Example of a EMODnet Human Activities DescribeFeature request:

https://ows.emodnet-humanactivities.eu/wfs?SERVICE=WFS&VERSION=1.1.0&request=describeFeatureType&typeName=shellfish&bbox=-1.3,0.3,49.2,49.9 

A GetFeature request returns a selection of features from a data source including geometry and attribute values.

Example of a EMODnet Human Activities GetFeature request:

https://ows.emodnet-humanactivities.eu/wfs?SERVICE=WFS&VERSION=1.1.0&request=getFeature&typeName=shellfish&bbox=-1.3,0.3,49.2,49.9&outputFormat=application/json

The EMODnet portals provide Web Coverage Services (WCS) to support requests for coverage data (rasters) or gridded data products. Enter one of the following addresses into your WCS client:

A WCS GetCapabilities request retrieves a list of the server’s data, as well as valid WCS operations and parameters.

Example of an EMODnet Bathymetry WCS GetCapabilities request:

https://ows.emodnet-bathymetry.eu/wcs?service=wcs&version=1.0.0&request=GetCapabilities 

A WCS DescribeCoverage request retrieves a metadata (XML) document that fully describes the requested coverages.

Example of an EMODnet Bathymetry DescribeCoverage request:

https://ows.emodnet-bathymetry.eu/wcs?service=wcs&version=1.0.0&request=DescribeCoverage&coverage=emodnet:mean

A WCS GetCoverage request returns a coverage in a well known format. Like a WMS GetMap request, but with several extensions to support the retrieval of coverages.

Example of an EMODnet Bathymetry GetCoverage request:

https://ows.emodnet-bathymetry.eu/wcs?service=wcs&version=1.0.0&request=getcoverage&coverage=emodnet:mean&crs=EPSG:4326&BBOX=-2.08542634499992,45.656434726,-1.08908527599993,46.4319841780001&format=image/tiff&interpolation=nearest&resx=0.00208333&resy=0.00208333

The mandatory GetCapabilities operation allows CSW clients to retrieve service metadata from a server. The response to a GetCapabilities request shall be an XML document containing service metadata about the server. This subclause specifies the XML document that a CSW server shall return to describe its capabilities.

Example of CMEMS CSW GetCapabilities:

https://cmems-catalog-ro.cls.fr/geonetwork/srv/eng/csw-MYOCEAN-CORE-PRODUCTS?request=GetCapabilities&service=CSW

GetRecords request allows to query the catalogue metadata records.

The recommended outputSchema is http://www.isotc211.org/2005/gmd as it is the one containing all the information.

Example of CMEMS CSW GetRecords:

https://cmems-catalog-ro.cls.fr/geonetwork/srv/eng/csw-MYOCEAN-CORE-PRODUCTS?SERVICE=CSW&REQUEST=GetRecords&VERSION=2.0.2&ElementSetName=full&resultType=results&maxRecords=10&outputSchema=http://www.isotc211.org/2005/gmd&outputFormat=application/xml

Example of CMEMS CSW GetRecords with a request on AlternateTitle parameter through an OCG Filter:

https://cmems-catalog-ro.cls.fr/geonetwork/srv/eng/csw-MYOCEAN-CORE-PRODUCTS?SERVICE=CSW&REQUEST=GetRecords&VERSION=2.0.2&ElementSetName=full&resultType=results&maxRecords=1&outputSchema=http://www.isotc211.org/2005/gmd&outputFormat=application/xml&typeNames=gmd:MD_Metadata&constraintLanguage=CQL_TEXT&constraint_language_version=1.1.0&constraint=AlternateTitle+like+%27ARCTIC_ANALYSIS_FORECAST_PHYS_002_001_A*%27

GetRecordById requests retrieves catalogue metadata records using their identifier.

The recommended outputSchema is http://www.isotc211.org/2005/gmd as it is the one containing all the information.

Example of a CSW GetRecordById:

https://cmems-catalog-ro.cls.fr/geonetwork/srv/eng/csw-MYOCEAN-CORE-PRODUCTS?SERVICE=CSW&REQUEST=GetRecordById&version=2.0.2&id=eec7a997-c57e-4dfa-9194-4c72154f5cc5&ElementSetName=full&resultType=results&outputSchema=http://www.isotc211.org/2005/gmd&outputFormat=application/xml

The mandatory GetCapabilities operation allows WMS clients to retrieve service metadata from a server. The response to a GetCapabilities request shall be an XML document containing metadata of the service (proposed layers, associated projections, author ...).

Example of a CMEMS GetCapabilities request:

http://nrt.cmems-du.eu/thredds/wms/global-analysis-forecast-phy-001-024-monthly?request=GetCapabilities&service=WMS

Using the information given in the GetCapabilities request, the GetMap request returns a raster map containing the requested data layer selected from all the available layers as defined in the XML document. The elements such as the data layer, region, projection, size of the returned image, image format, etc. are defined in the form of arguments.

Example of CMEMS WMS GetMap request with some NCWMS parameters:

http://nrt.cmems-du.eu/thredds/wms/global-analysis-forecast-phy-001-024-monthly?FORMAT=image%2Fpng&TRANSPARENT=TRUE&STYLES=boxfill%2Frainbow&LAYERS=mlotst&TIME=2019-06-16T00%253A00%253A00.000Z&COLORSCALERANGE=10%2C1150&NUMCOLORBANDS=250&ABOVEMAXCOLOR=0x000000&BELOWMINCOLOR=0x000000&LOGSCALE=true&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=155.04970170455,-90,322.57455255682,77.524850852273&WIDTH=256&HEIGHT=256

Returns:

The GetFeatureInfo operation returns the attributes of the requested entities in a map in different formats such as HTML, XML, and plain text.

Example of CMEMS GetFeatureInfo request:

http://nrt.cmems-du.eu/thredds/wms/global-analysis-forecast-phy-001-024-monthly?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetFeatureInfo&LAYERS=mlotst&QUERY_LAYERS=mlotst&STYLES=boxfill/rainbow&BBOX=-180,-125.87879,179.916672,135.87879&FEATURE_COUNT=1&HEIGHT=400&WIDTH=550&FORMAT=image/png&INFO_FORMAT=text/xml&SRS=EPSG:4326&X=251&Y=240&TIME=2019-06-16T00:00:00.000Z

Motu is a robust web server allowing users to download data through the web. It provides two main functionalities: Subsetter and Direct-Get-File which use an API in line with OGC spirit (close to OGC SPS + OGC WCS). These two interfaces share the same API but have not the same functionalities:

• Subsetter allows users to extract the data of a dataset, with geospatial, temporal and variable criteria.
• Direct get file allows users to access data of a dataset function of temporal criteria.

A graphic web interface and machine to machine interfaces allow to access data, information on data and on the usage of the server.

MOTU is an Open source software developed by CLS:  https://github.com/clstoulouse/motu.

On CMEMS project there are two ways to access (describe product, getsize) and download (download product) the product data:

• Using the MOTU interface. In this case the user must go through a Central Authentication Service (CAS), before accessing to the different services.
• Using the Python motu-client script which integrate the API and allows an automatically access or download product. The access through the CAS is transparent for users.

The download service provided by the Web-Portal relies on a MOTU request named DescribeProduct. This feature gives all the dataset description including the dataset ID, the time coverage, the geospatial coverage, the variable(s) and potentially the vertical coverage.

Example of MOTU Describe product request:

http://nrt.cmems-du.eu/motu-web/Motu?action=describeproduct&service=GLOBAL_ANALYSIS_FORECAST_PHY_001_024-TDS&product=global-analysis-forecast-phy-001-024

Returns:

Getsize request permits to recover the size of a download request, through graphic web interface, or through machine to machine interface.

Example of MOTU Getsize request:

http://nrt.cmems-du.eu/motu-web/Motu?action=getsize&service=GLOBAL_ANALYSIS_FORECAST_PHY_001_024-TDS&product=global-analysis-forecast-phy-001-024&x_lo=-100&x_hi=100&y_lo=-80&y_hi=90&t_lo=2019-06-16+12%3A00%3A00&t_hi=2019-06-16+12%3A00%3A00&z_lo=0.49&z_hi=1&variable=thetao

Returns:

ProductDownload request permits to download data of dataset, through graphic web interface, or through machine to machine interface.

Example of MOTU ProductDownload request:

http://nrt.cmems-du.eu/motu-web/Motu?action=productdownload&service=GLOBAL_ANALYSIS_FORECAST_PHY_001_024-TDS&product=global-analysis-forecast-phy-001-024&x_lo=-100&x_hi=100&y_lo=-80&y_hi=90&t_lo=2019-06-16+12%3A00%3A00&t_hi=2019-06-16+12%3A00%3A00&z_lo=0.49&z_hi=1&variable=thetao

A multi-platform command-line matching your data extraction (also referred as data selection or data query), which can be executed from the shell of your system (Windows/MacOS/*nix) thanks to a Python package called motuclient (https://github.com/clstoulouse/motu-client-python).

It's intended for both Beginners and Expert users and is particularly useful to automate retrieval of NearRealTime/Forecast data production and to optimize heavy requests such as long timeseries.

Example of command to download data in Subsetter:

python -m motuclient --motu http://nrt.cmems-du.eu/motu-web/Motu --service-id GLOBAL_ANALYSIS_FORECAST_BIO_001_028-TDS --product-id global-analysis-forecast-bio-001-028-monthly --longitude-min -180 --longitude-max 179.75 --latitude-min -80 --latitude-max 90 --date-min "2019-06-16 00:00:00" --date-max "2019-06-16 00:00:00" --depth-min 0.493 --depth-max 0.4942 --variable chl --variable phyc --variable o2 --variable no3 --variable po4 --variable si --variable fe --variable nppv --variable ph --variable spco2 --out-dir <OUTPUT_DIRECTORY> --out-name <OUTPUT_FILENAME> --user <USERNAME> --pwd <PASSWORD>

Example of command to download data in Direct-Get-File:

python -m motuclient --motu http://nrt.cmems-du.eu/motu-web/Motu --service-id GLOBAL_ANALYSIS_FORECAST_BIO_001_028-DGF --product-id global-analysis-forecast-bio-001-028-monthly --date-min "2019-06-16 00:00:00" --date-max "2019-06-16 00:00:00" --out-dir <OUTPUT_DIRECTORY> --out-name <OUTPUT_FILENAME> --user <USERNAME> --pwd <PASSWORD>

To obtain the describeproduct or getsize answers, use the -- describe-product or --size options in the command above.

The Sensor Observation Service (SOS) is a web service to query real-time sensor data and sensor data time series: https://www.opengeospatial.org/standards/requests/32

A multi-platform command-line matching your data extraction (also referred as data selection or data query), which can be executed from the shell of your system (Windows/MacOS/*nix) thanks to a Python script called sosclient and accessible in the following Zip package: ftp://ftp.ifremer.fr/ifremer/sismer/oceanotron/sosClientPython/sos-client-python.zip

Example of command to download data in SOS

python <PATH_TO_SOSCLIENT_DIR>/sos_client.py --user=<USERNAME> --pwd=<PASSWORD> --quiet --server=http://nrt.cmems-du.eu/oceanotron/SOS/default --offering-id=glo_multiparameter_nrt_profile_latest --longitude-min=-179.967 --longitude-max=179.986 --latitude-min=-78.01 --latitude-max=81.8618 --date-min="2019-07-12T02:58:20.000Z" --date-max="2019-07-12T02:58:20.000Z" --observed-property=sea_water_temperature --observed-property=sea_water_practical_salinity --out-dir=<OUTPUT_DIRECTORY> --out-name=<OUTPUT_FILENAME>

Most of CMEMS products raw data can be downloaded through a FTP server (login required). All CMEMS data are encapsulated in NetCDF3 or NetCDF4 data (https://www.unidata.ucar.edu/software/netcdf/).

Example of FTP data access:

ftp://nrt.cmems-du.eu/Core/GLOBAL_ANALYSIS_FORECAST_PHY_001_024/global-analysis-forecast-phy-001-024-monthly 

Resulting tree:

All the ICES data portals offer download services. These services are provided in standard formats (Web APIs, Soap etc). A list of those services along with descriptions and guidelines on how to use them can be found in the ices metadata catalogue:

http://gis.ices.dk/geonetwork/srv/eng/catalog.search#/search?facet.q=type%2Fservice

 Examples:

Stock assessment data for Herring:

http://sg.ices.dk/StandardGraphsWebServices.asmx/getStockDownloadData?AssessmentKey=7919

List of all stocks in 2013:

http://sg.ices.dk/StandardGraphsWebServices.asmx/getListStocks?year=2013

ICES hosts services for spatial datasets. All of ICES reference layers such as the ICES Areas, ICES Statistical rectangles, ICES Ecoregions can be found there. All of the standard OGC formats (WMS, WFS) are available through a single map server that can be found here:

http://gis.ices.dk/gis/rest/services

A user can navigate through the services directory and get more information on each service by using standard url queries. As an example, the following link is a getCapabilities request for the ICES Areas:

http://gis.ices.dk/gis/services/ICES_reference_layers/ICES_Areas/MapServer/WMSServer?request=GetCapabilities&service=WMS

Spatial datasets, just like the other ICES datasets, are accompanied with INSPIRE compliant matadata that can be found on the ICES metadata catalogue.