We are happy to announce a new release of our ecmwfr R package, version 2.0.0. This new release anticipates the retirement of the the old API, on September 16th, in favour of the new beta API. The package is mostly backwards compatible with previous code (more on this below), although WebAPI support was lost, but CEMS support was gained. You can install the latest package from CRAN using:
install.packages("ecmwfr")
With the move to a new API come some changes to the package itself. To update to the new beta API you will have to set a new Personal Access Token (PAT).
# set your PAT
wf_set_key(
key = "asfasdf-128242349-asdfasdf-asdfad"
)
In version 2.0.0 you do not have to set a user or a services. Updating old download code is as simple as removing the user or service arguments from your code if your PAT is updated. All old v1.x.x requests should translate for the most part unaltered to v2.x.x. An example of both v1.x.x and v2.x.x requests is given below.
# requests remain (mostly) unaltered between v1.x.x and v2.x.x
request <- list(
dataset_short_name = "reanalysis-era5-pressure-levels",
product_type = "reanalysis",
variable = "temperature",
year = "2000",
month = "04",
day = "04",
time = "00:00",
pressure_level = "850",
data_format = "netcdf",
download_format = "unarchived",
area = c(70, -20, 60, 30),
target = "era5-demo.nc"
)
# The original v1.x.x call
# this request for data will
# fail due to the spurious
# user name
wf_request(
request,
user = "your_id"
)
# The new v2.x.x call
wf_request(
request
)
Critical notes on (breaking) changes
Login credentials are now consistent across all ECMWF services using a single Personal Access Token (PAT). This change led to a simplification of the login procedure (see above). This consolidation of the user account allows for dynamic context switching between services and therefore allows for the mixing of CDS and ADS requests in batches.
With the move to the new API come changes to how netCDF files are handled. The latter is especially important for those not within the climate science community, where the grib format is the preferred standard. NetCDF support seems to be inconsistent at the time of writing. Although {ecmwfr} tries to provide backward compatibility and consistency to the community in querying the data the same is not true for the returned data itself. In practices, this means, although little change will be required to change the query of your data in old scripts (see above), that you might be faced with considerable code rewrites to adjust to the new netCDF format (potentially breaking downstream packages). We had hoped that the community would have been consulted, or at least parity in terms of returned data would be ensured.
We also note that the python CDSAPI does not address the conflict between the various API login methods, as {ecmwfr} does. The python API requires the PAT and service URL to stored in the same .cdsapirc credentials file, for all APIs. Therefore, this file can only be used by one API at the time. Resolving this issue requires hard coded PATs in code. Hard coding API keys into code is a big security issue. Although all data are technically open, this still establishes a poor credential management practices within the larger user community. An external solution is provided by a user, but this could have been fixed on an API interface library level. Luckily the latter does not impact R users!
In closing, we suggest the ECMWF to:
- Improve NetCDF Support: Ensure consistent and reliable netCDF support.
- Enhance Flexibility in the Python CDSAPI: Explore alternative methods for login credentials, to allow for more flexible API usage.
- Improve Documentation: Provide detailed and accessible documentation that caters to users of all levels, including in-depth explanations for low-level users.
- Communicate Breaking Changes Effectively: Implement a clear and transparent communication strategy to inform users about upcoming breaking changes well in advance, allowing them to plan and adapt accordingly with generous roll over periods.
