Title: | Download CMIP6 Data |
---|---|
Description: | Search and download CMIP6 data. |
Authors: | Elio Campitelli [cre, aut] |
Maintainer: | Elio Campitelli <[email protected]> |
License: | GPL-3 |
Version: | 0.0.2.9000 |
Built: | 2024-11-15 05:07:21 UTC |
Source: | https://github.com/eliocamp/rcmip6 |
Reads files in your local CMIP6 directory and parses the contents into a data.frame
cmip_available(root = cmip_root_get()) cmip_available_legacy(root = cmip_root_get())
cmip_available(root = cmip_root_get()) cmip_available_legacy(root = cmip_root_get())
root |
Root folder to download and organise the data. |
cmip_available_legacy()
reads from the older way of storing information
and will only work for files downloaded with version <= 0.0.2 of the package.
A data.frame.
Downloads CMIP data
cmip_download( results, root = cmip_root_get(), user = Sys.info()[["user"]], comment = NULL, year_range = c(-Inf, Inf), check_diskspace = TRUE, download_config = cmip_download_config(), ... ) cmip_download_config( delay = 0.5, retry = 5, total_connections = 1, host_connections = 1, low_speed_limit = 100, low_speed_time = 30 )
cmip_download( results, root = cmip_root_get(), user = Sys.info()[["user"]], comment = NULL, year_range = c(-Inf, Inf), check_diskspace = TRUE, download_config = cmip_download_config(), ... ) cmip_download_config( delay = 0.5, retry = 5, total_connections = 1, host_connections = 1, low_speed_limit = 100, low_speed_time = 30 )
results |
A list of search results from |
root |
Root folder to download and organise the data. |
user , comment
|
Deprecated. |
year_range |
An integer vector of length 2, indicating the start and end range of years. Restricts the download of model output with files that include some data within this range of years. Defaults to c(-Inf, Inf) to include all possible files |
check_diskspace |
Logical indicating whether to check if location has enough space to download all the requested files. |
download_config |
a list of arguments to configure the behaviour of downloads. |
... |
Ignored |
delay |
delay in seconds between retries. The actual delay adds a bit of randomness. |
retry |
number of retries before giving up on a download. |
total_connections |
maximum number of total concurrent connections. |
host_connections |
maximum number concurrent connections per host. |
low_speed_limit , low_speed_time
|
the download will fail if it downloads
at below |
A list of files.
Checks the online status of data nodes and returns filtered results so that there is a single replica of each instance that is hosted on an online node.
cmip_filter_replicas(results)
cmip_filter_replicas(results)
results |
A list of search results from |
A data.table.
Set or get the root folder for CMIP data.
cmip_root_set(root, mode = "default") cmip_root_get()
cmip_root_set(root, mode = "default") cmip_root_get()
root |
Root folder. |
mode |
Mode that defines folder permissions. Can be an umask valid string or the aliases:
|
Query CMIP data
cmip_search(query) cmip_info(results) cmip_url_to_list(url)
cmip_search(query) cmip_info(results) cmip_url_to_list(url)
query |
A list that defines the search parameters. Optionally, a character vector of instances IDs to be searched. |
results |
A list of search results from |
url |
The URL of the JSON query to convert into a list. See details. |
The easiest way to get a valid query
list is to use the search portal at
(https://esgf-node.llnl.gov/search/cmip6/)
to make a search that approximates what you are looking for. Then, under the
number of results there's a link that reads "return results as JSON".
Copy that link and pass it to [cmip_url_to_list()]
.
On RStudio you can also use the AddIn.
A data.table of results.
Removes a lot of less important columns from the output of
cmip_search()
. The full dataset is stored in the hidden column
full_info
.
Use cmip_unsimplify()
to return to the original format
cmip_simplify(results) cmip_unsimplify(data)
cmip_simplify(results) cmip_unsimplify(data)
results |
A list of search results from |
data |
A simplifided output from |
Computes the total size of a search result in Mb.
cmip_size(results)
cmip_size(results)
results |
A list of search results from |
Finds the URLs that cmip_download would obtain from a search results object.
cmip_urls(results)
cmip_urls(results)
results |
A list of search results from |
If a valid URL is not found the return value has an NA
in its place.
Note that the versions to download have "/fileServer/" in the path, whereas those for the remote access have "/dodsC/" instead (and can be string-substituted). This works for some sources we have seen (GDAL for example can within limitations read remotely from the 'fileServer' URLs, but the NetCDF library needs the 'dodsC' version, again this a pattern observed on some sources not an authorative claim).
It takes some time to process and find these so they aren't automatically added to the results from a search.
character vector of URLs