-
Notifications
You must be signed in to change notification settings - Fork 0
/
07-appendices.Rmd
142 lines (88 loc) · 16.4 KB
/
07-appendices.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
# Appendices {#appendices}
Below are several resources and definitions that may be helpful in developing a data management plan. This is by no means an exhaustive list. These are resources that we have either curated specifically for TBEP or resources we've found helpful in our own journey.
## List of resources
### Bulding Data Management Plans
* [Environmental Data Initiative **Data Management Resources**](https://environmentaldatainitiative.org/dm-resources/){target="_blank"}
* [University of California **DMPTool**](https://dmptool.org/){target="_blank"}
* [US Geological Survey resources for **Metadata Creation**](https://www.usgs.gov/products/data-and-tools/data-management/metadata-creation){target="_blank"}
* [ELIXIR and others **Data Stewardship Wizard**](https://ds-wizard.org/){target="_blank"}
### TBEP Open Science Trainings
* [CERF 2021 Open Science workshop](https://tbep-tech.github.io/cerf-os-workshop/){target="_blank"}
* [TBEP 2023 Open Science workshop](https://tbep-tech.github.io/tbep-os-workshop/){target="_blank"}
* [FLAFS 2023 Open Science workshop](https://tbep-tech.github.io/flafs-os-workshop/){target="_blank"}
### TBEP R Trainings
* [Peconic Estuary Program R training](https://tbep-tech.github.io/pep-r-training){target="_blank"}, [recording](https://drive.google.com/file/d/1ZjVHFrVpw2uTKZw-BmD29umdbl6viutM/view?usp=sharing){target="_blank"}
* [NERRS ecosystem metabolism R training ](https://tbep-tech.github.io/ecometab-r-training){target="_blank"}
[TBEP June 2020 R training](https://tbep-tech.github.io/tbep-r-training/){target="_blank"}, [recordings](https://www.youtube.com/watch?v=_RI4XMRWeV0&list=PLfJ6-D-exF9RM5TPtT4T0nxieqpr_R4pJ){target="_blank"}
* [Writing functions in R](https://tbep-tech.github.io/tbep-r-training/functions.html){target="_blank"}
* [R package development workflow](https://tbep-tech.github.io/tbep-r-training/packages.html){target="_blank"}
* [A soft introduction to Shiny](https://tbep-tech.github.io/tbep-r-training/shiny.html){target="_blank"}
* [CERF 2023 R Shiny workshop](https://tbep-tech.github.io/shiny-workshop/){target="_blank"}
### R Lessons & Tutorials
* [Software Carpentry: **R for Reproducible Scientific Analysis**](http://data-lessons.github.io/gapminder-R/){target="_blank"}
* [Data Carpentry: **Geospatial Workshop**](https://datacarpentry.org/geospatial-workshop/){target="_blank"}
* [Data Carpentry: **R for Data Analysis and Visualization of Ecological Data**](http://www.datacarpentry.org/R-ecology-lesson/){target="_blank"}
* [Data Carpentry: **Data Organization in Spreadsheets**](http://www.datacarpentry.org/spreadsheet-ecology-lesson/){target="_blank"}
* [**Reproducible Reporting with R (R$^3$)** for Marine Ecological Indicators](https://noaa-iea.github.io/r3-train/){target="_blank"}
* [**R for Water Resources Data Science**](https://www.r4wrds.com/){target="_blank"}
* [**RStudio Webinars**, many topics](https://www.rstudio.com/resources/webinars/){target="_blank"}
* [R For Cats: **Basic introduction site, with cats!**](http://rforcats.net/){target="_blank"}
* [Topical cheatsheets from **RStudio**, also viewed from the help menu](http://www.rstudio.com/resources/cheatsheets/){target="_blank"}
* [Cheatsheet from CRAN of **base R functions**](http://cran.r-project.org/doc/contrib/Short-refcard.pdf){target="_blank"}
* [Totally awesome **R-related artwork** by Allison Horst](https://github.com/allisonhorst/stats-illustrations){target="_blank"}
* [**Color reference** PDF with text names](http://www.stat.columbia.edu/~tzheng/files/Rcolor.pdf){target="_blank"}, [**Color cheatsheet** PDF from NCEAS](https://www.nceas.ucsb.edu/sites/default/files/2020-04/colorPaletteCheatsheet.pdf){target="_blank"}
### R eBooks/Courses
* [Jenny Bryan's **Stat545.com**](http://stat545.com/){target="_blank"}
* [Garrett Grolemund and Hadley Wickham's **R For Data Science**](http://r4ds.had.co.nz/){target="_blank"}
* [Chester Ismay and Albert Y. Kim's **Modern DiveR**](https://ismayc.github.io/moderndiver-book/){target="_blank"}
* [Julia Silge and David Robinson **Text Mining with R**](http://tidytextmining.com/){target="_blank"}
* [Hadley Wickham's **Advanced R**](http://adv-r.had.co.nz/){target="_blank"}
* [Hadley Wickham's **R for Data Science**](https://r4ds.had.co.nz){target="_blank"}
* [Yihui Xie **R Markdown: The Definitive Guide**](https://bookdown.org/yihui/rmarkdown/){target="_blank"}
* [Winston Chang **R Graphics Cookbook**](http://www.cookbook-r.com/){target="_blank"}
* [Wegman et al. **Remote Sensing and GIS for Ecologists: Using Open Source Software**](http://book.ecosens.org/RSEbook/){target="_blank"}
* [Lovelace et al. **Geocomputation with R**](https://geocompr.robinlovelace.net/){target="_blank"}
* [Edszer Pebesma and Roger Bivand **Spatial Data Science**](https://keen-swartz-3146c4.netlify.app/index.html){target="_blank"}
### Git/Github
* [Jenny Bryan's **Happy Git and Github for the useR**](http://happygitwithr.com/){target="_blank"}
* [Coding Club **Intro to Github**](https://ourcodingclub.github.io/2017/02/27/git.html){target="_blank"}
## Definitions
__CI/CD__: Continuous Integration/Continuous Deployment, describes web services that can be used to automate data checks, tests, or workflows. These are often integrated into existing platforms, such as GitHub Actions on GitHub.
__Dashboard__: Interactive and dynamic user interfaces available online that can be created to facilitate understanding or to inform decision-making. Many flavors exist, including [R Shiny](https://shiny.rstudio.com/), [Python Dash](https://dash.plotly.com/), or [ArcGIS storymaps](https://www.esri.com/en-us/arcgis/products/arcgis-storymaps/overview).
__Data__: A general term describing a variety of informational products that can support environmental decision-making or be used to test research hypotheses. Data can be as simple as a single spreadhsheet or more complex products such as model output or parameters.
__Database__: An organized collection of data, where each piece of data can be linked and accessed electronically through keys that act as unique identifiers for units of information. These are often called relational databases.
__Data Dictionary__: An informal description of the information included in a dataset, often used to place boundaries on expected values or data types. This includes column names, types of data stored in each column, and expected values for each data type.
__DOI__: Digital Object Identifier, a unique and permanent name (persistent identifier) for a dataset or other resource, used for archiving and discovery through online queries. Services like [Zenodo](https://zenodo.org/) can be linked to GitHub to create a DOI for a repository.
__FAIR__: Findable, Accessible, Interoperable, and Reusable, describes general guidelines for creating open data products or assessing the openness of existing products [@Wilkinson16].
__Flat File__: The simplest form of tabular data, often as a rectangular grid of information stored as [ASCII](https://en.wikipedia.org/wiki/ASCII) text in a non-proprietary file format. There is no information stored in each cell other than the observational values.
__Keys__: Identifiers that can be used to link data between tables. They are often used to identify unique rows of data, such as a station name or station name/date combination.
__Federated Repository__: An online network of connected repositories that use similar standards to collectively store data for discovery and access. Uploading a dataset to one node of a repository will make it available through all other nodes.
__Metadata__: A suite of industry or disciplinary standards as well as additional internal and external documentation and other data necessary for the identification, representation, interoperability, technical management, performance, and use of data contained in an information system [@Gilliland16]. Simply put, the who, what, when, where, why, and how of data.
__Model__: A general term describing a theoretical representation of a real-world phenomenon. It can be as simple as a statistical linear regression model (i.e., y varies as a function of x in a linear fashion) or a more involved mechanistic model with linked equations that describe real-world processes occurring through space and time.
__Open Science__: A philosophy and set of tools to make research reproducible and transparent, in addition to having long-term value through effective data preservation and sharing [@Beck20].
__Open Source__: Software with code that is freely available under a license that typically grants users the rights to modify or distribute to others for any purpose. The [R](https://cran.r-project.org/) statistical programming language is one example of open source software used in the environmental sciences.
__Provenance__: The history of a dataset, including its origin, purpose, and metadata. Formally, this can include the records of inputs, software, and steps of analysis used to create a dataset. The intent is to establish context and also allow reproducibility.
__Synthesis__: The collection and combination of datasets from different sources, often for research or to inform decision-making. The synthesis product may be considered a novel dataset in itself if the steps in its creation produce novel information not available from its source data.
__Tidy Data__: A set of simple rules for storing tabular data, including each variable in its own column, each observation in its own row, and each value in its own cell [@Wickham14c]
__Version Control__: A formal software or code development system that tracks and documents changes to create a record that describes the development history and that can be accessed at any time so that previous changes are never lost. [Git](https://git-scm.com/) is version control software, as compared to [GitHub](https://github.com/) which is an online platform for hosting projects using Git.
## RFP language
The following includes standard language and links to SOP sections that can be used in competitive requests for proposal (RFPs) issued by the TBEP to support use of open science and best practices for data management as described in this SOP. This language can be used as an expression for partners to encourage use of descriptions and approaches in this document for competitive criteria used to assess proposal qualifications.
### Best practices for open data
Respondents should demonstrate adequate hardware and software capabilities and personnel trained to receive, store, disseminate, print, summarize, and transmit open standard GIS and statistical package files. All final datasets, statistical assessments, data reduction methods and syntheses should be delivered to the TBEP in open standard formats using open science frameworks, such as those developed through the RMarkdown language, R or Rstudio statistical analysis platforms, Python scripts, or the QGIS environment. All data products and spatial coverages will include appropriate Federal Geographic Data Committee (FGDC) standards, Ecological Metadata Language (EML) compliant metadata, or other descriptive information appropriate for metadata (\@ref(metadatadesc)).
Respondents must demonstrate an understanding of tidy data formats as applied to tabular data and an ability to deliver data products in non-proprietary file formats (e.g., csv flat files, shapefiles, or KML). Use of online sharing platforms that enable data products to be findable, accessible, interoperable, and reusable (\@ref(fair)) are strongly encouraged. These can include formal data repositories that employ standardized metadata schema (e.g., [DataOne](https://www.dataone.org/)). Small datasets (<100 Mb) can also be hosted on the TBEP GitHub website (https://github.com/tbep-tech) by contacting staff for access. View the guidance [here](https://tbep-tech.github.io/data-management-sop/keys.html#datahome) for other acceptable alternatives.
## Contract language
The following includes some standard language and links to SOP sections that can be used in contracts for deliverables for data products. The text here describes general expectations of how data products should be delivered to TBEP in as open a format as possible following concepts and descriptions of approaches in this document. The intent is to contractually obligate our partners to adopt best practices for data management following an open science ethos.
### Declaration of use
All data deliverables are to be licensed under permissive terms that enable reuse in the public domain. This could include the [MIT](https://choosealicense.com/licenses/mit/) or [CC0](https://choosealicense.com/licenses/cc0-1.0/) licenses provided as stand-alone text files included with data deliverables. Exceptions to the permissive reuse terms in these licenses will be handled on a case by case basis, e.g., if data are sensitive or include information on human subjects. Additional guidance on choosing a license can be found [here](https://choosealicense.com/).
### General
An approach for data management should be identified as soon as possible after project inception (\@ref(howyou)). This will include identifying appropriate data deliverables at the beginning of the project using guidance in the SOP (\@ref(contribs)). Plans for documenting and delivering identified data products to the TBEP should also be identified, which primarily includes appropriate data formats (\@ref(whytidy)), metadata documentation (\@ref(metadatadesc)), and where data are hosted (\@ref(datahome)).
### Tabular data
All tabular data are to be provided in a tidy format to the extent possible. This includes following simple rules for delivering data as 1) each variable in its own column, 2) one observation per row, and 3) one value per cell (\@ref(whytidy)). Tidy data must be delivered in a non-proprietary or open-source compatible format, such as .csv flat files. Datasets using .xlsx file formats or that require proprietary software to access (e.g., SAS) will not be accepted unless project managers can provide justification for not using open formats.
### Geospatial data
Geospatial data must conform to Federal Geographic Data Committee ([FGDC](https://www.fgdc.gov/standards)) standards and other open source formats as compatible with the Geographic Data Abstraction Library ([GDAL](https://gdal.org/)) for spatial analysis (this includes most ESRI data products). Horizontal and vertical datums must be explicitly specified in any metadata, as well as any information describing attribute data accompanying spatial information. Appropriate datums should limit spatial distortion to the region of interest (e.g., [NAD83 / Florida West (ftUS)](https://epsg.io/2237-1714) for Florida).
### Source code
Source code used to generate a data product or analyze/evaluate a data product can be supplied as a text file to allow reproducibility of information and conclusions relevant for the project. To the extent possible, this will include identification of data inputs required for the analysis software, dependent software packages (and versions) used for analysis, and generated output from the source code. This information can be identified with inline comments in the source code. Use of version control software as hosted on GitHub is strongly encouraged for the delivery of source code. Project managers can work with TBEP staff to create a shared repository at https://github.com/tbep-tech.
### Metadata requirements
All data products are to be accompanied with appropriate metadata. Formal metadata schema can be used following the EML standard (preferred, \@ref(eml)) or as more general text files (acceptable but not preferred). A workflow for metadata generation as described in the SOP can be used (\@ref(metadata-workflow)). Plain text descriptions for metadata can include data dictionaries to aid in the interpretation of data deliverables (\@ref(dict))
### Delivery of data products
Data can be delivered through online sharing platforms that enable products to be findable, accessible, interoperable, and reusable (\@ref(fair)). These can include formal data repositories that employ standardized metadata schema (e.g., [DataOne](https://www.dataone.org/)). Small datasets (<100 Mb) can also be hosted on the TBEP GitHub website (https://github.com/tbep-tech) by contacting staff for access. View the guidance [here](https://tbep-tech.github.io/data-management-sop/keys.html#datahome) for other acceptable alternatives. All data products should have an associated DOI as a globally unique and persistent identifier (through https://zenodo.org/). An appropriate license should also be attached to the data.