Skip to content

Latest commit

 

History

History
404 lines (271 loc) · 14 KB

CONTRIBUTING.md

File metadata and controls

404 lines (271 loc) · 14 KB
Raw

Contribution Guide

Thanks a lot for your interest and taking the time to contribute to the surfgeopy project!

This document provides guidelines for contributing to the surfgeopy project.

Installation

This installation guide is focused on development. For installing surfgeopy in production runs check out the README.md.

In order to get the source of latest release, clone the surfgeopy repository from the [HZDR GitLab]:

git clone https://gitlab.hzdr.de/interpol/surfgeopy.git

By default, the cloned branch is the main branch.

To get the latest development version, checkout to the dev branch:

git checkout dev

We recommend to always pull the latest commit:

git pull origin dev

You are not allowed to directly push to dev or main branch. Please follow the instructions under Branching workflow.

Virtual environments

Following a best practice in Python development, we strongly encourage you to create and use virtual environments for development and production runs. A virtual environment encapsulates the package and all dependencies without messing up your other Python installations.

The following instructions should be executed from the surfgeopy source directory.

Using venv from the python standard library:

  1. Build a virtual environment:

    python -m venv <your_venv_name>

    Replace <you_venv_name> with an environment name of your choice.

  2. Activate the environment you just created:

    source <your_venv_name>/bin/activate

    as before replace <you_venv_name> with the environment name.

  3. To deactivate the virtual environment, type:

    deactivate

Using virtualenv:

  1. Building a virtual environment:

    virtualenv <your_venv_name>

    Replace <you_venv_name> with an environment name of your choice.

  2. Activate the environment:

    source <your_venv_name>/bin/activate

    As before replace <you_venv_name> with the environment name.

  3. To deactivate the virtual environment, type:

    deactivate

Using pyenv-virtualenv:

  1. Building the virtual environment:

    pyenv virtualenv 3.8 <your_venv_name>

    Replace <you_venv_name> with an environment name of your choice.

  2. Activate the newly created environment in the current local directory:

    pyenv local <your_venv_name>

    As before replace <you_venv_name> with the environment name.

    The above command creates a hidden file .python_version containing a "link" to the actual virtual environment managed by pyenv.

  3. To "deactivate" the virtual environment just remove this hidden file:

    rm .python_version

Using conda:

  1. Create an environment surfgeopy with the help of [conda]https://conda.io/projects/conda/en/latest/index.html) and the file environment.yaml (included in the source distribution of surfgeopy):

    conda env create -f environment.yaml

    The command creates a new conda environment called surfgeopy.

  2. Activate the new environment with:

    conda activate surfgeopy

    You may need to initialize conda env; follow the instructions printed out or read the conda docs.

  3. To deactivate the conda environment, type:

    conda deactivate

Installation

We recommend using pip from within a virtual environment (see above) to install surfgeopy.

To install surfgeopy, type:

pip install [-e] .[all,dev,docs]

where the flag -e means the package is directly linked into the Python site-packages. The options [all,dev,docs] refer to the requirements defined in the options.extras_require section in setup.cfg.

You must not use python setup.py install, since the file setup.py will not be present for every build of the package.

Troubleshooting: pytest with venv (not conda)

After installation, you might need to restart your virtual environment since the pytest command uses the PYTHONPATH environment variable which not automatically change to your virtual environment.

To restart your virtual environment created by venv, type:

deactivate && source <your_venv_name>/bin/activate

or run hash -r instead.

This problem does not seem to appear for virtual environments created by conda.

Dependency Management & Reproducibility (conda)

Here are a few recommendations for managing dependency and maintaining the reproducibility of your surfgeopy development environment:

  1. Always keep your abstract (unpinned) dependencies updated in environment.yaml and eventually in setup.cfg if you want to ship and install your package via pip later on.

  2. Create concrete dependencies as environment.lock.yaml for the exact reproduction of your environment with:

    conda env export -n surfgeopy -f environment.lock.yaml

    For multi-OS development, consider using --no-builds during the export.

  3. Update your current environment with respect to a new environment.lock.yaml using:

    conda env update -f environment.lock.yaml --prune

Running the unit tests

We use pytest to run the unit tests of surfgeopy. The unit tests themselves must always be placed into the tests directory. To run all tests, type:

pytest tests/test.py

from within the surfgeopy source directory.

When you run pytest, the coverage test is also done automatically. A summary of the coverage test is printed out in the terminal.

Writing new tests

We strongly encourage you to use the capabilities of pytest for writing the unit tests

It is highly recommended to use the capabilities of pytest for writing unittests.

Be aware of the following points:

  • the developer of the code should write the tests
  • test the behavior you expect from your code, not breaking points
  • use as small samples as possible
  • unit tests do not test if the code works, they test if the code still works
  • the coverage shall always be as high as possible
  • BUT, even 100% coverage does not mean, there is nothing missed (buzz: edge case!)

For additional reference on how to write tests, have a look at the following resources:

Documentation

This section provides some information about contributing to the docs.

Install dependencies

Building the docs requires additional dependencies. If you follow the above installation steps, the dependencies are satisfied. Otherwise you need to install them separately via:

pip install .[docs]

from the surfgeopy source directory.

Building the documentation

We use sphinx to build the surfgeopy docs. To build the docs in HTML format, run the following command:

sphinx-build -M html docs docs/build

Alternatively you can build the docs using the supplied Makefile. For that, you need to navigate to the docs directory and run the make command in Linux/mac OS or make.bat in Windows:

cd docs
make html

The command builds the docs and stores it in in docs/build. You may open the docs using a web browser of your choice by opening docs/build/html/index.html.

Design of the docs

The source files for the docs are stored in the docs directory. The Sphinx configuration file is docs/conf.py and the main index file of the docs is docs/index.rst.

The docs itself contains two different main sections:

  • The examples (docs/examples) contains all the tutorials of surfgeopy.
  • The modules(docs/modules) contains the reference to all exposed components of surfgeopy.

Pre-commit

For further developments, it is recommended to run all pre-commit-hooks every time before committing some changes to your branch.

To enable this, type:

pre-commit install

If you want to disable the pre-commit script, type:

pre-commit uninstall

To run all hooks defined in .pre-commit-config.yaml, type:

pre-commit run --all-files # DON'T DO THIS IF YOU DON'T KNOW WHAT HAPPENS

In the current state of the code, you should use this with caution since it might change code in the manner that it breaks (see below).

Down the road, we shall try to fulfill the full set of pre-commit hoos. However, further developments shall try to fulfil the full set of pre-commit-hooks.

Currently defined hooks

The following hooks are defined:

  • black: a straightforward code formatter; it modifies the code in order to fulfill the format requirement.
  • pre-commit-hooks: A collection of widely used hooks; see their repository for more informations.
  • isort: sorts the import statements; changes the code !!!DO NOT RUN THIS: IT WILL BREAK THE CURRENT VERSION!!!
  • pyupgrade: convert the syntax from Python2 to Python3. It's nice if you use code from an old post in stackoverflow ;-)
  • setup-cfg-fmt: formats the setup.cfg file for consistency.
  • flake8: a collection of hooks to ensure most of the PEP8 guidelines are satisfied. The concrete checks are defined in the setup.cfg[flake8].
  • mypy: a static type checker; mypy itself is configured in the setup.cfg[mypy-*].
  • check-manifest: checks if the MANIFEST.in is in a proper state. This ensures proper builds for uploading the package to PyPI. This is configured in setup.cfg[check-manifest].

Code development

Version control

We only use git to version control surfgeopy. The main repository for development is place on HZDR GitLab. Moreover, the releases and the development branch are also mirrored into the CASUS GitHub repository.

We are currently considering to upload the builds of surfgeopy to PyPI and conda-forge to make the code more accessible.

Branching workflow

We loosely follow the structure of Gitflow for our branching workflow. There are three types of branches in this workflow:

  1. main branch: On this branch, only the releases are stored. This means, on this branch, one has only fully tested, documented and cleaned up code.

  2. dev branch: On this branch, the development version are stored. At any given time, the branch must pass all the tests. This also means that on this branch, there is always a running version of surfgeopy even if the code and the docs are not in a "release state."

  3. feature branches: On these branches, all the features and code developments happen. These branches must always be branched from the dev branch (not from main).

Based on this workflow, you can freely push, change, and merge only on the feature branches. Furthermore, your feature branch is open to every developers in the surfgeopy project.

Once the implementation of a feature is finished, you can merge the feature branch to the dev branch via a merge request. The project maintainers will merge your merge request once the request is reviewed. In general, you cannot merge your feature branch directly to the dev branch.

Furthermore, as a contributor, you cannot merge directly to the main branch and you cannot make a merge request for that. Only the project maintainers can merge the dev to the main branch following the release procedure of Gitflow.

We manage the bug fixes on every branch separately with the relevant developers, usually via hotfix branches to implement the patches.

In the future, we may set up a continuous integration and development (CI/CD) on HZDR GitLab.

Project Organization

├── AUTHORS.md              <- List of developers and maintainers.
├── CHANGELOG.md            <- Changelog to keep track of new features and fixes.
├── LICENSE                 <- License as chosen on the command-line.
├── README.md               <- The top-level README for developers.
├── docs                    <- Directory for Sphinx documentation in rst or md.
├── environment.yaml        <- The conda environment file for reproducibility.
├── setup.cfg               <- Declarative configuration of your project.
├── setup.py                <- Use `python setup.py develop` to install for development or
|                              or create a distribution with `python setup.py bdist_wheel`.
├── surfgeopy            <- Actual Python package where the main functionality goes.
├── tests                   <- Unit tests which can be run with `py.test`.
├── pyproject.toml          <- Specification build requirements
├── MANIFEST.in             <- Keep track of (minimal) source distribution files
├── CONTRIBUTING.md         <- Contribution guidelines.
├── .readthedocs.yml        <- Configuration of readthedocs support
├── .gitignore              <- ignored files/directories if `git add/commit`
└── .pre-commit-config.yaml <- Configuration of pre-commit git hooks.