Development

Codebase

The pygeoapi codebase exists at https://github.com/geopython/pygeoapi.

Testing

pygeoapi uses pytest for managing its automated tests. Tests exist in /tests and are developed for providers, formatters, processes, as well as the overall API.

Tests can be run locally as part of development workflow. They are also run on pygeoapi’s GitHub Actions setup against all commits and pull requests to the code repository.

To run all tests, simply run pytest in the repository. To run a specific test file, run pytest tests/test_api.py, for example.

CQL extension lifecycle

Limitations

This workflow is valid only for the CQL-JSON format.

Schema

The Common Query Language (CQL) is the part 3 of the standard OGC API - Features. This extension has its specification available at OGC API - Features - Part 3: Filtering and the Common Query Language (CQL) and the schema exists in development at cql.json.

Model generation

pygeoapi uses a class-based Python model interface to translate the schema into Python objects defined by pydantic models. The model is generated with the pre-processing of the schema through the utility datamodel-codegen, which is part of the datamodel-code-generator package:

# Generate from local downloaded json schema file
datamodel-codegen  --input ~/Download/cql-schema.json --input-file-type jsonschema --output ./pygeoapi/models/cql_update.py --class-name CQLModel

Note that datamodel-code-generator must be explicitly installed, as it is not a pygeoapi runtime dependency

How to merge

Once the new pydantic models have been generated then the content of the Python file cql_update.py can be used to replace the old classes within the cql.py file. Update everything above the function get_next_node and then verify if the tests for the CQL are still passing, for example test_post_cql_json_between_query in tests/test_elasticsearch__provider.py.

Working with Spatialite on OSX

Using pyenv

It is common among OSX developers to use the package manager homebrew for the installation of pyenv to being able to manage multiple versions of Python. They can encounter errors about the load of some SQLite extensions that pygeoapi uses for handling spatial data formats. In order to run properly the server you are required to follow these steps below carefully.

Make Homebrew and pyenv play nicely together:

# see https://github.com/pyenv/pyenv/issues/106
alias brew='env PATH=${PATH//$(pyenv root)\/shims:/} brew'

Install Python with the option to enable SQLite extensions:

LDFLAGS="-L/usr/local/opt/sqlite/lib -L/usr/local/opt/zlib/lib" CPPFLAGS="-I/usr/local/opt/sqlite/include -I/usr/local/opt/zlib/include" PYTHON_CONFIGURE_OPTS="--enable-loadable-sqlite-extensions" pyenv install 3.7.6

Configure SQLite from Homebrew over that one shipped with the OS:

export PATH="/usr/local/opt/sqlite/bin:$PATH"

Install Spatialite from Homebrew:

brew update
brew install spatialite-tools
brew libspatialite

Set the variable for the Spatialite library under OSX:

SPATIALITE_LIBRARY_PATH=/usr/local/lib/mod_spatialite.dylib

Using pre-commit

You may optionally use pre-commit in order to check for linting and other static issues before committing changes. Pygeoapi’s repo includes a .pre-commit.yml file, check the pre-commit docs on how to set it up - in a nutshell:

  • pre-commit is mentioned in pygeoapi’s requirements-dev.txt file, so it will be included when you pip install those

  • run pre-commit install once in order to install its git commit hooks.

  • optionally, run pre-commit run --all-files, which will run all pre-commit hooks for all files in the repo. This also prepares the pre-commit environment.

  • from now on, whenever you do a git commit, the pre-commit hooks will run and the commit will only be done if all checks pass