Customizing pygeoapi: plugins

In this section we will explain how pygeoapi provides plugin architecture for data providers, formatters and processes.

Plugin development requires knowledge of how to program in Python as well as Python’s package/module system.

Overview

pygeoapi provides a robust plugin architecture that enables developers to extend functionality. Infact, pygeoapi itself implements numerous formats, data providers and the process functionality as plugins.

The pygeoapi architecture supports the following subsystems:

  • data providers

  • output formats

  • processes

The core pygeoapi plugin registry can be found in pygeoapi.plugin.PLUGINS.

Each plugin type implements its relevant base class as the API contract:

  • data providers: pygeoapi.provider.base

  • output formats: pygeoapi.formatter.base

  • processes: pygeoapi.process.base

Todo

link PLUGINS to API doc

Plugins can be developed outside of the pygeoapi codebase and be dynamically loaded by way of the pygeoapi configuration. This allows your custom plugins to live outside pygeoapi for easier maintenance of software updates.

Note

It is recommended to store pygeoapi plugins outside of pygeoapi for easier software updates and package management

Example: custom pygeoapi vector data provider

Lets consider the steps for a vector data provider plugin (source code is located here: Provider).

Python code

The below template provides a minimal example (let’s call the file mycoolvectordata.py:

from pygeoapi.provider.base import BaseProvider

class MyCoolVectorDataProvider(BaseProvider):
    """My cool vector data provider"""

    def __init__(self, provider_def):
        """Inherit from parent class"""

        super().__init__(provider_def)

    def get_fields(self):

        # open dat file and return fields and their datatypes
        return {
            'field1': 'string',
            'field2': 'string'
        }

    def query(self, offset=0, limit=10, resulttype='results',
              bbox=[], datetime_=None, properties=[], sortby=[],
              select_properties=[], skip_geometry=False, **kwargs):

        # optionally specify the output filename pygeoapi can use as part
        # of the response (HTTP Content-Disposition header)
        self.filename = "my-cool-filename.dat"

        # open data file (self.data) and process, return
        return {
            'type': 'FeatureCollection',
            'features': [{
                'type': 'Feature',
                'id': '371',
                'geometry': {
                    'type': 'Point',
                    'coordinates': [ -75, 45 ]
                },
                'properties': {
                    'stn_id': '35',
                    'datetime': '2001-10-30T14:24:55Z',
                    'value': '89.9'
                }
            }]
        }

    def get_schema():
        # return a `dict` of a JSON schema (inline or reference)
        return ('application/geo+json', {'$ref': 'https://geojson.org/schema/Feature.json'})

For brevity, the above code will always return the single feature of the dataset. In reality, the plugin developer would connect to a data source with capabilities to run queries and return a relevant result set, as well as implement the get method accordingly. As long as the plugin implements the API contract of its base provider, all other functionality is left to the provider implementation.

Each base class documents the functions, arguments and return types required for implementation.

Note

You can add language support to your plugin using these guides.

Connecting to pygeoapi

The following methods are options to connect the plugin to pygeoapi:

Option 1: Update in core pygeoapi:

  • copy mycoolvectordata.py into pygeoapi/provider

  • update the plugin registry in pygeoapi/plugin.py:PLUGINS['provider'] with the plugin’s shortname (say MyCoolVectorData) and dotted path to the class (i.e. pygeoapi.provider.mycoolvectordata.MyCoolVectorDataProvider)

  • specify in your dataset provider configuration as follows:

providers:
    - type: feature
      name: MyCoolVectorData
      data: /path/to/file
      id_field: stn_id

Option 2: implement outside of pygeoapi and add to configuration (recommended)

  • create a Python package of the mycoolvectordata.py module (see Cookiecutter as an example)

  • install your Python package onto your system (python setup.py install). At this point your new package should be in the PYTHONPATH of your pygeoapi installation

  • specify in your dataset provider configuration as follows:

providers:
    - type: feature
      name: mycooldatapackage.mycoolvectordata.MyCoolVectorDataProvider
      data: /path/to/file
      id_field: stn_id

Note

The United States Geological Survey has created a Cookiecutter project for creating pygeoapi plugins. See the pygeoapi-plugin-cookiecutter project to get started.

Example: custom pygeoapi raster data provider

Lets consider the steps for a raster data provider plugin (source code is located here: Provider).

Python code

The below template provides a minimal example (let’s call the file mycoolrasterdata.py:

from pygeoapi.provider.base import BaseProvider

class MyCoolRasterDataProvider(BaseProvider):
    """My cool raster data provider"""

    def __init__(self, provider_def):
        """Inherit from parent class"""

        super().__init__(provider_def)
        self.num_bands = 4
        self.axes = ['Lat', 'Long']

    def get_coverage_domainset(self):
        # return a CIS JSON DomainSet

    def get_coverage_rangetype(self):
        # return a CIS JSON RangeType

    def query(self, bands=[], subsets={}, format_='json', **kwargs):
        # process bands and subsets parameters
        # query/extract coverage data

        # optionally specify the output filename pygeoapi can use as part
        of the response (HTTP Content-Disposition header)
        self.filename = "my-cool-filename.dat"

        if format_ == 'json':
            # return a CoverageJSON representation
            return {'type': 'Coverage', ...}  # trimmed for brevity
        else:
            # return default (likely binary) representation
            return bytes(112)

For brevity, the above code will always JSON for metadata and binary or CoverageJSON for the data. In reality, the plugin developer would connect to a data source with capabilities to run queries and return a relevant result set, As long as the plugin implements the API contract of its base provider, all other functionality is left to the provider implementation.

Each base class documents the functions, arguments and return types required for implementation.

Example: custom pygeoapi formatter

Python code

The below template provides a minimal example (let’s call the file mycooljsonformat.py:

import json
from pygeoapi.formatter.base import BaseFormatter

class MyCoolJSONFormatter(BaseFormatter):
    """My cool JSON formatter"""

    def __init__(self, formatter_def):
        """Inherit from parent class"""

        super().__init__({'name': 'cooljson', 'geom': None})
        self.mimetype = 'application/json; subtype:mycooljson'

    def write(self, options={}, data=None):
        """custom writer"""

        out_data {'rows': []}

        for feature in data['features']:
            out_data.append(feature['properties'])

        return out_data

Processing plugins

Processing plugins are following the OGC API - Processes development. Given that the specification is under development, the implementation in pygeoapi/process/hello_world.py provides a suitable example for the time being.