Periodically refresh Datasette metadata from a remote URL

Install this plugin in the same environment as Datasette.

$ datasette install datasette-remote-metadata

Add the following to your metadata.json:

{
    "plugins": {
        "datasette-remote-metadata": {
            "url": "https://example.com/remote-metadata.yml"
        }
    }
}

The plugin will fetch the specified metadata from that URL at startup and combine it with any existing metadata. You can use a URL to either a JSON file or a YAML file.

It will periodically refresh that metadata - by default every 30 seconds, unless you specify an alternative "ttl" value in the plugin configuration.

Available configuration options are as follows:

• "url" - the URL to retrieve remote metadata from. Can link to a JSON or a YAML file.
• "ttl" - integer value in secords: how frequently should the script check for fresh metadata. Defaults to 30 seconds.
• "headers" - a dictionary of additional request headers to send.
• "cachebust" - if true, a random ?0.29508 value will be added to the query string of the remote metadata to bust any intermediary caches.

This example metadata.json configuration refreshes every 10 seconds, uses cache busting and sends an Authorization: Bearer xyz header with the request:

{
    "plugins": {
        "datasette-remote-metadata": {
            "url": "https://example.com/remote-metadata.yml",
            "ttl": 10,
            "cachebust": true,
            "headers": {
                "Authorization": "Bearer xyz"
            }
        }
    }
}

This example if you are using metadata.yaml for configuration:

plugins:
  datasette-remote-metadata:
    url: https://example.com/remote-metadata.yml
    ttl: 10
    cachebust: true
    headers:
      Authorization: Bearer xyz

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd datasette-remote-metadata
python3 -mvenv venv
source venv/bin/activate

Or if you are using pipenv:

pipenv shell

Now install the dependencies and test dependencies:

pip install -e '.[test]'

To run the tests:

pytest