django-geostore¶

Dynamic geographic datastore with Vector Tiles generation from PostGIS and json schema definition and validation.

Installation¶

Requirements¶

Minimum configuration :

Python 3.6+
PostgreSQL 10
PostGIS 2.4

And if you want to use Routing :

PgRouting 2.5 + django-geostore-routing

Recommended configuration :

Python 3.8
PostgreSQL 11
PostGIS 2.5

And if you want to use Routing :

PgRouting 2.6 + django-geostore-routing

Your final django project should use django.contrib.gis.backend.postgis as default DATABASE backend

USING database docker image :

https://hub.docker.com/r/postgis

or

https://hub.docker.com/r/pgrouting

SYSTEM REQUIREMENTS¶

these are debian packages required

libpq-dev (psycopg2)
gettext (translations)
binutils (django.contrib.gis)
libproj-dev (django.contrib.gis)
gdal-bin (django.contrib.gis)

recommended

postgresql-client (if you want to use ./manage.py dbshell command)

With pip¶

From Pypi:

pip install django-geostore

From Github:

pip install -e https://github.com/Terralego/django-geostore.git@master#egg=geostore

With git¶

git clone https://github.com/Terralego/django-geostore.git
cd django-geostore
python setup.py install

Configuration¶

In your project :

Add geostore to your INSTALLED_APPS :

# install required apps
INSTALLED_APPS = [
    ...
    'django.contrib.gis',  # assume contrib.gis is installed
    ...
    'rest_framework',
    'rest_framework_gis',
    'geostore',
    ...
]

Settings¶

Warning

Geostore will change the geojson serializer on app loading.

INTERNAL_GEOMETRY_SRID¶

Default: 4326

It’s the installation SRID, it must be set before the first migration and never change after installation, else you must create your own migrations to change your database SRID.

HOSTNAME¶

Default: empty

Used to feed TERRA_TILES_HOSTNAMES setting

TERRA_TILES_HOSTNAMES¶

Default: [HOSTNAME, ]

It contains the list of base URLs where are served the vector tiles. Since web browsers limit the number of connections to one domain name, a workaround is to use many domains to serve vector tiles, so browser will create more tcp connections, and the tiles loading will be faster.

MAX_TILE_ZOOM¶

Default: 15

It represent the max authorized zoom, if a tile with a zoom above this setting is requested, geostore will refuse to serve it.

MIN_TILE_ZOOM¶

Default: 10

Like for MAX_TILE_ZOOM setting, if a tile of a lesser zoom than this setting is requested, backend will refuse to serve it.

GEOSTORE_LAYER_VIEWSSET¶

Default: ‘geostore.views.LayerViewSet’

Python dotted path to LayerViewSet. Can be any class inherited from ‘geostore.views.LayerViewSet’

GEOSTORE_LAYER_SERIALIZER¶

Default: ‘geostore.serializers.LayerSerializer’

Python dotted path to LayerSerializer. Can be any class inherited from ‘geostore.serializers.LayerSerializer’

GEOSTORE_EXPORT_CELERY_ASYNC¶

Default: False If your project use a celery worker, set to True to enable async exports. URLS will be provided in API, calling these urls will launch asynchronous exports and send email with a link for user download.

URLs¶

Add to you urls.py file this pattern:

urlpatterns = [
    ...
    path('', include('geostore.urls', namespace='geostore')),
    ...
]

You can customize default url and namespace by including geostore.views directly

Admin¶

you can disable and / or customize admin

QUICK START¶

Manage layers¶

The simplest way to create a geographic data layer :

from geostore import GeometryTypes
from geostore.models import Layer

layer = Layer.objects.create(name='Mushroom spot',
                             geom_type=GeometryTypes.Point)

Geometry type validation¶

Layer support these geometry types :

Supported types¶

geostore.GeometryTypes

GeometryCollection = 7 LineString = 1 MultiLineString = 5 MultiPoint = 4 MultiPolygon = 6 Point = 0 Polygon = 3

Define a geometry type to layer to force feature geometry validation.

Without validation¶

from geostore.models import Layer, Feature
from geostore import GeometryTypes
from django.contrib.geos.geometries import GEOSGeometry

layer = Layer.objects.create(name='Mushroom spot 2')
feature = Feature(layer=layer,
                  geom=GEOSGeometry("POINT(0 0)")
feature.clean()  # ok
# then, you can save
feature.save()
feature = Feature(layer=layer,
                  geom=GEOSGeometry("LINESTRING((0 0), (1 1))")

feature.clean()  # ok too
feature.save()

With validation¶

from geostore.models import Layer, Feature
from geostore import GeometryTypes
from django.contrib.geos.geometries import GEOSGeometry

layer = Layer.objects.create(name='Mushroom spot 3',
                             geom_type=GeometryTypes.Point)
feature = Feature(layer=layer,
                  geom=GEOSGeometry("POINT(0 0)")

feature.clean()  # ok
feature.save()
feature = Feature(layer=layer,
                  geom=GEOSGeometry("LINESTRING((0 0), (1 1))")
feature.clean()  # validation error !

JSON schema definition / validation¶

You can use json schema definition to describe your data content, and improve feature properties validation.

https://json-schema.org/ https://rjsf-team.github.io/react-jsonschema-form/

from geostore.models import Layer, Feature
from geostore import GeometryTypes
from django.contrib.geos.geometries import GEOSGeometry

layer = Layer.objects.create(name='Mushroom spot 4',
                             geom_type=GeometryTypes.Point,
                             schema={
                               "required": ["name", "age"],
                               "properties": {
                                 "name": {
                                   "type": "string",
                                   "title": "Name"
                                 },
                                 "age": {
                                   "type": "integer",
                                   "title": "Age"
                                 }
                               }
                             })
feature = Feature(layer=layer,
                  geom=GEOSGeometry("POINT(0 0)")
feature.clean()  # Validation Error ! name and age are required

feature = Feature(layer=layer,
                  geom=GEOSGeometry("POINT(0 0)",
                  properties={
                      "name": "Arthur",
                  })
feature.clean()  # Validation Error ! age is required

feature = Feature(layer=layer,
                  geom=GEOSGeometry("POINT(0 0)",
                  properties={
                    "name": "Arthur",
                    "age": "ten",
                  })
feature.clean()  # Validation Error ! age should be integer

feature = Feature(layer=layer,
                  geom=GEOSGeometry("POINT(0 0)",
                  properties={
                    "name": "Arthur",
                    "age": 10
                  })
feature.clean()  # ok !
feature.save()

Vector tiles¶

geostore provide endpoint to generate and cache MVT based on your data.

You can access these tiles through Layer and LayerGroup features.

On layers¶

On group of layers¶

Relations¶

You can define relations between layers (and linked features)

Warning

Compute relations need celery project and worker configured in your project. Run at least 1 worker. You need to fix settings explicitly to enable asynchronous tasks. GEOSTORE_RELATION_CELERY_ASYNC = True

Manual relation¶

No automatic links between features. You need to create yourself FeatureRelation between Features.

Automatic relations¶

If any celery project worker is available, and GEOSTORE_RELATION_CELERY_ASYNC settings set to True, each layer relation creation or feature edition will launch async task to update relation between linked features.

Intersects¶

By selecting intersects, each feature in origin layer intersecting geometry features in destination layer, will be linked to them.

Distance¶

By selecting distance, each feature in origin layer with distance max geometry features in destination layer, will be linked to them.

Warning

You need to define distance in settings: {“distance”: 10000} # for 10km

Data import¶

ShapeFile¶

GeoJSON¶

Data export¶

API endpoints¶

Vector Tiles¶

Vector tiles are served following the Mapbox Vector Tiles standard, and using the ST_AsMVT Postgis method.

Most of the work is done in the geostore.tiles.helpers module.

Settings¶

Vector tiles can be served in many ways, and it generation can be configured. This allow you to manage which data is returned, but also some tunning settings.

The Layer models has a settings attribute which is a JSONField.

Here we describe available json keys and its content, then we provide your an example.

metadata¶

Contains all data metadata that can be added to tile content, it allows you to store it in a convenient way.

attribution¶

Default: None

Attribution of the layer’s data. Must be a dict like this:

{'name': 'OSM contributors', href='http://openstreetmap.org'}

licence¶

Default: None

String containing the layer’s data licence. i.e.: ODbL, CC-BY, Public Domain, …

description¶

Default: None

Text that describe the data.

tiles¶

minzoom¶

Default: 0

Min zoom when the layer is served in tiles. Must be higher or equal to MIN_ZOOM setting.

maxzoom¶

Default: 22

Max zoom when the layer is served in tiles. Must be lower or equal to MAX_ZOOM setting.

pixel_buffer¶

Default: 4

Buffer size around a tile, to match more features and clip features at a larger size than the tile.

Mostly, the default value is enough, but sometimes, depending of the display style (width border of lines or polygons), you will need to increase this value.

features_filter¶

Default: None

Filter the features queryset, by this value. Could be used to not return all features of your layers on the tiles.

The complete object is passed to a filter(properties__contains) method

properties_filter¶

Default: None

List of allowed properties in tiles. This must be a list of properties that will be the only one present in vector tiles. If set to None, all properties will be returned, else only properties present in the list will be returned.

features_limit¶

Default: 10000

Maximal number of features in a tile. Used to prevent tiles to have too much data, since MVT standard tells a tile must not be high than 500ko.

Example¶

{
      'metadata': {
          'attribution': {'name': 'OSM contributors', href='http://openstreetmap.org'}
          'licence': 'ODbL,
          'description': "Good Licence",
      },
      # Tilesets attributes
      'tiles': {
          'minzoom': 10,
          'maxzoom': 14,
          'pixel_buffer': 4,
          'features_filter': 500,
          'properties_filter': ['my_property', ],
          'features_limit': 10000,
      }
}

Vector Tiles Group Access¶

Django-Geostore has a mecanism to authorize only some django’s user Groups to access layer’s on vector tiles.

This can be used to manage layer access through vector tiles.

Here we’re going to describe how it works.

Where to add a group¶

Each layer has a ManyToMany relationship to django’s Group model, that authorized only users present is thoses groups to have access to thoses layers through vector tiles.

You can add a group, with the normal django’s ORM API:

from django.contrib.auth.models import Group
from geostore.models import Layer
g = Group.objects.first()
l = Layer.objects.first()

l.authorized_groups.add(g)

Then ?¶

Then, you can generate the autheticated URL by using a QueryString like above, where user_groups are a list of user_groups names, and layergroup is the group of the layer:

from geostore.tokens import tiles_token_generator
querystring = QueryDict(mutable=True)
querystring.update(
{
    "idb64": tiles_token_generator.token_idb64(
        user_groups, layergroup
    ),
    "token": tiles_token_generator.make_token(
        user_groups, layergroup
    ),
}
)

tilejson_url = reverse("group-tilejson", args=(layergroup.slug,))
authenticated_url = f"{tilejson_url}?{querystring.urlencode()}"

You’ll have available an authenticated url, this will filter layers in tiles that are accessible to the authenticated user groups.

All authenticated informations will be provided by the authenticated tilejson, that will provide to frontend all authenticated urls.

Usually, mapbox needs only the tilejson, geostore will do all the remaining work.

Plugins¶

Routing¶

django-geostore-routing integrate a way to use your LineString layer as a routing one. It uses pgRouting as backend.

Install it with :

pip install django-geostore-routing

and enable-it in your INSTALLED_APPS :

INSTALLED_APPS = (
    ...
    'geostore',
    'geostore_routing',
    ...
)

Full documentation : https://django-geostore-routing.readthedocs.io/

Repository : https://github.com/Terralego/django-geostore-routing