django-geostore¶
Dynamic geographic datastore with Vector Tiles generation from PostGIS and json schema definition and validation.
Installation¶
Requirements¶
- Minimum configuration :
- Python 3.6+
- PostgreSQL 10
- PostGIS 2.4
And if you want to use Routing :
- PgRouting 2.5 + django-geostore-routing
- Recommended configuration :
- Python 3.8
- PostgreSQL 11
- PostGIS 2.5
And if you want to use Routing :
- PgRouting 2.6 + django-geostore-routing
Your final django project should use django.contrib.gis.backend.postgis as default DATABASE backend
USING database docker image :
https://hub.docker.com/r/postgis
or
https://hub.docker.com/r/pgrouting
SYSTEM REQUIREMENTS¶
these are debian packages required
- libpq-dev (psycopg2)
- gettext (translations)
- binutils (django.contrib.gis)
- libproj-dev (django.contrib.gis)
- gdal-bin (django.contrib.gis)
recommended
- postgresql-client (if you want to use ./manage.py dbshell command)
With pip¶
From Pypi:
pip install django-geostore
From Github:
pip install -e https://github.com/Terralego/django-geostore.git@master#egg=geostore
With git¶
git clone https://github.com/Terralego/django-geostore.git
cd django-geostore
python setup.py install
Configuration¶
In your project :
Add geostore to your INSTALLED_APPS
:
# install required apps
INSTALLED_APPS = [
...
'django.contrib.gis', # assume contrib.gis is installed
...
'rest_framework',
'rest_framework_gis',
'geostore',
...
]
Settings¶
Warning
Geostore will change the geojson serializer on app loading.
INTERNAL_GEOMETRY_SRID¶
Default: 4326
It’s the installation SRID, it must be set before the first migration and never change after installation, else you must create your own migrations to change your database SRID.
TERRA_TILES_HOSTNAMES¶
Default: [HOSTNAME, ]
It contains the list of base URLs where are served the vector tiles. Since web browsers limit the number of connections to one domain name, a workaround is to use many domains to serve vector tiles, so browser will create more tcp connections, and the tiles loading will be faster.
MAX_TILE_ZOOM¶
Default: 15
It represent the max authorized zoom, if a tile with a zoom above this setting is requested, geostore will refuse to serve it.
MIN_TILE_ZOOM¶
Default: 10
Like for MAX_TILE_ZOOM
setting, if a tile of a lesser zoom than this setting is requested, backend will refuse to serve it.
GEOSTORE_LAYER_VIEWSSET¶
Default: ‘geostore.views.LayerViewSet’
Python dotted path to LayerViewSet. Can be any class inherited from ‘geostore.views.LayerViewSet’
GEOSTORE_LAYER_SERIALIZER¶
Default: ‘geostore.serializers.LayerSerializer’
Python dotted path to LayerSerializer. Can be any class inherited from ‘geostore.serializers.LayerSerializer’
GEOSTORE_EXPORT_CELERY_ASYNC¶
Default: False If your project use a celery worker, set to True to enable async exports. URLS will be provided in API, calling these urls will launch asynchronous exports and send email with a link for user download.
QUICK START¶
Manage layers¶
The simplest way to create a geographic data layer :
from geostore import GeometryTypes
from geostore.models import Layer
layer = Layer.objects.create(name='Mushroom spot',
geom_type=GeometryTypes.Point)
Geometry type validation¶
Layer support these geometry types :
Supported types¶
geostore.GeometryTypes
GeometryCollection = 7 LineString = 1 MultiLineString = 5 MultiPoint = 4 MultiPolygon = 6 Point = 0 Polygon = 3
Define a geometry type to layer to force feature geometry validation.
Without validation¶
from geostore.models import Layer, Feature
from geostore import GeometryTypes
from django.contrib.geos.geometries import GEOSGeometry
layer = Layer.objects.create(name='Mushroom spot 2')
feature = Feature(layer=layer,
geom=GEOSGeometry("POINT(0 0)")
feature.clean() # ok
# then, you can save
feature.save()
feature = Feature(layer=layer,
geom=GEOSGeometry("LINESTRING((0 0), (1 1))")
feature.clean() # ok too
feature.save()
With validation¶
from geostore.models import Layer, Feature
from geostore import GeometryTypes
from django.contrib.geos.geometries import GEOSGeometry
layer = Layer.objects.create(name='Mushroom spot 3',
geom_type=GeometryTypes.Point)
feature = Feature(layer=layer,
geom=GEOSGeometry("POINT(0 0)")
feature.clean() # ok
feature.save()
feature = Feature(layer=layer,
geom=GEOSGeometry("LINESTRING((0 0), (1 1))")
feature.clean() # validation error !
JSON schema definition / validation¶
You can use json schema definition to describe your data content, and improve feature properties validation.
https://json-schema.org/ https://rjsf-team.github.io/react-jsonschema-form/
from geostore.models import Layer, Feature
from geostore import GeometryTypes
from django.contrib.geos.geometries import GEOSGeometry
layer = Layer.objects.create(name='Mushroom spot 4',
geom_type=GeometryTypes.Point,
schema={
"required": ["name", "age"],
"properties": {
"name": {
"type": "string",
"title": "Name"
},
"age": {
"type": "integer",
"title": "Age"
}
}
})
feature = Feature(layer=layer,
geom=GEOSGeometry("POINT(0 0)")
feature.clean() # Validation Error ! name and age are required
feature = Feature(layer=layer,
geom=GEOSGeometry("POINT(0 0)",
properties={
"name": "Arthur",
})
feature.clean() # Validation Error ! age is required
feature = Feature(layer=layer,
geom=GEOSGeometry("POINT(0 0)",
properties={
"name": "Arthur",
"age": "ten",
})
feature.clean() # Validation Error ! age should be integer
feature = Feature(layer=layer,
geom=GEOSGeometry("POINT(0 0)",
properties={
"name": "Arthur",
"age": 10
})
feature.clean() # ok !
feature.save()
Vector tiles¶
geostore provide endpoint to generate and cache MVT based on your data.
You can access these tiles through Layer and LayerGroup features.
On layers¶
On group of layers¶
Relations¶
- You can define relations between layers (and linked features)
Warning
Compute relations need celery project and worker configured in your project. Run at least 1 worker. You need to fix settings explicitly to enable asynchronous tasks. GEOSTORE_RELATION_CELERY_ASYNC = True
Manual relation¶
No automatic links between features. You need to create yourself FeatureRelation between Features.
Automatic relations¶
If any celery project worker is available, and GEOSTORE_RELATION_CELERY_ASYNC settings set to True, each layer relation creation or feature edition will launch async task to update relation between linked features.
Intersects¶
By selecting intersects, each feature in origin layer intersecting geometry features in destination layer, will be linked to them.
Distance¶
By selecting distance, each feature in origin layer with distance max geometry features in destination layer, will be linked to them.
Warning
You need to define distance in settings: {“distance”: 10000} # for 10km
Data export¶
API endpoints¶
Vector Tiles¶
Vector tiles are served following the Mapbox Vector Tiles standard, and using the ST_AsMVT Postgis method.
Most of the work is done in the geostore.tiles.helpers
module.
Settings¶
Vector tiles can be served in many ways, and it generation can be configured. This allow you to manage which data is returned, but also some tunning settings.
The Layer
models has a settings
attribute which is a JSONField
.
Here we describe available json keys and its content, then we provide your an example.
metadata¶
Contains all data metadata that can be added to tile content, it allows you to store it in a convenient way.
attribution¶
Default: None
Attribution of the layer’s data. Must be a dict like this:
{'name': 'OSM contributors', href='http://openstreetmap.org'}
licence¶
Default: None
String containing the layer’s data licence. i.e.: ODbL, CC-BY, Public Domain, …
tiles¶
minzoom¶
Default: 0
Min zoom when the layer is served in tiles. Must be higher or equal to MIN_ZOOM
setting.
maxzoom¶
Default: 22
Max zoom when the layer is served in tiles. Must be lower or equal to MAX_ZOOM
setting.
pixel_buffer¶
Default: 4
Buffer size around a tile, to match more features and clip features at a larger size than the tile.
Mostly, the default value is enough, but sometimes, depending of the display style (width border of lines or polygons), you will need to increase this value.
features_filter¶
Default: None
Filter the features queryset, by this value. Could be used to not return all features of your layers on the tiles.
The complete object is passed to a filter(properties__contains)
method
properties_filter¶
Default: None
List of allowed properties in tiles. This must be a list of properties that will be the only one present in vector tiles.
If set to None
, all properties will be returned, else only properties present in the list will be returned.
features_limit¶
Default: 10000
Maximal number of features in a tile. Used to prevent tiles to have too much data, since MVT standard tells a tile must not be high than 500ko.
Example¶
{
'metadata': {
'attribution': {'name': 'OSM contributors', href='http://openstreetmap.org'}
'licence': 'ODbL,
'description': "Good Licence",
},
# Tilesets attributes
'tiles': {
'minzoom': 10,
'maxzoom': 14,
'pixel_buffer': 4,
'features_filter': 500,
'properties_filter': ['my_property', ],
'features_limit': 10000,
}
}
Vector Tiles Group Access¶
Django-Geostore has a mecanism to authorize only some django’s user Groups to access layer’s on vector tiles.
This can be used to manage layer access through vector tiles.
Here we’re going to describe how it works.
Where to add a group¶
Each layer has a ManyToMany relationship to django’s Group model, that authorized only users present is thoses groups to have access to thoses layers through vector tiles.
You can add a group, with the normal django’s ORM API:
from django.contrib.auth.models import Group
from geostore.models import Layer
g = Group.objects.first()
l = Layer.objects.first()
l.authorized_groups.add(g)
Then ?¶
Then, you can generate the autheticated URL by using a QueryString like above, where user_groups are a list of user_groups names, and layergroup is the group of the layer:
from geostore.tokens import tiles_token_generator
querystring = QueryDict(mutable=True)
querystring.update(
{
"idb64": tiles_token_generator.token_idb64(
user_groups, layergroup
),
"token": tiles_token_generator.make_token(
user_groups, layergroup
),
}
)
tilejson_url = reverse("group-tilejson", args=(layergroup.slug,))
authenticated_url = f"{tilejson_url}?{querystring.urlencode()}"
You’ll have available an authenticated url, this will filter layers in tiles that are accessible to the authenticated user groups.
All authenticated informations will be provided by the authenticated tilejson, that will provide to frontend all authenticated urls.
Usually, mapbox needs only the tilejson, geostore will do all the remaining work.
Plugins¶
Routing¶
django-geostore-routing integrate a way to use your LineString layer as a routing one. It uses pgRouting as backend.
Install it with :
pip install django-geostore-routing
and enable-it in your INSTALLED_APPS :
INSTALLED_APPS = (
...
'geostore',
'geostore_routing',
...
)
Full documentation : https://django-geostore-routing.readthedocs.io/
Repository : https://github.com/Terralego/django-geostore-routing