Graphene Elasticsearch (DSL) integration
Project description
Elasticsearch (DSL) integration for Graphene.
Prerequisites
Graphene 2.x. Support for Graphene 1.x is not planned, but might be considered.
Python 3.6, 3.7. Support for Python 2 is not intended.
Elasticsearch 6.x, 7.x. Support for Elasticsearch 5.x is not intended.
Main features and highlights
Implemented ElasticsearchConnectionField and ElasticsearchObjectType are the core classes to work with graphene.
Pluggable backends for searching, filtering, ordering, etc. Don’t like existing ones? Override, extend or write your own.
Implemented search backend.
Implemented filter backend.
See the Road-map for what’s yet planned to implemented.
Documentation
Documentation is available on Read the Docs.
Installation
Install latest stable version from PyPI:
pip install graphene-elasticOr latest development version from GitHub:
pip install https://github.com/barseghyanartur/graphene-elastic/archive/master.zipExamples
Install requirements
pip install -r requirements.txtPopulate sample data
The following command will create indexes for User and Post documents and populate them with sample data:
./scripts/populate_elasticsearch_data.shSample document definition
search_index/documents/post.py
See examples/search_index/documents/post.py for full example.
import datetime
from elasticsearch_dsl import (
Boolean,
Date,
Document,
InnerDoc,
Keyword,
Nested,
Text,
Integer,
)
class Comment(InnerDoc):
author = Text(fields={'raw': Keyword()})
content = Text(analyzer='snowball')
created_at = Date()
def age(self):
return datetime.datetime.now() - self.created_at
class Post(Document):
title = Text(
fields={'raw': Keyword()}
)
content = Text()
created_at = Date()
published = Boolean()
category = Text(
fields={'raw': Keyword()}
)
comments = Nested(Comment)
tags = Text(
analyzer=html_strip,
fields={'raw': Keyword(multi=True)},
multi=True
)
num_views = Integer()
class Index:
name = 'blog_post'
settings = {
'number_of_shards': 1,
'number_of_replicas': 1,
'blocks': {'read_only_allow_delete': None},
}
def add_comment(self, author, content):
self.comments.append(
Comment(
author=author,
content=content,
created_at=datetime.datetime.now()
)
)
def add_tag(self, name):
self.tags.append(name)
def save(self, ** kwargs):
self.created_at = datetime.datetime.now()
return super().save(** kwargs)Sample apps
Sample Flask app
Run the sample Flask app:
./scripts/run_flask.shOpen Flask graphiql client
http://127.0.0.1:8001/graphqlSample Django app
Run the sample Django app:
./scripts/run_django.sh runserverOpen Flask graphiql client
http://127.0.0.1:8000/graphqlConnectionField example
ConnectionField is the most flexible and feature rich solution you have. It uses filter backends which you can tie to your needs the way you want in a declarative manner.
Sample schema definition
import graphene
from graphene_elastic import (
ElasticsearchObjectType,
ElasticsearchConnectionField,
)
from graphene_elastic.filter_backends import (
FilteringFilterBackend,
SearchFilterBackend,
)
from graphene_elastic.constants import (
LOOKUP_FILTER_PREFIX,
LOOKUP_FILTER_TERM,
LOOKUP_FILTER_TERMS,
LOOKUP_FILTER_WILDCARD,
LOOKUP_QUERY_EXCLUDE,
LOOKUP_QUERY_IN,
)
# Object type definition
class Post(ElasticsearchObjectType):
class Meta(object):
document = PostDocument
interfaces = (Node,)
filter_backends = [
FilteringFilterBackend,
SearchFilterBackend,
OrderingFilterBackend,
]
# For `FilteringFilterBackend` backend
filter_fields = {
'title': {
'field': 'title.raw',
'lookups': [
LOOKUP_FILTER_TERM,
LOOKUP_FILTER_TERMS,
LOOKUP_FILTER_PREFIX,
LOOKUP_FILTER_WILDCARD,
LOOKUP_QUERY_IN,
LOOKUP_QUERY_EXCLUDE,
],
'default_lookup': LOOKUP_FILTER_TERM,
},
'category': 'category.raw',
'tags': 'tags.raw',
'num_views': 'num_views',
}
# For `SearchFilterBackend` backend
search_fields = {
'title': {'boost': 4},
'content': {'boost': 2},
'category': None,
}
# For `OrderingFilterBackend` backend
ordering_fields = {
'id': None,
'title': 'title.raw',
'created_at': 'created_at',
'num_views': 'num_views',
}
# Query definition
class Query(graphene.ObjectType):
all_post_documents = ElasticsearchConnectionField(Post)
# Schema definition
schema = graphene.Schema(query=Query)Filter
Sample queries
Since we didn’t specify any lookups on category, by default all lookups are available and the default lookup would be term. Note, that in the {value:"Elastic"} part, the value stands for default lookup, whatever it has been set to.
query PostsQuery {
allPostDocuments(filter:{category:{value:"Elastic"}}) {
edges {
node {
id
title
category
content
createdAt
comments
}
}
}
}But, we could use another lookup (in example below - terms). Note, that in the {terms:["Elastic", "Python"]} part, the terms is the lookup name.
query PostsQuery {
allPostDocuments(filter:{
category:{terms:["Elastic", "Python"]}
}) {
edges {
node {
id
title
category
content
createdAt
comments
}
}
}
}Or apply a gt (range) query in addition to filtering:
{
allPostDocuments(filter:{
category:{term:"Python"},
numViews:{gt:"700"}
}) {
edges {
node {
category
title
comments
numViews
}
}
}
}Implemented filter lookups
The following lookups are available:
contains
ends_with (or endsWith for camelCase)
exclude
exists
geo_bounding_box (or geoBoundingBox for camelCase)
geo_distance (or geoDistance for camelCase)
geo_polygon (or geoPolygon for camelCase)
gt
gte
in
is_null (or isNull for camelCase)
lt
lte
prefix
range
starts_with (or startsWith for camelCase)
term
terms
wildcard
See dedicated documentation on filter lookups for more information.
Search
query {
allPostDocuments(
search:{
title:{value:"Release", boost:1},
content:{value:"Box"}
}}
) {
edges {
node {
category
title
comments
}
}
}
}Ordering
Possible choices are ASC and DESC.
{
allPostDocuments(filter:{
tags:{in:["photography", "models"]},
ordering:{title:ASC}
}) {
edges {
node {
category
title
content
numViews
tags
}
}
}
}Road-map
Road-map and development plans.
Lots of features are planned to be released in the upcoming Beta releases:
Ordering backend
Geo-spatial backend
Aggregations (faceted search) backend
Post-filter backend
Nested backend
Highlight backend
Suggester backend
Global aggregations backend
More-like-this backend
Complex search backends, such as Simple query search
Source filter backend
Stay tuned or reach out if you want to help.
Testing
Project is covered with tests.
By defaults tests are executed against the Elasticsearch 7.x.
Running Elasticsearch
Run Elasticsearch 7.x with Docker
docker-compose up elasticsearchRunning tests
Make sure you have the test requirements installed:
pip install -r requirements/test.txtTo test with all supported Python versions type:
toxTo test against specific environment, type:
tox -e py37To test just your working environment type:
./runtests.pyTo run a single test module in your working environment type:
./runtests.py src/graphene_elastic/tests/test_filter_backend.pyTo run a single test class in a given test module in your working environment type:
./runtests.py src/graphene_elastic/tests/test_filter_backend.py::FilterBackendElasticTestCaseDebugging
For development purposes, you could use the flask app (easy to debug). Standard pdb works (import pdb; pdb.set_trace()). If ipdb does not work well for you, use ptpdb.
Writing documentation
Keep the following hierarchy.
=====
title
=====
header
======
sub-header
----------
sub-sub-header
~~~~~~~~~~~~~~
sub-sub-sub-header
^^^^^^^^^^^^^^^^^^
sub-sub-sub-sub-header
++++++++++++++++++++++
sub-sub-sub-sub-sub-header
**************************License
GPL-2.0-only OR LGPL-2.1-or-later
Support
For any issues contact me at the e-mail given in the Author section.
