Batch Processing
In order to answer queries, JanusGraph has to perform queries against the storage backend. In general, there are two ways of doing this:
- Once data from the backend is needed, execute a backend query and continue with the result.
- Maintain a list of what data is needed. Once the list reaches a certain size, execute a batched backend query to fetch all of it at once.
The first option tends to be more responsive and consume less memory because the query can emit the first results very early without waiting for larger batches of queries to complete. It however sends many small queries to the storage backend for traversals that traverse a high number of vertices which leads to poor performance. That is why JanusGraph uses batch processing by default. Both of these options are described in greater detail below, including information about configuring batch processing.
Note
The default setting was changed in version 1.0.0. Older versions of JanusGraph used no batch processing (first option) by default.
No Batch Processing
In terms of graph traversals, the execution of queries is loosely coupled to the principle of Depth-First-Search.
Use this configuration in use cases where for example ...
- ... each query only accesses few vertices of the graph.
- ... your application does not need the full result set immediately but rather requires a low latency for the first results to arrive.
Possible limitations
- Traversing large neighborhoods can make the query slow.
Steps to explicitly configure this option:
- Ensure
query.batch.enabled
is set tofalse
Unrestricted Batch Processing
Using this configuration, each step which traverses the Graph
starting from a vertex (so e.g. in()
, outE()
and values()
but not inV()
or otherV()
and also not valueMap()
, see
#2444)
becomes a blocking operator which means that it produces no
results until all the results of the previous step are known.
Only then, a single backend query is executed and the results
are passed to the next step. Manual barrier()
steps do not
affect this in any meaningful way.
This way of execution can be thought of as a
Breadth-First-Search.
Use this configuration in use cases where for example ...
- ... your queries are likely to access multiple vertices in each step.
- ... there is a significant network latency between JanusGraph and the storage backend.
Possible limitations
- Increased memory consumption
- If limit steps occur late in the query, there might be an unnecessary overhead produced by the steps before the limit step.
- Performing very large backend queries could stress the storage backend.
Steps to explicitly configure this option:
- Ensure
query.batch.enabled
is set totrue
- Ensure
query.batch.limited
is set tofalse
Limited Batch Processing
Using this configuration, each step which traverses the Graph
starting from a vertex (so e.g. in()
, outE()
and values()
but not inV()
or otherV()
) aggregates a number of vertices
first, before executing a batched backend query.
This aggregation phase and backend query phase will repeat
until all vertices are processed.
In contrast to unrestricted batch processing where one batch
corresponds to one step in the query, this approach can
construct multiple batches per step.
This is the default configuration of JanusGraph since version 1.0.0.
Configuring the batch size
Although batch size does not necessarily need to be configured,
it can provide an additional tuning parameter to improve the
performance of a query.
By default, the batch size for TinkerPop's barrier step
will be provided by LazyBarrierStrategy
, which is currently at 2500
.
For batchable cases where LazyBarrierStrategy
doesn't inject any barrier
steps,
the barrier step will be ingected with the size configured via query.batch.limited-size
(which defaults to 2500
, same as with LazyBarrierStrategy
).
The batch size of each vertex step can be individually
configured by prepending a barrier(<size>)
step.
For example, in the query below, the first out()
step would
use the default batch size of 2500 and the second out()
step
would use a manually configured batch size of 1234:
g.V(list_of_vertices).out().barrier(1234).out()
For local traversals which start with a vertex step, the limit is best configured outside the local traversal, as seen below:
g.V(list_of_vertices).out().barrier(1234).where(__.out())
barrier(1234)
step would not be allowed to aggregate multiple
traversers.
A special case applies to repeat()
steps.
Because the local traversal of a repeat()
step has two inputs
(first, the step before the repeat()
step and second, the
last step of the repeated traversal, which feeds the result
back to the beginning), two limits can be configured here.
g.V(list_of_vertices).barrier(1234).repeat(__.barrier(2345).out()).times(5)
barrier(1234)
step in front of the local
traversal can only aggregate traversers once they enter the
repeat
step for the first time. For each iteration, the inner
barrier(2345)
is used to aggregate traversers from the
previous iteration.
Use this configuration in use cases where for example ...
- ... you have a mixture of traversals that traverse a high number of vertices and traversals that only access few vertices of the graph.
Possible limitations
- Increased memory consumption (compared to no batch processing)
- The performance of queries depends on the configured batch size. If you use this configuration, make sure that the latency and throughput of your queries meet your requirements and if not, tweak the batch size accordingly.
Steps to explicitly configure this option:
- Ensure
query.batch.enabled
is set totrue
- Ensure
query.batch.limited
is set totrue
Batched Query Processing Flow
Whenever query.batch.enabled
is set to true
steps compatible with batch processing are going to be
executed in batched fashion. Each storage backend may differently execute such batches, but usually
it means requesting data in parallel for multiple vertices which usually improves query performance
when the query is accessing many vertices.
Batched query processing takes into account two types of steps:
- Batch compatible step. This is the step which will execute batch requests. Currently, the list of such steps
is the next:
out()
,in()
,both()
,inE()
,outE()
,bothE()
,has()
,values()
,properties()
,valueMap()
,propertyMap()
,elementMap()
,label()
. - Parent step. This is a parent step which has local traversals with the same start. Such parent steps also implement the
interface
TraversalParent
. There are many such steps, but as for an example those could be:and(...)
,or(...)
,not(...)
,order().by(...)
,project("valueA", "valueB", "valueC").by(...).by(...).by(...)
,union(..., ..., ...)
,choose(..., ..., ...)
,coalesce(..., ...)
,where(...)
, etc. Start of such local steps should be the same, thus, the only exception currently are stepsrepeat()
andmatch()
(see below on how they are processed).
Parent steps register their vertices for later processing with the batch compatible start step. For example,
g.V(v1, v2, v3).union(out("knows"), in("follows"))
v1
, v2
, and v3
will be registered with out("knows")
and in("follows")
steps for batch processing because their parent step (union
) registers any input with the batch compatible child start
steps. Moreover, parent steps can register vertices for batch processing even with deep nested batch compatible start steps. For example,
g.V(v1, v2, v3).
and(
union(out("edge1"), in("edge2")),
or(
union(out("edge3"), in("edge4").optional(out("edge5"))),
optional(out("edge6")).in("edge7")))
v1
, v2
, and v3
will be registered with out("edge1")
, in("edge2")
, out("edge3")
,
in("edge4")
, and out("edge6")
steps for batch processing because they all can be considered as starts of the most root parent step (and
step).
That said, those vertices won't be registered for batch processing with steps out("edge5")
or in("edge7")
because those steps
are either not starting steps or starting steps of other parent steps. As such, out("edge5")
will be registered with
any vertex returned from in("edge4")
step, and in("edge7")
will be registered with any vertex returned from optional(out("edge6"))
step.
Batch processing for repeat
step
Repeat step doesn't follow the rules of other parent steps and registers vertices to child steps differently. Currently, TinkerPop's default implementation is using Breadth-First Search instead of Depth-Fist Search (as used for other steps).
JanusGraph applies repeat step vertices to the start of local repeat
step, start of local emit
step in case it is
placed before repeat
step, and start of local until
step in case it is placed before repeat
step.
Moreover, for any next iteration JanusGraph applies result of the local repeat
step (end step) to the beginning of the
local repeat
step (start step) as well as start steps of emit
and until
traversals.
Use-cases for batch requests per level (loop
):
-
Simple example.
In the above example verticesg.V(v1, v2, v3).repeat(out("knows")).emit()
v1
,v2
, andv3
will be registered without("knows")
step because it's the start step with batch support. Moreover, the result of all iterations on the same level (loop
) ofout("knows")
will be registered back toout("knows")
for the next level (loop
) iterations and so on untilout("knows")
stops emitting any results. -
Example with custom
emit
traversal afterrepeat
.The above example's vertices registration flow is the same as in exampleg.V(v1, v2, v3).repeat(out("knows")).emit(out("follows"))
1
, but the difference is thatout("follows")
will receive vertices for registration fromout("knows")
the same way asout("knows")
step itself receives vertices for registration from itself. Notice, the same logic would apply foruntil
step if it wereuntil
instead ofemit
here. -
Example with custom
emit
traversal beforerepeat
.The above example's vertices registration flow is the same as in exampleg.V(v1, v2, v3).emit(out("follows")).repeat(out("knows"))
2
, but the difference is thatout("follows")
will receive vertices for registration both fromout("knows")
and the start verticesv1
,v2
,v3
. In other words, vertices registration sources forout("knows")
andout("follows")
are the same in this case. Notice, the same logic would apply foruntil
step if it wereuntil
instead ofemit
here. -
Example with custom
emit
anduntil
traversals beforerepeat
.In the above example all 3 stepsg.V(v1, v2, v3).emit(out("follows")).until(out("feeds")).repeat(out("knows"))
out("follows")
,out("feeds")
, andout("knows")
have the same vertices registration flow where they receive vertices both from query start (v1
,v2
,v3
) and from local repeat end step (out("knows")
). -
Example with custom
emit
anduntil
traversals afterrepeat
.The above example's vertices registration flow is the same as in exampleg.V(v1, v2, v3).repeat(out("knows")).emit(out("follows")).until(out("feeds"))
4
, but the difference is thatout("follows")
andout("feeds")
won't receive vertices registration from the query start (v1
,v2
,v3
). -
Example with custom
until
traversal beforerepeat
andemit(true)
afterrepeat
.The above example's vertices registration flow is the same as in exampleg.V(v1, v2, v3).until(out("feeds")).repeat(out("knows")).emit()
4
, except that theemit
traversal doesn't have any start step which supports batching. Thus,emit
traversal doesn't receive batched vertices registration.
Use-cases for batch requests per iteration:
In most cases (like the above examples 1 - 6
and other cases) TinkerPop's default repeat
step implementation executes
the local repeat
traversal for the whole level (loop
) before emit
or until
is executed. In other words repeat
traversal
is executed multiple times (multiple iterations on the same loop
) before emit
or until
first execution on the current loop
.
This gives JanusGraph possibility to make larger batch requests containing vertices from multiple repeat
traversal iterations
which efficiently executes batch requests.
That said, there are 3 use-cases when the execution flow is different and TinkerPop executes until
or emit
traversals after each
repeat
traversal iteration. In such case until
or emit
steps will execute batch requests for vertices collected on the
current iteration only and not for vertices collected from all iterations of the same level (loop
).
g.V(v1, v2, v3).emit().repeat(out("knows")).until(out("feeds"))
g.V(v1, v2, v3).emit(out("follows")).repeat(out("knows")).until(out("feeds"))
g.V(v1, v2, v3).until(out("feeds")).repeat(out("knows")).emit(out("follows"))
The above 3 examples show the pattern when until
or emit
executes batches per iteration instead of per level.
In case any emit
step is placed before repeat
step while until
step is placed after repeat
step. In case
until
step is placed before repeat
step while non-true emit
step is placed after repeat
step.
In all other cases repeat
step will be executed for the whole loop
and only after that emit
or until
will be executed.
These limitations might be resolved after JanusGraph adds support for DFS repeat step execution (see issue #3787).
Multi-nested repeat
step modes:
By default, in cases when batch start steps have multiple repeat
step parents the batch registration is considering all repeat
parent steps.
However, in cases when transaction cache is small and repeat step traverses more than one level
deep, it could result for some vertices to be re-fetched again or vertices which don't need to be fetched due to early
cycle end could potentially be fetched into the transaction cache. It would mean a waste of operation when it isn't necessary.
Thus, JanusGraph provides a configuration option query.batch.repeat-step-mode
to control multi-repeat step behaviour:
closest_repeat_parent
(default option) - consider the closestrepeat
step only.In the example above,g.V().repeat(and(repeat(out("knows")).emit())).emit()
out("knows")
will be receiving vertices for batching from theand
step input for the first iterations as well as theout("knows")
step output for the next iterations.all_repeat_parents
- consider registering vertices from the start and end of eachrepeat
step parent.In the example above,g.V().repeat(and(repeat(out("knows")).emit())).emit()
out("knows")
will be receiving vertices for batching from the most outerrepeat
step input (for the first iterations), the most outerrepeat
step output (which isand
output) (for the first iterations),
theand
step input (for the first iterations), and from theout("knows")
output (for the next iterations).starts_only_of_all_repeat_parents
- consider registering vertices from the start of eachrepeat
step parent.In the example above,g.V().repeat(and(repeat(out("knows")).emit())).emit()
out("knows")
will be receiving vertices for batching from the most outerrepeat
step input (for the first iterations), theand
step input (for the first iterations), and from theout("knows")
output (for the next iterations).
Batch processing for match
step
Currently, JanusGraph supports vertices registration for batch processing inside individual local traversals of the match
step, but not between those local traversals. Also, JanusGraph doesn't register start of the match
step with any
of the local traversals of the match
step. Thus, performance for match
step might be limited. This is a temporary
limitation until this feature is implemented (see issue #3788).
Batch processing for properties
Some of the Gremlin steps with enabled optimization may prefetch vertex properties in batches.
As for now, JanusGraph uses slice queries to query part of the row data. A single-slice query contains
the start key and the end key to define a slice of data JanusGraph is interested in.
As JanusGraph doesn't support multi-range slice queries right now it can either fetch a single property
in a single Slice query or all properties in a single slice query. Thus, users have to decide the tradeoff between
different properties fetching approaches and decide when they want to fetch all properties in a single slice query
(which is usually faster but unnecessary properties might be fetched) or to fetch only requested properties in
separate slice query per each property (might be slightly slower but will fetch only the requested properties).
See issue #3816 which will allow fetching only requested properties via a single slice query.
See configuration option query.fast-property
which may be used to pre-fetch all properties on a first singular property
access when direct vertex properties are requested (for example vertex.properties("foo")
).
See configuration option query.batch.has-step-mode
to control properties pre-fetching behaviour for has
step.
See configuration option query.batch.properties-mode
to control properties pre-fetching behaviour for values
,
properties
, valueMap
, propertyMap
, and elementMap
steps.
See configuration option query.batch.label-step-mode
to control labels pre-fetching behaviour for label
step.