X - the type that is returned by the currently set of mapper function. the next added
mapper function will be called with a parameter of this type as inputpublic abstract class MapReducer<X> extends Object implements Mappable<X>, Serializable
It accepts a list of filters, transformation `map` functions a produces a result when calling the `reduce` method (or one of its shorthand versions like `sum`, `count`, etc.).
You can set a list of filters that are applied on the raw OSM data, for example you can filter:
Depending on the used data "view", the MapReducer produces either "snapshots" or evaluated all modifications ("contributions") of the matching raw OSM data.
These data can then be transformed arbitrarily by user defined `map` functions (which take one of these entity snapshots or modifications as input an produce an arbitrary output) or `flatMap` functions (which can return an arbitrary number of results per entity snapshot/contribution). It is possible to chain together any number of transformation functions.
Finally, one can either use one of the pre-defined result-generating functions (e.g. `sum`, `count`, `average`, `uniq`), or specify a custom `reduce` procedure.
If one wants to get results that are aggregated by timestamp (or some other index), one can use the `aggregateByTimestamp` or `aggregateBy` functionality that automatically handles the grouping of the output data.
For more complex analyses, it is also possible to enable the grouping of the input data by the respective OSM ID. This can be used to view at the whole history of entities at once.
| Modifier and Type | Field and Description |
|---|---|
protected OSHDBBoundingBox |
bboxFilter |
protected OSHDBJdbc |
keytables |
protected OSHDBDatabase |
oshdb |
protected Long |
timeout |
protected OSHDBTimestampList |
tstamps |
protected EnumSet<OSMType> |
typeFilter |
| Modifier | Constructor and Description |
|---|---|
protected |
MapReducer(MapReducer<?> obj) |
protected |
MapReducer(OSHDBDatabase oshdb,
Class<? extends OSHDBMapReducible> forClass) |
| Modifier and Type | Method and Description |
|---|---|
<U extends Comparable<U> & Serializable> |
aggregateBy(SerializableFunction<X,U> indexer)
Sets a custom aggregation function that is used to group output results into.
|
<U extends Comparable<U> & Serializable> |
aggregateBy(SerializableFunction<X,U> indexer,
Collection<U> zerofill)
Sets a custom aggregation function that is used to group output results into.
|
<U extends Comparable<U> & Serializable,P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> |
aggregateByGeometry(Map<U,P> geometries)
Sets up automatic aggregation by geometries.
|
MapAggregator<OSHDBTimestamp,X> |
aggregateByTimestamp()
Sets up automatic aggregation by timestamp.
|
MapAggregator<OSHDBTimestamp,X> |
aggregateByTimestamp(SerializableFunction<X,OSHDBTimestamp> indexer)
Sets up aggregation by a custom time index.
|
MapReducer<X> |
areaOfInterest(OSHDBBoundingBox bboxFilter)
Set the area of interest to the given bounding box.
|
<P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> |
areaOfInterest(P polygonFilter)
Set the area of interest to the given polygon.
|
Double |
average()
Calculates the averages of the results.
|
<R extends Number> |
average(SerializableFunction<X,R> mapper)
Calculates the average of the results provided by a given `mapper` function.
|
List<X> |
collect()
Collects all results into a List.
|
protected abstract MapReducer<X> |
copy() |
Integer |
count()
Counts the number of results.
|
Integer |
countUniq()
Counts all unique values of the results.
|
Double |
estimatedMedian()
Returns an estimate of the median of the results.
|
<R extends Number> |
estimatedMedian(SerializableFunction<X,R> mapper)
Returns an estimate of the median of the results after applying the given map function.
|
Double |
estimatedQuantile(double q)
Returns an estimate of a requested quantile of the results.
|
<R extends Number> |
estimatedQuantile(SerializableFunction<X,R> mapper,
double q)
Returns an estimate of a requested quantile of the results after applying the given map
function.
|
DoubleUnaryOperator |
estimatedQuantiles()
Returns a function that computes estimates of arbitrary quantiles of the results.
|
List<Double> |
estimatedQuantiles(Iterable<Double> q)
Returns an estimate of the quantiles of the results.
|
<R extends Number> |
estimatedQuantiles(SerializableFunction<X,R> mapper)
Returns a function that computes estimates of arbitrary quantiles of the results after applying
the given map function.
|
<R extends Number> |
estimatedQuantiles(SerializableFunction<X,R> mapper,
Iterable<Double> q)
Returns an estimate of the quantiles of the results after applying the given map function.
|
MapReducer<X> |
filter(SerializablePredicate<X> f)
Adds a custom arbitrary filter that gets executed in the current transformation chain.
|
<R> MapReducer<R> |
flatMap(SerializableFunction<X,Iterable<R>> flatMapper)
Set an arbitrary `flatMap` transformation function, which returns list with an arbitrary number
of results per input data entry.
|
protected <R,S> S |
flatMapReduceCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<R>> mapper,
SerializableSupplier<S> identitySupplier,
SerializableBiFunction<S,R,S> accumulator,
SerializableBinaryOperator<S> combiner)
Generic "flat" version of the map-reduce used by the `OSMContributionView`, with by-osm-id
grouped input to the `mapper` function.
|
protected <R,S> S |
flatMapReduceCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<R>> mapper,
SerializableSupplier<S> identitySupplier,
SerializableBiFunction<S,R,S> accumulator,
SerializableBinaryOperator<S> combiner)
Generic "flat" version of the map-reduce used by the `OSMEntitySnapshotView`, with by-osm-id
grouped input to the `mapper` function.
|
protected Stream<X> |
flatMapStreamCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<X>> mapper) |
protected Stream<X> |
flatMapStreamCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<X>> mapper) |
void |
forEach(SerializableConsumer<X> action)
Deprecated.
only for testing purposes, use `.stream().forEach()` instead
|
protected Iterable<XYGridTree.CellIdRange> |
getCellIdRanges() |
protected CellIterator.OSMEntityFilter |
getFilter() |
protected <P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> |
getPolyFilter() |
protected CellIterator.OSHEntityFilter |
getPreFilter() |
protected TagInterpreter |
getTagInterpreter() |
protected TagTranslator |
getTagTranslator() |
MapReducer<List<X>> |
groupByEntity()
Groups the input data (osm entity snapshot or contributions) by their respective entity's ids
before feeding them into further transformation functions.
|
boolean |
isCancelable()
Returns if the current backend can be canceled (e.g.
|
MapReducer<X> |
keytables(OSHDBJdbc keytables)
Sets the keytables database to use in the calculations to resolve strings (osm tags, roles)
into internally used identifiers.
|
<R> MapReducer<R> |
map(SerializableFunction<X,R> mapper)
Set an arbitrary `map` transformation function.
|
protected <R,S> S |
mapReduceCellsOSMContribution(SerializableFunction<OSMContribution,R> mapper,
SerializableSupplier<S> identitySupplier,
SerializableBiFunction<S,R,S> accumulator,
SerializableBinaryOperator<S> combiner)
Generic map-reduce used by the `OSMContributionView`.
|
protected <R,S> S |
mapReduceCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,R> mapper,
SerializableSupplier<S> identitySupplier,
SerializableBiFunction<S,R,S> accumulator,
SerializableBinaryOperator<S> combiner)
Generic map-reduce used by the `OSMEntitySnapshotView`.
|
protected Stream<X> |
mapStreamCellsOSMContribution(SerializableFunction<OSMContribution,X> mapper) |
protected Stream<X> |
mapStreamCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,X> mapper) |
MapReducer<X> |
osmEntityFilter(SerializablePredicate<OSMEntity> f)
Adds a custom arbitrary filter that gets executed for each osm entity and determines if it
should be considered for this analyis or not.
|
MapReducer<X> |
osmTag(Collection<? extends OSMTagInterface> tags)
Adds an osm tag filter: The analysis will be restricted to osm entities that have at least one
of the supplied tags (key=value pairs or key=*).
|
MapReducer<X> |
osmTag(OSMTagInterface tag)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
(with an arbitrary value), or this tag key and value.
|
MapReducer<X> |
osmTag(String key)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
(with an arbitrary value).
|
MapReducer<X> |
osmTag(String key,
Collection<String> values)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
and one of the given values.
|
MapReducer<X> |
osmTag(String key,
Pattern valuePattern)
Adds an osm tag filter: The analysis will be restricted to osm entities that have a tag with
the given key and whose value matches the given regular expression pattern.
|
MapReducer<X> |
osmTag(String key,
String value)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
and value.
|
default M |
osmType(OSMType type1,
OSMType... otherTypes)
Limits the analysis to the given osm entity types.
|
MapReducer<X> |
osmType(Set<OSMType> typeFilter)
Limits the analysis to the given osm entity types.
|
default M |
osmTypes(EnumSet<OSMType> typeFilter)
Deprecated.
replaced by
osmType(OSMType, OSMType...) |
default M |
osmTypes(OSMType type1,
OSMType... otherTypes)
Deprecated.
replaced by
osmType(OSMType, OSMType...) |
<S> S |
reduce(SerializableSupplier<S> identitySupplier,
SerializableBiFunction<S,X,S> accumulator,
SerializableBinaryOperator<S> combiner)
Generic map-reduce routine.
|
X |
reduce(SerializableSupplier<X> identitySupplier,
SerializableBinaryOperator<X> accumulator)
Generic map-reduce routine (shorthand syntax).
|
Stream<X> |
stream()
Returns all results as a Stream.
|
Number |
sum()
Sums up the results.
|
<R extends Number> |
sum(SerializableFunction<X,R> mapper)
Sums up the results provided by a given `mapper` function.
|
MapReducer<X> |
tagInterpreter(TagInterpreter tagInterpreter)
Sets the tagInterpreter to use in the analysis.
|
MapReducer<X> |
timestamps(OSHDBTimestampList tstamps)
Set the timestamps for which to perform the analysis.
|
MapReducer<X> |
timestamps(String isoDate)
Sets a single timestamp for which to perform the analysis at.
|
MapReducer<X> |
timestamps(String isoDateStart,
String isoDateEnd)
Sets two timestamps (start and end date) for which to perform the analysis.
|
MapReducer<X> |
timestamps(String isoDateStart,
String isoDateEnd,
OSHDBTimestamps.Interval interval)
Set the timestamps for which to perform the analysis in a regular interval between a start and
end date.
|
MapReducer<X> |
timestamps(String isoDateFirst,
String isoDateSecond,
String... isoDateMore)
Sets multiple arbitrary timestamps for which to perform the analysis.
|
Set<X> |
uniq()
Gets all unique values of the results.
|
<R> Set<R> |
uniq(SerializableFunction<X,R> mapper)
Gets all unique values of the results provided by a given mapper function.
|
Double |
weightedAverage(SerializableFunction<X,WeightedValue> mapper)
Calculates the weighted average of the results provided by the `mapper` function.
|
default M |
where(Collection<OSMTag> keyValuePairs)
Deprecated.
replaced by
osmTag(Collection) |
default M |
where(OSMTag tag)
Deprecated.
replaced by
osmTag(OSMTagInterface) |
default M |
where(OSMTagKey key)
Deprecated.
replaced by
osmTag(OSMTagInterface) |
default M |
where(SerializablePredicate<OSMEntity> f)
|
default M |
where(String key)
Deprecated.
replaced by
osmTag(String) |
default M |
where(String key,
Collection<String> values)
Deprecated.
replaced by
osmTag(String, Collection) |
default M |
where(String key,
Pattern valuePattern)
Deprecated.
replaced by
osmTag(String, Pattern) |
default M |
where(String key,
String value)
Deprecated.
replaced by
osmTag(String, String) |
protected OSHDBDatabase oshdb
protected transient OSHDBJdbc keytables
protected Long timeout
protected OSHDBTimestampList tstamps
protected OSHDBBoundingBox bboxFilter
protected MapReducer(OSHDBDatabase oshdb, Class<? extends OSHDBMapReducible> forClass)
protected MapReducer(MapReducer<?> obj)
public boolean isCancelable()
@NotNull protected abstract MapReducer<X> copy()
@Contract(pure=true) public MapReducer<X> keytables(OSHDBJdbc keytables)
keytables - the database to use for resolving strings into internal identifiers@Contract(pure=true) public MapReducer<X> tagInterpreter(TagInterpreter tagInterpreter)
tagInterpreter - the tagInterpreter object to use in the processing of osm entities@Contract(pure=true) public MapReducer<X> areaOfInterest(@NotNull OSHDBBoundingBox bboxFilter)
bboxFilter - the bounding box to query the data in@Contract(pure=true) public <P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> MapReducer<X> areaOfInterest(@NotNull P polygonFilter)
polygonFilter - the bounding box to query the data in@Contract(pure=true) public MapReducer<X> timestamps(OSHDBTimestampList tstamps)
Depending on the *View*, this has slightly different semantics:
tstamps - an object (implementing the OSHDBTimestampList interface) which provides the
timestamps to do the analysis for@Contract(pure=true) public MapReducer<X> timestamps(String isoDateStart, String isoDateEnd, OSHDBTimestamps.Interval interval)
See timestamps(OSHDBTimestampList) for further information.
isoDateStart - an ISO 8601 date string representing the start date of the analysisisoDateEnd - an ISO 8601 date string representing the end date of the analysisinterval - the interval between the timestamps to be used in the analysis@Contract(pure=true) public MapReducer<X> timestamps(String isoDate)
Useful in combination with the OSMEntitySnapshotView when not performing further aggregation by timestamp.
See timestamps(OSHDBTimestampList) for further information.
isoDate - an ISO 8601 date string representing the date of the analysis@Contract(pure=true) public MapReducer<X> timestamps(String isoDateStart, String isoDateEnd)
Useful in combination with the OSMContributionView when not performing further aggregation by timestamp.
See timestamps(OSHDBTimestampList) for further information.
isoDateStart - an ISO 8601 date string representing the start date of the analysisisoDateEnd - an ISO 8601 date string representing the end date of the analysis@Contract(pure=true) public MapReducer<X> timestamps(String isoDateFirst, String isoDateSecond, String... isoDateMore)
Note for programmers wanting to use this method to supply an arbitrary number (n>=1) of
timestamps: You may supply the same time string multiple times, which will be de-duplicated
internally. E.g. you can call the method like this:
.timestamps(dateArr[0], dateArr[0], dateArr)
See timestamps(OSHDBTimestampList) for further information.
isoDateFirst - an ISO 8601 date string representing the start date of the analysisisoDateSecond - an ISO 8601 date string representing the second date of the analysisisoDateMore - more ISO 8601 date strings representing the remaining timestamps of the
analysis@Contract(pure=true) public MapReducer<X> osmType(Set<OSMType> typeFilter)
typeFilter - the set of osm types to filter (e.g. `EnumSet.of(OSMType.WAY)`)@Contract(pure=true) public MapReducer<X> osmEntityFilter(SerializablePredicate<OSMEntity> f)
f - the filter function to call for each osm entity@Contract(pure=true) public MapReducer<X> osmTag(String key)
key - the tag key to filter the osm entities for@Contract(pure=true) public MapReducer<X> osmTag(OSMTagInterface tag)
tag - the tag (key, or key and value) to filter the osm entities for@Contract(pure=true) public MapReducer<X> osmTag(String key, String value)
key - the tag to filter the osm entities forvalue - the tag value to filter the osm entities for@Contract(pure=true) public MapReducer<X> osmTag(String key, Collection<String> values)
key - the tag key to filter the osm entities forvalues - an array of tag values to filter the osm entities for@Contract(pure=true) public MapReducer<X> osmTag(String key, Pattern valuePattern)
key - the tag key to filter the osm entities forvaluePattern - a regular expression which the tag value of the osm entity must match@Contract(pure=true) public MapReducer<X> osmTag(Collection<? extends OSMTagInterface> tags)
tags - the tags (key/value pairs or key=*) to filter the osm entities for@Contract(pure=true) public <R> MapReducer<R> map(SerializableFunction<X,R> mapper)
map in interface Mappable<X>R - an arbitrary data type which is the return type of the transformation `map` functionmapper - function that will be applied to each data entry (osm entity snapshot or
contribution)@Contract(pure=true) public <R> MapReducer<R> flatMap(SerializableFunction<X,Iterable<R>> flatMapper)
flatMap in interface Mappable<X>R - an arbitrary data type which is the return type of the transformation `map` functionflatMapper - function that will be applied to each data entry (osm entity snapshot or
contribution) and returns a list of results@Contract(pure=true) public MapReducer<X> filter(SerializablePredicate<X> f)
@Contract(pure=true) public MapReducer<List<X>> groupByEntity() throws UnsupportedOperationException
This needs to be called before any `map` or `flatMap` transformation functions have been set. Otherwise a runtime exception will be thrown.
UnsupportedOperationException - if this is called after some map (or flatMap) functions
have already been setUnsupportedOperationException - if this is called when a grouping has already been
activated@Contract(pure=true) public <U extends Comparable<U> & Serializable> MapAggregator<U,X> aggregateBy(SerializableFunction<X,U> indexer, Collection<U> zerofill)
U - the data type of the values used to aggregate the output. has to be a comparable
typeindexer - a function that will be called for each input element and returns a value that
will be used to group the results byzerofill - a collection of values that are expected to be present in the result@Contract(pure=true) public <U extends Comparable<U> & Serializable> MapAggregator<U,X> aggregateBy(SerializableFunction<X,U> indexer)
U - the data type of the values used to aggregate the output. has to be a comparable
typeindexer - a function that will be called for each input element and returns a value that
will be used to group the results by@Contract(pure=true) public MapAggregator<OSHDBTimestamp,X> aggregateByTimestamp() throws UnsupportedOperationException
In the OSMEntitySnapshotView, the snapshots' timestamp will be used directly to aggregate results into. In the OSMContributionView, the timestamps of the respective data modifications will be matched to corresponding time intervals (that are defined by the `timestamps` setting here).
Cannot be used together with the `groupByEntity()` setting enabled.
UnsupportedOperationException - if this is called when the `groupByEntity()` mode has
been activatedpublic MapAggregator<OSHDBTimestamp,X> aggregateByTimestamp(SerializableFunction<X,OSHDBTimestamp> indexer) throws UnsupportedOperationException
The timestamps returned by the supplied indexing function are matched to the corresponding time intervals.
indexer - a callback function that return a timestamp object for each given data. Note
that if this function returns timestamps outside of the supplied timestamps()
interval results may be undefinedUnsupportedOperationException@Contract(pure=true) public <U extends Comparable<U> & Serializable,P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> MapAggregator<U,X> aggregateByGeometry(Map<U,P> geometries) throws UnsupportedOperationException
Cannot be used together with the `groupByEntity()` setting enabled.
UnsupportedOperationException - if this is called when the `groupByEntity()` mode has
been activatedUnsupportedOperationException - when called after any map or flatMap functions are set@Contract(pure=true) public <S> S reduce(SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,X,S> accumulator, SerializableBinaryOperator<S> combiner) throws Exception
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.
S - the data type used to contain the "reduced" (intermediate and final) resultsidentitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <R>)
and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and
returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter
(mutate) the state of the accumulation value (e.g. directly adding new values to an
existing Set object)combiner - a function that calculates the "sum" of two <S> values; this function
must be pure (have no side effects), and is not allowed to alter the state of the two
input objects it gets!UnsupportedOperationException - if the used oshdb database backend doesn't implement
the required reduce operation.Exception - if during the reducing operation an exception happens (see the respective
implementations for details).@Contract(pure=true) public X reduce(SerializableSupplier<X> identitySupplier, SerializableBinaryOperator<X> accumulator) throws Exception
This variant is shorter to program than `reduce(identitySupplier, accumulator, combiner)`, but can only be used if the result type is the same as the current `map`ped type <X>. Also this variant can be less efficient since it cannot benefit from the mutability freedoms the accumulator+combiner approach has.
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator) interface.
identitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <X>)
and an accumulation value (also of type <X>, e.g. the result of
`identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this
function is not to alter (mutate) the state of the accumulation value (e.g. directly
adding new values to an existing Set object)Exception@Contract(pure=true) public Number sum() throws Exception
The current data values need to be numeric (castable to "Number" type), otherwise a runtime exception will be thrown.
UnsupportedOperationException - if the data cannot be cast to numbersException@Contract(pure=true) public <R extends Number> R sum(SerializableFunction<X,R> mapper) throws Exception
This is a shorthand for `.map(mapper).sum()`, with the difference that here the numerical return type of the `mapper` is ensured.
R - the numeric type that is returned by the `mapper` functionmapper - function that returns the numbers to sum upException@Contract(pure=true) public Integer count() throws Exception
Exception@Contract(pure=true) public Set<X> uniq() throws Exception
For example, this can be used together with the OSMContributionView to get the total amount of unique users editing specific feature types.
Exception@Contract(pure=true) public <R> Set<R> uniq(SerializableFunction<X,R> mapper) throws Exception
This is a shorthand for `.map(mapper).uniq()`.
R - the type that is returned by the `mapper` functionmapper - function that returns some valuesException@Contract(pure=true) public Integer countUniq() throws Exception
For example, this can be used together with the OSMContributionView to get the number of unique users editing specific feature types.
Exception@Contract(pure=true) public Double average() throws Exception
The current data values need to be numeric (castable to "Number" type), otherwise a runtime exception will be thrown.
UnsupportedOperationException - if the data cannot be cast to numbersException@Contract(pure=true) public <R extends Number> Double average(SerializableFunction<X,R> mapper) throws Exception
R - the numeric type that is returned by the `mapper` functionmapper - function that returns the numbers to averageException@Contract(pure=true) public Double weightedAverage(SerializableFunction<X,WeightedValue> mapper) throws Exception
The mapper must return an object of the type `WeightedValue` which contains a numeric value associated with a (floating point) weight.
mapper - function that gets called for each entity snapshot or modification, needs to
return the value and weight combination of numbers to averageException@Contract(pure=true) public Double estimatedMedian() throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
Exception@Contract(pure=true) public <R extends Number> Double estimatedMedian(SerializableFunction<X,R> mapper) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the mean forException@Contract(pure=true) public Double estimatedQuantile(double q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
q - the desired quantile to calculate (as a number between 0 and 1)Exception@Contract(pure=true) public <R extends Number> Double estimatedQuantile(SerializableFunction<X,R> mapper, double q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the quantile forq - the desired quantile to calculate (as a number between 0 and 1)Exception@Contract(pure=true) public List<Double> estimatedQuantiles(Iterable<Double> q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
q - the desired quantiles to calculate (as a collection of numbers between 0 and 1)Exception@Contract(pure=true) public <R extends Number> List<Double> estimatedQuantiles(SerializableFunction<X,R> mapper, Iterable<Double> q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the quantiles forq - the desired quantiles to calculate (as a collection of numbers between 0 and 1)Exception@Contract(pure=true) public DoubleUnaryOperator estimatedQuantiles() throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
Exception@Contract(pure=true) public <R extends Number> DoubleUnaryOperator estimatedQuantiles(SerializableFunction<X,R> mapper) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the quantiles forException@Deprecated public void forEach(SerializableConsumer<X> action) throws Exception
This method can be handy for testing purposes. But note that since the `action` doesn't produce a return value, it must facilitate its own way of producing output.
If you'd like to use such a "forEach" in a non-test use case, use `.stream().forEach()` instead.
action - function that gets called for each transformed data entryException@Contract(pure=true) public List<X> collect() throws Exception
Exception@Contract(pure=true) public Stream<X> stream() throws Exception
If the used oshdb database backend doesn't implement the stream operation directly, this will fall back to executing `.collect().stream()` instead, which buffers all results in memory first before returning them as a stream.
Exceptionprotected Stream<X> mapStreamCellsOSMContribution(SerializableFunction<OSMContribution,X> mapper) throws Exception
Exceptionprotected Stream<X> flatMapStreamCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<X>> mapper) throws Exception
Exceptionprotected Stream<X> mapStreamCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,X> mapper) throws Exception
Exceptionprotected Stream<X> flatMapStreamCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<X>> mapper) throws Exception
Exceptionprotected <R,S> S mapReduceCellsOSMContribution(SerializableFunction<OSMContribution,R> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) throws Exception
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.
R - the data type returned by the `mapper` functionS - the data type used to contain the "reduced" (intermediate and final) resultsmapper - a function that's called for each `OSMContribution`identitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <R>)
and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and
returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter
(mutate) the state of the accumulation value (e.g. directly adding new values to an
existing Set object)combiner - a function that calculates the "sum" of two <S> values; this function
must be pure (have no side effects), and is not allowed to alter the state of the two
input objects it gets!Exceptionprotected <R,S> S flatMapReduceCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<R>> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) throws Exception
Contrary to the "normal" map-reduce, the "flat" version adds the possibility to return any number of results in the `mapper` function. Additionally, this interface provides the `mapper` function with a list of all `OSMContribution`s of a particular OSM entity. This is used to do more complex analyses that require the full edit history of the respective OSM entities as input.
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.
R - the data type returned by the `mapper` functionS - the data type used to contain the "reduced" (intermediate and final) resultsmapper - a function that's called for all `OSMContribution`s of a particular OSM entity;
returns a list of results (which can have any number of entries).identitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <R>)
and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and
returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter
(mutate) the state of the accumulation value (e.g. directly adding new values to an
existing Set object)combiner - a function that calculates the "sum" of two <S> values; this function
must be pure (have no side effects), and is not allowed to alter the state of the two
input objects it gets!Exceptionprotected <R,S> S mapReduceCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,R> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) throws Exception
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.
R - the data type returned by the `mapper` functionS - the data type used to contain the "reduced" (intermediate and final) resultsmapper - a function that's called for each `OSMEntitySnapshot`identitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <R>)
and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and
returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter
(mutate) the state of the accumulation value (e.g. directly adding new values to an
existing Set object)combiner - a function that calculates the "sum" of two <S> values; this function
must be pure (have no side effects), and is not allowed to alter the state of the two
input objects it gets!Exceptionprotected <R,S> S flatMapReduceCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<R>> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) throws Exception
Contrary to the "normal" map-reduce, the "flat" version adds the possibility to return any number of results in the `mapper` function. Additionally, this interface provides the `mapper` function with a list of all `OSMContribution`s of a particular OSM entity. This is used to do more complex analyses that require the full list of snapshots of the respective OSM entities as input.
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.
R - the data type returned by the `mapper` functionS - the data type used to contain the "reduced" (intermediate and final) resultsmapper - a function that's called for all `OSMEntitySnapshot`s of a particular OSM entity;
returns a list of results (which can have any number of entries)identitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <R>)
and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and
returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter
(mutate) the state of the accumulation value (e.g. directly adding new values to an
existing Set object)combiner - a function that calculates the "sum" of two <S> values; this function
must be pure (have no side effects), and is not allowed to alter the state of the two
input objects it gets!Exceptionprotected TagInterpreter getTagInterpreter() throws org.json.simple.parser.ParseException, SQLException, IOException
org.json.simple.parser.ParseExceptionSQLExceptionIOExceptionprotected TagTranslator getTagTranslator()
protected CellIterator.OSHEntityFilter getPreFilter()
protected CellIterator.OSMEntityFilter getFilter()
protected Iterable<XYGridTree.CellIdRange> getCellIdRanges()
protected <P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> P getPolyFilter()
public M osmType(OSMType type1, OSMType... otherTypes)
type1 - the set of osm types to filter (e.g. `OSMType.NODE`)otherTypes - more osm types which should be analyzed@Deprecated public M osmTypes(EnumSet<OSMType> typeFilter)
osmType(OSMType, OSMType...)@Deprecated public M osmTypes(OSMType type1, OSMType... otherTypes)
osmType(OSMType, OSMType...)@Deprecated public M where(SerializablePredicate<OSMEntity> f)
osmEntityFilter(org.heigit.bigspatialdata.oshdb.api.generic.function.SerializablePredicate<org.heigit.bigspatialdata.oshdb.osm.OSMEntity>)@Deprecated public M where(OSMTagKey key)
osmTag(OSMTagInterface)@Deprecated public M where(String key)
osmTag(String)@Deprecated public M where(OSMTag tag)
osmTag(OSMTagInterface)@Deprecated public M where(String key, String value)
osmTag(String, String)@Deprecated public M where(String key, Collection<String> values)
osmTag(String, Collection)@Deprecated public M where(String key, Pattern valuePattern)
osmTag(String, Pattern)@Deprecated public M where(Collection<OSMTag> keyValuePairs)
osmTag(Collection)Copyright © 2016–2020 HeiGIT. All rights reserved.