X - the type that is returned by the currently set of mapper function. the next added
mapper function will be called with a parameter of this type as inputU - the type of the index values returned by the `mapper function`, used to group resultspublic class MapAggregator<U extends Comparable<U> & Serializable,X> extends Object implements Mappable<X>
This class provides similar functionality as a MapReducer, with the difference that here the `reduce` does automatic aggregation of results by the values returned by an arbitrary indexing function.
All results for which the set `indexer` returns the same value are aggregated into separate "bins". This can be used to aggregate results by timestamp, geographic region, user id, osm tag, etc.
Internally, this wraps around an existing MapReducer object, which still continues to be responsible for all actual calculations.
| Modifier and Type | Method and Description |
|---|---|
<V extends Comparable<V> & Serializable> |
aggregateBy(SerializableFunction<X,V> indexer)
Sets up aggregation by another custom index.
|
<V extends Comparable<V> & Serializable> |
aggregateBy(SerializableFunction<X,V> indexer,
Collection<V> zerofill)
Sets up aggregation by another custom index.
|
<V extends Comparable<V> & Serializable,P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> |
aggregateByGeometry(Map<V,P> geometries)
Aggregates the results by sub-regions as well, in addition to the timestamps.
|
MapAggregator<OSHDBCombinedIndex<U,OSHDBTimestamp>,X> |
aggregateByTimestamp(SerializableFunction<X,OSHDBTimestamp> indexer)
Sets up aggregation by a custom time index.
|
MapAggregator<U,X> |
areaOfInterest(OSHDBBoundingBox bboxFilter)
Set the area of interest to the given bounding box.
|
<P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> |
areaOfInterest(P polygonFilter)
Set the area of interest to the given polygon.
|
SortedMap<U,Double> |
average()
Calculates the averages of the results.
|
<R extends Number> |
average(SerializableFunction<X,R> mapper)
Calculates the average of the results provided by a given `mapper` function.
|
SortedMap<U,List<X>> |
collect()
Collects the results of this data aggregation into Lists.
|
SortedMap<U,Integer> |
count()
Counts the number of results.
|
SortedMap<U,Integer> |
countUniq()
Counts all unique values of the results.
|
SortedMap<U,Double> |
estimatedMedian()
Returns an estimate of the median of the results.
|
<R extends Number> |
estimatedMedian(SerializableFunction<X,R> mapper)
Returns an estimate of the median of the results after applying the given map function.
|
SortedMap<U,Double> |
estimatedQuantile(double q)
Returns an estimate of a requested quantile of the results.
|
<R extends Number> |
estimatedQuantile(SerializableFunction<X,R> mapper,
double q)
Returns an estimate of a requested quantile of the results after applying the given map
function.
|
SortedMap<U,DoubleUnaryOperator> |
estimatedQuantiles()
Returns a function that computes estimates of arbitrary quantiles of the results.
|
SortedMap<U,List<Double>> |
estimatedQuantiles(Iterable<Double> q)
Returns an estimate of the quantiles of the results.
|
<R extends Number> |
estimatedQuantiles(SerializableFunction<X,R> mapper)
Returns a function that computes estimates of arbitrary quantiles of the results after applying
the given map function.
|
<R extends Number> |
estimatedQuantiles(SerializableFunction<X,R> mapper,
Iterable<Double> q)
Returns an estimate of the quantiles of the results after applying the given map function.
|
MapAggregator<U,X> |
filter(SerializablePredicate<X> f)
Adds a custom arbitrary filter that gets executed in the current transformation chain.
|
<R> MapAggregator<U,R> |
flatMap(SerializableFunction<X,Iterable<R>> flatMapper)
Set an arbitrary `flatMap` transformation function, which returns list with an arbitrary number
of results per input data entry.
|
void |
forEach(SerializableBiConsumer<U,List<X>> action)
Deprecated.
only for testing purposes. use `.collect().forEach()` or `.stream().forEach()`
instead
|
<R> MapAggregator<U,R> |
map(SerializableFunction<X,R> mapper)
Set an arbitrary `map` transformation function.
|
MapAggregator<U,X> |
osmEntityFilter(SerializablePredicate<OSMEntity> f)
Adds a custom arbitrary filter that gets executed for each osm entity and determines if it
should be considered for this analyis or not.
|
MapAggregator<U,X> |
osmTag(Collection<? extends OSMTagInterface> tags)
Adds an osm tag filter: The analysis will be restricted to osm entities that have at least one
of the supplied tags (key=value pairs or key=*).
|
MapAggregator<U,X> |
osmTag(OSMTagInterface tag)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
(with an arbitrary value), or this tag key and value.
|
MapAggregator<U,X> |
osmTag(String key)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
(with an arbitrary value).
|
MapAggregator<U,X> |
osmTag(String key,
Collection<String> values)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
and one of the
given values.
|
MapAggregator<U,X> |
osmTag(String key,
Pattern valuePattern)
Adds an osm tag filter: The analysis will be restricted to osm entities that have a tag with
the given key and whose value matches the given regular expression pattern.
|
MapAggregator<U,X> |
osmTag(String key,
String value)
Adds an osm tag filter: The analysis will be restricted to osm entities that have this tag key
and value.
|
default M |
osmType(OSMType type1,
OSMType... otherTypes)
Limits the analysis to the given osm entity types.
|
MapAggregator<U,X> |
osmType(Set<OSMType> typeFilter)
Limits the analysis to the given osm entity types.
|
default M |
osmTypes(EnumSet<OSMType> typeFilter)
Deprecated.
replaced by
osmType(OSMType, OSMType...) |
default M |
osmTypes(OSMType type1,
OSMType... otherTypes)
Deprecated.
replaced by
osmType(OSMType, OSMType...) |
<S> SortedMap<U,S> |
reduce(SerializableSupplier<S> identitySupplier,
SerializableBiFunction<S,X,S> accumulator,
SerializableBinaryOperator<S> combiner)
Map-reduce routine with built-in aggregation.
|
SortedMap<U,X> |
reduce(SerializableSupplier<X> identitySupplier,
SerializableBinaryOperator<X> accumulator)
Map-reduce routine with built-in aggregation (shorthand syntax).
|
Stream<Map.Entry<U,X>> |
stream()
Returns all results as a Stream.
|
SortedMap<U,Number> |
sum()
Sums up the results.
|
<R extends Number> |
sum(SerializableFunction<X,R> mapper)
Sums up the results provided by a given `mapper` function.
|
SortedMap<U,Set<X>> |
uniq()
Gets all unique values of the results.
|
<R> SortedMap<U,Set<R>> |
uniq(SerializableFunction<X,R> mapper)
Gets all unique values of the results provided by a given mapper function.
|
SortedMap<U,Double> |
weightedAverage(SerializableFunction<X,WeightedValue> mapper)
Calculates the weighted average of the results provided by the `mapper` function.
|
default M |
where(Collection<OSMTag> keyValuePairs)
Deprecated.
replaced by
osmTag(Collection) |
default M |
where(OSMTag tag)
Deprecated.
replaced by
osmTag(OSMTagInterface) |
default M |
where(OSMTagKey key)
Deprecated.
replaced by
osmTag(OSMTagInterface) |
default M |
where(SerializablePredicate<OSMEntity> f)
|
default M |
where(String key)
Deprecated.
replaced by
osmTag(String) |
default M |
where(String key,
Collection<String> values)
Deprecated.
replaced by
osmTag(String, Collection) |
default M |
where(String key,
Pattern valuePattern)
Deprecated.
replaced by
osmTag(String, Pattern) |
default M |
where(String key,
String value)
Deprecated.
replaced by
osmTag(String, String) |
@Contract(pure=true) public <V extends Comparable<V> & Serializable> MapAggregator<OSHDBCombinedIndex<U,V>,X> aggregateBy(SerializableFunction<X,V> indexer, Collection<V> zerofill)
indexer - a callback function that returns an index object for each given datazerofill - a collection of values that are expected to be present in the result@Contract(pure=true) public <V extends Comparable<V> & Serializable> MapAggregator<OSHDBCombinedIndex<U,V>,X> aggregateBy(SerializableFunction<X,V> indexer)
indexer - a callback function that returns an index object for each given data.@Contract(pure=true) public MapAggregator<OSHDBCombinedIndex<U,OSHDBTimestamp>,X> aggregateByTimestamp(SerializableFunction<X,OSHDBTimestamp> indexer)
The timestamps returned by the supplied indexing function are matched to the corresponding time intervals
indexer - a callback function that returns a timestamp object for each given data.
Note that if this function returns timestamps outside of the supplied
timestamps() interval results may be undefined@Contract(pure=true) public <V extends Comparable<V> & Serializable,P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> MapAggregator<OSHDBCombinedIndex<U,V>,X> aggregateByGeometry(Map<V,P> geometries) throws UnsupportedOperationException
Cannot be used together with the `groupByEntity()` setting enabled.
UnsupportedOperationException - if this is called when the `groupByEntity()` mode has
been activatedUnsupportedOperationException - when called after any map or flatMap functions are set@Contract(pure=true) public MapAggregator<U,X> areaOfInterest(OSHDBBoundingBox bboxFilter)
Only objects inside or clipped by this bbox will be passed on to the analysis' `mapper` function.
bboxFilter - the bounding box to query the data in@Contract(pure=true) public <P extends org.locationtech.jts.geom.Geometry & org.locationtech.jts.geom.Polygonal> MapAggregator<U,X> areaOfInterest(P polygonFilter)
polygonFilter - the bounding box to query the data in@Contract(pure=true) public MapAggregator<U,X> osmType(Set<OSMType> typeFilter)
typeFilter - the set of osm types to filter (e.g. `EnumSet.of(OSMType.WAY)`)@Contract(pure=true) public MapAggregator<U,X> osmEntityFilter(SerializablePredicate<OSMEntity> f)
f - the filter function to call for each osm entity@Contract(pure=true) public MapAggregator<U,X> osmTag(OSMTagInterface tag)
tag - the tag (key, or key and value) to filter the osm entities for@Contract(pure=true) public MapAggregator<U,X> osmTag(String key)
key - the tag key to filter the osm entities for@Contract(pure=true) public MapAggregator<U,X> osmTag(String key, String value)
key - the tag key to filter the osm entities forvalue - the tag value to filter the osm entities for@Contract(pure=true) public MapAggregator<U,X> osmTag(String key, Collection<String> values)
key - the tag key to filter the osm entities forvalues - an array of tag values to filter the osm entities for@Contract(pure=true) public MapAggregator<U,X> osmTag(String key, Pattern valuePattern)
key - the tag key to filter the osm entities forvaluePattern - a regular expression which the tag value of the osm entity must match@Contract(pure=true) public MapAggregator<U,X> osmTag(Collection<? extends OSMTagInterface> tags)
tags - the tags (key/value pairs or key=*) to filter the osm entities for@Contract(pure=true) public SortedMap<U,Number> sum() throws Exception
The current data values need to be numeric (castable to "Number" type), otherwise a runtime exception will be thrown.
UnsupportedOperationException - if the data cannot be cast to numbersException@Contract(pure=true) public <R extends Number> SortedMap<U,R> sum(SerializableFunction<X,R> mapper) throws Exception
This is a shorthand for `.map(mapper).sum()`, with the difference that here the numerical return type of the `mapper` is ensured.
R - the numeric type that is returned by the `mapper` functionmapper - function that returns the numbers to sum upException@Contract(pure=true) public SortedMap<U,Integer> count() throws Exception
Exception@Contract(pure=true) public SortedMap<U,Set<X>> uniq() throws Exception
For example, this can be used together with the OSMContributionView to get the total amount of unique users editing specific feature types.
Exception@Contract(pure=true) public <R> SortedMap<U,Set<R>> uniq(SerializableFunction<X,R> mapper) throws Exception
This is a shorthand for `.map(mapper).uniq()`.
R - the type that is returned by the `mapper` functionmapper - function that returns some valuesException@Contract(pure=true) public SortedMap<U,Integer> countUniq() throws Exception
For example, this can be used together with the OSMContributionView to get the number of unique users editing specific feature types.
Exception@Contract(pure=true) public SortedMap<U,Double> average() throws Exception
The current data values need to be numeric (castable to "Number" type), otherwise a runtime exception will be thrown.
UnsupportedOperationException - if the data cannot be cast to numbersException@Contract(pure=true) public <R extends Number> SortedMap<U,Double> average(SerializableFunction<X,R> mapper) throws Exception
R - the numeric type that is returned by the `mapper` functionmapper - function that returns the numbers to averageException@Contract(pure=true) public SortedMap<U,Double> weightedAverage(SerializableFunction<X,WeightedValue> mapper) throws Exception
The mapper must return an object of the type `WeightedValue` which contains a numeric value associated with a (floating point) weight.
mapper - function that gets called for each entity snapshot or modification, needs to
return the value and weight combination of numbers to averageException@Contract(pure=true) public SortedMap<U,Double> estimatedMedian() throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
Exception@Contract(pure=true) public <R extends Number> SortedMap<U,Double> estimatedMedian(SerializableFunction<X,R> mapper) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the mean forException@Contract(pure=true) public SortedMap<U,Double> estimatedQuantile(double q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
q - the desired quantile to calculate (as a number between 0 and 1)Exception@Contract(pure=true) public <R extends Number> SortedMap<U,Double> estimatedQuantile(SerializableFunction<X,R> mapper, double q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the quantile forq - the desired quantile to calculate (as a number between 0 and 1)Exception@Contract(pure=true) public SortedMap<U,List<Double>> estimatedQuantiles(Iterable<Double> q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
q - the desired quantiles to calculate (as a collection of numbers between 0 and 1)Exception@Contract(pure=true) public <R extends Number> SortedMap<U,List<Double>> estimatedQuantiles(SerializableFunction<X,R> mapper, Iterable<Double> q) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the quantiles forq - the desired quantiles to calculate (as a collection of numbers between 0 and 1)Exception@Contract(pure=true) public SortedMap<U,DoubleUnaryOperator> estimatedQuantiles() throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
Exception@Contract(pure=true) public <R extends Number> SortedMap<U,DoubleUnaryOperator> estimatedQuantiles(SerializableFunction<X,R> mapper) throws Exception
Uses the t-digest algorithm to calculate estimates for the quantiles in a map-reduce system: https://raw.githubusercontent.com/tdunning/t-digest/master/docs/t-digest-paper/histo.pdf
mapper - function that returns the numbers to generate the quantiles forException@Deprecated public void forEach(SerializableBiConsumer<U,List<X>> action) throws Exception
This method can be handy for testing purposes. But note that since the `action` doesn't produce a return value, it must facilitate its own way of producing output.
If you'd like to use such a "forEach" in a non-test use case, use `.collect().forEach()` or `.stream().forEach()` instead.
action - function that gets called for each transformed data entryException@Contract(pure=true) public SortedMap<U,List<X>> collect() throws Exception
Exception@Contract(pure=true) public Stream<Map.Entry<U,X>> stream() throws Exception
Exception@Contract(pure=true) public <R> MapAggregator<U,R> map(SerializableFunction<X,R> mapper)
map in interface Mappable<X>R - an arbitrary data type which is the return type of the transformation `map` functionmapper - function that will be applied to each data entry (osm entity snapshot or
contribution)@Contract(pure=true) public <R> MapAggregator<U,R> flatMap(SerializableFunction<X,Iterable<R>> flatMapper)
The results of this function will be "flattened", meaning that they can be for example transformed again by setting additional `map` functions.
flatMap in interface Mappable<X>R - an arbitrary data type which is the return type of the transformation `map` functionflatMapper - function that will be applied to each data entry (osm entity snapshot or
contribution) and returns a list of results@Contract(pure=true) public MapAggregator<U,X> filter(SerializablePredicate<X> f)
@Contract(pure=true) public <S> SortedMap<U,S> reduce(SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,X,S> accumulator, SerializableBinaryOperator<S> combiner) throws Exception
This can be used to perform an arbitrary reduce routine whose results are aggregated separately according to some custom index value.
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.
S - the data type used to contain the "reduced" (intermediate and final) resultsidentitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <R>)
and an accumulation value (type <S>, e.g. the result of `identitySupplier()`)
and returns the "sum" of the two; contrary to `combiner`, this function is allowed to
alter (mutate) the state of the accumulation value (e.g. directly adding new values to
an existing Set object)combiner - a function that calculates the "sum" of two <S> values; this function
must be pure (have no side effects), and is not allowed to alter the state of the two
input objects it gets!Exception@Contract(pure=true) public SortedMap<U,X> reduce(SerializableSupplier<X> identitySupplier, SerializableBinaryOperator<X> accumulator) throws Exception
This can be used to perform an arbitrary reduce routine whose results are aggregated separately according to some custom index value.
This variant is shorter to program than `reduce(identitySupplier, accumulator, combiner)`, but can only be used if the result type is the same as the current `map`ped type <X>. Also this variant can be less efficient since it cannot benefit from the mutability freedoms the accumulator+combiner approach has.
The combination of the used types and identity/reducer functions must make "mathematical" sense:
Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.
identitySupplier - a factory function that returns a new starting value to reduce results
into (e.g. when summing values, one needs to start at zero)accumulator - a function that takes a result from the `mapper` function (type <X>)
and an accumulation value (also of type <X>, e.g. the result of
`identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this
function is not to alter (mutate) the state of the accumulation value (e.g. directly
adding new values to an existing Set object)Exceptionpublic M osmType(OSMType type1, OSMType... otherTypes)
type1 - the set of osm types to filter (e.g. `OSMType.NODE`)otherTypes - more osm types which should be analyzed@Deprecated public M osmTypes(EnumSet<OSMType> typeFilter)
osmType(OSMType, OSMType...)@Deprecated public M osmTypes(OSMType type1, OSMType... otherTypes)
osmType(OSMType, OSMType...)@Deprecated public M where(SerializablePredicate<OSMEntity> f)
osmEntityFilter(org.heigit.bigspatialdata.oshdb.api.generic.function.SerializablePredicate<org.heigit.bigspatialdata.oshdb.osm.OSMEntity>)@Deprecated public M where(OSMTagKey key)
osmTag(OSMTagInterface)@Deprecated public M where(String key)
osmTag(String)@Deprecated public M where(OSMTag tag)
osmTag(OSMTagInterface)@Deprecated public M where(String key, String value)
osmTag(String, String)@Deprecated public M where(String key, Collection<String> values)
osmTag(String, Collection)@Deprecated public M where(String key, Pattern valuePattern)
osmTag(String, Pattern)@Deprecated public M where(Collection<OSMTag> keyValuePairs)
osmTag(Collection)Copyright © 2016–2020 HeiGIT. All rights reserved.