org.heigit.bigspatialdata.oshdb.api.mapreducer.backend

Class MapReducerJdbcSinglethread<X>

java.lang.Object

org.heigit.bigspatialdata.oshdb.api.mapreducer.MapReducer<X>

org.heigit.bigspatialdata.oshdb.api.mapreducer.backend.MapReducerJdbcSinglethread<X>

All Implemented Interfaces:

Serializable, Mappable<X>

public class MapReducerJdbcSinglethread<X>
extends MapReducer<X>

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected long	executionStartTimeMillis Stores the start time of reduce/stream operation as returned by System.currentTimeMillis().

Fields inherited from class org.heigit.bigspatialdata.oshdb.api.mapreducer.MapReducer

bboxFilter, keytables, oshdb, timeout, tstamps, typeFilter

Constructor Summary

Constructors

Constructor and Description
MapReducerJdbcSinglethread(OSHDBDatabase oshdb, Class<? extends OSHDBMapReducible> forClass)

Method Summary

Modifier and Type	Method and Description
protected MapReducer<X>	copy()
protected <R,S> S	flatMapReduceCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<R>> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) Generic "flat" version of the map-reduce used by the `OSMContributionView`, with by-osm-id grouped input to the `mapper` function.
protected <R,S> S	flatMapReduceCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<R>> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) Generic "flat" version of the map-reduce used by the `OSMEntitySnapshotView`, with by-osm-id grouped input to the `mapper` function.
protected Stream<X>	flatMapStreamCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<X>> mapper)
protected Stream<X>	flatMapStreamCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<X>> mapper)
protected ResultSet	getOshCellsRawDataFromDb(XYGridTree.CellIdRange cellIdRange)
protected Stream<GridOSHEntity>	getOshCellsStream(XYGridTree.CellIdRange cellIdRange)
boolean	isActive()
boolean	isCancelable() Returns if the current backend can be canceled (e.g.
protected <R,S> S	mapReduceCellsOSMContribution(SerializableFunction<OSMContribution,R> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) Generic map-reduce used by the `OSMContributionView`.
protected <R,S> S	mapReduceCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,R> mapper, SerializableSupplier<S> identitySupplier, SerializableBiFunction<S,R,S> accumulator, SerializableBinaryOperator<S> combiner) Generic map-reduce used by the `OSMEntitySnapshotView`.
protected Stream<X>	mapStreamCellsOSMContribution(SerializableFunction<OSMContribution,X> mapper)
protected Stream<X>	mapStreamCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,X> mapper)
default M	osmType(OSMType type1, OSMType... otherTypes) Limits the analysis to the given osm entity types.
default M	osmTypes(EnumSet<OSMType> typeFilter) Deprecated. replaced by osmType(OSMType, OSMType...)
default M	osmTypes(OSMType type1, OSMType... otherTypes) Deprecated. replaced by osmType(OSMType, OSMType...)
protected GridOSHEntity	readOshCellRawData(ResultSet oshCellsRawData) Returns data of one cell from the raw data stream.
default M	where(Collection<OSMTag> keyValuePairs) Deprecated. replaced by osmTag(Collection)
default M	where(OSMTag tag) Deprecated. replaced by osmTag(OSMTagInterface)
default M	where(OSMTagKey key) Deprecated. replaced by osmTag(OSMTagInterface)
default M	where(SerializablePredicate<OSMEntity> f) Deprecated. replaced by osmEntityFilter(org.heigit.bigspatialdata.oshdb.api.generic.function.SerializablePredicate<org.heigit.bigspatialdata.oshdb.osm.OSMEntity>)
default M	where(String key) Deprecated. replaced by osmTag(String)
default M	where(String key, Collection<String> values) Deprecated. replaced by osmTag(String, Collection)
default M	where(String key, Pattern valuePattern) Deprecated. replaced by osmTag(String, Pattern)
default M	where(String key, String value) Deprecated. replaced by osmTag(String, String)

Methods inherited from class org.heigit.bigspatialdata.oshdb.api.mapreducer.MapReducer

aggregateBy, aggregateBy, aggregateByGeometry, aggregateByTimestamp, aggregateByTimestamp, areaOfInterest, areaOfInterest, average, average, collect, count, countUniq, estimatedMedian, estimatedMedian, estimatedQuantile, estimatedQuantile, estimatedQuantiles, estimatedQuantiles, estimatedQuantiles, estimatedQuantiles, filter, flatMap, forEach, getCellIdRanges, getFilter, getPolyFilter, getPreFilter, getTagInterpreter, getTagTranslator, groupByEntity, keytables, map, osmEntityFilter, osmTag, osmTag, osmTag, osmTag, osmTag, osmTag, osmType, reduce, reduce, stream, sum, sum, tagInterpreter, timestamps, timestamps, timestamps, timestamps, timestamps, uniq, uniq, weightedAverage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

executionStartTimeMillis

protected long executionStartTimeMillis

Stores the start time of reduce/stream operation as returned by System.currentTimeMillis(). Used to determine query timeouts.

Constructor Detail

MapReducerJdbcSinglethread

public MapReducerJdbcSinglethread(OSHDBDatabase oshdb,
                                  Class<? extends OSHDBMapReducible> forClass)

Method Detail

copy

@NotNull
protected MapReducer<X> copy()

Specified by:

copy in class MapReducer<X>

isCancelable

public boolean isCancelable()

Description copied from class: MapReducer

Returns if the current backend can be canceled (e.g. in a query timeout).

Overrides:

isCancelable in class MapReducer<X>

mapReduceCellsOSMContribution

protected <R,S> S mapReduceCellsOSMContribution(SerializableFunction<OSMContribution,R> mapper,
                                                SerializableSupplier<S> identitySupplier,
                                                SerializableBiFunction<S,R,S> accumulator,
                                                SerializableBinaryOperator<S> combiner)
                                         throws Exception

Description copied from class: MapReducer

Generic map-reduce used by the `OSMContributionView`.

The combination of the used types and identity/reducer functions must make "mathematical" sense:

• the accumulator and combiner functions need to be associative,
• values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
• the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.

Overrides:

mapReduceCellsOSMContribution in class MapReducer<X>

Type Parameters:

R - the data type returned by the `mapper` function

S - the data type used to contain the "reduced" (intermediate and final) results

Parameters:

mapper - a function that's called for each `OSMContribution`

identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)

accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)

combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!

Returns:

the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)

Throws:

Exception

flatMapReduceCellsOSMContributionGroupedById

protected <R,S> S flatMapReduceCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<R>> mapper,
                                                               SerializableSupplier<S> identitySupplier,
                                                               SerializableBiFunction<S,R,S> accumulator,
                                                               SerializableBinaryOperator<S> combiner)
                                                        throws Exception

Description copied from class: MapReducer

Generic "flat" version of the map-reduce used by the `OSMContributionView`, with by-osm-id grouped input to the `mapper` function.

Contrary to the "normal" map-reduce, the "flat" version adds the possibility to return any number of results in the `mapper` function. Additionally, this interface provides the `mapper` function with a list of all `OSMContribution`s of a particular OSM entity. This is used to do more complex analyses that require the full edit history of the respective OSM entities as input.

The combination of the used types and identity/reducer functions must make "mathematical" sense:

• the accumulator and combiner functions need to be associative,
• values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
• the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.

Overrides:

flatMapReduceCellsOSMContributionGroupedById in class MapReducer<X>

Type Parameters:

R - the data type returned by the `mapper` function

S - the data type used to contain the "reduced" (intermediate and final) results

Parameters:

mapper - a function that's called for all `OSMContribution`s of a particular OSM entity; returns a list of results (which can have any number of entries).

identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)

accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)

combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!

Returns:

the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)

Throws:

Exception

mapReduceCellsOSMEntitySnapshot

protected <R,S> S mapReduceCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,R> mapper,
                                                  SerializableSupplier<S> identitySupplier,
                                                  SerializableBiFunction<S,R,S> accumulator,
                                                  SerializableBinaryOperator<S> combiner)
                                           throws Exception

Description copied from class: MapReducer

Generic map-reduce used by the `OSMEntitySnapshotView`.

The combination of the used types and identity/reducer functions must make "mathematical" sense:

• the accumulator and combiner functions need to be associative,
• values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
• the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.

Overrides:

mapReduceCellsOSMEntitySnapshot in class MapReducer<X>

Type Parameters:

R - the data type returned by the `mapper` function

S - the data type used to contain the "reduced" (intermediate and final) results

Parameters:

mapper - a function that's called for each `OSMEntitySnapshot`

identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)

accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)

combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!

Returns:

the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)

Throws:

Exception

flatMapReduceCellsOSMEntitySnapshotGroupedById

protected <R,S> S flatMapReduceCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<R>> mapper,
                                                                 SerializableSupplier<S> identitySupplier,
                                                                 SerializableBiFunction<S,R,S> accumulator,
                                                                 SerializableBinaryOperator<S> combiner)
                                                          throws Exception

Description copied from class: MapReducer

Generic "flat" version of the map-reduce used by the `OSMEntitySnapshotView`, with by-osm-id grouped input to the `mapper` function.

Contrary to the "normal" map-reduce, the "flat" version adds the possibility to return any number of results in the `mapper` function. Additionally, this interface provides the `mapper` function with a list of all `OSMContribution`s of a particular OSM entity. This is used to do more complex analyses that require the full list of snapshots of the respective OSM entities as input.

The combination of the used types and identity/reducer functions must make "mathematical" sense:

• the accumulator and combiner functions need to be associative,
• values generated by the identitySupplier factory must be an identity for the combiner function: `combiner(identitySupplier(),x)` must be equal to `x`,
• the combiner function must be compatible with the accumulator function: `combiner(u, accumulator(identitySupplier(), t)) == accumulator.apply(u, t)`

Functionally, this interface is similar to Java8 Stream's reduce(identity,accumulator,combiner) interface.

Overrides:

flatMapReduceCellsOSMEntitySnapshotGroupedById in class MapReducer<X>

Type Parameters:

R - the data type returned by the `mapper` function

S - the data type used to contain the "reduced" (intermediate and final) results

Parameters:

mapper - a function that's called for all `OSMEntitySnapshot`s of a particular OSM entity; returns a list of results (which can have any number of entries)

identitySupplier - a factory function that returns a new starting value to reduce results into (e.g. when summing values, one needs to start at zero)

accumulator - a function that takes a result from the `mapper` function (type <R>) and an accumulation value (type <S>, e.g. the result of `identitySupplier()`) and returns the "sum" of the two; contrary to `combiner`, this function is allowed to alter (mutate) the state of the accumulation value (e.g. directly adding new values to an existing Set object)

combiner - a function that calculates the "sum" of two <S> values; this function must be pure (have no side effects), and is not allowed to alter the state of the two input objects it gets!

Returns:

the result of the map-reduce operation, the final result of the last call to the `combiner` function, after all `mapper` results have been aggregated (in the `accumulator` and `combiner` steps)

Throws:

Exception

mapStreamCellsOSMContribution

protected Stream<X> mapStreamCellsOSMContribution(SerializableFunction<OSMContribution,X> mapper)
                                           throws Exception

Overrides:

mapStreamCellsOSMContribution in class MapReducer<X>

Throws:

Exception

flatMapStreamCellsOSMContributionGroupedById

protected Stream<X> flatMapStreamCellsOSMContributionGroupedById(SerializableFunction<List<OSMContribution>,Iterable<X>> mapper)
                                                          throws Exception

Overrides:

flatMapStreamCellsOSMContributionGroupedById in class MapReducer<X>

Throws:

Exception

mapStreamCellsOSMEntitySnapshot

protected Stream<X> mapStreamCellsOSMEntitySnapshot(SerializableFunction<OSMEntitySnapshot,X> mapper)
                                             throws Exception

Overrides:

mapStreamCellsOSMEntitySnapshot in class MapReducer<X>

Throws:

Exception

flatMapStreamCellsOSMEntitySnapshotGroupedById

protected Stream<X> flatMapStreamCellsOSMEntitySnapshotGroupedById(SerializableFunction<List<OSMEntitySnapshot>,Iterable<X>> mapper)
                                                            throws Exception

Overrides:

flatMapStreamCellsOSMEntitySnapshotGroupedById in class MapReducer<X>

Throws:

Exception

isActive

public boolean isActive()

getOshCellsRawDataFromDb

protected ResultSet getOshCellsRawDataFromDb(XYGridTree.CellIdRange cellIdRange)
                                      throws SQLException

Throws:

SQLException

readOshCellRawData

protected GridOSHEntity readOshCellRawData(ResultSet oshCellsRawData)
                                    throws IOException,
                                           ClassNotFoundException,
                                           SQLException

Returns data of one cell from the raw data stream.

Throws:

IOException

ClassNotFoundException

SQLException

getOshCellsStream

@Nonnull
protected Stream<GridOSHEntity> getOshCellsStream(XYGridTree.CellIdRange cellIdRange)

osmType

public M osmType(OSMType type1,
                 OSMType... otherTypes)

Limits the analysis to the given osm entity types.

Parameters:

type1 - the set of osm types to filter (e.g. `OSMType.NODE`)

otherTypes - more osm types which should be analyzed

Returns:

`this` mapReducer (can be used to chain multiple commands together)

osmTypes

@Deprecated
public M osmTypes(EnumSet<OSMType> typeFilter)

Deprecated. replaced by osmType(OSMType, OSMType...)

deprecated.

osmTypes

@Deprecated
public M osmTypes(OSMType type1,
                              OSMType... otherTypes)

Deprecated. replaced by osmType(OSMType, OSMType...)

deprecated.

where

@Deprecated
public M where(SerializablePredicate<OSMEntity> f)

Deprecated. replaced by osmEntityFilter(org.heigit.bigspatialdata.oshdb.api.generic.function.SerializablePredicate<org.heigit.bigspatialdata.oshdb.osm.OSMEntity>)

deprecated.

where

@Deprecated
public M where(OSMTagKey key)

Deprecated. replaced by osmTag(OSMTagInterface)

deprecated.

where

@Deprecated
public M where(String key)

Deprecated. replaced by osmTag(String)

deprecated.

where

@Deprecated
public M where(OSMTag tag)

Deprecated. replaced by osmTag(OSMTagInterface)

deprecated.

where

@Deprecated
public M where(String key,
                           String value)

Deprecated. replaced by osmTag(String, String)

deprecated.

where

@Deprecated
public M where(String key,
                           Collection<String> values)

Deprecated. replaced by osmTag(String, Collection)

deprecated.

where

@Deprecated
public M where(String key,
                           Pattern valuePattern)

Deprecated. replaced by osmTag(String, Pattern)

deprecated.

where

@Deprecated
public M where(Collection<OSMTag> keyValuePairs)

Deprecated. replaced by osmTag(Collection)

deprecated.