org.openstreetmap.atlas.generator

Class AtlasGenerator

java.lang.Object

org.openstreetmap.atlas.utilities.runtime.Command

org.openstreetmap.atlas.generator.tools.spark.SparkJob

org.openstreetmap.atlas.generator.AtlasGenerator

All Implemented Interfaces:

java.io.Serializable

public class AtlasGenerator
extends SparkJob

Generate Atlas Shards for a specific version and a specific set of countries

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.openstreetmap.atlas.utilities.runtime.Command

org.openstreetmap.atlas.utilities.runtime.Command.Flag, org.openstreetmap.atlas.utilities.runtime.Command.Optionality, org.openstreetmap.atlas.utilities.runtime.Command.Switch<T>, org.openstreetmap.atlas.utilities.runtime.Command.SwitchList

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ATLAS_FOLDER
static java.lang.String	COUNTRY_STATISTICS_FOLDER
static java.lang.String	RAW_ATLAS_FOLDER
static java.lang.String	SHARD_DELTAS_ADDED_FOLDER
static java.lang.String	SHARD_DELTAS_CHANGED_FOLDER
static java.lang.String	SHARD_DELTAS_FOLDER
static java.lang.String	SHARD_DELTAS_REMOVED_FOLDER
static java.lang.String	SHARD_STATISTICS_FOLDER
static java.lang.String	SLICED_RAW_ATLAS_FOLDER

Fields inherited from class org.openstreetmap.atlas.generator.tools.spark.SparkJob

ADDITIONAL_SPARK_OPTIONS, COMPRESS_OUTPUT, EXITED_FILE, FAILED_FILE, INPUT, MASTER, OUTPUT, SAVING_SEPARATOR, SENSITIVE_CONFIGURATION_PATTERN, SPARK_CONTEXT_PROVIDER, SPARK_OPTIONS, STARTED_FILE, STARTED_FOLDER, SUCCESS_FILE

Constructor Summary

Constructors

Constructor and Description
AtlasGenerator()

Method Summary

Modifier and Type	Method and Description
protected static java.util.List<org.openstreetmap.atlas.generator.AtlasGenerationTask>	generateTasks(org.openstreetmap.atlas.utilities.collections.StringList countries, org.openstreetmap.atlas.geography.boundary.CountryBoundaryMap boundaryMap, org.openstreetmap.atlas.geography.sharding.Sharding sharding) Generates a List of AtlasGenerationTasks for given countries using given CountryBoundaryMap and Sharding strategy.
java.lang.String	getName()
static void	main(java.lang.String[] args)
protected java.util.List<java.lang.String>	outputToClean(org.openstreetmap.atlas.utilities.runtime.CommandMap command) Define all the folders to clean before a run.
void	start(org.openstreetmap.atlas.utilities.runtime.CommandMap command) The spark Job
protected org.openstreetmap.atlas.utilities.runtime.Command.SwitchList	switches()

Methods inherited from class org.openstreetmap.atlas.generator.tools.spark.SparkJob

configuration, configurationMap, getAlternateParallelFolderOutput, getAlternateSubFolderOutput, getContext, input, onRun, output, resource, resource, setContext, splitAndSaveAsHadoopFile

Methods inherited from class org.openstreetmap.atlas.utilities.runtime.Command

commandSummary, getCommandMap, lastRawCommand, run, runWithoutQuitting

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ATLAS_FOLDER

public static final java.lang.String ATLAS_FOLDER

See Also:

Constant Field Values

RAW_ATLAS_FOLDER

public static final java.lang.String RAW_ATLAS_FOLDER

See Also:

Constant Field Values

SLICED_RAW_ATLAS_FOLDER

public static final java.lang.String SLICED_RAW_ATLAS_FOLDER

See Also:

Constant Field Values

SHARD_STATISTICS_FOLDER

public static final java.lang.String SHARD_STATISTICS_FOLDER

See Also:

Constant Field Values

COUNTRY_STATISTICS_FOLDER

public static final java.lang.String COUNTRY_STATISTICS_FOLDER

See Also:

Constant Field Values

SHARD_DELTAS_FOLDER

public static final java.lang.String SHARD_DELTAS_FOLDER

See Also:

Constant Field Values

SHARD_DELTAS_ADDED_FOLDER

public static final java.lang.String SHARD_DELTAS_ADDED_FOLDER

See Also:

Constant Field Values

SHARD_DELTAS_CHANGED_FOLDER

public static final java.lang.String SHARD_DELTAS_CHANGED_FOLDER

See Also:

Constant Field Values

SHARD_DELTAS_REMOVED_FOLDER

public static final java.lang.String SHARD_DELTAS_REMOVED_FOLDER

See Also:

Constant Field Values

Constructor Detail

AtlasGenerator

public AtlasGenerator()

Method Detail

main

public static void main(java.lang.String[] args)

generateTasks

protected static java.util.List<org.openstreetmap.atlas.generator.AtlasGenerationTask> generateTasks(org.openstreetmap.atlas.utilities.collections.StringList countries,
                                                                                                     org.openstreetmap.atlas.geography.boundary.CountryBoundaryMap boundaryMap,
                                                                                                     org.openstreetmap.atlas.geography.sharding.Sharding sharding)

Generates a List of AtlasGenerationTasks for given countries using given CountryBoundaryMap and Sharding strategy.

Parameters:

countries - Countries to generate tasks for

boundaryMap - CountryBoundaryMap to read country boundaries

sharding - Sharding strategy

Returns:

List of AtlasGenerationTasks

getName

public java.lang.String getName()

Specified by:

getName in class SparkJob

Returns:

The name of the job

start

public void start(org.openstreetmap.atlas.utilities.runtime.CommandMap command)

Description copied from class: SparkJob

The spark Job

Specified by:

start in class SparkJob

Parameters:

command - The arguments passed to the main method

outputToClean

protected java.util.List<java.lang.String> outputToClean(org.openstreetmap.atlas.utilities.runtime.CommandMap command)

Description copied from class: SparkJob

Define all the folders to clean before a run.

Overrides:

outputToClean in class SparkJob

Parameters:

command - The command parameters sent to the main class.

Returns:

All the paths to clean

switches

protected org.openstreetmap.atlas.utilities.runtime.Command.SwitchList switches()

Overrides:

switches in class SparkJob