eu.europeana.cloud.migrator

Class DefaultResourceProvider

java.lang.Object

eu.europeana.cloud.migrator.DefaultResourceProvider

All Implemented Interfaces:

ResourceProvider

Direct Known Subclasses:

EuropeanaNewspapersResourceProvider, FoodAndDrinkResourceProvider, RoyalArmoryResourceProvider

public abstract class DefaultResourceProvider
extends Object
implements ResourceProvider

Field Summary

Fields

Modifier and Type	Field and Description
static String	CONTACT_PERSON_KEY
protected String	dataProviderId
static String	DEFAULT_CONTACT_PERSON
static String	DEFAULT_DIGITAL_LIBRARY_WEBSITE
static String	DEFAULT_DIGITAL_LIBRARY_WEBSITE_URL
static String	DEFAULT_OFFICIAL_ADDRESS
static String	DEFAULT_ORGANISATION_NAME
static String	DEFAULT_ORGANISATION_WEBSITE
static String	DEFAULT_ORGANISATION_WEBSITE_URL
static String	DEFAULT_REMARKS
static String	DIGITAL_LIBRARY_WEBSITE_KEY
static String	DIGITAL_LIBRARY_WEBSITE_URL_KEY
protected boolean	local
protected List<URI>	locations
protected String	mappingFile
static String	OFFICIAL_ADDRESS_KEY
static String	ORGANISATION_NAME_KEY
static String	ORGANISATION_WEBSITE_KEY
static String	ORGANISATION_WEBSITE_URL_KEY
static String	PROPERTIES_EXTENSION
static String	REMARKS_KEY
protected String	representationName

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	DefaultResourceProvider(String representationName, String mappingFile, String locations, String dataProviderId)

Method Summary

Methods

Modifier and Type	Method and Description
protected eu.europeana.cloud.common.model.DataProviderProperties	getDataProviderPropertiesFromFile(File dpFile)
protected eu.europeana.cloud.common.model.DataProviderProperties	getDefaultDataProviderProperties()
int	getFileCount(String localId) Determine the number of files that should be added for the specified local identifier.
String	getFilename(String location, String path) Default implementation of determining filename from path.
List<URI>	getLocations() Return resource provider location as URI.
protected FilePaths	getProviderPaths(String location, String providerId, Map<String,List<FilePaths>> paths)
URI	getProvidersLocation(String providerId)
String	getRepresentationName() Return the default representation name of records added for this resource provider.
Map<String,String>	getReversedMapping() Mapping file can associate local record identifier with a set of paths to files.
boolean	isLocal() Checks whether the provider location is local.
Map<String,List<FilePaths>>	scan() Scan location in case it's local.
List<FilePaths>	split(List<FilePaths> paths) Resource provider implementation may split the file paths that will be migrated into several parts.
boolean	usePathsFile() Indicates whether file paths to process should be stored in a file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface eu.europeana.cloud.migrator.ResourceProvider

getDataProviderId, getDataProviderProperties, getLocalIdentifier, getResourceProviderId

Field Detail

ORGANISATION_NAME_KEY

public static final String ORGANISATION_NAME_KEY

See Also:

Constant Field Values

OFFICIAL_ADDRESS_KEY

public static final String OFFICIAL_ADDRESS_KEY

See Also:

Constant Field Values

ORGANISATION_WEBSITE_KEY

public static final String ORGANISATION_WEBSITE_KEY

See Also:

Constant Field Values

ORGANISATION_WEBSITE_URL_KEY

public static final String ORGANISATION_WEBSITE_URL_KEY

See Also:

Constant Field Values

DIGITAL_LIBRARY_WEBSITE_KEY

public static final String DIGITAL_LIBRARY_WEBSITE_KEY

See Also:

Constant Field Values

DIGITAL_LIBRARY_WEBSITE_URL_KEY

public static final String DIGITAL_LIBRARY_WEBSITE_URL_KEY

See Also:

Constant Field Values

CONTACT_PERSON_KEY

public static final String CONTACT_PERSON_KEY

See Also:

Constant Field Values

REMARKS_KEY

public static final String REMARKS_KEY

See Also:

Constant Field Values

DEFAULT_ORGANISATION_NAME

public static final String DEFAULT_ORGANISATION_NAME

See Also:

Constant Field Values

DEFAULT_OFFICIAL_ADDRESS

public static final String DEFAULT_OFFICIAL_ADDRESS

See Also:

Constant Field Values

DEFAULT_ORGANISATION_WEBSITE

public static final String DEFAULT_ORGANISATION_WEBSITE

See Also:

Constant Field Values

DEFAULT_ORGANISATION_WEBSITE_URL

public static final String DEFAULT_ORGANISATION_WEBSITE_URL

See Also:

Constant Field Values

DEFAULT_DIGITAL_LIBRARY_WEBSITE

public static final String DEFAULT_DIGITAL_LIBRARY_WEBSITE

See Also:

Constant Field Values

DEFAULT_DIGITAL_LIBRARY_WEBSITE_URL

public static final String DEFAULT_DIGITAL_LIBRARY_WEBSITE_URL

See Also:

Constant Field Values

DEFAULT_CONTACT_PERSON

public static final String DEFAULT_CONTACT_PERSON

See Also:

Constant Field Values

DEFAULT_REMARKS

public static final String DEFAULT_REMARKS

See Also:

Constant Field Values

PROPERTIES_EXTENSION

public static final String PROPERTIES_EXTENSION

See Also:

Constant Field Values

representationName

protected String representationName

mappingFile

protected String mappingFile

locations

protected List<URI> locations

local

protected boolean local

dataProviderId

protected String dataProviderId

Constructor Detail

DefaultResourceProvider

protected DefaultResourceProvider(String representationName,
                       String mappingFile,
                       String locations,
                       String dataProviderId)

Method Detail

getDataProviderPropertiesFromFile

protected eu.europeana.cloud.common.model.DataProviderProperties getDataProviderPropertiesFromFile(File dpFile)

getDefaultDataProviderProperties

protected eu.europeana.cloud.common.model.DataProviderProperties getDefaultDataProviderProperties()

getRepresentationName

public String getRepresentationName()

Description copied from interface: ResourceProvider

Return the default representation name of records added for this resource provider. The representation name is the same for all records within one run of migration tool.

Specified by:

getRepresentationName in interface ResourceProvider

Returns:

default representation name

getLocations

public List<URI> getLocations()

Description copied from interface: ResourceProvider

Return resource provider location as URI. This may indicate directory or remote resource URL.

Specified by:

getLocations in interface ResourceProvider

Returns:

resource provider locaction as URI

isLocal

public boolean isLocal()

Description copied from interface: ResourceProvider

Checks whether the provider location is local.

Specified by:

isLocal in interface ResourceProvider

Returns:

true when location is local or false otherwise

scan

public Map<String,List<FilePaths>> scan()

Description copied from interface: ResourceProvider

Scan location in case it's local. Otherwise return an empty map. The returned map should associate provider identifier (it can be determined from the path) with the list of files to add.

Specified by:

scan in interface ResourceProvider

Returns:

map of provider identifiers associated to the list of paths to files

getProviderPaths

protected FilePaths getProviderPaths(String location,
                         String providerId,
                         Map<String,List<FilePaths>> paths)

getProvidersLocation

public URI getProvidersLocation(String providerId)

getFilename

public String getFilename(String location,
                 String path)

Default implementation of determining filename from path. The result is either path without location, part of the specified path after "/" or "\" character or path itself.

Specified by:

getFilename in interface ResourceProvider

Parameters:

location - location where path is placed, usually path starts with location

path - path to file either local or remote in URI syntax

Returns:

filename to be stored in ECloud

getFileCount

public int getFileCount(String localId)

Description copied from interface: ResourceProvider

Determine the number of files that should be added for the specified local identifier. It may be done using the mapping file or any other way proper for the resource provider. When it is impossible to determine -1 is returned.

Specified by:

getFileCount in interface ResourceProvider

Parameters:

localId - local identifier of the record

Returns:

number of files the record should have or -1 if it's impossible to determine

split

public List<FilePaths> split(List<FilePaths> paths)

Description copied from interface: ResourceProvider

Resource provider implementation may split the file paths that will be migrated into several parts. The way of splitting is specific for a resource provider. The idea of splitting the file paths is to use many threads during the migration process. For each FilePaths object there will be a separate thread.

Specified by:

split in interface ResourceProvider

Parameters:

paths - paths to files that will be migrated

Returns:

a list of lists

usePathsFile

public boolean usePathsFile()

Description copied from interface: ResourceProvider

Indicates whether file paths to process should be stored in a file. If false they are stored in memory.

Specified by:

usePathsFile in interface ResourceProvider

Returns:

true when paths should be stored in a file or false otherwise

getReversedMapping

public Map<String,String> getReversedMapping()

Description copied from interface: ResourceProvider

Mapping file can associate local record identifier with a set of paths to files. Internally mapping is stored in a reversed form since the lookups are done using path to file and not the local identifier. This method returns reversed mapping.

Specified by:

getReversedMapping in interface ResourceProvider

Returns:

reversed mapping