io.cdap.wrangler.dataset.schema

Class SchemaRegistry

java.lang.Object

io.cdap.wrangler.dataset.schema.SchemaRegistry

public final class SchemaRegistry
extends Object

This class SchemaRegistry is responsible for managing the schema registry store. A schema is uniquely identified by an ID and contains other metadata about the schema, such as a name, description, type, etc. By itself, a schema does not contain an actual schema object. It is more like a group of schema entries. A schema entry contains the actual schema bytes and is uniquely identified by the schema it belongs to and a version number. A schema entry is added to a schema and can also be removed. In addition, each schema keeps track of the latest schema entry. Schema information is stored in two tables, one for the schema metadata and one for the schema entries. The schema metadata table contains nine columns: namespace, id, name, description, created, updated, type, auto, current The namespace and id columns form the primary key for the metadata table. The current column points to the latest schema entry version, and the auto column is used for generating version numbers for any new schema entries. The schema entry table contains four columns: namespace, id, version, schema The namespace, id, and version columns form the primary key, while the schema column contains the actual schema.

Field Summary

Fields

Modifier and Type	Field and Description
static io.cdap.cdap.spi.data.table.StructuredTableSpecification	ENTRY_TABLE_SPEC
static io.cdap.cdap.spi.data.table.StructuredTableSpecification	META_TABLE_SPEC

Constructor Summary

Constructors

Constructor and Description
SchemaRegistry(io.cdap.cdap.spi.data.StructuredTable metaTable, io.cdap.cdap.spi.data.StructuredTable entryTable)

Method Summary

Modifier and Type	Method and Description
long	add(NamespacedId id, byte[] specification) Adds a new entry to the schema.
void	delete(NamespacedId id) Deletes the schema and all associated entries.
static SchemaRegistry	get(io.cdap.cdap.spi.data.StructuredTableContext context)
SchemaEntry	getEntry(NamespacedId id) Get the latest entry of the specified schema if it exists
SchemaEntry	getEntry(NamespacedId id, long version) Get a specific version of the specified schema.
Set<Long>	getVersions(NamespacedId id) Return all versions of the specified schema.
boolean	hasSchema(NamespacedId id) Checks if there is a schema entry, its not necessary that there are any version of schema registered.
boolean	hasSchema(NamespacedId id, long version) Checks if schema id and version combination exists in the registry.
void	remove(NamespacedId id, long version) Deletes a specified version of the schema.
void	write(SchemaDescriptor schemaDescriptor) Writes an entry in the schema registry.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

META_TABLE_SPEC

public static final io.cdap.cdap.spi.data.table.StructuredTableSpecification META_TABLE_SPEC

ENTRY_TABLE_SPEC

public static final io.cdap.cdap.spi.data.table.StructuredTableSpecification ENTRY_TABLE_SPEC

Constructor Detail

SchemaRegistry

public SchemaRegistry(io.cdap.cdap.spi.data.StructuredTable metaTable,
                      io.cdap.cdap.spi.data.StructuredTable entryTable)

Method Detail

get

public static SchemaRegistry get(io.cdap.cdap.spi.data.StructuredTableContext context)

write

public void write(SchemaDescriptor schemaDescriptor)
           throws IOException

Writes an entry in the schema registry. If the schema already exists, it is overwritten.

Parameters:

schemaDescriptor - information about the schema to write

Throws:

IOException - if there was an error reading from or writing to the storage system

delete

public void delete(NamespacedId id)
            throws IOException

Deletes the schema and all associated entries.

Parameters:

id - of the schema to delete

Throws:

IOException - if there was an error reading from or writing to the storage system

add

public long add(NamespacedId id,
                byte[] specification)
         throws IOException

Adds a new entry to the schema.

Parameters:

id - the id of the schema to add

specification - the schema to be added

Throws:

SchemaNotFoundException - if the schema does not exist

IOException - if there was an error reading from or writing to the storage system

remove

public void remove(NamespacedId id,
                   long version)
            throws IOException

Deletes a specified version of the schema. TODO: (CDAP-14661) update latest version pointer if the latest version was removed

Parameters:

id - of the schema to be deleted.

version - of the schema to be deleted.

Throws:

SchemaNotFoundException - if the schema does not exist

IOException - if there was an error reading from or writing to the storage system

hasSchema

public boolean hasSchema(NamespacedId id,
                         long version)
                  throws IOException

Checks if schema id and version combination exists in the registry.

Parameters:

id - of the schema to be checked

version - version of the schema to be checked.

Returns:

true if id and version matches, else false.

Throws:

SchemaNotFoundException - if the schema does not exist

IOException - if there was an error reading from or writing to the storage system

hasSchema

public boolean hasSchema(NamespacedId id)
                  throws IOException

Checks if there is a schema entry, its not necessary that there are any version of schema registered.

Parameters:

id - the id of the schema to check

Returns:

true if it exists, false otherwise

Throws:

IOException - if there was an error reading from or writing to the storage system

getVersions

public Set<Long> getVersions(NamespacedId id)
                      throws IOException

Return all versions of the specified schema.

Parameters:

id - the schema id

Returns:

list of schema versions

Throws:

SchemaNotFoundException - if the schema does not exist

IOException - if there was an error reading from or writing to the storage system

getEntry

public SchemaEntry getEntry(NamespacedId id,
                            long version)
                     throws IOException

Get a specific version of the specified schema.

Parameters:

id - the schema id

version - the entry version to get

Returns:

the schema entry

Throws:

SchemaNotFoundException - if the schema does not exist

IOException - if there was an error reading from or writing to the storage system

getEntry

public SchemaEntry getEntry(NamespacedId id)
                     throws IOException

Get the latest entry of the specified schema if it exists

Parameters:

id - the schema id

Returns:

the latest entry of the specified schema

Throws:

SchemaNotFoundException - if the schema or its latest entry could not be found

IOException - if there was an error reading from or writing to the storage system