public class RepositoryAdmin extends Object
discovery and full-schema
export/import
facilities; it is used as a complement to the standard
Admin interface to provide for maintenance of optional
ColumnDefinition structures and querying of ColumnAuditor structures.| Constructor and Description |
|---|
RepositoryAdmin(Connection connection)
Initialize a RepositoryAdmin object using a Connection provided by either a
ConnectionFactory
or MConnectionFactory. |
| Modifier and Type | Method and Description |
|---|---|
void |
addColumnAuditor(HTableDescriptor htd,
HColumnDescriptor hcd,
ColumnAuditor colAuditor)
Add (or modify, if already existing) the submitted
ColumnAuditor to the submitted
Table and Column Family. |
void |
addColumnAuditor(TableName tableName,
byte[] colFamily,
ColumnAuditor colAuditor)
Add (or modify, if already existing) the submitted
ColumnAuditor to the submitted
Table and Column Family. |
void |
addColumnAuditors(HTableDescriptor htd,
HColumnDescriptor hcd,
List<ColumnAuditor> colAuditors)
Add (or modify, if already existing) the submitted
ColumnAuditors to the submitted
Table and Column Family. |
void |
addColumnAuditors(TableName tableName,
byte[] colFamily,
List<ColumnAuditor> colAuditors)
Add (or modify, if already existing) the submitted
ColumnAuditors to the submitted
Table and Column Family. |
void |
addColumnDefinition(HTableDescriptor htd,
HColumnDescriptor hcd,
ColumnDefinition colDefinition)
Add (or modify, if already existing) the submitted
ColumnDefinition to the submitted
Table and Column Family. |
void |
addColumnDefinition(TableName tableName,
byte[] colFamily,
ColumnDefinition colDefinition)
Add (or modify, if already existing) the submitted
ColumnDefinition to the submitted
Table and Column Family. |
void |
addColumnDefinitions(HTableDescriptor htd,
HColumnDescriptor hcd,
List<ColumnDefinition> colDefinitions)
Add (or modify, if already existing) the submitted
ColumnDefinitions to the submitted
Table and Column Family. |
void |
addColumnDefinitions(TableName tableName,
byte[] colFamily,
List<ColumnDefinition> colDefinitions)
Add (or modify, if already existing) the submitted
ColumnDefinitions to the submitted
Table and Column Family. |
boolean |
columnDefinitionsEnforced(TableName tableName,
byte[] colFamily)
Ascertains whether or not column definition enforcement is enabled for a specified Column
Family.
|
HTableMultiplexer |
createHTableMultiplexer(int perRegionServerBufferQueueSize)
NOTE: THIS METHOD IS NOT YET SUPPORTED!
Create an HTableMultiplexer object. SPECIAL NOTE: An HTableMultiplexer returned by this method will (1) validate submitted Column qualifiers and values (if ColumnDefinitionsEnforced is set to true for the related Column Family), (2)
process "put" requests in a standard manner (queuing them for subsequent asynchronous
processing by HBase) and then (3) perform synchronous ColumnManager Repository
processing upon the metadata of each successfully queued "put". |
void |
deleteColumnDefinition(HTableDescriptor htd,
HColumnDescriptor hcd,
byte[] colQualifier)
Delete the
ColumnDefinition (pertaining to the submitted Column Qualifier)
from the submitted Table and Column Family. |
void |
deleteColumnDefinition(TableName tableName,
byte[] colFamily,
byte[] colQualifier)
Delete the
ColumnDefinition (pertaining to the submitted Column Qualifier)
from the submitted Table and Column Family. |
static void |
deleteTempReportTables(Admin hbaseAdmin)
Delete temp report Tables which might remain from any abnormally terminated
invocations of the
outputReportOnInvalidColumn* methods. |
void |
discoverColumnMetadata(boolean includeAllCells,
boolean useMapreduce)
Performs discovery of Column metadata for all
ColumnManager-included user Tables,
storing the results for each unique Column Qualifier as a
ColumnAuditor object in
the ColumnManager Repository; all such metadata is then retrievable via the
getColumnAuditors and
getColumnQualifiers methods. |
void |
discoverColumnMetadata(String namespace,
boolean includeAllCells,
boolean useMapreduce)
Performs discovery of Column metadata for all
ColumnManager-included user Tables in the
submitted namespace,
storing the results for each unique Column Qualifier as a
ColumnAuditor object in
the ColumnManager Repository; all such metadata is then retrievable via the
getColumnAuditors and
getColumnQualifiers methods. |
void |
discoverColumnMetadata(TableName tableName,
boolean includeAllCells,
boolean useMapreduce)
Performs discovery of Column metadata for the specified Table,
storing the results for each unique Column Qualifier as a
ColumnAuditor object in
the ColumnManager Repository; all such metadata is then retrievable via the
getColumnAuditors and
getColumnQualifiers methods. |
void |
dumpRepositoryTable()
Causes the complete contents (all cells) of the Repository table to be "dumped" (written) to
the logging facility, for maintenance or debugging purposes.
|
void |
enableColumnAliases(boolean enabled,
TableName tableName,
byte[] colFamily)
Enable/disable column aliasing for the specified Column Family.
|
void |
enableColumnDefinitionEnforcement(boolean enabled,
TableName tableName,
byte[] colFamily)
Enable/disable column definition enforcement for the specified Column Family.
|
void |
exportSchema(File targetFile)
Creates an external HBaseSchemaArchive (HSA) file in XML format* containing the complete
metadata contents (i.e., all
Namespace,
Table, Column Family,
ColumnAuditor, and ColumnDefinition metadata) of the
ColumnManager metadata Repository. |
void |
exportSchema(File targetFile,
String sourceNamespaceName)
Creates an external HBaseSchemaArchive (HSA) file in XML format* containing the complete
schema contents of the specified HBase Namespace.
|
void |
exportSchema(File targetFile,
TableName sourceTableName)
Creates an external HBaseSchemaArchive (HSA) file in XML format* containing the complete
schema contents of the specified HBase Table.
|
static String |
generateHsaFileSummary(File sourceHsaFile)
Generates a hierarchically-indented, text-based summary report of the contents of an external
HBaseSchemaArchive (HSA) XML file*.
|
ChangeEventMonitor |
getChangeEventMonitor()
Get a
ChangeEventMonitor object |
Set<ColumnAuditor> |
getColumnAuditors(HTableDescriptor htd,
HColumnDescriptor hcd)
Get
ColumnAuditor objects from the Repository for a specified Table and
Column Family. |
Set<ColumnAuditor> |
getColumnAuditors(TableName tableName,
byte[] colFamily)
Get
ColumnAuditor objects from the Repository for a specified Table and
Column Family. |
Collection<ColumnDefinition> |
getColumnDefinitions(HTableDescriptor htd,
HColumnDescriptor hcd)
Get
ColumnDefinitions belonging to the submitted Table and
Column Family. |
Collection<ColumnDefinition> |
getColumnDefinitions(TableName tableName,
byte[] colFamily)
Get
ColumnDefinitions belonging to the submitted Table and
Column Family. |
Set<byte[]> |
getColumnQualifiers(HTableDescriptor htd,
HColumnDescriptor hcd)
Get Column Qualifiers from the
ColumnAuditor metadata in the Repository for a specified
Table and Column Family. |
Set<byte[]> |
getColumnQualifiers(TableName tableName,
byte[] colFamily)
Get Column Qualifiers from the
ColumnAuditor metadata in the Repository for a specified
Table and Column Family. |
static int |
getRepositoryMaxVersions(Admin hbaseAdmin)
Get the maxVersions setting for the Repository table (maximum versions for Repository to retain
of each schema metadata attribute).
|
void |
importColumnDefinitions(File sourceHsaFile)
Import into the ColumnManager Repository all
ColumnDefinitions that are found in the
submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing. |
void |
importColumnDefinitions(File sourceHsaFile,
String namespace)
Import into the ColumnManager Repository all of a specified Namespace's
ColumnDefinitions that are found in the submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing. |
void |
importColumnDefinitions(File sourceHsaFile,
TableName tableName)
Import into the ColumnManager Repository all of a specified Table's
ColumnDefinitions that are found in the submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing. |
void |
importColumnDefinitions(File sourceHsaFile,
TableName tableName,
byte[] colFamily)
Import into the ColumnManager Repository all of a specified ColumnFamily's
ColumnDefinitions that are found in the submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing. |
void |
importSchema(File sourceHsaFile,
boolean includeColumnAuditors)
Import into HBase the complete contents of an external HBaseSchemaArchive (HSA) XML file*;
this process will NOT overlay any existing Namespace and Table structures in HBase.
|
void |
importSchema(File sourceHsaFile,
String namespaceName,
boolean includeColumnAuditors)
Import a Namespace into HBase with all its component schema-objects as serialized in an
external HBaseSchemaArchive (HSA) XML file*; this process will NOT overlay any existing
Namespace and Table structures in HBase.
|
void |
importSchema(File sourceHsaFile,
TableName tableName,
boolean includeColumnAuditors)
Import a Table into HBase with all its component schema-objects as serialized in an external
HBaseSchemaArchive (HSA) XML file*; this process will NOT overlay an existing Table
(and its component structures) in HBase.
|
static void |
installRepositoryStructures(Admin hbaseAdmin)
Create namespace and table structures for Repository; note that these structures will
automatically be created with the first invocation of
MConnectionFactory.createConnection() if the structures do not already exist. |
boolean |
namespaceExists(NamespaceDescriptor nd)
Namespace existence verification.
|
boolean |
namespaceExists(String namespaceName)
Namespace existence verification.
|
boolean |
outputReportOnColumnQualifiers(File targetFile,
String namespace)
Generates and outputs a CSV-formatted report on all
ColumnAuditors that have been
captured or
discovered
pertaining to all ColumnManager-included user
Tables within the submitted Namespace. |
boolean |
outputReportOnColumnQualifiers(File targetFile,
TableName tableName)
Generates and outputs a CSV-formatted report on all
ColumnAuditors that have been
captured or
discovered
pertaining to the submitted user Table. |
boolean |
outputReportOnColumnQualifiers(File targetFile,
TableName tableName,
byte[] colFamily)
Generates and outputs a CSV-formatted report on all
ColumnAuditors that have been
captured or
discovered
pertaining to the submitted Column Family within the submitted user Table. |
boolean |
outputReportOnInvalidColumnLengths(File targetFile,
TableName tableName,
boolean verbose,
boolean includeAllCells,
boolean useMapreduce)
Generates and outputs a CSV-formatted report of all invalid lengths of column values stored
in a Table, as stipulated by the
ColumnLength
settings in the Table's ColumnDefinitions. |
boolean |
outputReportOnInvalidColumnLengths(File targetFile,
TableName tableName,
byte[] colFamily,
boolean verbose,
boolean includeAllCells,
boolean useMapreduce)
Generates and outputs a CSV-formatted report of all invalid lengths of column values stored
in a Column Family, as stipulated by the
ColumnLength
settings in the Column Family's ColumnDefinitions. |
boolean |
outputReportOnInvalidColumnQualifiers(File targetFile,
TableName tableName,
boolean verbose,
boolean useMapreduce)
Generates and outputs a CSV-formatted report of all invalid column qualifiers stored in a
Table, as stipulated by the Table's
ColumnDefinitions. |
boolean |
outputReportOnInvalidColumnQualifiers(File targetFile,
TableName tableName,
byte[] colFamily,
boolean verbose,
boolean useMapreduce)
Generates and outputs a CSV-formatted report of all invalid column qualifiers stored in a
Column Family, as stipulated by the Column Family's
ColumnDefinitions. |
boolean |
outputReportOnInvalidColumnValues(File targetFile,
TableName tableName,
boolean verbose,
boolean includeAllCells,
boolean useMapreduce)
Generates and outputs a CSV-formatted report of all invalid column values stored
in a Table, as stipulated by the
ValidationRegex settings in the Table's ColumnDefinitions. |
boolean |
outputReportOnInvalidColumnValues(File targetFile,
TableName tableName,
byte[] colFamily,
boolean verbose,
boolean includeAllCells,
boolean useMapreduce)
Generates and outputs a CSV-formatted report of all invalid column values stored
in a Column Family, as stipulated by the
ValidationRegex settings in the Column Family's ColumnDefinitions. |
static void |
setRepositoryMaxVersions(Admin hbaseAdmin,
int maxVersions)
Allows administrative override of the default maxVersions setting for the Repository table.
|
static void |
uninstallRepositoryStructures(Admin hbaseAdmin)
Disable and delete Repository table and drop Repository namespace (for uninstall or reinstall
of ColumnManager).
|
public RepositoryAdmin(Connection connection) throws IOException
ConnectionFactory
or MConnectionFactory.connection - An HBase Connection.IOException - if a remote or network exception occurspublic static void installRepositoryStructures(Admin hbaseAdmin) throws IOException
MConnectionFactory.createConnection() if the structures do not already exist.hbaseAdmin - standard HBase Admin objectIOException - if a remote or network exception occurspublic static void setRepositoryMaxVersions(Admin hbaseAdmin, int maxVersions) throws IOException
hbaseAdmin - Standard Admin objectmaxVersions - Maximum versions for Repository to retain of each schema metadata attributeIOException - if a remote or network exception occurspublic static int getRepositoryMaxVersions(Admin hbaseAdmin) throws IOException
hbaseAdmin - Standard Admin objectIOException - if a remote or network exception occurspublic static void uninstallRepositoryStructures(Admin hbaseAdmin) throws IOException
hbaseAdmin - standard HBase AdminIOException - if a remote or network exception occurspublic static void deleteTempReportTables(Admin hbaseAdmin) throws IOException
outputReportOnInvalidColumn* methods. Note that invocation of this method will cause any
currently-running invocations of the
outputReportOnInvalidColumn* methods to abnormally terminate.hbaseAdmin - standard HBase AdminIOException - if a remote or network exception occurspublic void dumpRepositoryTable()
throws IOException
IOException - if a remote or network exception occurspublic void addColumnAuditor(TableName tableName, byte[] colFamily, ColumnAuditor colAuditor) throws IOException, TableNotIncludedForProcessingException
ColumnAuditor to the submitted
Table and Column Family.tableName - name of Table to which ColumnAuditor is to be added/modifiedcolFamily - Column Family to which ColumnAuditor is to be added/modifiedcolAuditor - ColumnAuditor to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void addColumnAuditor(HTableDescriptor htd, HColumnDescriptor hcd, ColumnAuditor colAuditor) throws IOException, TableNotIncludedForProcessingException
ColumnAuditor to the submitted
Table and Column Family.htd - Table to which ColumnAuditor is to be added/modifiedhcd - Column Descriptor (Column Family) to which ColumnAuditor is to be
added/modifiedcolAuditor - ColumnAuditor to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void addColumnAuditors(TableName tableName, byte[] colFamily, List<ColumnAuditor> colAuditors) throws IOException, TableNotIncludedForProcessingException
ColumnAuditors to the submitted
Table and Column Family.tableName - name of Table to which ColumnAuditors are to be added/modifiedcolFamily - Column Family to which ColumnAuditors are to be added/modifiedcolAuditors - List of ColumnAuditors to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void addColumnAuditors(HTableDescriptor htd, HColumnDescriptor hcd, List<ColumnAuditor> colAuditors) throws IOException, TableNotIncludedForProcessingException
ColumnAuditors to the submitted
Table and Column Family.htd - Table to which ColumnAuditors are to be added/modifiedhcd - ColumnAuditor [Family] Descriptor to which ColumnAuditors are to
be added/modifiedcolAuditors - List of ColumnAuditors to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic Set<ColumnAuditor> getColumnAuditors(HTableDescriptor htd, HColumnDescriptor hcd) throws IOException, TableNotIncludedForProcessingException
ColumnAuditor objects from the Repository for a specified Table and
Column Family.htd - Table Descriptorhcd - Column [Family] DescriptorColumnAuditors, or null if Table or Column Family not
foundIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic Set<ColumnAuditor> getColumnAuditors(TableName tableName, byte[] colFamily) throws IOException, TableNotIncludedForProcessingException
ColumnAuditor objects from the Repository for a specified Table and
Column Family.tableName - Table namecolFamily - Column FamilyColumnAuditors, or null if tableName or colFamily not foundIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic Set<byte[]> getColumnQualifiers(HTableDescriptor htd, HColumnDescriptor hcd) throws IOException, TableNotIncludedForProcessingException
ColumnAuditor metadata in the Repository for a specified
Table and Column Family.htd - Table Descriptorhcd - Column [Family] DescriptorIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic Set<byte[]> getColumnQualifiers(TableName tableName, byte[] colFamily) throws IOException, TableNotIncludedForProcessingException
ColumnAuditor metadata in the Repository for a specified
Table and Column Family.tableName - Table namecolFamily - Column FamilyIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void addColumnDefinition(TableName tableName, byte[] colFamily, ColumnDefinition colDefinition) throws IOException, TableNotIncludedForProcessingException
ColumnDefinition to the submitted
Table and Column Family.tableName - name of Table to which ColumnDefinition is to be addedcolFamily - Column Family to which ColumnDefinition is to be addedcolDefinition - ColumnDefinition to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void addColumnDefinition(HTableDescriptor htd, HColumnDescriptor hcd, ColumnDefinition colDefinition) throws IOException, TableNotIncludedForProcessingException
ColumnDefinition to the submitted
Table and Column Family.htd - Table to which ColumnDefinition is to be addedhcd - ColumnAuditor [Family] Descriptor to which ColumnDefinition is to be
addedcolDefinition - ColumnDefinition to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void addColumnDefinitions(TableName tableName, byte[] colFamily, List<ColumnDefinition> colDefinitions) throws IOException, TableNotIncludedForProcessingException
ColumnDefinitions to the submitted
Table and Column Family.tableName - name of Table to which ColumnDefinitions are to be addedcolFamily - Column Family to which ColumnDefinitions are to be addedcolDefinitions - List of ColumnDefinitions to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void addColumnDefinitions(HTableDescriptor htd, HColumnDescriptor hcd, List<ColumnDefinition> colDefinitions) throws IOException, TableNotIncludedForProcessingException
ColumnDefinitions to the submitted
Table and Column Family.htd - Table to which ColumnDefinitions are to be addedhcd - ColumnAuditor [Family] Descriptor to which ColumnDefinitions are to
be addedcolDefinitions - List of ColumnDefinitions to be added or modifiedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic Collection<ColumnDefinition> getColumnDefinitions(HTableDescriptor htd, HColumnDescriptor hcd) throws IOException, TableNotIncludedForProcessingException
ColumnDefinitions belonging to the submitted Table and
Column Family.htd - Table to which ColumnDefinitions belonghcd - ColumnAuditor [Family] Descriptor to which ColumnDefinitions belongColumnDefinitionsIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic Collection<ColumnDefinition> getColumnDefinitions(TableName tableName, byte[] colFamily) throws IOException, TableNotIncludedForProcessingException
ColumnDefinitions belonging to the submitted Table and
Column Family.tableName - name of Table to which ColumnDefinitions belongcolFamily - Column Family to which ColumnDefinitions belongColumnDefinitionsIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void deleteColumnDefinition(TableName tableName, byte[] colFamily, byte[] colQualifier) throws IOException, TableNotIncludedForProcessingException
ColumnDefinition (pertaining to the submitted Column Qualifier)
from the submitted Table and Column Family.tableName - name of Table from which ColumnDefinition is to be deletedcolFamily - Column Family from which ColumnDefinition is to be deletedcolQualifier - qualifier which uniquely identifies the ColumnDefinition to be
deletedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void deleteColumnDefinition(HTableDescriptor htd, HColumnDescriptor hcd, byte[] colQualifier) throws IOException, TableNotIncludedForProcessingException
ColumnDefinition (pertaining to the submitted Column Qualifier)
from the submitted Table and Column Family.htd - Table from which ColumnDefinition is to be deletedhcd - ColumnAuditor [Family] Descriptor from which ColumnDefinition is to
be deletedcolQualifier - qualifier which uniquely identifies ColumnDefinition to be deletedIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic boolean columnDefinitionsEnforced(TableName tableName, byte[] colFamily) throws IOException, TableNotIncludedForProcessingException
tableName - Name of Table to which Column Family belongs.colFamily - Name of Column Family for which column definition enforcement is to be
ascertainedtrue if column definition enforcement is enabled for the specified Column
Family; false if column definition enforcement is not enabledIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void enableColumnDefinitionEnforcement(boolean enabled,
TableName tableName,
byte[] colFamily)
throws IOException,
TableNotIncludedForProcessingException
Mutation to the
Table and
Column Family (via the HBase API) must correspond to an existing
ColumnDefinition, and the corresponding Column value submitted must pass all
validations (if any) stipulated by the ColumnDefinition.enabled - if true, column definition enforcement is enabled; if false, it
is disabledtableName - Name of Table to which Column Family belongscolFamily - Name of Column Family for which column definition enforcement is to be
enabled or disabledIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void enableColumnAliases(boolean enabled,
TableName tableName,
byte[] colFamily)
throws IOException,
TableNotIncludedForProcessingException
enabled - if true, column aliasing will be enabled; if false, it
will be disabledtableName - Name of Table to which Column Family belongscolFamily - Name of Column Family for which column aliasing is to be
enabled or disabledIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic ChangeEventMonitor getChangeEventMonitor() throws IOException
ChangeEventMonitor objectChangeEventMonitor objectIOException - if a remote or network exception occurspublic HTableMultiplexer createHTableMultiplexer(int perRegionServerBufferQueueSize) throws IOException
ColumnDefinitionsEnforced is set to true for the related Column Family), (2)
process "put" requests in a standard manner (queuing them for subsequent asynchronous
processing by HBase) and then (3) perform synchronous ColumnManager Repository
processing upon the metadata of each successfully queued "put". Be advised that the third
step will not take into account any potentially failed "put" transactions among those queued in
the second step; instead it assumes that all "put" transactions succeeded, updating the
ColumnManager Repository accordingly.perRegionServerBufferQueueSize - determines the max number of the buffered Put ops for
each region server before dropping a request.IOException - if a remote or network exception occurspublic void discoverColumnMetadata(boolean includeAllCells,
boolean useMapreduce)
throws Exception
ColumnAuditor object in
the ColumnManager Repository; all such metadata is then retrievable via the
getColumnAuditors and
getColumnQualifiers methods. The discovery process entails a
KeyOnlyFilter scan of the Tables, either
via direct scanning or via mapreduce. Note that when the mapreduce option is
utilized, the default row-cache setting is 500, which may be overridden by setting the
Configuration parameter
TableInputFormat.SCAN_CACHEDROWS.includeAllCells - if true, analysis will include ALL cells for each column;
otherwise, analysis will only include the most recent cell.useMapreduce - if true, discovery is done via mapreduce; otherwise, discovery
is done via direct-scanIOException - if a remote or network exception occursExceptionpublic void discoverColumnMetadata(String namespace, boolean includeAllCells, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ColumnAuditor object in
the ColumnManager Repository; all such metadata is then retrievable via the
getColumnAuditors and
getColumnQualifiers methods. The discovery process entails a
KeyOnlyFilter scan of the Tables, either
via direct scanning or via mapreduce. Note that when the mapreduce option is
utilized, the default row-cache setting is 500, which may be overridden by setting the
Configuration parameter
TableInputFormat.SCAN_CACHEDROWS.namespace - Namespace for which schema metadata is to be discovered; submitted
namespace must have at least one Table that is
included in ColumnManager processingincludeAllCells - if true, analysis will include ALL cells for each column;
otherwise, analysis will only include the most recent cell.useMapreduce - if true, discovery is done via mapreduce; otherwise, discovery
is done via direct-scanIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if no Tables from the Namespace are
included in ColumnManager processingExceptionpublic void discoverColumnMetadata(TableName tableName, boolean includeAllCells, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ColumnAuditor object in
the ColumnManager Repository; all such metadata is then retrievable via the
getColumnAuditors and
getColumnQualifiers methods. The discovery process entails a
KeyOnlyFilter scan of the Table,
either via direct scanning or via mapreduce. Note that when the mapreduce option is
utilized, the default row-cache setting is 500, which may be overridden by setting the
Configuration parameter
TableInputFormat.SCAN_CACHEDROWS.tableName - Table for which schema metadata is to be discovered; submitted
Table must be
included in ColumnManager processingincludeAllCells - if true, analysis will include ALL cells for each column;
otherwise, analysis will only include the most recent cell.useMapreduce - if true, discovery is done via mapreduce; otherwise, discovery
is done via direct-scanIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingExceptionpublic void exportSchema(File targetFile) throws IOException, JAXBException, XMLStreamException
Namespace,
Table, Column Family,
ColumnAuditor, and ColumnDefinition metadata) of the
ColumnManager metadata Repository. This constitutes an XML-formatted serialization of
objects of the following classes: NamespaceDescriptor, HTableDescriptor,
HColumnDescriptor, ColumnAuditor, and ColumnDefinition.
targetFile - target fileIOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingXMLStreamException - if an exception occurs in the context of JAXB processingpublic void exportSchema(File targetFile, String sourceNamespaceName) throws IOException, JAXBException, XMLStreamException, TableNotIncludedForProcessingException
NamespaceDescriptor, HTableDescriptor, HColumnDescriptor,
ColumnAuditor, and ColumnDefinition.
targetFile - target filesourceNamespaceName - namespace from which to export schema entitiesIOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingXMLStreamException - if an exception occurs in the context of JAXB processingTableNotIncludedForProcessingException - if no Tables from the Namespace are
included in ColumnManager processingpublic void exportSchema(File targetFile, TableName sourceTableName) throws IOException, JAXBException, XMLStreamException, TableNotIncludedForProcessingException
HTableDescriptor, HColumnDescriptor, ColumnAuditor, and
ColumnDefinition.
targetFile - target FilesourceTableName - table to export (along with its component schema-entities)IOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingXMLStreamException - if an exception occurs in the context of JAXB processingTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void importSchema(File sourceHsaFile, boolean includeColumnAuditors) throws IOException, JAXBException
#exportSchema methods and adheres to
the XML Schema layout in
HBaseSchemaArchive.xsd.xml.sourceHsaFile - source fileincludeColumnAuditors - if true, import ColumnAuditor metadata from the
HBaseSchemaArchive file into the ColumnManager Repository.IOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingpublic void importSchema(File sourceHsaFile, String namespaceName, boolean includeColumnAuditors) throws IOException, JAXBException, TableNotIncludedForProcessingException
#exportSchema methods and adheres to
the XML Schema layout in
HBaseSchemaArchive.xsd.xml.sourceHsaFile - source filenamespaceName - namespace to import.includeColumnAuditors - if true, import ColumnAuditor metadata from the
HBaseSchemaArchive file into the ColumnManager Repository.IOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingTableNotIncludedForProcessingException - if no Tables from the Namespace are
included in ColumnManager processingpublic void importSchema(File sourceHsaFile, TableName tableName, boolean includeColumnAuditors) throws IOException, JAXBException, TableNotIncludedForProcessingException
#exportSchema methods and adheres to
the XML Schema layout in
HBaseSchemaArchive.xsd.xml.includeColumnAuditors - if true, import ColumnAuditor metadata from the
HBaseSchemaArchive file into the ColumnManager Repository.tableName - Name of table to be imported.sourceHsaFile - source fileIOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void importColumnDefinitions(File sourceHsaFile) throws IOException, JAXBException
ColumnDefinitions that are found in the
submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing.
#exportSchema methods and adheres to
the XML Schema layout in
HBaseSchemaArchive.xsd.xml.sourceHsaFile - source fileIOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingpublic void importColumnDefinitions(File sourceHsaFile, String namespace) throws IOException, JAXBException, TableNotIncludedForProcessingException
ColumnDefinitions that are found in the submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing.
#exportSchema methods and adheres to
the XML Schema layout in
HBaseSchemaArchive.xsd.xml.namespace - namespace for which ColumnDefinitions are to be importedsourceHsaFile - source fileIOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingTableNotIncludedForProcessingException - if no Tables from the Namespace are
included in ColumnManager processingpublic void importColumnDefinitions(File sourceHsaFile, TableName tableName) throws IOException, JAXBException, TableNotIncludedForProcessingException
ColumnDefinitions that are found in the submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing.
#exportSchema methods and adheres to
the XML Schema layout in
HBaseSchemaArchive.xsd.xml.tableName - table for which ColumnDefinitions are to be importedsourceHsaFile - source fileIOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic void importColumnDefinitions(File sourceHsaFile, TableName tableName, byte[] colFamily) throws IOException, JAXBException, TableNotIncludedForProcessingException
ColumnDefinitions that are found in the submitted HBaseSchemaArchive (HSA) XML file*;
for a ColumnDefinition to be imported, it must belong to an existing
Table/ColumnFamily which is
included in ColumnManager processing.
#exportSchema methods and adheres to
the XML Schema layout in
HBaseSchemaArchive.xsd.xml.tableName - table of ColumnFamily for which ColumnDefinitions are to be importedcolFamily - ColumnFamily for which ColumnDefinitions are to be importedsourceHsaFile - source fileIOException - if a remote or network exception occursJAXBException - if an exception occurs in the context of JAXB processingTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic static String generateHsaFileSummary(File sourceHsaFile) throws JAXBException
sourceHsaFile - source HBaseSchemaArchive fileJAXBException - if an exception occurs in the context of JAXB processingpublic boolean outputReportOnColumnQualifiers(File targetFile, String namespace) throws IOException, TableNotIncludedForProcessingException
ColumnAuditors that have been
captured or
discovered
pertaining to all ColumnManager-included user
Tables within the submitted Namespace.targetFile - file to which the CSV file is to be outputtednamespace - namespace for which ColumnAuditor report is to be generated/outputtedtrue if ColumnAuditors found and reported on; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if no Tables from the Namespace are
included in ColumnManager processingpublic boolean outputReportOnColumnQualifiers(File targetFile, TableName tableName) throws IOException, TableNotIncludedForProcessingException
ColumnAuditors that have been
captured or
discovered
pertaining to the submitted user Table.targetFile - file to which the CSV file is to be outputtedtableName - name of the Table for which ColumnAuditor report is to be
generated/outputtedtrue if ColumnAuditors found and reported on; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic boolean outputReportOnColumnQualifiers(File targetFile, TableName tableName, byte[] colFamily) throws IOException, TableNotIncludedForProcessingException
ColumnAuditors that have been
captured or
discovered
pertaining to the submitted Column Family within the submitted user Table.targetFile - file to which the CSV file is to be outputtedtableName - name of the Table for which ColumnAuditor report is to be
generated/outputtedcolFamily - name of the Column Family within the Table for which ColumnAuditor
report is to be generated/outputtedtrue if ColumnAuditors found and reported on; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingpublic boolean outputReportOnInvalidColumnQualifiers(File targetFile, TableName tableName, boolean verbose, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ColumnDefinitions.
If no ColumnDefinitions exist for any of the Table's Column Families, a
ColumnDefinitionNotFoundException will be thrown.tableName - name of the Table to be reported ontargetFile - file to which the CSV file is to be outputtedverbose - if true the outputted CSV file will include an entry (identified by
the fully-qualified column name and rowId) for each explicit invalid column qualifier that is
found; otherwise the report will contain a summary, with one line for each invalid column
qualifier found, along with a count of the number of rows which contain that same invalid
column qualifier.useMapreduce - if true, analysis will be done on servers via mapreduce jobs;
otherwise, analysis is done via direct-scantrue if invalid column qualifiers found; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingExceptionpublic boolean outputReportOnInvalidColumnQualifiers(File targetFile, TableName tableName, byte[] colFamily, boolean verbose, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ColumnDefinitions.
If no ColumnDefinitions exist for the Column Family, a
ColumnDefinitionNotFoundException will be thrown.targetFile - file to which the CSV file is to be outputtedtableName - name of the Table containing the Column Family to be reported oncolFamily - name of the Column Family to be reported onverbose - if true the outputted CSV file will include an entry (identified by
the fully-qualified column name and rowId) for each explicit invalid column qualifier that is
found; otherwise the report will contain a summary, with one line for each invalid column
qualifier found, along with a count of the number of rows which contain that same invalid
column qualifier.useMapreduce - if true, analysis will be done on servers via mapreduce jobs;
otherwise, analysis is done via direct-scantrue if invalid column qualifiers found; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingExceptionpublic boolean outputReportOnInvalidColumnLengths(File targetFile, TableName tableName, boolean verbose, boolean includeAllCells, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ColumnLength
settings in the Table's ColumnDefinitions.
If no ColumnDefinitions exist for any of the Table's Column Families, a
ColumnDefinitionNotFoundException will be thrown.tableName - name of the Table to be reported ontargetFile - file to which the CSV file is to be outputtedverbose - if true the outputted CSV file will include an entry (identified by
the fully-qualified column name and rowId) for each explicit invalid column length that is
found; otherwise the report will contain a summary, with one line for each column found
to contain at least a single instance of an invalid column length, along with a count of the
number of rows in which that column contains an invalid column length.includeAllCells - if true, analysis will include ALL cells for each column;
otherwise, analysis will only include the most recent cell.useMapreduce - if true, analysis will be done on servers via mapreduce jobs;
otherwise, analysis is done via direct-scantrue if invalid column qualifiers found; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingExceptionpublic boolean outputReportOnInvalidColumnLengths(File targetFile, TableName tableName, byte[] colFamily, boolean verbose, boolean includeAllCells, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ColumnLength
settings in the Column Family's ColumnDefinitions.
If no ColumnDefinitions exist for the Column Family, a
ColumnDefinitionNotFoundException will be thrown.targetFile - file to which the CSV file is to be outputtedtableName - name of the Table containing the Column Family to be reported oncolFamily - name of the Column Family to be reported onverbose - if true the outputted CSV file will include an entry (identified by
the fully-qualified column name and rowId) for each explicit invalid column length that is
found; otherwise the report will contain a summary, with one line for each column found
to contain at least a single instance of an invalid column length, along with a count of the
number of rows in which that column contains an invalid column length.includeAllCells - if true, analysis will include ALL cells for each column;
otherwise, analysis will only include the most recent cell.useMapreduce - if true, analysis will be done on servers via mapreduce jobs;
otherwise, analysis is done via direct-scantrue if invalid column qualifiers found; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingExceptionpublic boolean outputReportOnInvalidColumnValues(File targetFile, TableName tableName, boolean verbose, boolean includeAllCells, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ValidationRegex settings in the Table's ColumnDefinitions.
If no ColumnDefinitions exist for any of the Table's Column Families, a
ColumnDefinitionNotFoundException will be thrown.tableName - name of the Table to be reported ontargetFile - file to which the CSV file is to be outputtedverbose - if true the outputted CSV file will include an entry (identified by
the fully-qualified column name and rowId) for each explicit invalid column value that is
found; otherwise the report will contain a summary, with one line for each column found
to contain at least a single instance of an invalid column value, along with a count of the
number of rows in which that column contains an invalid column value.includeAllCells - if true, analysis will include ALL cells for each column;
otherwise, analysis will only include the most recent cell.useMapreduce - if true, analysis will be done on servers via mapreduce jobs;
otherwise, analysis is done via direct-scantrue if invalid column qualifiers found; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingExceptionpublic boolean outputReportOnInvalidColumnValues(File targetFile, TableName tableName, byte[] colFamily, boolean verbose, boolean includeAllCells, boolean useMapreduce) throws Exception, TableNotIncludedForProcessingException
ValidationRegex settings in the Column Family's ColumnDefinitions.
If no ColumnDefinitions exist for the Column Family, a
ColumnDefinitionNotFoundException will be thrown.targetFile - file to which the CSV file is to be outputtedtableName - name of the Table containing the Column Family to be reported oncolFamily - name of the Column Family to be reported onverbose - if true the outputted CSV file will include an entry (identified by
the fully-qualified column name and rowId) for each explicit invalid column value that is
found; otherwise the report will contain a summary, with one line for each column found
to contain at least a single instance of an invalid column value, along with a count of the
number of rows in which that column contains an invalid column value.includeAllCells - if true, analysis will include ALL cells for each column;
otherwise, analysis will only include the most recent cell.useMapreduce - if true, analysis will be done on servers via mapreduce jobs;
otherwise, analysis is done via direct-scantrue if invalid column qualifiers found; otherwise, falseIOException - if a remote or network exception occursTableNotIncludedForProcessingException - if Table not
included in ColumnManager processingExceptionpublic boolean namespaceExists(NamespaceDescriptor nd) throws IOException
nd - NamespaceDescriptorIOException - if a remote or network exception occurspublic boolean namespaceExists(String namespaceName) throws IOException
namespaceName - name of namespaceIOException - if a remote or network exception occursCopyright © 2016. All rights reserved.