@NotThreadSafe final class SourceInfo extends Object
The source partition information describes the database whose log is being consumed. Typically, the
database is identified by the host address port number of the MySQL server and the name of the database. Here's a JSON-like
representation of an example database:
{
"server" : "production-server"
}
The source offset information describes how much of the database's binary log the source the change detector
has already processed, and it includes the binlog filename, the next
position in the binlog to start reading, and the next event row number. Here's a JSON-like
representation of an example:
{
"server_id": 112233,
"ts_sec": 1465937,
"gtid": "db58b0ae-2c10-11e6-b284-0242ac110002:199",
"file": "mysql-bin.000003",
"pos" = 990,
"row": 0,
"snapshot": true
}
The "gtids" field only appears in offsets produced when GTIDs are enabled. The "snapshot" field only appears in
offsets produced when the connector is in the middle of a snapshot. And finally, the "ts" field contains the
seconds since Unix epoch (since Jan 1, 1970) of the MySQL event; the message envelopes also have a
timestamp, but that timestamp is the milliseconds since since Jan 1, 1970.
The source struct appears in each message envelope and contains MySQL information about the event. It is
a mixture the field from the partition (which is renamed in the source to make more sense), the
position of the event (and row number within the event) inside
the binlog file. When GTIDs are enabled, it also includes the GTID of the transaction in which the
event occurs. Like with the offset, the "snapshot" field only appears for events produced when the connector is in the
middle of a snapshot. Here's a JSON-like representation of the source for an event that corresponds to the above partition and
offset:
{
"name": "production-server",
"server_id": 112233,
"ts_sec": 1465937,
"gtid": "db58b0ae-2c10-11e6-b284-0242ac110002:199",
"file": "mysql-bin.000003",
"pos" = 1081,
"row": 0,
"snapshot": true
}
| Modifier and Type | Field and Description |
|---|---|
static String |
BINLOG_EVENT_ROW_NUMBER_OFFSET_KEY |
static String |
BINLOG_FILENAME_OFFSET_KEY |
static String |
BINLOG_POSITION_OFFSET_KEY |
private String |
binlogFilename |
private String |
binlogGtid |
private long |
binlogTimestampSeconds |
static String |
GTID_KEY |
static String |
GTID_SET_KEY |
private String |
gtidSet |
private long |
lastBinlogPosition |
private int |
lastEventRowNumber |
private boolean |
lastSnapshot |
private long |
nextBinlogPosition |
private int |
nextEventRowNumber |
private boolean |
nextSnapshot |
static org.apache.kafka.connect.data.Schema |
SCHEMA
|
static String |
SERVER_ID_KEY |
static String |
SERVER_NAME_KEY |
static String |
SERVER_PARTITION_KEY |
private long |
serverId |
private String |
serverName |
static String |
SNAPSHOT_KEY |
private Map<String,String> |
sourcePartition |
static String |
TIMESTAMP_KEY |
| Constructor and Description |
|---|
SourceInfo() |
| Modifier and Type | Method and Description |
|---|---|
String |
binlogFilename()
Get the name of the MySQL binary log file that has been processed.
|
private boolean |
booleanOffsetValue(Map<String,?> values,
String key) |
void |
completeSnapshot()
Denote that a snapshot has completed successfully.
|
String |
gtidSet()
Get the string representation of the GTID range for the MySQL binary log file.
|
static boolean |
isPositionAtOrBefore(Document recorded,
Document desired)
|
boolean |
isSnapshotInEffect()
Determine whether a snapshot is currently in effect.
|
long |
lastBinlogPosition()
Get the position within the MySQL binary log file of the most recently processed event.
|
int |
lastEventRowNumber()
Get the previous row within the event at the
position within the binary log file
. |
private long |
longOffsetValue(Map<String,?> values,
String key) |
void |
markLastSnapshot()
Denote that a snapshot will be complete after one last record.
|
long |
nextBinlogPosition()
Get the position within the MySQL binary log file of the next event to be processed.
|
int |
nextEventRowNumber()
Get the next row within the event at the
position within the binary
log file
. |
Map<String,?> |
offset()
Get the Kafka Connect detail about the source "offset", which describes the position within the source where we last
have last read.
|
Map<String,?> |
offsetForRow(int eventRowNumber,
int totalNumberOfRows)
Set the current row number within a given event, and then get the Kafka Connect detail about the source "offset", which
describes the position within the source where we have last read.
|
private Map<String,?> |
offsetUsingPosition(long binlogPosition) |
Map<String,String> |
partition()
Get the Kafka Connect detail about the source "partition", which describes the portion of the source that we are
consuming.
|
org.apache.kafka.connect.data.Schema |
schema()
|
String |
serverName()
Get the logical identifier of the database that is the source of the events.
|
void |
setBinlogServerId(long serverId)
Set the server ID as found within the MySQL binary log file.
|
void |
setBinlogStartPoint(String binlogFilename,
long positionOfFirstEvent)
Set the name of the MySQL binary log file.
|
void |
setBinlogTimestampSeconds(long timestampInSeconds)
Set the number of seconds since Unix epoch (January 1, 1970) as found within the MySQL binary log file.
|
void |
setEventPosition(long positionOfCurrentEvent,
long eventSizeInBytes)
Set the position within the MySQL binary log file of the current event.
|
void |
setGtid(String gtid)
Set the latest GTID from the MySQL binary log file.
|
void |
setGtidSet(String gtidSet)
Set the set of GTIDs known to the MySQL server.
|
void |
setOffset(Map<String,?> sourceOffset)
Set the source offset, as read from Kafka Connect.
|
void |
setServerName(String logicalId)
Set the database identifier.
|
void |
startSnapshot()
Denote that a snapshot is being (or has been) started.
|
org.apache.kafka.connect.data.Struct |
struct()
|
String |
toString() |
public static final String SERVER_ID_KEY
public static final String SERVER_NAME_KEY
public static final String SERVER_PARTITION_KEY
public static final String GTID_SET_KEY
public static final String GTID_KEY
public static final String BINLOG_FILENAME_OFFSET_KEY
public static final String BINLOG_POSITION_OFFSET_KEY
public static final String BINLOG_EVENT_ROW_NUMBER_OFFSET_KEY
public static final String TIMESTAMP_KEY
public static final String SNAPSHOT_KEY
public static final org.apache.kafka.connect.data.Schema SCHEMA
private String gtidSet
private String binlogGtid
private String binlogFilename
private long lastBinlogPosition
private int lastEventRowNumber
private long nextBinlogPosition
private int nextEventRowNumber
private String serverName
private long serverId
private long binlogTimestampSeconds
private boolean lastSnapshot
private boolean nextSnapshot
public void setServerName(String logicalId)
logicalId - the logical identifier for the database; may not be nullpublic Map<String,String> partition()
database server.
The resulting map is mutable for efficiency reasons (this information rarely changes), but should not be mutated.
public Map<String,?> offset()
public Map<String,?> offsetForRow(int eventRowNumber, int totalNumberOfRows)
This method should always be called before struct().
eventRowNumber - the 0-based row number within the event being processedtotalNumberOfRows - the total number of rows within the event being processedpublic org.apache.kafka.connect.data.Schema schema()
Schema; never nullstruct()public org.apache.kafka.connect.data.Struct struct()
Struct representation of the source partition() and offset() information. The Struct
complies with the SCHEMA for the MySQL connector.
This method should always be called after offsetForRow(int, int).
Struct; never nullschema()public boolean isSnapshotInEffect()
true if a snapshot is in effect, or false otherwisepublic void setGtid(String gtid)
gtid - the string representation of a specific GTID; may not be nullpublic void setGtidSet(String gtidSet)
gtidSet - the string representation of GTID set; may not be nullpublic void setBinlogStartPoint(String binlogFilename, long positionOfFirstEvent)
binlogFilename - the name of the binary log file; may not be nullpositionOfFirstEvent - the position in the binary log file to begin processingpublic void setEventPosition(long positionOfCurrentEvent,
long eventSizeInBytes)
positionOfCurrentEvent - the position within the binary log file of the current eventeventSizeInBytes - the size in bytes of this eventpublic void setBinlogServerId(long serverId)
serverId - the server ID found within the binary log filepublic void setBinlogTimestampSeconds(long timestampInSeconds)
timestampInSeconds - the timestamp in seconds found within the binary log filepublic void startSnapshot()
public void markLastSnapshot()
public void completeSnapshot()
public void setOffset(Map<String,?> sourceOffset)
sourceOffset - the previously-recorded Kafka Connect source offsetorg.apache.kafka.connect.errors.ConnectException - if any offset parameter values are missing, invalid, or of the wrong typepublic String gtidSet()
public String binlogFilename()
setpublic long nextBinlogPosition()
setpublic long lastBinlogPosition()
setpublic int nextEventRowNumber()
position within the binary
log file
.public int lastEventRowNumber()
position within the binary log file
.public String serverName()
setpublic static boolean isPositionAtOrBefore(Document recorded, Document desired)
offset is at or before the point in time of the second
offset, where the offsets are given in JSON representation of the maps returned by offset().
This logic makes a significant assumption: once a MySQL server/cluster has GTIDs enabled, they will never be disabled. This is the only way to compare a position with a GTID to a position without a GTID, and we conclude that any position with a GTID is *after* the position without.
When both positions have GTIDs, then we compare the positions by using only the GTIDs. Of course, if the GTIDs are the same, then we also look at whether they have snapshots enabled.
recorded - the position obtained from recorded history; never nulldesired - the desired position that we want to obtain, which should be after some recorded positions,
at some recorded positions, and before other recorded positions; never nulltrue if the recorded position is at or before the desired position; or false otherwiseCopyright © 2016 JBoss by Red Hat. All rights reserved.