parReg

Class ParRegUtil

java.lang.Object

parReg.ParRegUtil

public class ParRegUtil
extends Object

Helper methods for partitioned region testing. Contains methods that invoke internal hooks into the partitioned region implementation.

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	diskRecoveryKey
static int	EQUAL
static int	EQUIVALENT
protected int	verifyBucketCopiesBucketId
protected boolean	verifyBucketCopiesCompleted
protected StringBuffer	verifyBucketCopiesErrStr

Constructor Summary

Constructors

Constructor and Description
ParRegUtil()

Method Summary

Modifier and Type	Method and Description
static String	_verifyPrimaries(com.gemstone.gemfire.cache.Region aRegion, int numExtraCopies, boolean logProgress) Verify there is only one primary per bucket and that the current VM's view of each bucket's primary agrees.
protected static String	compareBucketMaps(Object dump1, Object dump2) Compare the contents of 2 bucket maps.
static com.gemstone.gemfire.cache.Region	createClientRegion(String cacheConfig, String regionConfig, String xmlFile) This differs from ParRegUtil.createRegion() in that it works on the ClientCache and ClientRegionFactory created regions.
static void	createDiskStoreIfNecessary(String hydraRegionConfigName) Creates a DiskStore for the given hydra region config name, setting the diskDirs to existing disk directories if ParRegPrm-recoverFromDisk is true.
static com.gemstone.gemfire.cache.Region	createRegion(String cacheDescriptName, String regionDescriptName, String xmlFile) This differs from util.CacheUtil.createRegion(String, String, String) in that this can use disk directories previously written to the blackboard by ParRegUtil.writeDiskDirsToBB and also can call ParRegUtil.createDiskStoreIfNecessary.
static void	doRebalance() Do a rebalance and verify balance was improved.
static void	doRebalance(com.gemstone.gemfire.cache.GemFireCache cache)
static void	dumpAllDataStores(com.gemstone.gemfire.cache.Region aRegion) Dump all data stores for the given partitioned region.
static void	dumpAllPartitionedRegions() Dumps all partitioned regions of the current cache in this member.
static Boolean[]	getBucketStatus(com.gemstone.gemfire.cache.Region aRegion, int bucketId) Return the current status of the given bucketID in the current jvm.
static Object	getExistingKey(com.gemstone.gemfire.internal.cache.PartitionedRegion parReg, boolean uniqueKeys, int numThreads) Return a random key currently in the given region.
static Object	getExistingKey(com.gemstone.gemfire.cache.Region aRegion, boolean uniqueKeys, int numThreads, boolean useServerKeys) Return a random key currently in the given region.
static List	getExistingKeys(com.gemstone.gemfire.internal.cache.PartitionedRegion parReg, boolean uniqueKeys, int numThreads, int numKeysToGet) Return a list of random keys currently in the given region.
static List	getExistingKeys(com.gemstone.gemfire.cache.Region aRegion, boolean uniqueKeys, int numThreads, int numKeysToGet, boolean useServerKeys) Return a random key currently in the given region.
protected static Object	getKey(Iterator it, boolean uniqueKeys, int numThreads) Given an iterator on a set of keys, return one of them honoring uniqueKeys.
protected static List	getKeys(Iterator it, boolean uniqueKeys, int numThreads, int numKeysToGet) Given an iterator on a set of keys, return a list of them honoring uniqueKeys.
static long	getLocalSize(com.gemstone.gemfire.cache.Region aRegion) Use internal hooks to get the number of entries in this VM, but not the number of entries in the entire partitioned region.
static int	getRegionCountFromClientMetaData() Get the number of regions from ClientMetadataService
static void	HydraTask_rebalance() Hydra task to rebalance and verify.
static void	HydraTask_recreateBucketsFromHDFS() todo@lhughes -- temporary wait to recreate buckets for HDFS data
static void	HydraTask_recreateBucketsFromHDFS(com.gemstone.gemfire.cache.GemFireCache cache)
static void	HydraTask_waitForRecovery() Waits for recovery in the current vm for each root region.
static void	HydraTask_waitForRecovery(com.gemstone.gemfire.cache.GemFireCache cache)
static void	registerInterest(com.gemstone.gemfire.cache.Region aRegion) Register interest with ALL_KEYS, and InterestPolicyResult = KEYS_VALUES which is equivalent to a full GII.
static void	registerInterest(com.gemstone.gemfire.cache.Region aRegion, boolean loopUntilSuccess) Register interest with ALL_KEYS, and InterestPolicyResult = KEYS_VALUES which is equivalent to a full GII.
static void	validateAfterRecovery(com.gemstone.gemfire.cache.Region reg) Run validation on the PR after recovery (the PR must contain data)
static void	verifyBucketCopies(com.gemstone.gemfire.cache.Region aRegion, int numExtraCopies) Internal consistency check for PR; ensure that all copies of the PR's data are consistent.
void	verifyBucketCopiesBatched(com.gemstone.gemfire.cache.Region aRegion, int numExtraCopies) Internal consistency check for PR; ensure that all copies of the PR's data are consistent.
static void	verifyBucketsOnUniqueHosts(com.gemstone.gemfire.cache.Region aRegion) Verify that each bucket copy is on a different host.
static void	verifyContainsKey(com.gemstone.gemfire.cache.Region aRegion, Object key, boolean expected) Verify containsKey for the given region and key.
static void	verifyContainsValueForKey(com.gemstone.gemfire.cache.Region aRegion, Object key, boolean expected) Verify containsValueForKey for the given region and key.
static void	verifyMyValue(Object key, Object expectedValue, Object valueToCheck, int compareStrategy) Verify that the given object is an instance of ValueHolder with expectedValue as the myValue field.
static void	verifyPrimaries(com.gemstone.gemfire.cache.Region aRegion, int numExtraCopies) Verify there is only one primary per bucket and that the current VM's view of each bucket's primary agrees.
static void	verifyPrimariesWithWait(com.gemstone.gemfire.cache.Region aRegion, int numExtraCopies) Verify there is only one primary per bucket and that the current VM's view of each bucket's primary agrees.
static void	verifyPRMetaData(com.gemstone.gemfire.cache.Region aRegion) Verify consistency for PR metadata.
static void	verifySize(com.gemstone.gemfire.cache.Region aRegion, int expectedSize) Verify that the size of the given region is expectedSize.
static void	verifySize(com.gemstone.gemfire.cache.Region aRegion, int expectedSize, Map regionSnapshot) Verify that the size of the given region is expectedSize.
protected static void	verifyValue(Object key, Object obj1, Object obj2, int compareStrategy) Verify that the two given objects are equal, giving special consideration to instances of ValueHolder.
static void	writeDiskDirsToBB(com.gemstone.gemfire.cache.Region reg) For disk tests, write the diskDir names to the blackboard for later use by recovery.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EQUAL

public static int EQUAL

EQUIVALENT

public static int EQUIVALENT

verifyBucketCopiesBucketId

protected int verifyBucketCopiesBucketId

verifyBucketCopiesErrStr

protected StringBuffer verifyBucketCopiesErrStr

verifyBucketCopiesCompleted

protected boolean verifyBucketCopiesCompleted

diskRecoveryKey

protected static final String diskRecoveryKey

See Also:

Constant Field Values

Constructor Detail

ParRegUtil

public ParRegUtil()

Method Detail

getExistingKey

public static Object getExistingKey(com.gemstone.gemfire.cache.Region aRegion,
                                    boolean uniqueKeys,
                                    int numThreads,
                                    boolean useServerKeys)

Return a random key currently in the given region.

Parameters:

aRegion - The region to use for getting an existing key (may or may not be a partitioned region).

uniqueKeys - True if each thread operates on a unique set of keys.

numThreads - Used if uniqueKeys is true. This is the number of threads using uniqueKeys.

getExistingKey

public static Object getExistingKey(com.gemstone.gemfire.internal.cache.PartitionedRegion parReg,
                                    boolean uniqueKeys,
                                    int numThreads)

Return a random key currently in the given region.

Parameters:

aRegion - The PartitionedRegion to use for getting an existing key.

uniqueKeys - True if each thread operates on a unique set of keys.

numThreads - Used if uniqueKeys is true. This is the number of threads using uniqueKeys.

getKey

protected static Object getKey(Iterator it,
                               boolean uniqueKeys,
                               int numThreads)

Given an iterator on a set of keys, return one of them honoring uniqueKeys.

Parameters:

it - An iterator on a set of keys.

uniqueKeys - True if each thread operates on a unique set of keys.

numThreads - Used if uniqueKeys is true. This is the number of threads using uniqueKeys.

getLocalSize

public static long getLocalSize(com.gemstone.gemfire.cache.Region aRegion)

Use internal hooks to get the number of entries in this VM, but not the number of entries in the entire partitioned region. aRegion must be a PartitionedRegion.

getExistingKeys

public static List getExistingKeys(com.gemstone.gemfire.cache.Region aRegion,
                                   boolean uniqueKeys,
                                   int numThreads,
                                   int numKeysToGet,
                                   boolean useServerKeys)

Return a random key currently in the given region.

Parameters:

aRegion - The region to use for getting an existing key (may or may not be a partitioned region).

uniqueKeys - True if each thread operates on a unique set of keys.

numThreads - Used if uniqueKeys is true. This is the number of threads using uniqueKeys.

numKeysToGet - Return a list of this many keys, or fewer if there aren't enough keys available.

useServerKeys - Use keysOnServer to get existing keys.

getExistingKeys

public static List getExistingKeys(com.gemstone.gemfire.internal.cache.PartitionedRegion parReg,
                                   boolean uniqueKeys,
                                   int numThreads,
                                   int numKeysToGet)

Return a list of random keys currently in the given region.

Parameters:

aRegion - The PartitionedRegion to use for getting an existing key.

uniqueKeys - True if each thread operates on a unique set of keys.

numThreads - Used if uniqueKeys is true. This is the number of threads using uniqueKeys.

numKeysToGet - Return a list of this many keys, or fewer if there aren't enough keys available.

getKeys

protected static List getKeys(Iterator it,
                              boolean uniqueKeys,
                              int numThreads,
                              int numKeysToGet)

Given an iterator on a set of keys, return a list of them honoring uniqueKeys.

Parameters:

it - An iterator on a set of keys.

uniqueKeys - True if each thread operates on a unique set of keys.

numThreads - Used if uniqueKeys is true. This is the number of threads using uniqueKeys.

numKeysToGet - Return a list of this many keys, or fewer if there aren't enough keys available.

getRegionCountFromClientMetaData

public static int getRegionCountFromClientMetaData()

Get the number of regions from ClientMetadataService

verifyMyValue

public static void verifyMyValue(Object key,
                                 Object expectedValue,
                                 Object valueToCheck,
                                 int compareStrategy)

Verify that the given object is an instance of ValueHolder with expectedValue as the myValue field.

Parameters:

key - The key whose value we are checking.

expectedValue - The expected myValue field of a ValueHolder in aRegion, or null if the expected value should be null.

valuetoCheck - This is expected to be a ValueHolder or PdxInstance representing a ValueHolder, whose myValue field compares to expectedValue, according to comparStrategy

compareStrategy - Whether the compare is equals or equivalent (for ValueHolders)

Throws:

TestException - if the result of a get on key does not have the expected value.

verifyValue

protected static void verifyValue(Object key,
                                  Object obj1,
                                  Object obj2,
                                  int compareStrategy)

Verify that the two given objects are equal, giving special consideration to instances of ValueHolder.

Parameters:

key - The key whose value we are checking.

obj1 - One object to compare.

obj2 - The other object to compare.

compareStrategy - Whether the compare is equals or equivalent (for ValueHolders)

Throws:

TestException - if the result of a get on key does not have the expected value.

verifyContainsKey

public static void verifyContainsKey(com.gemstone.gemfire.cache.Region aRegion,
                                     Object key,
                                     boolean expected)

Verify containsKey for the given region and key.

Parameters:

aRegion - The region to verify.

key - The key in aRegion to verify.

expected - The expected value of containsKey()

Throws:

TestException - if containsKey() has the wrong value

verifyContainsValueForKey

public static void verifyContainsValueForKey(com.gemstone.gemfire.cache.Region aRegion,
                                             Object key,
                                             boolean expected)

Verify containsValueForKey for the given region and key.

Parameters:

aRegion - The region to verify.

key - The key in aRegion to verify.

expected - The expected value of containsKey()

Throws:

TestException - if containsValueforKey() has the wrong value

verifySize

public static void verifySize(com.gemstone.gemfire.cache.Region aRegion,
                              int expectedSize)

Verify that the size of the given region is expectedSize.

Parameters:

aRegion - The region to verify.

expectedSize - The expected size of aRegion.

Throws:

TestException - if size() has the wrong value

verifySize

public static void verifySize(com.gemstone.gemfire.cache.Region aRegion,
                              int expectedSize,
                              Map regionSnapshot)

Verify that the size of the given region is expectedSize.

Parameters:

aRegion - The region to verify.

expectedSize - The expected size of aRegion.

Throws:

TestException - if size() has the wrong value

dumpAllDataStores

public static void dumpAllDataStores(com.gemstone.gemfire.cache.Region aRegion)

Dump all data stores for the given partitioned region.

dumpAllPartitionedRegions

public static void dumpAllPartitionedRegions()

Dumps all partitioned regions of the current cache in this member.

verifyPRMetaData

public static void verifyPRMetaData(com.gemstone.gemfire.cache.Region aRegion)

Verify consistency for PR metadata.

verifyPrimaries

public static void verifyPrimaries(com.gemstone.gemfire.cache.Region aRegion,
                                   int numExtraCopies)

Verify there is only one primary per bucket and that the current VM's view of each bucket's primary agrees.

Parameters:

numExtraCopies - - The number of expected extra copies of data for aRegion (in addition to the primary copy). of data for aRegion. If this is -1, then this will expect that we have less than full redundancy at the time of this check.

verifyPrimariesWithWait

public static void verifyPrimariesWithWait(com.gemstone.gemfire.cache.Region aRegion,
                                           int numExtraCopies)

Verify there is only one primary per bucket and that the current VM's view of each bucket's primary agrees. Due to the asychronous nature of electing primaries, the test might need to wait a bit before all primaries are selected. If we get a primary failure, sleep, then validate primaries again until 1) we pass the validation 2) we have been trying to pass the validation for N millis.

Parameters:

numExtraCopies - - The number of expected extra copies of data for aRegion (in addition to the primary copy). If this is -1, then this will expect that we have less than full redundancy at the time of this check.

_verifyPrimaries

public static String _verifyPrimaries(com.gemstone.gemfire.cache.Region aRegion,
                                      int numExtraCopies,
                                      boolean logProgress)

Verify there is only one primary per bucket and that the current VM's view of each bucket's primary agrees.

Parameters:

numExtraCopies - - The number of expected extra copies of data for aRegion (in addition to the primary copy). If this is -1, then this will expect that we have less than full redundancy at the time of this check.

logProgress - If true, then log each bucket as it is verified, otherwise run silent.

verifyBucketCopies

public static void verifyBucketCopies(com.gemstone.gemfire.cache.Region aRegion,
                                      int numExtraCopies)

Internal consistency check for PR; ensure that all copies of the PR's data are consistent. This returns when all buckets are verified. Compare with verifyBucketCopiesBatched which can be called repeatedly in a batched task and throws StopSchedulingTaskOnClientOrder to signal it has completed all verification. Note this is a static method.

Parameters:

aRegion - - A partitioned region.

numExtraCopies - - The number of expected extra copies of data for aRegion (in addition to the primary copy). If this is -1, then this will expect that we have less than full redundancy at the time of this check.

verifyBucketCopiesBatched

public void verifyBucketCopiesBatched(com.gemstone.gemfire.cache.Region aRegion,
                                      int numExtraCopies)

Internal consistency check for PR; ensure that all copies of the PR's data are consistent. This must be called repeatedly on the same ParRegUtil instance by the same thread until StopSchedulingTaskOnClientOrder is thrown to signal completion. This method will return after running for TestHelperPrms.minTaskGranularitySec seconds to enable batching. Compare with verifyBucketCopies, which verifies all buckets in one call. Note this is an instance method.

Parameters:

aRegion - - A partitioned region.

numExtraCopies - - The number of expected extra copies of data for aRegion (in addition to the primary copy). If this is -1, then this will expect that we have less than full redundancy at the time of this check.

compareBucketMaps

protected static String compareBucketMaps(Object dump1,
                                          Object dump2)

Compare the contents of 2 bucket maps. Return a string describing any descrepancies.

Parameters:

map1 - A map of keys/values, ie a bucket.

map2 - A map of keys/values, ie a bucket.

registerInterest

public static void registerInterest(com.gemstone.gemfire.cache.Region aRegion,
                                    boolean loopUntilSuccess)

Register interest with ALL_KEYS, and InterestPolicyResult = KEYS_VALUES which is equivalent to a full GII.

Parameters:

aRegion - The region to register interest on.

loopUntilSuccess - If true, loop until registerInterest is successful while allowing remote PartitionOfflineExceptions.

registerInterest

public static void registerInterest(com.gemstone.gemfire.cache.Region aRegion)

Register interest with ALL_KEYS, and InterestPolicyResult = KEYS_VALUES which is equivalent to a full GII.

Parameters:

aRegion - The region to register interest on.

verifyBucketsOnUniqueHosts

public static void verifyBucketsOnUniqueHosts(com.gemstone.gemfire.cache.Region aRegion)

Verify that each bucket copy is on a different host.

doRebalance

public static void doRebalance()

Do a rebalance and verify balance was improved. If evictionPercentage > 0 (the default) then we have heapLRU and this can cause simulate and rebalance results to differ if eviction kicks in between. (See BUG 44899).

doRebalance

public static void doRebalance(com.gemstone.gemfire.cache.GemFireCache cache)

HydraTask_rebalance

public static void HydraTask_rebalance()

Hydra task to rebalance and verify. This will only run in one vm, even if many vms execute this task. This is coordinated by a blackboard counter. Once this has executed in a single vm, it won't rebalance again (unless the counter is zeroed).

HydraTask_waitForRecovery

public static void HydraTask_waitForRecovery()

Waits for recovery in the current vm for each root region.

HydraTask_waitForRecovery

public static void HydraTask_waitForRecovery(com.gemstone.gemfire.cache.GemFireCache cache)

HydraTask_recreateBucketsFromHDFS

public static void HydraTask_recreateBucketsFromHDFS()

todo@lhughes -- temporary wait to recreate buckets for HDFS data

HydraTask_recreateBucketsFromHDFS

public static void HydraTask_recreateBucketsFromHDFS(com.gemstone.gemfire.cache.GemFireCache cache)

writeDiskDirsToBB

public static void writeDiskDirsToBB(com.gemstone.gemfire.cache.Region reg)

For disk tests, write the diskDir names to the blackboard for later use by recovery. Each time this is called, the diskDirs will be written to the bb shared map under a new key.

validateAfterRecovery

public static void validateAfterRecovery(com.gemstone.gemfire.cache.Region reg)

Run validation on the PR after recovery (the PR must contain data)

createDiskStoreIfNecessary

public static void createDiskStoreIfNecessary(String hydraRegionConfigName)

Creates a DiskStore for the given hydra region config name, setting the diskDirs to existing disk directories if ParRegPrm-recoverFromDisk is true. Disk dirs are set to dirs previously written to the blackboard by ParRegUtil.writediskDirsToBB. A different set of diskDirs is assigned to the diskStore ech time this is called. If no recovery is specified, then creates the DiskStore without explicitly setting disk dirs. If no diskStore is specified in the hydraRRegionConfigName, then no DiskStore is created.

Parameters:

hydraRegionConfigName - The hydra config name for the region to be defined for recovery.

createRegion

public static com.gemstone.gemfire.cache.Region createRegion(String cacheDescriptName,
                                                             String regionDescriptName,
                                                             String xmlFile)

This differs from util.CacheUtil.createRegion(String, String, String) in that this can use disk directories previously written to the blackboard by ParRegUtil.writeDiskDirsToBB and also can call ParRegUtil.createDiskStoreIfNecessary. Some tests use end tasks to bring up a new set of vms after the regular task vms shutdown to cause a disk recovery. The end tasks have different vmIDs than the regular tasks, so end tasks must read the disk directory names created by the regular tasks from the blackboard (and contain the regular task's vmIds) and make the end tasks use them for recovery. Create a region either by creating the cache with a declarative xml file or create the region programmatically. Which method chosen depends on the hydra parameter CachePrms.useDeclarativeXmlFile and whether this VM already has a connection to the distributed system.

Parameters:

cacheDescriptName - The name of the cache description to use to create the cache.

regionDescriptName - The name of the region description to use if the region is created programmatically.

xmlFile - The xmlFile to use if the region is created with the xml file. Note: This can only honor CachePrms.useDeclarativeXmlFile or the xmlFile parameter if we are not already connected. If we already have a connection to the distributed system, then creating the cache will use any declarative xml file used when the connection was established, if any declarative xml file was set during the connect. See the comments in the code below.

createClientRegion

public static com.gemstone.gemfire.cache.Region createClientRegion(String cacheConfig,
                                                                   String regionConfig,
                                                                   String xmlFile)

This differs from ParRegUtil.createRegion() in that it works on the ClientCache and ClientRegionFactory created regions.

getBucketStatus

public static Boolean[] getBucketStatus(com.gemstone.gemfire.cache.Region aRegion,
                                        int bucketId)

Return the current status of the given bucketID in the current jvm.

Parameters:

aRegion - The region to get bucket status on.

bucketId - The bucket ID of interest.

Returns:

null if no buckets exist anywere for the given bucketId OR Boolean[] where [0] true if bucket exists locally, false otherwise [1] true if the local bucket is primary, false if not primary or the bucket is not hosted locally (meaning that [0] is false