public class AwsProvisioner extends TaskClusterRequestHandler
The AWS Provisioner is responsible for provisioning instances on EC2 for use in TaskCluster. The provisioner maintains a set of worker configurations which can be managed with an API that is typically available at aws-provisioner.taskcluster.net. This API can also perform basic instance management tasks in addition to maintaining the internal state of worker type configuration information.
The Provisioner runs at a configurable interval. Each iteration of the provisioner fetches a current copy the state that the AWS EC2 api reports. In each iteration, we ask the Queue how many tasks are pending for that worker type. Based on the number of tasks pending and the scaling ratio, we may submit requests for new instances. We use pricing information, capacity and utility factor information to decide which instance type in which region would be the optimal configuration.
Each EC2 instance type will declare a capacity and utility factor. Capacity is the number of tasks that a given machine is capable of running concurrently. Utility factor is a relative measure of performance between two instance types. We multiply the utility factor by the spot price to compare instance types and regions when making the bidding choices.
| Modifier and Type | Field and Description |
|---|---|
protected static String |
defaultBaseURL |
| Constructor and Description |
|---|
AwsProvisioner() |
AwsProvisioner(String baseURL) |
AwsProvisioner(String clientId,
String accessToken) |
AwsProvisioner(String clientId,
String accessToken,
String certificate) |
| Modifier and Type | Method and Description |
|---|---|
CallSummary<EmptyPayload,EmptyPayload> |
apiReference()
Get an API reference!
|
CallSummary<EmptyPayload,EmptyPayload> |
awsState()
DEPRECATED.
|
CallSummary<GetSecretRequest,EmptyPayload> |
createSecret(String token,
GetSecretRequest payload)
Insert a secret into the secret storage.
|
CallSummary<CreateWorkerTypeRequest,GetWorkerTypeRequest> |
createWorkerType(String workerType,
CreateWorkerTypeRequest payload)
Create a worker type and ensure that all EC2 regions have the required KeyPair
|
CallSummary<EmptyPayload,Object> |
getLaunchSpecs(String workerType)
Return the EC2 LaunchSpecifications for all combinations of regions and instance types or a list of reasons why the launch specifications are not valid
|
CallSummary<EmptyPayload,GetSecretResponse> |
getSecret(String token)
Retreive a secret from storage.
|
CallSummary<EmptyPayload,EmptyPayload> |
instanceStarted(String instanceId,
String token)
An instance will report in by giving its instance id as well as its security token.
|
CallSummary<EmptyPayload,String[]> |
listWorkerTypes()
List all known WorkerType names
|
CallSummary<EmptyPayload,EmptyPayload> |
ping()
Documented later…
|
CallSummary<EmptyPayload,EmptyPayload> |
removeSecret(String token)
Remove a secret.
|
CallSummary<EmptyPayload,EmptyPayload> |
removeWorkerType(String workerType)
Delete a WorkerType definition, submits requests to kill all instances and delete the KeyPair from all configured EC2 regions
|
CallSummary<EmptyPayload,EmptyPayload> |
state(String workerType)
Return the state of a given workertype as stored by the provisioner.
|
CallSummary<CreateWorkerTypeRequest,GetWorkerTypeRequest> |
updateWorkerType(String workerType,
CreateWorkerTypeRequest payload)
Update a workerType and ensure that all regions have the require KeyPair
|
CallSummary<EmptyPayload,GetWorkerTypeRequest> |
workerType(String workerType)
Retreive a WorkerType definition
|
apiCall, configureHawk, setBaseURLprotected static final String defaultBaseURL
public AwsProvisioner(String baseURL)
public AwsProvisioner()
public CallSummary<CreateWorkerTypeRequest,GetWorkerTypeRequest> createWorkerType(String workerType, CreateWorkerTypeRequest payload) throws APICallFailure
Create a worker type and ensure that all EC2 regions have the required KeyPair
See http://docs.taskcluster.net/aws-provisioner/api-docs/#createWorkerType
APICallFailurepublic CallSummary<CreateWorkerTypeRequest,GetWorkerTypeRequest> updateWorkerType(String workerType, CreateWorkerTypeRequest payload) throws APICallFailure
Update a workerType and ensure that all regions have the require KeyPair
See http://docs.taskcluster.net/aws-provisioner/api-docs/#updateWorkerType
APICallFailurepublic CallSummary<EmptyPayload,GetWorkerTypeRequest> workerType(String workerType) throws APICallFailure
Retreive a WorkerType definition
See http://docs.taskcluster.net/aws-provisioner/api-docs/#workerType
APICallFailurepublic CallSummary<EmptyPayload,EmptyPayload> removeWorkerType(String workerType) throws APICallFailure
Delete a WorkerType definition, submits requests to kill all instances and delete the KeyPair from all configured EC2 regions
See http://docs.taskcluster.net/aws-provisioner/api-docs/#removeWorkerType
APICallFailurepublic CallSummary<EmptyPayload,String[]> listWorkerTypes() throws APICallFailure
List all known WorkerType names
See http://docs.taskcluster.net/aws-provisioner/api-docs/#listWorkerTypes
APICallFailurepublic CallSummary<GetSecretRequest,EmptyPayload> createSecret(String token, GetSecretRequest payload) throws APICallFailure
Insert a secret into the secret storage. This should not normally be done through this API, but is provided for testing and completeness
See http://docs.taskcluster.net/aws-provisioner/api-docs/#createSecret
APICallFailurepublic CallSummary<EmptyPayload,GetSecretResponse> getSecret(String token) throws APICallFailure
Retreive a secret from storage. It is important that this secret is deleted by the consumer, or else the secrets will be visible to any process which can read HTTP on the worker localhost interface.
See http://docs.taskcluster.net/aws-provisioner/api-docs/#getSecret
APICallFailurepublic CallSummary<EmptyPayload,EmptyPayload> instanceStarted(String instanceId, String token) throws APICallFailure
An instance will report in by giving its instance id as well as its security token. The token is given and checked to ensure that it matches a real token that exists to ensure that random machines do not check in. We could generate a different token but that seems like overkill
See http://docs.taskcluster.net/aws-provisioner/api-docs/#instanceStarted
APICallFailurepublic CallSummary<EmptyPayload,EmptyPayload> removeSecret(String token) throws APICallFailure
Remove a secret. It is very important that the consumer of a secret delete the secret from storage before handing over control to another process or else it could read the HTTP UserData endpoint and use the getSecrete() api here to get the secrets
See http://docs.taskcluster.net/aws-provisioner/api-docs/#removeSecret
APICallFailurepublic CallSummary<EmptyPayload,Object> getLaunchSpecs(String workerType) throws APICallFailure
Return the EC2 LaunchSpecifications for all combinations of regions and instance types or a list of reasons why the launch specifications are not valid
This API end-point is experimental and may be subject to change without warning.
See http://docs.taskcluster.net/aws-provisioner/api-docs/#getLaunchSpecs
APICallFailurepublic CallSummary<EmptyPayload,EmptyPayload> awsState() throws APICallFailure
APICallFailurepublic CallSummary<EmptyPayload,EmptyPayload> state(String workerType) throws APICallFailure
Return the state of a given workertype as stored by the provisioner. This state is stored as three lists: 1 for all instances, 1 for requests which show in the ec2 api and 1 list for those only tracked internally in the provisioner.
See http://docs.taskcluster.net/aws-provisioner/api-docs/#state
APICallFailurepublic CallSummary<EmptyPayload,EmptyPayload> ping() throws APICallFailure
Documented later…
Warning this api end-point is not stable.
See http://docs.taskcluster.net/aws-provisioner/api-docs/#ping
APICallFailurepublic CallSummary<EmptyPayload,EmptyPayload> apiReference() throws APICallFailure
Get an API reference!
Warning this api end-point is not stable.
See http://docs.taskcluster.net/aws-provisioner/api-docs/#apiReference
APICallFailureCopyright © 2014–2015 Mozilla. All rights reserved.