Spot Instances

Overview

PyStratus supports launching clusters using EC2 spot instances (hadoop plugin only). This is activated on a per-cluster basis.

Required Settings

To enable spot instances for a cluster, add the following attributes to the cluster definition.

• spot_cluster=true set this to tell PyStratus that you would like this cluster to be launched with spot instances; also see An Alternative Approach below.
• max_price=.1 set this to the maximum price you are willing to pay for your spot instances
• launch_group=name-of-launch-group set this to the name of the launch group for your spot instances

Optional Settings

By default the master instance will not be launched as a spot instance to give a bit more stability in the likely event that your spot instances are terminated. If you would like to bypass this and have all instances in a cluster use spots, you can set master_spot=true in the cluster definition.

An Alternative Approach

If you have a cluster that you would like to launch as spot instances sometimes, you can add the max_price and launch_group to the cluster definition. This alone will leave the cluster using On Demand instances. If you would like to launch it with spot instances, simply set an environment variable SPOT_CLUSTER=true before executing stratus.

E.g.

# will launch spot instances assuming max_price and launch_group have been provided
SPOT_CLUSTER=true stratus exec CLUSTERNAME launch-cluster 3