Persistent coasters documentation

persistent_coaster.asc

Coasters

Coasters is the resource manager for swift. By default, swift uses automatic-coasters, which provisions compute resources for the duration of a swift run based on the directives in the sites file. Persistent coasters, on the other hand allows you to retain an active service separate from the swift runtime across several swift runs. This can be used to hold on to resources in a queue based system, string together a collection of nodes for use as a quick ad-hoc cluster, or help work-around networking issues.

Modes of operation

There are 3 modes of interest when using persistent-coasters

Coaster-service in active mode.

Here the coaster-service alone is persistent, the workers are started per run, based on the configurations set in the sites file used by the Swift run. This mode helps simplify the networking aspects of the early stages of swift initialization.

Let’s say you want to run swift on "lumiere" where you data resides, and your compute nodes are attached to the head node "fram". For this setup you’ll need swift installed on both machines. 1. Login to fram and run :

# Provide a port which you've confirmed to be not fire-walled.
# Assume 50005 is available
coaster-service -p 50005 -nosec

1. Note the line "Started coaster service: http://140.221.71.82:50005" This url should be used in the sites file used by swift.

2. Login to lumiere and edit the sites file:

<?xml version="1.0" encoding="UTF-8"?>
<config xmlns="http://www.ci.uchicago.edu/swift/SwiftSites">
 <pool handle="fram">
   <!-- paste the Coaster service URL here -->
   <execution provider="coaster-persistent" url="http://140.221.71.82:50015" jobmanager="local:pbs"/>
   <profile namespace="globus" key="jobsPerNode">24</profile>
   <profile key="jobThrottle" namespace="karajan">1</profile>
   <profile namespace="globus" key="maxtime">300</profile>
   <profile namespace="globus" key="maxwalltime">00:03:00</profile>
   <profile namespace="karajan" key="initialScore">10000</profile>
   <profile namespace="globus" key="slots">1</profile>
   <profile namespace="globus" key="nodeGranularity">1</profile>
   <profile namespace="globus" key="maxNodes">1</profile>
   <profile namespace="globus" key="queue">batch</profile>
   <filesystem provider="local" url="none" />
   <workdirectory>/home/linjiao/tmp/swift</workdirectory>
 </pool>
</config>

1. Run swift on lumiere with the above sites file

Coasters in passive mode

Via start-coaster-service