fsxn-pcluster.md

Mount FSx Netapp ONTAP with AWS ParallelCluster

FSx Netapp is a multi-protocol filesystem. It mounts on Windows as SMB, Linux as NFS and Mac. This allows cluster users to bridge their Windows and Linux machines with the same filesystem, potentially running both windows and linux machines for a post-processing workflow.

Pros

• Multi-Protocol
• Hybrid support
• Multi-AZ (for High Availibility)

Cons

• Not as fast as FSx Lustre
• Harder to Setup with AWS ParallelCluster

In this guide we walk through how to create a FSx Netapp filesystem and how to mount it to parallelcluster. We will cover the steps needed to bridge this filesystem to on-prem in another doc.

1. Create Filesystem

1. Go to the FSx Console > Create Filesystem > Select FSx Netapp

2. Now set the filesystem name, select the same VPC as the cluster, and set the storage size:

Wait ~15 minutes for the filesystem to create.

2. Modify Route Table

1. Go to the FSx Console > Select the filesystem
2. Click on the Route Table:

3. Modify the Subnet association for the Route Table
4. Add the subnet that the cluster is launched

3. Modify Security Group

Next we're going to add a new rule to FSxN's Security Group. To find the security group we need to scroll down the

1. Go to the FSx Console > Select the filesystem
2. Then scroll down to the Preferred subnet > Click on the Elastic Network Interface (ENI):

3. Now check the box next to the ENI and scroll down to find the associated Security Group:

4. On that Security Group, Add a Ingress rule that allows all traffic from the VPC's CIDR range. If you'd like to be more specific, you can only allow the ports specificied in the FSxN docs.

Rule	CIDR Range	Description
All Traffic	172.31.0.0/16	FSx Netapp Ingress

4. Mount to Cluster

Next we'll mount the filesystem on the cluster using a Slurm Prolog script. This is only required to run on the head node.

1. First create a script called mount-fsxn.sh with the following content:

#!/bin/bash

# usage: mount-fsxn.sh svm-0b28f18aab8cea77a.fs-0464c49bc5b02f3c4.fsx.us-east-2.amazonaws.com /fsx

FSX_DNS=$1
MOUNT_DIR=$2

. /etc/parallelcluster/cfnconfig
test "$cfn_node_type" != "HeadNode" && exit

# create a directory
mkdir -p ${MOUNT_DIR}

# mount on head node
sudo mount -t nfs $FSX_DNS:/vol1 $MOUNT_DIR

cat << EOF > /opt/slurm/etc/prolog.sh
#!/bin/sh

if mount | /bin/grep -q ${MOUNT_DIR} ; then
  exit 0
else
  # create a directory
  mkdir -p ${MOUNT_DIR}

  # mount on compute node
  mount -t nfs $FSX_DNS:/vol1 $MOUNT_DIR
fi
EOF
chmod 744 /opt/slurm/etc/prolog.sh

echo "Prolog=/opt/slurm/etc/prolog.sh" >> /opt/slurm/etc/slurm.conf
systemctl restart slurmctld

2. Upload mount-fsxn.sh to a S3 bucket:

aws s3 cp mount.sh s3://bucket/mount-fsxn.sh

3. Include the following in your cluster's config in the HeadNode/CustomActions section:

        OnNodeConfigured:
          Script: s3://bucket/mount-fsxn.sh
          Args:
            - svm-0b28f18aab8cea77a.fs-0464c49bc5b02f3c4.fsx.us-east-2.amazonaws.com
            - /fsx
      Iam:
        S3Access:
          - BucketName: bucket