Skip to content

Instantly share code, notes, and snippets.

Last active September 3, 2021 04:49
What would you like to do?


apt-get install fuse

chmod +x goofys-latest
cp goofys-latest /usr/local/bin/goofys

mkdir -p ~/.aws
echo '[default]' > ~/.aws/credentials
echo 'aws_access_key_id = XKXXXXXXXXXXXXXXXXXXXXXX' >> ~/.aws/credentials
echo 'aws_secret_access_key = YYYYYYYYYYYYYYYYYYYYYYYYYYYYY ' >> ~/.aws/credentials

mkdir -p /goofys_mountpoint
chown clickhouse:clickhouse /goofys_mountpoint

# one-time mount (just to test)
goofys -f -o allow_other --uid=111 --gid=115 --file-mode=0755 --dir-mode=0755 -o noatime --region=us-west-2 --debug_s3 s3_bucket_name /goofys_mountpoint

# edit /etc/fstab like that (to remount on each restart)
goofys#s3_bucket_name /goofys_mountpoint fuse _netdev,allow_other,noatime,--uid=111,--gid=115,--file-mode=0755,--dir-mode=755 0 0

# df -h 


Project page:

  1. install package:
apt-get update
apt-get install s3fs
  1. create & chmod file with credentials
echo 'KEYID:ACCESS_KEY' > /etc/passwd-s3fs
chmod 600 /etc/passwd-s3fs
  1. create mounting point
mkdir -p /s3fsmount
chown clickhouse:clickhouse /s3fsmount
  1. check/debug if you are able to mount in verbose mode
# S3 doesn't have normal file atributes & owners like in linux, so to make it accessible for `clickhouse`
# user we need to fix the owners and attributes.
# 1. check uid / gid of clickhouse user first, you will need them in later commands (in uid / gid)
id -u clickhouse
id -g clickhouse

# I've met that problem when changing bucket to us-west2:
s3fs bucketname /s3fsmount/ -o dbglevel=info -f -o url= -o allow_other -o umask=227 -o uid=111 -o gid=115 -o noatime

# or - even more verbose:
# s3fs bucketname /s3fsmount/ -o dbglevel=debug -f -o curldbg -o url= -o allow_other -o umask=227 -o uid=111 -o gid=115

# you can just leave it running attached to some terminal and to all FS checks in another window.
# press Ctrl+C to exit that, after you see that everything is mounted & working
  1. After it's working - you can add the entry to /etc/fstab (to mount filesystem everytime when system starts)
#edit /etc/fstab - add a line:
s3fs#bucketname /s3fsmount fuse _netdev,allow_other,umask=227,uid=111,gid=115,noatime 0 0

# other samples (also with cache)
#s3fs#s3fsmount /s3fsmount fuse _netdev,allow_other,umask=227,uid=111,gid=115,url=,use_cache=/root/cache 0 0

after that do

# to mount all disks in fstab
mount -a

# check if mounted:
df -h
s3fs            256T     0  256T   0% /s3fsmount

# to unmount the disk
umount /s3fsmount
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment