This might help someone who wants to set up GoCD (https://www.gocd.io/) on an Amazon EC2 machine with the Amazon Linux AMI. I'm doing this from a Mac.
I'm not great at this stuff but I don't think I've done anything here terribly wrong. It will at least get you started. A person who is better at this sort of thing would probably use Docker.
In the Amazon console, select the Amazon Linux AMI. Choose the m4.xlarge instance type. I'm running the Go server and two agents on this machine. It is likely overkill -- I'm going to monitor the machine and downgrade if it makes sense, or install a third agent.
On the 'Add Storage' step, add a new 8 GB volume with the device /dev/sdb selected. We will use this to store Go's artifacts, as their docs recommend storing artifacts away from the rest of Go.
Add security group rules for HTTP / TCP / 80 / 0.0.0.0/0
and HTTPS / TCP / 443 / 0.0.0.0/0
.
When you go to launch the instance, a popup will ask you to create a new key pair or select an existing one. To create a new one, give it a name (e.g., 'gocd') and click the download button. Save it in your Downloads folder. Click 'Launch Instance'.
While the instance is launching, copy the private key file to your ssh directory and restrict the permissions:
cp ~/Downloads/gocd.pem ~/.ssh
chmod 400 ~/.ssh/gocd.pem
Go grab the 'IPv4 Public IP' from your instance's description in the AWS console. Edit the file ~/.ssh/config and add this:
Host gocd
HostName [your IPv4 Public IP]
User ec2-user
IdentityFile ~/.ssh/gocd.pem
This will let you type ssh gocd
to log into your machine. Do that now to make sure everything is working. Apply available updates by typing sudo yum update
.
This is optional but the Apache instructions below will need to be changed if you choose not to do it.
Go to the Elastic IPs section in the Amazon console and allocate a new address. Associate it to your GoCD EC2 instance. This wipes out your existing public IP address, so edit ~/.ssh/config and put your new Elastic IP in for the HostName.
Use your DNS provider to set up a new A record to point gocd.yourdomain.com at the Elastic IP address.
First install Java 1.8 with sudo yum install -y java-1.8.0-openjdk
. Then uninstall Java 1.7 with sudo yum remove java-1.7.0-openjdk
. My understanding is that the order matters because uninstalling Java 1.7 first will also uninstall dependencies that we want to hold onto.
Install the Go server:
echo "
[gocd]
name = GoCD YUM Repository
baseurl = https://download.gocd.io
enabled = 1
gpgcheck = 1
gpgkey = https://download.gocd.io/GOCD-GPG-KEY.asc
" | sudo tee /etc/yum.repos.d/gocd.repo
sudo yum install -y go-server
sudo yum install -y httpd24
sudo service httpd start
sudo chkconfig httpd on
That last line is to make it start up automatically on system boot.
You should now be able to go to http://gocd.yourdomain.com and see the Apache HTTP server test page.
Make an httpd configuration file (sudo vi /etc/httpd/conf.d/gocd.conf
) and put this in it:
<VirtualHost *:80>
ServerName gocd.yourdomain.com
ProxyPass "/" "http://localhost:8153/"
ProxyPassReverse "/" "http://localhost:8153/"
ProxyPreserveHost On
</VirtualHost>
Restart httpd: sudo service httpd restart
.
Start the Go server: sudo /etc/init.d/go-server start
If you go to http://gocd.yourdomain.com, you should now see the Go server's 'add pipeline' page.
We will add SSL capability later.
I recommend using the oauth plugin (see https://github.com/gocd-contrib/gocd-oauth-login), but even if you use that you're instructed to first set up an admin account using Go's built-in password mechanism. Here's how:
sudo htpasswd -c -s /etc/go/passwd yourusername
That will prompt you to enter a password. Once you're done, in your Go server UI go to the 'Server Configuration' page and set the Password File Path to /etc/go/passwd
. Save and it will redirect you to the authentication page. Check that you can log in.
Based on the instructions at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html, prepare the xvdb volume and mount it at the '/artifacts' dir:
sudo mkfs -t ext4 /dev/xvdb
sudo mkdir /artifacts
sudo mount /dev/xvdb /artifacts
To make this happen on bootup, back up the fstab file (sudo cp /etc/fstab /etc/fstab.orig
) and then grab the UUID that sudo file -s /dev/xvdb
spits out. Edit the fstab file sudo vi /etc/fstab
and put this line at the end:
UUID=[your_uuid] /artifacts ext4 defaults,nofail 0 2
Run sudo mount -a
and if it doesn't print anything out, it worked.
In the Go server configuration UI, set the artifacts directory to be /artifacts
. Make the go user the owner: sudo chown go:go /artifacts
. Then restart the Go server: sudo /etc/init.d/go-server restart
.
The first agent is easy to install: sudo yum install -y go-agent
. Start it: sudo /etc/init.d/go-agent
. You should now see it listed at http://gocd.yourdomain.com/go/agents
For the second one, do this (based on https://docs.gocd.io/current/advanced_usage/admin_install_multiple_agents.html)
sudo ln -s /etc/init.d/go-agent /etc/init.d/go-agent-1
sudo ln -s /usr/share/go-agent /usr/share/go-agent-1
sudo cp /etc/default/go-agent /etc/default/go-agent-1
sudo mkdir /var/{lib,log}/go-agent-1
sudo chown go:go /var/{lib,log}/go-agent-1
sudo chgrp go /etc/default/go-agent-1
Then start it: sudo /etc/init.d/go-agent-1
Make it start automatically on boot: sudo chkconfig go-agent-1 on
.
This is based on the instructions here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-an-instance.html
Install mod_ssl: sudo yum install -y mod24_ssl
Then get an SSL cert for gocd.yourdomain.com and upload the files to the server:
scp -i ~/.ssh/gocd.pem gocd_yourdomain_com.crt gocd_yourdomain_com.key gocd_yourdomain_com_bundle.pem ec2-user@yourpublicip:~
Copy your certificate files to the appropriate places and fix up their permissions:
sudo cp gocd_yourdomain_com.key /etc/pki/tls/private
sudo cp gocd_yourdomain_com.crt /etc/pki/tls/certs
sudo cp gocd_yourdomain_com_bundle.pem /etc/pki/tls/certs
sudo chmod 600 /etc/pki/tls/certs/gocd_yourdomain_com.crt
sudo chmod 644 /etc/pki/tls/certs/gocd_yourdomain_com_bundle.pem
sudo chmod 600 /etc/pki/tls/private/gocd_yourdomain_com.key
Edit the relevant lines in /etc/httpd/conf.d/ssl.conf
to match:
SSLCertificateFile /etc/pki/tls/certs/gocd_yourdomain_com.crt
SSLCertificateKeyFile /etc/pki/tls/private/gocd_yourdomain_com.key
SSLCertificateChainFile /etc/pki/tls/certs/gocd_yourdomain_com_bundle.pem
Edit /etc/httpd/conf.d/gocd.conf
and add this block at the end:
<VirtualHost *:443>
ServerName gocd.yourdomain.com
# Proxy everything over to the GoCD server
ProxyPass / http://localhost:8153/
ProxyPassReverse / http://localhost:8153/
ProxyPreserveHost On
RequestHeader set X-Forwarded-Proto "https"
<Location />
Order allow,deny
Allow from all
</Location>
# SSL configuration
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/gocd_yourdomain_com.crt
SSLCertificateKeyFile /etc/pki/tls/private/gocd_yourdomain_com.key
SSLCertificateChainFile /etc/pki/tls/certs/gocd_yourdomain_com_bundle.pem
</VirtualHost>
Restart httpd: sudo service httpd restart
Going to https://gocd.yourdomain.com should work.
You'll need to install whatever software your agents need to run your tests (e.g., a database, ruby, etc.) and configure pipelines in Go.
Thanks a lot for these instructions!
You seem to be missing
sudo /etc/init.d/go-server start
in the Install and configure Apache section