Lets spin up a cloud VM so we can talk about other stuff while we wait.
- Deploy an Ubuntu 18.04 instance.
- Make sure the keypair is injected
- Make sure to associate the static IP to the one used for DNS (see below)
- Don't forget to set a disk size!!!! (20GB will do)
I saw a talk by Alan Vardy at the February 2019 YEGRB (Edmonton Ruby Meetup) on deploying Rails applications to Digital Ocean using Dokku. That talk inspired this presentation, and I am stealing a bunch of his stuff/steps. Here is a link to Alan's blog post on this subject:
https://www.alanvardy.com/posts/6
- Set up Dokku on a cloud VM
- Create a Ruby-on-Rails application and deploy to the cloud using Dokku
- Create a flask application and deploy to the cloud using Dokku
- Get an SSL cert
- Play around with containers
- Discuss some service implications
https://github.com/dokku/dokku
16k stars, first commit was June 2013.
- Dokku is a PaaS (platform-as-a-service) layer on top of IaaS (infrastucture-as-a-service, e.g., our cloud)
- Dokku is inspired by the commercial hosting service Heroku
- It uses Docker containers for managing services (e.g., application server, database server)
- It uses git for deployment.
- Magical container swapping ensures zero downtime during deployment (if new containers are broken, old containers keep serving application)
- Ideally, we will be following the best practices of a '12 Factor App` (in particular, configuration through environment variables): https://12factor.net/
Note that on the commercial Heroku service, there is less setup required (the platform is all you have access to, not SSH access to a UNIX account). Setting up Dokku on an OpenStack cloud VM takes time so one has to ask 'Is it worth it?'.
Lets SSH into our new app server VM (account ubuntu) and install Dokku:
wget https://raw.githubusercontent.com/dokku/dokku/v0.14.5/bootstrap.sh;
sudo DOKKU_TAG=v0.14.5 bash bootstrap.sh
In theory we could have done the following to get an older version of Dokku:
apt-get install dokku
I won't demo the installation of these prerequisites.
I have set up three domain names (A records) pointing to the same floating IP in Google Cloud Platform (manages c3.ca):
whatever.uofa.c3.cawhatever-rails.uofa.c3.cawhatever-flask.uofa.c3.ca
I am running a Mac laptop, and this is a list of things I needed to make this work:
- I probably need
homebrew - Ruby-on-Rails:
- Ruby version manager:
rvm - Once rvm installed, grab a recent Ruby:
rvm install ruby-2.5; rvm --default use 2.5 - Once ruby is installed, install Rails to create new projects. preferably in a gemset (e.g.,
gem install rails -v 5.2).
- Ruby version manager:
- Flask:
- Need python installed from somewhere (I use homebrew)
- I create a virtual environment:
virtualenv --no-download ~/virtualenv/whatever-flask - I activate the virtual environment:
source ~/virtualenv/whatever-flask/bin/activate - I use pip to install flask:
pip install flask
Some of these things are peculiar to my laptop, OS, and my work habits.
I visit http://whatever.uofa.c3.ca, which gives me a temporary setup webpage -- once the setup form is submitted, the page can't be accessed again.
(You can't visit it because of security rules, which will be loosened later.)
There is the opportunity to upload some public keys.
We can either run each application on a separate port, or as a subdomain of our server. We will choose the subdomain option.
I tell Dokku that the applications I run will be under uofa.c3.ca.
On the command line of the cloud VM (as user ubunutu), create two applications, one for rails, one for flask.
dokku apps:create whatever-rails
dokku apps:create whatever-flask
(Despite the naming, Dokku figures out what kind of app it is based on what sort of files we push to it).
Configure dokku to use postgres:
sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git
We'll provision a database and hook it up to the Rails app
dokku postgres:create whateverdb
dokku postgres:link whateverdb whatever-rails
Notice that the environment variable DATABASE_URL is now set for the Rails app.
We want to create a new rails application that uses PostgreSQL by default, and we want to delay deploying the bundle.
rails new whatever-rails --skip-bundle --database=postgresql
Rails new also initializes a new git repository, so lets put what we have in git:
git add .
git commit -m "First commit on new rails app"
We will run PostgreSQL in production, but SQLite3 for local development.
I had issues with the SQLite3 version, so need to be careful with the version
gem 'dokku-cli'
# Database-per-environment
group :development, :test do
gem 'sqlite3', '~> 1.3.6'
end
group :production do
gem 'pg', '>= 0.18', '< 2.0'
end
We need to also modify the development and production environments in config/database.yml. The development environment will use Sqlite3, and the production environment will be configured with the DATABASE_URL environment variable.
development:
<<: *default
adapter: sqlite3
database: db/development.sqlite3
...
production:
<<: *default
url: <%= ENV['DATABASE_URL'] %>
I had some trouble working with Bundler 2.x (Dokku got angry when I tried to deploy it) so I had to install an older version of bundler and made sure that it was the one that is being used when creating the bundle:
gem install bundler --version 1.15.2
bundle _1.15.2_ install --path vendor/bundle
Important: I don't want to accidentally commit the local development bundle, or the database, so I'll add to .gitignore:
# Don't commit bundle
vendor/bundle
# Don't commit database
db/*.sqlite3
Let's ensure the web app runs:
bundle exec rails s
Now we can visit http://localhost:3000 in a browser to check out the web page.
Add and commit to Git:
git add .
git commit -m 'Web app works in development'
The Rails welcome page doesn't work in a production environment so we will create a root webpage that controls a database table:
bundle exec rails generate scaffold Thing name:string quantity:integer
Migrate our dev database:
bundle exec rails db:migrate
To the config/routes.rb we add a line in the Rails.application.routes.draw block to set this page as what gets loaded when people visit the root URL:
root 'things#index'
Confirm that this root page works by starting the webserver, then commit to git.
The file CHECKS provides a test that our application is working when deployed to Dokku:
# CHECKS
WAIT=10
ATTEMPTS=6
/check.txt it_works
In order for the check to work, we add another route to config/routes.rb:
get '/check.txt', to: proc {[200, {}, ['it_works']]}
We need to create a file called app.json:
{
"name": "whatever-rails",
"description": "Whatever running on Dokku!",
"keywords": [
"dokku",
"rails"
],
"scripts": {
"dokku": {
"postdeploy": "bundle exec rails db:migrate"
}
}
}
Lets commit our app to git:
git add .
git commit -m 'Ready to deploy'
Add a git remote pointing to our app server:
git remote add dokku dokku@whatever.uofa.c3.ca:whatever-rails
Our SSH key controls the authorization.
Finally, we deploy by pushing our master branch:
git push dokku master
While that's deploying, lets go to the OpenStack dashboard and change our cloud security group to allow public access.
After deployment, we can visit our app at http://whatever-rails.uofa.c3.ca.
Create a directory somewhere on your work laptop for a flask project:
mkdir whatever-flask
cd whatever-flask
Create the python file whatever-flask.py
import os
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
return 'Like, whatever world!'
if __name__ == '__main__':
# Bind to PORT if defined, otherwise default to 5000.
port = int(os.environ.get('PORT', 5000))
app.run(host='127.0.0.1', port=port)We can now run our app locally: python whatever-flask.py (view it at http://localhost:5000).
We set up a requirements.txt file so that Dokku knows what python packages we need:
Flask==0.12.1
gunicorn==19.7.1
We create a Procfile so that Dokku knows how to start our application. We are using a Web-server gateway interface (WSGI) called 'Green Unicorn':
web: gunicorn whatever-flask:app --workers=4
Python dumps out lots of stuff we don't want in our repository, so update .gitignore:
__pycache__/
*.pyc
venv/
Create a git repository and commit to it:
git init
git add .
git commit -m "Deploy Flask with Dokku"
Set up our Dokku remote repository again:
git remote add dokku dokku@whatever.uofa.c3.ca:whatever-flask
Deploy by pushing:
git push dokku master
After deploy we visit http://whatever-flask.uofa.c3.ca.
IMPORTANT: Open VM security groups to public traffic first or cert request will fail
Dokku has a plugin to request and use a free SSL cert from Lets Encrypt. Via sudo on our app server, we install it:
sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
Lets setup our an email to use when requesting certificates for our rails app:
dokku config:set --no-restart whatever-rails DOKKU_LETSENCRYPT_EMAIL=REDACTED@ualberta.ca
Now lets turn it on:
dokku letsencrypt whatever-rails
Visit: https://whatever-rails.uofa.c3.ca
Lets Encrypt certs have a short life (30 days?) so we set up a cron job on the app server to auto-renew our certificate before it expires:
dokku letsencrypt:cron-job --add
Show all applications running:
dokku apps
More verbose information:
dokku apps:report
Check out some logs:
dokku logs whatever-rails
Restart an application:
dokku ps:restart whatever-rails
Checking out the scaling of an application:
dokku ps:scale whatever-rails
Rescale the rails application to have four containers working:
dokku ps:scale whatever-rails web=4
List all containers:
sudo docker ps
Run bash in a container:
dokku enter whatever-rails web.1
Get a rails console:
dokku run whatever-rails console
Stop an application:
dokku ps:stop whatever-rails
Recover all applications after reboot:
dokku ps:restore
Database export and import:
dokku postgres:export [db_name] > [db_name].dump
dokku postgres:import [db_name] < [db_name].dump
The rails app has a gem installed called dokku-cli which gives us access to some controls from the dev directory on the laptop:
bundle exec dokku ps
bundle exec dokku config
etc.
- Setup of dokku on a VM
- Creation of apps
- Nice deployment
- Nice connection of services
- Super-easy handling of SSL
- Lots of database creation, configuration stuff -- this was just easy
- Lots of service configuration
- Would be super-nice if some of the Dokku stuff had a web interface in front of it. There is a project called 'wharf' on github that tries to do this -- tried it, and it almost worked (didn't have time to correct problems encountered).
- Application deployment and SSL management is fantastic;
- How to fix things when things go wrong? Probably needs some heavy container expertise.
So what do we want to do with this?
Is this something that we could offer as a service to our users? What would the service look like?
User would need to supply:
- a pre-obtained hostname, or a subdomain of a domain we could provision for them;
- whether they need a database (and/or other datastore, like Redis);
- An email for LetsEncrypt.
We would give them:
- the git remote, e.g.
git remote add dokku dokku@super-science-thingy.c3.ca:super-science-thingy. - some assurances about database backup;
- logging?
- other admin tasks.
If we don't want to have a service offering, is this a path that we want to steer users down? Is it worth a workshop? Or have staff help with setup?
Or is this 'thanks, but no thanks' / 'not ready for prime time'? (or 'users can figure this out themselves'.)
REALLY DUMB STUFF:
- the local dev instance will now be totally broken, some additional work is needed (e.g., install local database or hook it up to local rails database)
- really, this is just an example that shows potential uses, not really a good recipe for anything
We'll modify the flask application to report the contents of the database connected to the rails application
dokku postgres:link whateverdb whatever-flask
Now flask app sees the DATABASE_URL environment variable
On local machine:
New requirements.txt contents:
Flask==0.12.1
gunicorn==19.7.1
psycopg2==2.7.7
That last line was added and interfaces with PostgreSQL.
New whatever-flask.py contents:
import os
import psycopg2
import urllib
from flask import Flask
url = urllib.parse.urlparse(os.environ.get('DATABASE_URL'))
db = "dbname=%s user=%s password=%s host=%s " % (url.path[1:], url.username, url.password, url.hostname)
conn = psycopg2.connect(db)
cur = conn.cursor()
app = Flask(__name__)
@app.route('/')
def hello():
cur.execute('SELECT * from things')
rows = cur.fetchall()
response = '<pre>\n'
for row in rows:
response += ' -- '.join(map(str, row)) + '\n'
response += '</pre>'
return response
if __name__ == '__main__':
# Bind to PORT if defined, otherwise default to 5000.
port = int(os.environ.get('PORT', 5000))
app.run(host='127.0.0.1', port=port)
Commit and push, should work!