Python is an incredibly versatile language. In fact, it is considered to be a staple of modern development, being used for the simplest of scripts to incredibly complex machine learning and neural network training algorithms. But perhaps the less-known usage of python is its utilization as a web server. Overshadowed by more popular frameworks such as node/express or rails, python is often overlooked as a web server choice for most developers. However, having a backend written in python is incredibly useful for several reasons, among which are
- It’s incredibly easy to step up from learning python as a regular scripting language to using it to make a backend
- It’s best to use if you plan on serving parts of your application that are already written in python (e.g. getting a form submission, evaluating the input via a tensorflow model, and returning the output to a use)
- It has an incredibly diverse ecosystem of packages and tools to help you with development, not to mention a great community of developers (since the language has been around so long)
The purpose of this article is to motivate how python in part to create a full stack web application. In this tutorial, I will be using Flask, the python “microframework” to developing web applications.
I would be remiss not to mention that there are some more popular tools out there such as Django, but Flask is incredibly useful for the budding developer since it is bare bones and requires developers to create every component of the app on their own (rather than calling some command line tool that generates 20 files automatically…lookin’ at you ruby on rails). Of course, I won’t be going through how to start a web app completely from scratch, rather I’ll give an intro to Flask and then move onto how you can use a project called flask-base to work off of in the future.
Flask is a microframework (read as: It doesn’t come with much) for web development in python. Before we do a deep(ish) dive, let's cover some basic concepts of backend development.
Let's imagine you're visiting apple.com and want to go to the Mac section (at https://www.apple.com/mac/). how do apple's servers know to serve you the specific page that shows the details about macs etc. It is most likely because they have a web app running on a server that knows when someone looks up mydomain.com and goes to the /mac/ section of my website, I am going to handle that request and send some pages back. The logic behind figuring out what to do when someone goes to /mac/ is done by a route!
So when I visit apple.com (implied apple.com/), the / route handles what is shown. If I go to apple.com/purchase, there is a /purchase route. If I go to apple.com/purchase/1 where 1 is some item identifier, there most likely is a generic route handler /purchase/<int:item-id> that handles that request. Routes can handle both GET and POST requests as well.
So how do we make a basic flask app that has routes? Well, let's take a look at the docs. Create a python file called hello.py that contains the following.
https://gist.github.com/d62bade33e8022b16f24942e772ea774
Let's break down what's happening here.
Line 1: We import our flask dependency
Line 2: We create an instance of a Flask app. The argument passed into the Flask instantiator (__name__) evaluate to a string that "names" the flask app. When run from the command line, __name__ == "__main__". You can set the first argument to whatever you want.
Line 3: We set up a route on our app that executes the function immediately below it when that route is visited (in this case http://blah.com/). Note that the function must return a string or a rendered template.
On the command line, let's set up something called a virtual environment (it will help us isolate our development environment and package installations from the rest of our system). If you haven't done so already, install pip via easy_install pip (you may need to run sudo in front of this if you are on a mac. Next, in the directory of your app run pip install virtualenv. Then create your virtual environment by running virtualenv venv (this creates a virtual environment in a folder called venv of the current directory). Lastly, we need to activate this virtual environment to install packages into it specifically. Run source venv/bin/activate. You can deactivate this virtual environment by running deactivate from your commandline. Pretty simple.
Now that our virtual environment is installed and activated, let's install flask. It's really simple, just run pip install Flask. We can then run the example from earlier by writing in our command line
https://gist.github.com/a03a59a93e8928e565491c9b7a5a822a
You should see something like * Running on http://localhost:5000/ in your terminal. And if you visit that link in your browser, you'll see a page with just Hello World.
Now let's figure out some project to create in order to demonstrate the full capaibilities of Flask. One recent project I came up with is a club rating app. Currently, I attend the University of Pennsylvania. One of the most common problems that freshmen on campus face is choosing which clubs to join on campus. This process is further complicated by the fact that some clubs are incredibly competitive to get into, have multiple interview rounds, and require a large time commitment. Often none of these aspects of clubs are discussed during club information sessions. So, in order to combat this issue, we can create an app where
- An administrator can set survey questions for users to answer about clubs.
- Users can view average ratings for each survey question for each club
- Users can view indiviudal responses for clubs. If a user chooses to submit another review, their previous response is overwritten.
- Users can suggest clubs for administrators to edit/approve to show in public (administrators should be notified via email when this happens)
-
A user or admin needs to be able to edit their own account information.
-
An administrator should have the ability to add/remove users from the system, add/remove survey questions, add/remove club categories, and adding/removing clubs.
In order to develop this app, we'll need to have some more components in addition to flask, such as a backing database, a login management system, some way to organize routes, and handle emailing. We could code this from scratch...however, there is already an amazing boilerplate that can give you a great starting place.
Flask-base is a project that my friends and I developed as part of a student run nonprofit called Hack4Impact. We work with nonprofits over a semester to develop technical projects that help them accomplish their mission, and often found that we would be repeating the same code across all of our applications. So we decided to create a single code base containing the most common parts of any app we made i.e. a user authentication scheme, account management, blueprints (to handle routes), a backing database, and emailing (with a redis queue). It recently became fairly popular, garnering 1200+ github stars over the course of a few months. This codebase is perfect for what we are trying to set up.
First let's clone flask-base. Follow the instructions on the README.md page. In a nutshell run the following (new line = new command)
https://gist.github.com/026553569d79f07a89389dfb2588c4e5
Okay...so what have we done here. First we clone our repository from github (i.e. download it) and then go into its directory. We then create a new virtual environment which is then activated. Additionally we read the package dependencies in the requirements.txt file and install all of them via pip. Lastly, we instantiate our database (recreate it) and also insert an administrator rule (via setup_dev).
Additionally, let's create a running db migration (this will keep track of changes in our database models without needing to recreate our database (i.e. remove all the information and then rebuild the db from scratch, migrations allows us to preserve the information). We can do this via
https://gist.github.com/95cf44b01dea9ed37657d8ba61e44780
To run the app, run honcho start -f Local (you'll need to install Honcho if you haven't already). If you have any issues, chances are they have been addressed in the README of flask-base already. Now you can visit localhost:5000 and pull up a running flask base application!
To log into the app as an administrator, go to the login link and type in for the username flask-base-admin@example.com with a password password. You can then invite new users into the application from the administrator screen. Note that before you do so, you'll need to create a config.env file that contains the following two variables:
https://gist.github.com/503bc48da8239cc9bb86cf1319248055
Upon creation of a user account, the account remains unconfirmed until the new invited user clicks a link sent to their email. Additionally, a user can register for the app and will go through a similar authentication flow with regards to confirmation.
Look through the flask-base documentation to get a better sense of some of the capabilities of flask-base out of the box. For now, we're going to move on to how we can use it to make our app.
All our database logic is wrapped by the SQLAlchemy ORM (so we don't have to make very verbose database statements anytime we want to run queries/additions/deletions of records) and all the database models (think that they are like classes) are contained within the app/models folder. Let's think of some models that are needed for the application itself.
So we need to have a Club model that contains the name of the club (String Type), a club description) (Text type), some boolean is_confirmed keeping track of whether a club that is suggested has been approved by an administrator to be shown. Additionally, we want some way to refer to the categories of a club, and another way to refer to the question answers that belong to a club.
Let's think about how Clubs and Club Categories should relate to each other. We can think of it as follows. A club has many categories (e.g. a club can be a Social Impact and Tech club) and a club category can belong to many clubs (e.g. there can be many Tech clubs on campus). The only attribute this ClubCategory has a category_name (which is a String).
We can create this relationship (a many to many relationship), via an association table.
Now how do we encode that logic into flask base? First, create a file called club.py in app/models. First let's create the Club and ClubCategory models.
https://gist.github.com/62267d77c8d912c9f1f0a323b71a4778
So now we have two models, but they aren't connected to each other. Each of them have individual attributes, but neither can be explicitly connected to each other. We make the connection via an association as I mentioned earlier. After the db import, add the following lines.
https://gist.github.com/4873bddc937859a54c4c5fad5d967936
What this does is create a new association table (an intermediary between the Club and ClubCategory model). There are two columns in this table club_id and club_category_id which refer to the respective ids of their respective models (note that the id attribute is a Primary Key within each model i.e. the thing that is unique for each record. But within the association table, we refer to these Primary Keys as Foreign Keys (because they are refering to other tables). Additionally, we need to add a line to the Club model at the bottom.
https://gist.github.com/c646a6348709caef658a00c9dac0807c
And this actually creates the bidirectional relationship between the Club and ClubCategory models. It says to set up a relationship between Club and ClubCategory using the club_category_assoc association table. The backref tells the ClubCategory model how to refer to the Club models. So, with a given club club, you can run club.categories to get an array of category object backs. With a given ClubCategory called category, you can get all the clubs in that category by doing category.clubs.
You can see this in action by doing the following:
In app/models/__init__.py add the line
https://gist.github.com/521d6e687f1348995f0e48e09885237f
And then run python manage.py shell. Run the following commands to interact with your database models (note that >>> indicates an input you put in).
https://gist.github.com/3c9e67189a01b7a224ac7e8bf7dda7ac
Great! We now have a working Club and ClubCategory model. Now let's move onto the Question and Answer models. For a question, we need to keep track of the content of the question (which will be a String Type containing the text of the question itself). We will also include a max_rating attribute that will contain the maximum rating an individual can give for the question (e.g. if the question content is "Rate the community of the club 10 is the best", we could set max_rating to be 10). Additionally, we'll keep track of a boolean free_response to determine whether we will allow people to include an optional extra response that is long form). Lastly, we will need to have a relation to the Answer model (this will be a one to many relation because a question can have multiple answers but an answer can only have one question).
The Answer model will have an answer attribute corresponding the the free response text of an answer (if the question allowed for free response), a rating (1 to whatever is the max rating for the question), a user_id relating to the user who wrote the question (once again a user can have many answers, but an answer can only have one user), a question_id referring to the question that the answer belongs to and the club_id referring to the club the answer belongs to.
Let's create a file question.py
https://gist.github.com/ca8a43a9197827b38f7119d17678e2bb
Most of the stuff in here is fairly straightforward except for the last line. The last line connects the Question and Answer models. It says to set up a relationship with the Answer model which can refer to the Question model via the keyword question (i.e. given an answer a, you can get the question via a.question and given a question q, you can get the answer associated with it via q.answers). Let's now set up the Answer model. Create a new file called answer.py in the models folder and paste in the following.
https://gist.github.com/e20f81d6153d84c27e7ce5e29770d8b8
So this file is much longer, but recall that there are many things an answer is related to. Let's start at the beginning, note that question_id refers to the Question model via the foreign key questions.id (the id column of the questions table (which contains records of instances of the Question model).
Note that we also have a user_id column that refers to a user. Let's go into user.py within the app/models folder and add the line
https://gist.github.com/a39beb508d677c8f06796c8728adbc32
to the line after the role_id declaration. This statement uses very similar syntax to that of the Question model.
Also note that there is a club_id attribute that refers to the club the answer is associated with. Edit the club.py file to include the line
https://gist.github.com/d4ab3eb9ab2d578e409b34c1dd7a820e
as the last attribute of the Club model.
Finally, add these two lines to __init__.py in app/models
https://gist.github.com/f02bdef56daf70aff838f4468ef7f99e
And now we should be able to play around with our databases as follows.
Lastly, let's address the newAnswer method. This method is used to insert new answers into the database while making sure that if a user has already answered that question, we delete it and insert the new response.
Once again, we can run python manage.py shell
https://gist.github.com/0f624ac23e8d9a554533f0f1bbe8da1d
There, we are now done with the models :)
Now the database stuff is out of way, let's create the way for users to interact with the application itself. First let's set up some blueprints.
Blueprints are a great way to organize you flask application. It allows you to mount all routes that are associated with each other in a single file. e.g. for all account things (like account management, user password reset/forgot password, etc), would be in the account blueprint.
Each blueprint has a folder associated with it under app (e.g. there is an account/ folder) and a folder under templates containing the actual html templates that will be rendered to the user.
Let's add some blueprints. Before the return app line of app/__init__.py add the following
https://gist.github.com/1167b4221196528e45f2f66976721216
These calls create blueprints mounted at the url prefixes /club, /question, and /category respectively. Let's create the folders club, question, and category for each of the blueprints. Within each of the folders create the files __init__.py, forms.py, and views.py.
I'll walk through how to set up the views/templates for club blueprint. The other views are fairly easy to understand from the code.
So within the club view, we want to have a few different things to show
-
If you are an administrator, you should be able to create a club (give it a name, description, and categories).
-
If you are an administrator, you should be able to view all the clubs (including ones that aren't confirmed)
-
If you are an administator or user, you should be able to view an individual club's information.
-
If you are an administator, you should be able to edit a club's information and delete a club.
Let's first create a couple of forms within forms.py that we will then pass to our views (specifically the view that handles create a new club and the one that edits club information).
In forms.py for club add the following lines:
https://gist.github.com/e187dc7138a6131a72d3d06e218e0338
Flask-base uses wtforms to create forms (wtforms allows us to create forms as a in an object oriented manner i.e. each form is a class).
So we create two forms, one called NewClubForm that extends the base wtforms Form class, and has 3 fields: name (text input), desc (text input containing the description of the club), and categories (a multiple select dropdown). With the categories field, we query the ClubCategory model (with a lambda function which is basically an anonymous function) for the category names and populate the category select field options with the results from that query.
Lastly, we have a submit field, so the submit button can be rendered.
Next, we have an EditClubForm which extends the NewClubForm field set by adding a new field called is_confirmed. Recall that is_confirmed in our Club model determines whether the given club instance can be shown or not shown to the public (we will be adding the function for a club to be suggested by users, and by default, suggested clubs are hidden until admin approval). We also overwrite the submit field to contain different text i.e. "Edit Club".
In views.py under club/, we create a few routes.
-
/new-club(GET, POST) LOGIN PROTECTED: The renders and accepts data from form for creating a new club. -
/clubs(GET) ADMIN PROTECTED: Renders all the clubs -
/<int:club_id>/(:info)(GET) LOGIN PROTECTED: Will render out info for a given club instance withid = club_id(can access route at e.g. /club/1 or /club/1/info. -
/<int:club_id>/change-club-details(GET, POST) ADMIN PROTECTED: Render and accept data from form for editing club information. -
/<int:club_id>/delete(GET) ADMIN PROTECTED: Render page to delete club -
/<int:club_id>/_delete(GET) ADMIN PROTECTED: Delete club with club id.
For the first route (/new-club), we want to also allow regular users to create a new club (hence why we only login protect it). Let's see how we can make a route for this.
https://gist.github.com/5a04df24ba97fe4cd1a100051cda2403
Breaking down the code. In line 1, we declare where the route will be accessible (i.e. it will be on the club blueprint at the sub-route /new-club, so the full URL it can be accessed at is basedomain.com/club/new-club.
We then put a route decorator @login_required on the route (this decorator will throw a 403 error if the user isn't logged in but will allow the user to view the route if they are logged in).
Next, we define a method to handle requests to the route (note that this name must be unique). This method can be referred to by club.new_club in Jinja templating.
We then instantiate our NewClubForm we created earlier. In the following line, we check to see if the form submission was valid (note that this route will also accept POST requests to it) via the form.validate_on_submit() method. If it is, then we create a new Club instance with name, description, and categories corresponding to the form fields. Note for is_confirmed we set it equal to whether the current user is an administrator or not (because if a regular user submits to this form, we want the new club to not appear to everyone, hence we set is_confirmed to False). We then add the new club instance to the database session and commit the session.
Lastly, if the user submitting the form is not an admin, we generate a link to send to the administrator of the form via email. This link should go directly to the admin change_club_details route which will allow the admin to toggle is_confirmed. We then look through the database for all users with an administrator role and add an emailing task to our redis queue. Within the get_queue() method, we enqueue the send_email job specifically, setting the recipient to the admin email, the subject equal to
https://gist.github.com/16963928c3141483529e4c3fb10f5c21
add the club instance (to be used as a templating variable), and the link (also to be used as a templating variable).
We also pass the template in which we create in app/templates/club/email/suggested_club.html and .txt. The content is as follows for the html file:
https://gist.github.com/c3e82bb3aa4d5faacecef15d184df184
and for the .txt file
https://gist.github.com/cf73b456d862eadefc543b6504e55cb1
Next we will take care of the /clubs route that renders all the clubs in a table. For the route handler, we can just pass in all the clubs into a template.
https://gist.github.com/7c6b2b387bdc364e8db860e2b10deef6
And the club template we render is located at app/templates/club/clubs.html with the following content.
https://gist.github.com/c1a29431f3dc63207cee2c9ca19f3f04
Most of this is fairly straightforward if you know Jinja (or any templating language). Basically, the page will go through all the clubs (with {% for c in clubs %} ... {% endfor %}) and for each club, it will render the club name {{ c.name }}, the club categories
https://gist.github.com/3f296f5ecc7660b2f1209fb82421adab
Note that for each of the clubs rendered, we also include a line:
https://gist.github.com/0f24f10ecb0d64a13055852ef716771d
This links to the individual club info page for the given club instance that is rendered. Let's move on to making that route.
Note that for this view we only need to pass in the club instance information to the manage_club view. We can do this easily via:
https://gist.github.com/732b9b04e90599ad696f0fbf0489ba9a
We can also set up a few other routes (because our manage_club.html page actually does display multiple routes).
Let's set up the /change-club-details route (which just renders and accepts info from the EditClubForm form.
https://gist.github.com/ce5c537c2e6918484c22d4d8def4b539
Note that when saving the club.is_confirmed field, we need to convert the string True and False values (as stated in the forms.py specification for EditClubForm) to their boolean counterparts. We do this via a custom defined function bool which is defined as follows:
https://gist.github.com/a2aa92693ed0cdc5e201354c3d503b52
(the python default bool will return True if any string is defined, including 'False', hence we need to define our own function.
We also define the delete (renders the delete page) and _delete functions (actually deletes the club instance)
https://gist.github.com/cf2310efe2cdfe191d4777eec125ef70
Note that for the _delete route, we have a redirect towards the clubs route that lists all the club instances.
Now we move to the manage_club.html template at app/templates/club/manage_club.html. The content of that is as follows:
https://gist.github.com/3713785a46b90cacfa03751f5ecf8d35
Let's break down file. On the first line we are just extending our base layout and then we import form macros (macros are basically methods for jinja).
We have a endpoints variable that will contain links to the different parts of the management page. In the navigation macro, we render all the individual elements of the list in the endpoints.
We also create a club_info macro that will contain the information related to the club (and all the answer associated with the club by doing the following)
https://gist.github.com/7b2bbe96e8e677b07f67ae1cbc2e96bf
The most interesting part here is how to calculate the average rating and pass that into the route. I create a list called all_c and for each of the clubs, I create a club_obj containing basic information for the club and for each of the answers for a club I add a new property of the club_obj correponding to the question (if one doesn't exist already) content and I append each of the ratings to a list. I then iterate through each of the properties of the club_obj. If the property has a value that is of type list, then I replace that list with the average of the ratings in that list. I then append club_obj to all_c and pass that into the template.
For the submit-review route, I need to create a form dynamically based on the questions that I have in my Question model. The code is as follows:
https://gist.github.com/ffe4e2133e7e325c0840440f951bd20c
We first create a dummy form class (inheriting from the base Form class). Then for each of the questions, we ceate new form fields setattr(F, ...) on the dummy form F. The setattr method takes as its second argument the name of the form field (which we set to the id of the question with _q appended to it corresponding to the rating and _resp corresponding to a free response if indicated). For the rating form field, we create a SelectField with choices from 1 to the max_rating.
To handle a form submission, we use the same if statement form.validate_on_submit() but instead of looking for specific named fields of the form, we instead iterate through all the fields of the form and create a new answer (using the newAnswer method which will delete any previous response before adding a new one for the user if they responded for this club).
Now that most of the app is done, we can launch this app on heroku.
First, if you haven't set up a git repo initially, run git init and log into your heroku account (go to heroku.com, sign up for an account, and install the CLI if you haven't done so).
Then, git add all relevant files (i.e. anything but config.env and venv/) and run pip freeze > requirements.txt to make sure that all the dependencies you have installed are included.
Run heroku create to make a new heroku instance and run git push heroku master to add your files to the heroku repository.
After that is done running, you'll need to set some environment variables with the following command
https://gist.github.com/7396f9676846c2e3ed222d9ae06d3099
Once that is done, run the following
https://gist.github.com/7c4195483759dc52b585195d02aececb
(this will create the database on heroku)
and
https://gist.github.com/1c8af6f742fdb6dab3961aaa8b0c5e3d
This creates the administrator account.
You'll need to also create a redis togo instance to handle the task queue with
https://gist.github.com/f9e67b6e176a71474ea60b1f89f7336e
and lastly run
https://gist.github.com/e56943694e211cafeb989cd4b51eae4a
which will tell heroku to spin up a dyno (read as sub-server) that handles our redis queue.
You can then run heroku open to open your running heroku app in a separate window.
It's pretty easy to copy the current application structure and add more information/routes to the app, just view any of the previous routes we have implemented. If, for some reason, you want to include file uploading of some type, you'll need to integrate the app with Amazon S3 if you plan to run the app on heroku (since it has an ephemeral file system).
Overall, flask-base provides a great starting point for making your flask application. Of course the backend may be fairly verbose to code, but as a result, it gives you very granular control over your app.