Skip to content

Instantly share code, notes, and snippets.

@Nilpo
Last active January 13, 2024 14:04
Show Gist options
  • Save Nilpo/8ed5e44be00d6cf21f22 to your computer and use it in GitHub Desktop.
Save Nilpo/8ed5e44be00d6cf21f22 to your computer and use it in GitHub Desktop.
Using Git to Manage a Live Web Site

Using Git to Manage a Live Web Site

Overview

As a freelancer, I build a lot of web sites. That's a lot of code changes to track. Thankfully, a Git-enabled workflow with proper branching makes short work of project tracking. I can easily see development features in branches as well as a snapshot of the sites' production code. A nice addition to that workflow is that ability to use Git to push updates to any of the various sites I work on while committing changes.

Contents

Prerequisites

  • Git, installed on both the development machine and the live server
  • a basic working knowledge of Git, beginners welcome!
  • passwordless SSH access to your server using pre-shared keys
  • a hobby (you'll have a lot of extra time on your hands with the quicker workflow)

back to top


Getting Started

Installing Git

Git is available for Windows, Linux, and Mac. Getting it installed and running is a pretty straightforward task. Check out the following link for updated instructions on getting Git up and running.

Getting Started Installing Git

You'll need to have Git installed on your development machines as well as on the server or servers where you wish to host your website. This process can even be adapted to work with multiple servers such as mirrors behind a load balancer.

back to top

Setting up Passwordless SSH Access

The process for updating a live web server relies on the use of post hooks within the Git environment. Since this is fully automated, there is no opportunity to enter login credentials while establishing the SSH connection to the remote server. To work around this, we are going to set up passwordless SSH access. To begin, you will need to SSH into your server.

ssh user@hostname

Next, you'll need to make sure you have a ~/.ssh in your user's home directory. If not, go ahead and create one now.

mkdir ~/.ssh

On Mac and Linux, you can harness the power of terminal to do both in one go.

if [ ! -d ~/.ssh ]; then mkdir ~/.ssh; fi

Next you'll need to generate a public SSH key if you don't already have one. List the files in your ~/.ssh directory to check.

ls -al ~/.ssh

The file you're looking for is usually named similarly to id_rsa.pub or id_ed25519. If you're not sure, you can generate a new one. The command below will create an SSH key using the provided email as a label.

ssh-keygen -t ed25519 -C "your_email@example.com"

You'll probably want to keep all of the default settings. This will should create a file named id_ed25519 in the ~/.ssh directory created earlier.

When prompted, be sure to provide a secure SSH passphrase.

If you had to create an SSH key, you'll need to configure the ssh-agent program to use it.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

If you know what you are doing, you can use an existing SSH key in your ~/.ssh directory by providing the private key file to ssh-agent.

If you're still not sure what's going on, you should two files in your ~/.ssh directory that correspond to the private and public key files. Typically, the public key will be a file by the same name with a .pub extension added. An example would be a private key file named id_ed25519 and a public key file named id_ed25519.pub.

Once you have generated an SSH key on your local machine, it's time to put the matching shared key file on the server.

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@hostname

This will add your public key to the authorized keys on the remote server. This process can be repeated from each development machine to add as many authorized keys as necessary to the server. You'll know you did it correctly when you close your connection and reconnect without being prompted for a password.

back to top


Configuring the Remote Server Repository

The machine you intend to use as a live production server needs to have a Git repository that can write to an appropriate web-accessible directory. The Git metadata (the .git directory) does not need to be in a web-accessible location. Instead, it can be anywhere that is user-writeable by your SSH user.

back to top

Setting up a Bare Repository

In order to push files to your web server, you'll need to have a copy of your repository on your web server. You'll want to start by creating a bare repository to house your web site. The repository should be set up somewhere outside of your web root. We'll instruct Git where to put the actual files later. Once you decide on a location for your repository, the following commands will create the bare repository.

git init --bare --shared mywebsite.git && cd $_

A bare repository contains all of the Git metadata without any HEAD. Essentially, this means that your repository has a .git directory, but does not have any working files checked out. The next step is to create a Git hook that will check out those files any time you instruct it to.

If you wish to run git commands from the detached work tree, you'll need to set the environmental variable GIT_DIR to the path of mywebsite.git before running any commands.

back to top

Add a Post-Receive Hook

Create a file named post-receive in the hooks directory of your repository with the following contents. You may optionally specify a branch name to checkout. If unspecified, the default master or main branch is used.

#! /bin/sh
GIT_WORK_TREE=/path/to/webroot/of/mywebsite git checkout -qf [--detach] [<branch>]

Once you create your hook, go ahead and mark it as executable.

chmod +x hooks/post-receive

GIT_WORK_TREE allows you to instruct Git where the working directory should be for a repository. This allows you to keep the repository outside of the web root with a detached work tree in a web accessible location. Make sure the path you specify exists, Git will not create it for you.

back to top


Configuring the Local Development Machine

The local development machine will house the web site repository. Relevant files will be copied to the live server whenever you choose to push those changes. This means you should keep a working copy of the repository on your development machine. You could also employ the use of any centralized repository including cloud-based ones such as GitHub or BitBucket. Your workflow is entirely up to you. Since all changes are pushed from the local repository, this process is not affected by how you choose to handle your project.

back to top

Setting up the Working Repository

On your development machine, you should have a working Git repository. If not, you can create one in an existing project directory with the following commands.

git init
git add -A
git commit -m "Initial Commit"

back to top

Add a Remote Repository Pointing to the Web Server

Once you have a working repository, you'll need to add a remote pointing to the one you set up on your server.

git remote add live ssh://server1.example.com/home/user/mywebsite.git

Make sure the hostname and path you provide point to the server and repository you set up previously. If your remote user name is different from that of your local machine, you'll need to modify the above command to include the remote user name.

git remote add live ssh://user@server1.example.com/home/user/mywebsite.git

Finally, it's time to push your current website to the live server for the first time.

git push live +master:refs/heads/master

This command instructs Git to push the current master branch to the live remote. (There's no need to send any other branches.) In the future, the server will only check out from the master branch so you won't need to specify that explicitly every time.

back to top


Build Something Beautiful

Everything is ready to go. It's time to let the creative juices flow! Your workflow doesn't need to change at all. Whenever you are ready, pushing changes to the live web server is as simple as running the following command.

git push live

Setting receive.denycurrentbranch to "ignore" on the server eliminates a warning issued by recent versions of Git when you push an update to a checked-out branch on the server.

If you ever need to force a reload on the server side without making any local changes, simply create an empty commit and push it!

git commit --allow-empty -m "trigger update"
git push live

back to top


Additional Tips

Here are a few more tips and tricks that you may find useful when employing this style of workflow.

back to top

Pushing Changes to Multiple Servers

You may find the need to push to multiple servers. Perhaps you have multiple testing servers or your live site is mirrored across multiple servers behind a load balancer. In any case, pushing to multiple servers is as easy as adding more urls to the [remote "live"] section in .git/config.

[remote "live"]
    url = ssh://server1.example.com/home/user/mywebsite.git
	url = ssh://server2.example.com/home/user/mywebsite.git

Now issuing the command git push live will update all of the urls you've added at one time. Simple!

back to top

Ignoring Local Changes to Tracked Files

From time to time you'll find there are files you want to track in your repository but don't wish to have changed every time you update your website. A good example would be configuration files in your web site that have settings specific to the server the site is on. Pushing updates to your site would ordinarily overwrite these files with whatever version of the file lives on your development machine. Preventing this is easy. SSH into the remote server and navigate into the Git repository. Enter the following command, listing each file you wish to ignore.

git update-index --assume-unchanged <file...>

This instruct Gits to ignore any changes to the specified files with any future checkouts. You can reverse this effect on one or more files any time you deem necessary.

git update-index --no-assume-unchanged <file...>

If you want to see a list of ignored files, that's easy too.

git ls-files -v | grep ^[a-z]

back to top


References

@vigilancetech-com
Copy link

I've tried to follow these instructions as closely as possible but when I get to the git push to populate the web server's git repo it says everything is okay but the actual index.html file doesn't copy across.

Any idea how to debug this?

@hansgv
Copy link

hansgv commented Sep 27, 2022

@Nilpo Is it possible to revert to the previous version in case there's an issue after git push live?

@Nilpo
Copy link
Author

Nilpo commented Sep 27, 2022

@Nilpo Is it possible to revert to the previous version in case there's an issue after git push live?

Sure. Just make any corrections locally and make a new commit or simply roll the pointer back to an earlier commit using any of git's history commands and then push live again. The live server will always pull the latest commit on the master branch. If you need more control, you can implement other branching methodologies, but the practice remains the same.

You can also point the live server to any commit you wish by modifying the line used in the tutorial.

 git push live +master:refs/heads/master

But this is an advanced concept and it can lead to unexpected results if you aren't careful. You're better off leaving your master branch (or whatever branch you choose) as a "production" branch and ensuring that is last commit is always a working state. Feature branch, test build, and merge working states into that branch. Rolling back a single commit should always return your application to a working state if anything goes wrong.

@rodneytamblyn
Copy link

You can also push a specific commit hash to remote like this (where deploy is the post-receive hook):

git push -f staging eb26cxxxxxxxx9da265577633xxxxxxx543:deploy

Works for me, but I am not an expert. Nilpo's best practice advice above is good.

@Nilpo
Copy link
Author

Nilpo commented Sep 28, 2022

You can also push a specific commit hash to remote like this (where deploy is the post-receive hook):

git push -f staging eb26cxxxxxxxx9da265577633xxxxxxx543:deploy

Works for me, but I am not an expert. Nilpo's best practice advice above is good.

You're absolutely right. I keep my recommendations as simple as possible since it's so hard to provide support here. But you are spot on. This can be expanded as far as your individual skill level will allow. My own implementation is a bit more complex as well, especially where production sites are concerned.

@Nilpo
Copy link
Author

Nilpo commented Sep 28, 2022

I've tried to follow these instructions as closely as possible but when I get to the git push to populate the web server's git repo it says everything is okay but the actual index.html file doesn't copy across.

Any idea how to debug this?

Git has a --verbose option for some commands that may help, but it's hard to say without getting an idea what your set up looks like. My first thougths would be to make sure your users have adequate permissions for the directories you are working with. You can also do a clone of your repo to your local machine or another server to test for any issues within the repo. If you've done any copy/pasting maybe you've gotten some non-printable characters and Git is choking on that. Otherwise, the only real issues I've run into were either permission-based or network problems.

@Nilpo
Copy link
Author

Nilpo commented Sep 28, 2022

Further to my question, my solution (I don't know if this is the best, but it seems to work) su -c "GIT_WORK_TREE=/PATH/WEBSITE git checkout -f" web ... where 'web' in this case is the user I want the files to be owned by.

To get this solution working you may need to change existing ownership and file permissions of the bare repository to allow the web user to read, as otherwise it will generate permissions errors, as you are "su" to the context of "web" user. so for example: chown root:web /path/to/barerepo.git chown root:web /path/to/website ... and chmod the file permissions.

Works for me, but there may be better ways - please advise if you know them.

Sorry I missed getting back to this. This is certainly a viable option. Although, it would be my preference to not run this as a root user an rely on su to take care of things. Privilege de-escalation is generally bad practice. We generally want to operate as exactly the correct user with no more permissions than is absolutely required. For me, that would mean allowing ssh for the web user temporarily and setting this up as that user. All files would then be owned by web without any hassles.

Of course you could set this up as root and do a chown -R web .git/.

Either way, after your initial pull to the server, the Post-Receive hook should always run as the web user anyway, especially if you are chrooted or otherwise sandboxed as with most typical server setups.

We would never want Git to make arbitrary changes to a server with root privileges. Especially since we're allowing it to execute a shell script essentially unattended.

@jenetiks
Copy link

jenetiks commented Dec 7, 2022

Could we update the keygen to utilize a stronger algorithm, such as ed25519? While RSA is sufficient for now, ed25519 is considered to be quantum resistant and better practice.

Either way, thank you for your contributions! Guides for newbies are always a good thing!

@Nilpo
Copy link
Author

Nilpo commented Dec 8, 2022

Could we update the keygen to utilize a stronger algorithm, such as ed25519?

Done! Thanks for the suggestion. The current recommendation is to use certficates instead of keys, but the process is much more involved so I've been putting off. The goal was to keep this beginner-friendly, and certificates woud definitely move this into an intermediate or advanced setup although it really is the most secure, especially if this system is used with multiple servers at once. Maybe I'll fork this and do both.

@Kwpa
Copy link

Kwpa commented Feb 6, 2023

Thanks for these instructions! I was wondering if the following is possible using your technique (I may have misunderstood some things):

  1. Set up any kind of static blog site (in my case I'd be using 11ty) with a 'posts' folder
  2. In a separate private Github repo create .md files
  3. Some .md files would be tagged in frontmatter as 'private', others 'public'
  4. Push changes to the repo
  5. Have the website connect to the Github account via SSH
  6. Update the 'posts' folder with only the .md files that are tagged 'public'
  7. Render those files into the static site as appropriate

Do your instructions cover this?

@Denis5161
Copy link

Hello,

this gist is amazing. I set it up on my server and it runs perfectly. I also have a .gitignore file in my repo, which I remove with rm <#PATH>/.gitignore.

Thank you!
Is there maybe an official command to checkout a working tree to another path? I have tried git checkout-index, but that doesn't work because it's a post-receive hook.

@minifiredragon
Copy link

minifiredragon commented Mar 12, 2023

Just getting into git myself atm, and as a side note, I use a self hosted gitlab instance and the instructions had me init the repository using main instead of master. So when I ran the

git push live +master:refs/heads/master

This of course led to nothing happing but an error message, and it took me some time to track down that the above line is what did it. Even though I re ran the line as

git push live +main:refs/heads/main

It never updated the HEAD file and nothing worked. I did some minor searches on how to fix this with a git command, but did not really find any way to fix it. So the short answer was I edited the HEAD file and changed master to main, and at that point all worked well.

If you could add a note to that particular section that master should be the name of the top branch (what ever the proper term is) it might help us newbs keep a little more hair. :)

@dwesolowski
Copy link

Kinda follow what needs to be done but I will be using a centralized repository (github)

Goals:

  1. Develop on one machine
  2. Push to Github
  3. Github push changes to my webserver IP?

Could you give a little guidance here?

@hayden-t
Copy link

hayden-t commented May 17, 2023

Could you give a little guidance here?

Hi, Time for an update on my method, I first wound up here too, when i had the idea to use git on live website. It works great, I still use it, and also connect to github for privately storing the code.

The key thing I do differently to how this guide recommends and how i started out, is to keep your git repo (.git folder) one level above the public_html folder. This way there is no chance of it accidentally becoming exposed and publicly accessible. (which has happened to me twice by accident)

The trick is to use .gitignore to ignore all and then add back the dirs you want, (most likely public_html, maybe more)

/*
!/public_html
!/.gitignore

then you can have github as a remote, and use the deploy key setting for the repo to allow you/others to push/pull from the live and dev website copies, and collaborate with others.

You can also set up github actions, to on push to live branch deploy the code to live hosting, by ssh in and run git fetch

name: Deploy via Git Pull
on:
  push:
    branches: main
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - run: mkdir "$HOME/.ssh"
      - run: echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
      - run: chmod 600 "$HOME/.ssh/key"
      - run: ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no ${{ secrets.USER }}@${{ secrets.HOST }} 'cd /home/homedir && git fetch && git reset --hard origin/main'

One of many good advantages to having your live site as a git repo, is that you get to track any changes or new files, in case something goes wrong or is done to the site.

@domvo
Copy link

domvo commented Jun 8, 2023

@u0421793
Regarding SSH key creation: you only have to do this on the client, not on the server. Basic instructions:

  1. Generate an ssh key on your local machine. Example: ssh-keygen -t ed25519 -C "your_email@example.com". Make sure to NOT enter a passphrase in this step. This adds an extra layer of security that we don't want in our automated scenario.
  2. Then, from your local machine run: ssh-copy-id -i ~/.ssh/id_ed25519.pub user@host. You will probably be asked to provide your ssh password for the user user. This is done only once.
  3. Afterwards verify that your passwordless connection is working by simply running: ssh user@host.

That's it.

Side note: Try to avoid the "whiny" tone and stay professional. This is a volunteer community.

@u0421793
Copy link

u0421793 commented Jun 8, 2023

@u0421793 Regarding SSH key creation: you only have to do this on the client, not on the server. Basic instructions:

  1. Generate an ssh key on your local machine. Example: ssh-keygen -t ed25519 -C "your_email@example.com". Make sure to NOT enter a passphrase in this step.

Ah, don't enter the password, I see – I was entering my password because it was asking

@Nilpo
Copy link
Author

Nilpo commented Jun 9, 2023

You can see from the many comments of successful implementations that this method does, in fact, work.

Yes, the instructions could be slightly clearer concerning what commands are on the client machine and which are on the server. To be honest, I keep them a little cryptic intentionally.

First, this is meant to be a general guideline, not a full instructional. If you know what's happening, these instructions are more than sufficient to make this work. If you can't make it work, it's probably because you shouldn't be trying. There are a lot of security concerns that must be taken into account when implementing this type of solution. Keeping the instructions slightly vague helps to ensure that you actually know what's going on.

When people ask for help, and they seem to have a concept of what they are doing, I always try to make sure that they get it working. But I won't handhold everyone who asks because it's probably not a good idea.

Finally, I've been asked by some to update this with more information, but I will not be doing that. This method is no longer considered best practice. The correct way to do this in 2023 and moving forward is with certificate-based authentication and GitHub Actions. I may put together instructions for this in the future, but I haven't to date because it's far more complicated.

I will continue to respond to requests here and I will also link the new method when I have time to put it together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment