Skip to content

Instantly share code, notes, and snippets.

@przemoc
Last active October 16, 2024 09:15
Show Gist options
  • Save przemoc/571091 to your computer and use it in GitHub Desktop.
Save przemoc/571091 to your computer and use it in GitHub Desktop.
Lockable script
#!/bin/bash
# SPDX-License-Identifier: MIT
## Copyright (C) 2009 Przemyslaw Pawelczyk <przemoc@gmail.com>
##
## This script is licensed under the terms of the MIT license.
## https://opensource.org/licenses/MIT
#
# Lockable script boilerplate
### HEADER ###
LOCKFILE="/var/lock/`basename $0`"
LOCKFD=99
# PRIVATE
_lock() { flock -$1 $LOCKFD; }
_no_more_locking() { _lock u; _lock xn && rm -f $LOCKFILE; }
_prepare_locking() { eval "exec $LOCKFD>\"$LOCKFILE\""; trap _no_more_locking EXIT; }
# ON START
_prepare_locking
# PUBLIC
exlock_now() { _lock xn; } # obtain an exclusive lock immediately or fail
exlock() { _lock x; } # obtain an exclusive lock
shlock() { _lock s; } # obtain a shared lock
unlock() { _lock u; } # drop a lock
### BEGIN OF SCRIPT ###
# Simplest example is avoiding running multiple instances of script.
exlock_now || exit 1
# Remember! Lock file is removed when one of the scripts exits and it is
# the only script holding the lock or lock is not acquired at all.
@blezek
Copy link

blezek commented Sep 24, 2012

Outstandingly useful script. Thanks!

@bendilley
Copy link

ditto @dblezek

@luislobo
Copy link

I don't know much about bash scripting, i have simple questions:

  1. my script should go at the end, after exlock_now || exit 1?
  2. do I have to call _no_more_locking explicitly at the end?

@przemoc
Copy link
Author

przemoc commented Dec 19, 2013

@luislobo:

  1. Yes.
  2. No, _no_more_locking is called automatically when script ends.

Sorry for late reply, but unfortunately gists lack any notification system...

Copy link

ghost commented Aug 15, 2015

Super helpful, thanks for this boilerplate. 🍹

@PiotrCzapla
Copy link

@przemoc really useful snippet! But have you thought about getting it on the MIT license?

The GPL v2 prevents me form using it, as I would basically have to publish the script somewhere and that would be quite useless given that the script will have probably 2 more lines :)

Let me know what you think.

@przemoc
Copy link
Author

przemoc commented Dec 13, 2015

@PiotrCzapla
There won't be a problem. I'll improve this boilerplate a bit by incorporating some of my comments you can find in http://stackoverflow.com/a/1985512/241521 and relicense it to MIT License (which I'm already using in my more recent creations). For the time being please use simply relicensed version I've sent to your github e-mail.

@TUNER88
Copy link

TUNER88 commented Dec 14, 2016

@przemoc any news about the improved version?

@reesd
Copy link

reesd commented Jan 10, 2017

Yes, would really like the MIT licensed version. "derivative work" can be a tricky with bash scripts.

As suggestion would be making it a sourceable library also that could be used across multiple scripts. Perhaps taking the lockname rather than using basename.

@przemoc
Copy link
Author

przemoc commented Jan 10, 2017

commit 6571db3000d4060e4168c3406f370a02a32b5045 (2017-01-10 23:54:05 +0100):

lockable_script_boilerplate.sh: Relicense to MIT.

I wanted to do it long time ago, along with some updates, but it's so
low on my todo list, that only comment on stackoverflow changed it:

    http://stackoverflow.com/a/1985512/241521

because I pasted the code there.

Sorry, no code updates yet.

@przemoc
Copy link
Author

przemoc commented Feb 6, 2018

commit 74a5939bc7729bb0b46de47ed0ef6f551c29a8f6
Author: Przemyslaw Pawelczyk <przemoc@gmail.com>
Date:   2018-02-06 14:12:23 +0100

    Add SPDX License Identifier.

    The Software Package Data Exchange (SPDX) is a good initiative, it has
    matured over time and deserves accelerated adoption in open-source.

    https://spdx.org/learn
    https://spdx.org/using-spdx
    https://spdx.org/license-list

@ao2
Copy link

ao2 commented May 7, 2018

Hi,

I was evaluating this solution, the code works in the common case, but IMHO it has some formal issues.

AFAIU synchronization via flock is performed on the actual file on the filesystem (or in memory), not on the file name nor on the file descriptor, these latter are just convenience mechanisms to access the file end hence the lock.

Some example to verify that:

  • using different file names for LOCKFILE would still work if they were (hard) linked to the same file, e.g.:
#this is just an example to show the point, cleanup is ignored.
ln "/tmp/EXISTING_FILE" "/tmp/lock.$$"
LOCKFILE="/tmp/lock.$$"
  • using different file descriptors across invocations would still work (this is rather obvious considering that file descriptors are unique only in the same process) as long as they were bound to the same file, e.g.:
LOCKFD=$(( (RANDOM % 100) + 1))

Conversely, if the same file name refers to different files at two different points in time, locking would not work as desired.

For example:

exec 9>/tmp/filename

# removing the filename, but the file is still accessible via fd 9
rm /tmp/filename

# same name, new, different file (also different fd but this is unrelated)
exec 10>/tmp/filename

echo "Hello" >> /proc/self/fd/9
echo "World" >> /proc/self/fd/10

# The two files are different
cat /proc/self/fd/9
cat /proc/self/fd/10

the two file descriptors refer to different files in memory, even if they were opened with the same file name.

Considering that, I think the proposed solution leaves a window between _prepare_locking() and _lock() which can be problematic.

If for some reason $LOCKFILE gets deleted by some other entity when the first invocation is between _prepare_locking() and _lock(), a second invocation would fail to synchronize, because this second _prepare_locking() would create a new file (even when using the same name), so it would execute regardless of the first invocation.

To reduce the window to the minimum _lock and _prepare_lock could be merged into one function, there will be a little extra work if the script locks and unlocks multiple times but it will be more robust.

Finally, about _no_more_locking() consider adding a comment to explain that it is like that to support exlock or shlock; in case only exlock_now() is required the function would not be needed and just a trap "rm -f \"$LOCKFILE\";" would suffice since the lock will be released automatically when the process exits and the file is closed.

FWIW I ended up using the mkdir approach (http://wiki.bash-hackers.org/howto/mutex) in my scripts which looks simpler and more robust.

Sorry for the long post.

Ciao,
Antonio

@justinc1
Copy link

An ugly test - if I kill -9 pid-of-my-script, then it doesn't have a chance to remove the lock.

@aquarion
Copy link

An ugly test - if I kill -9 pid-of-my-script, then it doesn't have a chance to remove the lock.

This will always be true. kill -9 is "non-catchable, non-ignorable kill", the OS won't let you do anything when you get one, it's the equivalent of pulling taking an axe to the power supply. It's why you should escalate to kill -9 when trying to fix something, not start there.

@clockworksoul
Copy link

This is beautiful. Thank you.

@wiedehopf
Copy link

wiedehopf commented Jun 16, 2024

This is very nice.
If you don't mind the lockfile staying around, this is very simple and easy in bash and probably fine in sh (not tested):

exec 9>>mylockfile

if ! flock --exclusive --nonblock 9; then
    echo "$(date -u +"%FT%T.%3NZ") $$: scriptname will wait for lock"
    flock --exclusive 9
    echo "$(date -u +"%FT%T.%3NZ") $$: scriptname finished waiting for lock"
fi

... this code is protected against concurrency ...

It's easy enough to adjust that to exit if the script is already running:

exec 9>>mylockfile

if ! flock --exclusive --nonblock 9; then
    echo "scriptname already running, exiting"
    exit 1
fi

... this code is protected against concurrency ...

The lock is automatically released once the file descriptor is closed (which is inherited by subshells or things you exec from the shellscript, so all that needs to close).
Thus the rest of the script is protected against concurrent execution.
There is no issues with power loss and the lockfile staying around either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment