Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Lockable script
#!/bin/bash
# SPDX-License-Identifier: MIT
## Copyright (C) 2009 Przemyslaw Pawelczyk <przemoc@gmail.com>
##
## This script is licensed under the terms of the MIT license.
## https://opensource.org/licenses/MIT
#
# Lockable script boilerplate
### HEADER ###
LOCKFILE="/var/lock/`basename $0`"
LOCKFD=99
# PRIVATE
_lock() { flock -$1 $LOCKFD; }
_no_more_locking() { _lock u; _lock xn && rm -f $LOCKFILE; }
_prepare_locking() { eval "exec $LOCKFD>\"$LOCKFILE\""; trap _no_more_locking EXIT; }
# ON START
_prepare_locking
# PUBLIC
exlock_now() { _lock xn; } # obtain an exclusive lock immediately or fail
exlock() { _lock x; } # obtain an exclusive lock
shlock() { _lock s; } # obtain a shared lock
unlock() { _lock u; } # drop a lock
### BEGIN OF SCRIPT ###
# Simplest example is avoiding running multiple instances of script.
exlock_now || exit 1
# Remember! Lock file is removed when one of the scripts exits and it is
# the only script holding the lock or lock is not acquired at all.
@blezek

This comment has been minimized.

Show comment Hide comment
@blezek

blezek Sep 24, 2012

Outstandingly useful script. Thanks!

blezek commented Sep 24, 2012

Outstandingly useful script. Thanks!

@bendilley

This comment has been minimized.

Show comment Hide comment
@bendilley

bendilley May 13, 2013

ditto @dblezek

ditto @dblezek

@luislobo

This comment has been minimized.

Show comment Hide comment
@luislobo

luislobo Jul 29, 2013

I don't know much about bash scripting, i have simple questions:

  1. my script should go at the end, after exlock_now || exit 1?
  2. do I have to call _no_more_locking explicitly at the end?

I don't know much about bash scripting, i have simple questions:

  1. my script should go at the end, after exlock_now || exit 1?
  2. do I have to call _no_more_locking explicitly at the end?
@przemoc

This comment has been minimized.

Show comment Hide comment
@przemoc

przemoc Dec 19, 2013

@luislobo:

  1. Yes.
  2. No, _no_more_locking is called automatically when script ends.

Sorry for late reply, but unfortunately gists lack any notification system...

Owner

przemoc commented Dec 19, 2013

@luislobo:

  1. Yes.
  2. No, _no_more_locking is called automatically when script ends.

Sorry for late reply, but unfortunately gists lack any notification system...

@ghost

This comment has been minimized.

Show comment Hide comment
@ghost

ghost Aug 15, 2015

Super helpful, thanks for this boilerplate. 🍹

ghost commented Aug 15, 2015

Super helpful, thanks for this boilerplate. 🍹

@PiotrCzapla

This comment has been minimized.

Show comment Hide comment
@PiotrCzapla

PiotrCzapla Dec 1, 2015

@przemoc really useful snippet! But have you thought about getting it on the MIT license?

The GPL v2 prevents me form using it, as I would basically have to publish the script somewhere and that would be quite useless given that the script will have probably 2 more lines :)

Let me know what you think.

@przemoc really useful snippet! But have you thought about getting it on the MIT license?

The GPL v2 prevents me form using it, as I would basically have to publish the script somewhere and that would be quite useless given that the script will have probably 2 more lines :)

Let me know what you think.

@przemoc

This comment has been minimized.

Show comment Hide comment
@przemoc

przemoc Dec 13, 2015

@PiotrCzapla
There won't be a problem. I'll improve this boilerplate a bit by incorporating some of my comments you can find in http://stackoverflow.com/a/1985512/241521 and relicense it to MIT License (which I'm already using in my more recent creations). For the time being please use simply relicensed version I've sent to your github e-mail.

Owner

przemoc commented Dec 13, 2015

@PiotrCzapla
There won't be a problem. I'll improve this boilerplate a bit by incorporating some of my comments you can find in http://stackoverflow.com/a/1985512/241521 and relicense it to MIT License (which I'm already using in my more recent creations). For the time being please use simply relicensed version I've sent to your github e-mail.

@TUNER88

This comment has been minimized.

Show comment Hide comment
@TUNER88

TUNER88 Dec 14, 2016

@przemoc any news about the improved version?

TUNER88 commented Dec 14, 2016

@przemoc any news about the improved version?

@reesd

This comment has been minimized.

Show comment Hide comment
@reesd

reesd Jan 10, 2017

Yes, would really like the MIT licensed version. "derivative work" can be a tricky with bash scripts.

As suggestion would be making it a sourceable library also that could be used across multiple scripts. Perhaps taking the lockname rather than using basename.

reesd commented Jan 10, 2017

Yes, would really like the MIT licensed version. "derivative work" can be a tricky with bash scripts.

As suggestion would be making it a sourceable library also that could be used across multiple scripts. Perhaps taking the lockname rather than using basename.

@przemoc

This comment has been minimized.

Show comment Hide comment
@przemoc

przemoc Jan 10, 2017

commit 6571db3000d4060e4168c3406f370a02a32b5045 (2017-01-10 23:54:05 +0100):

lockable_script_boilerplate.sh: Relicense to MIT.

I wanted to do it long time ago, along with some updates, but it's so
low on my todo list, that only comment on stackoverflow changed it:

    http://stackoverflow.com/a/1985512/241521

because I pasted the code there.

Sorry, no code updates yet.
Owner

przemoc commented Jan 10, 2017

commit 6571db3000d4060e4168c3406f370a02a32b5045 (2017-01-10 23:54:05 +0100):

lockable_script_boilerplate.sh: Relicense to MIT.

I wanted to do it long time ago, along with some updates, but it's so
low on my todo list, that only comment on stackoverflow changed it:

    http://stackoverflow.com/a/1985512/241521

because I pasted the code there.

Sorry, no code updates yet.
@przemoc

This comment has been minimized.

Show comment Hide comment
@przemoc

przemoc Feb 6, 2018

commit 74a5939bc7729bb0b46de47ed0ef6f551c29a8f6
Author: Przemyslaw Pawelczyk <przemoc@gmail.com>
Date:   2018-02-06 14:12:23 +0100

    Add SPDX License Identifier.

    The Software Package Data Exchange (SPDX) is a good initiative, it has
    matured over time and deserves accelerated adoption in open-source.

    https://spdx.org/learn
    https://spdx.org/using-spdx
    https://spdx.org/license-list
Owner

przemoc commented Feb 6, 2018

commit 74a5939bc7729bb0b46de47ed0ef6f551c29a8f6
Author: Przemyslaw Pawelczyk <przemoc@gmail.com>
Date:   2018-02-06 14:12:23 +0100

    Add SPDX License Identifier.

    The Software Package Data Exchange (SPDX) is a good initiative, it has
    matured over time and deserves accelerated adoption in open-source.

    https://spdx.org/learn
    https://spdx.org/using-spdx
    https://spdx.org/license-list
@ao2

This comment has been minimized.

Show comment Hide comment
@ao2

ao2 May 7, 2018

Hi,

I was evaluating this solution, the code works in the common case, but IMHO it has some formal issues.

AFAIU synchronization via flock is performed on the actual file on the filesystem (or in memory), not on the file name nor on the file descriptor, these latter are just convenience mechanisms to access the file end hence the lock.

Some example to verify that:

  • using different file names for LOCKFILE would still work if they were (hard) linked to the same file, e.g.:
#this is just an example to show the point, cleanup is ignored.
ln "/tmp/EXISTING_FILE" "/tmp/lock.$$"
LOCKFILE="/tmp/lock.$$"
  • using different file descriptors across invocations would still work (this is rather obvious considering that file descriptors are unique only in the same process) as long as they were bound to the same file, e.g.:
LOCKFD=$(( (RANDOM % 100) + 1))

Conversely, if the same file name refers to different files at two different points in time, locking would not work as desired.

For example:

exec 9>/tmp/filename

# removing the filename, but the file is still accessible via fd 9
rm /tmp/filename

# same name, new, different file (also different fd but this is unrelated)
exec 10>/tmp/filename

echo "Hello" >> /proc/self/fd/9
echo "World" >> /proc/self/fd/10

# The two files are different
cat /proc/self/fd/9
cat /proc/self/fd/10

the two file descriptors refer to different files in memory, even if they were opened with the same file name.

Considering that, I think the proposed solution leaves a window between _prepare_locking() and _lock() which can be problematic.

If for some reason $LOCKFILE gets deleted by some other entity when the first invocation is between _prepare_locking() and _lock(), a second invocation would fail to synchronize, because this second _prepare_locking() would create a new file (even when using the same name), so it would execute regardless of the first invocation.

To reduce the window to the minimum _lock and _prepare_lock could be merged into one function, there will be a little extra work if the script locks and unlocks multiple times but it will be more robust.

Finally, about _no_more_locking() consider adding a comment to explain that it is like that to support exlock or shlock; in case only exlock_now() is required the function would not be needed and just a trap "rm -f \"$LOCKFILE\";" would suffice since the lock will be released automatically when the process exits and the file is closed.

FWIW I ended up using the mkdir approach (http://wiki.bash-hackers.org/howto/mutex) in my scripts which looks simpler and more robust.

Sorry for the long post.

Ciao,
Antonio

ao2 commented May 7, 2018

Hi,

I was evaluating this solution, the code works in the common case, but IMHO it has some formal issues.

AFAIU synchronization via flock is performed on the actual file on the filesystem (or in memory), not on the file name nor on the file descriptor, these latter are just convenience mechanisms to access the file end hence the lock.

Some example to verify that:

  • using different file names for LOCKFILE would still work if they were (hard) linked to the same file, e.g.:
#this is just an example to show the point, cleanup is ignored.
ln "/tmp/EXISTING_FILE" "/tmp/lock.$$"
LOCKFILE="/tmp/lock.$$"
  • using different file descriptors across invocations would still work (this is rather obvious considering that file descriptors are unique only in the same process) as long as they were bound to the same file, e.g.:
LOCKFD=$(( (RANDOM % 100) + 1))

Conversely, if the same file name refers to different files at two different points in time, locking would not work as desired.

For example:

exec 9>/tmp/filename

# removing the filename, but the file is still accessible via fd 9
rm /tmp/filename

# same name, new, different file (also different fd but this is unrelated)
exec 10>/tmp/filename

echo "Hello" >> /proc/self/fd/9
echo "World" >> /proc/self/fd/10

# The two files are different
cat /proc/self/fd/9
cat /proc/self/fd/10

the two file descriptors refer to different files in memory, even if they were opened with the same file name.

Considering that, I think the proposed solution leaves a window between _prepare_locking() and _lock() which can be problematic.

If for some reason $LOCKFILE gets deleted by some other entity when the first invocation is between _prepare_locking() and _lock(), a second invocation would fail to synchronize, because this second _prepare_locking() would create a new file (even when using the same name), so it would execute regardless of the first invocation.

To reduce the window to the minimum _lock and _prepare_lock could be merged into one function, there will be a little extra work if the script locks and unlocks multiple times but it will be more robust.

Finally, about _no_more_locking() consider adding a comment to explain that it is like that to support exlock or shlock; in case only exlock_now() is required the function would not be needed and just a trap "rm -f \"$LOCKFILE\";" would suffice since the lock will be released automatically when the process exits and the file is closed.

FWIW I ended up using the mkdir approach (http://wiki.bash-hackers.org/howto/mutex) in my scripts which looks simpler and more robust.

Sorry for the long post.

Ciao,
Antonio

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment