Skip to content

Instantly share code, notes, and snippets.

@jamesls
Created December 4, 2018 05:08
Show Gist options
  • Save jamesls/9d52c7b3a19491f4d14c2d487515115f to your computer and use it in GitHub Desktop.
Save jamesls/9d52c7b3a19491f4d14c2d487515115f to your computer and use it in GitHub Desktop.
Slow rename()
SUMMARY
If you generate a lot of data and then immediately rename a file on top of an
existing file (even if the destination file is empty, you'll see several
seconds delay).
$ uname -a
Linux ip-10-0-0-127 4.15.0-1021-aws #21-Ubuntu SMP Tue Aug 28 10:23:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ grep ext4 < /proc/mounts
/dev/nvme0n1p1 / ext4 rw,relatime,discard,data=ordered 0 0
$ mkdir ~/slow-mv
$ cd ~/slow-mv
$ dd if=/dev/urandom of=5GB bs=1MB count=5120
5120+0 records in
5120+0 records out
5120000000 bytes (5.1 GB, 4.8 GiB) copied, 31.4953 s, 163 MB/s
NORMAL CASE
$ time mv 5GB 5GB.rename
real 0m0.001s
user 0m0.000s
sys 0m0.001s
$ time mv 5GB.rename 5GB
real 0m0.001s
user 0m0.001s
sys 0m0.000s
SLOW DOWN CASE
However, if we copy the file to a new file, and then rename it over the
original file, we see a slow down:
$ time cp 5GB 5GB.copy
real 0m27.597s
user 0m0.000s
sys 0m3.364s
This is the part I don't get:
$ time mv 5GB.copy 5GB
real 0m4.540s
user 0m0.000s
sys 0m0.321s
Even if the destination file is empty it still is slow:
$ cp 5GB 5GB.copy
$ rm 5GB
$ touch 5GB
$ time mv 5GB.copy 5GB
real 0m2.283s
user 0m0.000s
sys 0m0.041s
Just for completeness, the strace shows the slowdown
is in the rename syscall.
$ sudo strace -t mv 5GB.copy 5GB
....
05:05:10 stat("5GB", {st_mode=S_IFREG|0664, st_size=5120000000, ...}) = 0
05:05:10 lstat("5GB.copy", {st_mode=S_IFREG|0664, st_size=5120000000, ...}) = 0
05:05:10 lstat("5GB", {st_mode=S_IFREG|0664, st_size=5120000000, ...}) = 0
05:05:10 geteuid() = 0
*** 05:05:10 rename("5GB.copy", "5GB") = 0
*** 05:05:18 lseek(0, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
05:05:18 close(0) = 0
05:05:18 close(1) = 0
05:05:18 close(2) = 0
05:05:18 exit_group(0) = ?
05:05:18 +++ exited with 0 +++
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment