Created
August 20, 2018 14:04
-
-
Save tvass/4106056e3d645d5b226e2ef67100f67e to your computer and use it in GitHub Desktop.
Performance issue with Ansible when using 'stat' module
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pwd : /home/tvass/Documents/work/ansible-perf | |
uname : Linux XXXXX 4.17.9-200.fc28.x86_64 #1 SMP Mon Jul 23 21:41:29 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | |
Ansible version : ansible 2.6.2 (same issue w/ 2.5.8 and 2.4.6.0) | |
1) Generate 300 random files | |
--- | |
#!/bin/bash | |
# Generates ten data files, each 300 MB in size and filled with random data. | |
[[ ! -d "data/" ]] && mkdir "data/" | |
for i in {1..300}; do | |
dd if="/dev/urandom" of="data/file${i}" bs=1024 count=10240 | |
done | |
--- | |
2) Time with sha256sum | |
$ time sha256sum data/file* | |
[...] | |
real 0m9.400s | |
user 0m7.469s | |
sys 0m0.608s | |
3) Time with Python | |
--- | |
#!/usr/bin/env python | |
import hashlib | |
import sys | |
def sha256_checksum(filename, block_size=65536): | |
sha256 = hashlib.sha256() | |
with open(filename, 'rb') as f: | |
for block in iter(lambda: f.read(block_size), b''): | |
sha256.update(block) | |
return sha256.hexdigest() | |
def main(): | |
for f in sys.argv[1:]: | |
checksum = sha256_checksum(f) | |
print(checksum + ' ' + f) | |
if __name__ == '__main__': | |
main() | |
--- | |
$ time ./hash.py data/* | |
[...] | |
real 0m8.292s | |
user 0m6.800s | |
sys 0m0.630s | |
4) Time with Ansible | |
--- | |
- hosts: localhost | |
gather_facts: false | |
strategy: free | |
tasks: | |
- name: Check if package is present | |
stat: | |
path: "{{ item }}" | |
checksum_algorithm: sha256 | |
get_attributes: no | |
get_mime: no | |
get_md5: no | |
with_fileglob: | |
- "/home/tvass/Documents/work/ansible-perf/data/*" | |
--- | |
$ time ansible-playbook cs.yaml | |
[...] | |
real 0m41.888s | |
user 1m10.612s | |
sys 0m11.982s |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment