sanoj-unnikrishnan

# Below script has to purpose 
#  1. Display xattr of entire FS tree in a human readable form 
#  2. Display all the directory where contri and size mismatch.
#      (If there are any directory with contri and size mismatch that are not dirty 
#       then that highlights a propogation issue)
#  The script takes only one input LOG _FILE generated from the command,
#  find <brick_path> | xargs  getfattr -d -m. -e hex  > log_gluster_xattr

import re
import subprocess

sanoj-unnikrishnan

Current Design
-------------------------
Quota translator  : Ensures the limits are enforced. To know the current size enforce lookup is done on QUOTAD for that directory
                    The values are cached until soft/hard timeout (To reduce IPC) 
QUOTAD            : Seperate process that aggregates the size of directory from all the birck server processes.
Marker Translator : A server translator that accounts for and maintains the size of directory.

In the Current design the quota translator has to make IPC on a periodic basis based on timeouts configured at volume level. 
The timeouts also create a room for overshoot in quota limit.

sanoj-unnikrishnan

Directory quota in XFS is implemented through project quota feature. 
The directory on which quota has to be set is given a unique project id (which is tracked in inode)
Directory quota in XFS are journaled.

However behaviorally the directory quota in XFS is very different from that in GlusterFS. 

Pro's in GlusterFS quota over XFS:
GlusterFS will allow you to set a limit on /home and /home/user1 .
So if you write to /home/user1/file1 the limit for both its ancestors are checked and honored.
An admin who has to give storage to 50 users can configure /home to 1 TB and each user home to 25GB say

sanoj-unnikrishnan

Please refer to the explanation on current accounting scheme in
(https://gist.github.com/sanoj-unnikrishnan/f715a73afaa4af9c1fc81d05fab062f3).
I shall assume the same context here.

In the current design, we update the in core context and the on disk xattr at the same time.
While we do avoid multiple updates in case there are multiple fop on same inode, we still incur overhead from on disk updates.
We could decouple the on disk updates by doing them in a periodic (time interval ) manner.
The incore context would however be kept updated at all times.
In the current design, post a crash we could have the stale xattr at the top of hierarchy until lookup has happened at bottom.

sanoj-unnikrishnan

Marker translator is responsible for accounting of size in GlusterFS. 

Current accounting design.
----------------------------
updates on write

Accounting happens using a set of contri and size xattrs on each inode. 
Contri gives the contribution towards its parent while size give the size of the direcotry.
In a quiesced state Contri and size should be the same value.

sanoj-unnikrishnan

Glusterfs quota troubleshooting

Relevant Files
--------------
logs:
quotad log	
Brick log
Crawler log
Aux mount log