GSoC 2010 VFS Proposal

gistfile1.txt

Title: 	VFS Quota's
Student: 	Samuel Greear
Abstract: 	Implement filesystem-agnostic kernel quota support and tools.
Content: 	

Name: Samuel J. Greear
Email: sjg@thesjg.com
Project title: VFS Quota's

Project goals and description:

The goals of this project would be to implement a filesystem-agnostic quota
implementation and related tools. The core of this would exist in a new kernel
source file, vfs_quota.c, loosely modeled after the time tested UFS quota
internals. The VFS quota implementation will use the more general bytes
and files rather than the filesystem-specific blocks or inodes.

There are multiple ways this could be hooked into the kernel, the
three most obvious are: as a new vnode operations vector for each quota instance -
this would involve a discrete quota op responsible for calling (or not) the
underlying filesystem vfs operation. As a new null layer filesystem or
integrated with nullfs. Or, it could be integrated into the VFS wrapper
functions. All technical merit aside, this final approach would be the one I
would take given the limited nature (time-wise) of GSoC.

One hairy detail that will have to be dealt with in some manner is the
bootstrap problem. There are a variety of approaches that could be taken
in this regard but I would prefer to research other quota implementations
for precedent and thoroughly discuss the options with my assigned mentor
before coming to a determination.

The kernel side of this project is actually relatively simple. The majority
of the work is anticipated to be in userspace implementing the required tools.
These tools will be required to perform the functions of: enabling and
disabling quotas, showing quota usage on a per-filesystem basis for users
or groups, setting quota limits on a per-filesystem basis for users or groups.
These tools will also be required to be able to traverse a filesystem using
standard API's to generate a quota file containing current usages for all
users and groups.

Preliminary project timeline:

*** Community bonding period ***

- I am involved with the DragonFly BSD project outside of Summer of Code and have
been for some time. As such, I have a possibly larger interest in the success of
the DragonFly BSD Google Summer of Code projects than my fellow students. During the
community bonding period I will find time to confer with the project organizer. We
will use this time to determine methods by which I and my project can be used to set
a good example for other students to follow with their own projects.
- Get a head start on the research portion of the project

*** Week beginning May 24 ***
- Research
- Determine how to bootstrap quota support when mounting a filesystem
- Create a preliminary description of the userland tools and options
- Define quota file format
- Begin work

*** Weeks beginning May 31, June 7, 14 ***
- Implement kernel VFS quota support

*** Weeks beginning June 14, 21, 28 [overlaps above] ***
- Implement basic userland quota control and maintenance tools

*** Week beginning July 5 ***
- Rigorously test the new kernel code
- Ensure all code is clean and well commented, ready for community review.
- Make sure that at a minimum, rudimentary external documentation exists in
the form of manpages

*** July 12-16, Mid-term submission ***
- Ensure that I have supplied all code, patches, documentation and if necessary
a revised schedule with all previously agreed upon revisions by the 12'th so
that my mentor may complete my mid-term evaluation.
- Take advantage of midterms by having others review and test the cleaned up
and documented mid-term code.

*** Weeks beginning July 19, 26, August 2, 9 ***
- Complete implementation of userspace quota tools such that they and the
kernel code comprise a complete and useable system.
- Implement ideas or features and fix bugs brought to light during mid term
reviews.
- Ensure that documentation is adequate for system administrators to use
the quota system.

*** August 16, firm "pencils down" ***
- Ensure that my assigned mentor has all useful materials produced available
for review.

*** EXTRAS ***

If sufficient time remains, several projects might be attempted to improve
the system. The first of these is to research and implement attribute caching
in the kernel VFS quota code. It is anticipated that the high number of
VOP_GETATTR calls that the quota subsystem would be required to make will
incur a significant performance penalty. It may be beneficial to cache
attributes outside of the quota code, perhaps even in the vnode, but this
would require research.

Alternatively, or in addition to the above, it may be beneficial to provide
at least a roadmap for the eventual removal of UFS quota support. This would
likely have to include integrating the optimized quota check code which
operates directly on a UFS-formatted block device into the new quota tools.

An idea that has presented itself since this proposal is, in userspace there already exists an efficient method of traversing a UFS filesystem and calculating the blocks and inodes -- the existing quota code. The method it uses is to bypass the VFS layer altogether, open the actual device (as the superuser) and just ferret out what it needs in a linear fashion. This is much, much faster than any generic solution. It would be ideal if the userspace quota tools simply exposed a filesystem specific API, so that this optimized code could be retained for UFS. In addition, a generic method through the VFS layer supporting any filesystem type would have to be added.

As a future project, it is feasible that an optimized HAMMER solution could be implemented as well, by traversing the B-Tree (possibly through an existing mirror-stream ioctl) and synchronizing against HAMMER transaction id's.

An extensible file format would be mandatory for this -- or the individual quota scan/check implementations would have to implement their own format.