Flopsie design decisions

gistfile1.txt

Bedrock Linux 1.0alpha4 "Flopsie" will include a new fuse filesystem
(tentatively named 'brf') which somewhat resembles a union filesystem to
resolve the "/etc issue".  The way it will work is that it will redirect
filesystem calls to either the client-specific versions of the files or the
global, shared versions of the files, depending on the settings provided (i.e.,
the stuff in brclients.conf).  Instead of just using it to replace bind-mounts
for /etc, it could also be used to replace the other bind-mounts used to share
things in each client, which may be considered cleaner/simpler.  The downsides
of this are (1) historically union filesystems are very difficult to do well,
so this could be a dead-end, and (2) it will likely cause filesystem access
overhead.  Since this is alpha, we'll run with it and see what happens.  If it
fails, we can roll back to what is used in alpha3/Bosco or come up with other
ideas.

The filesystem will look something like this:

/
/bedrock
/bedrock/clients/
/bedrock/clients/arch <which then has brf mounted onto it>
/bedrock/clients/debian <which then has brf mounted onto it>
/bedrock/clients/bedrock <which is bind-mounted to the root>

brf can redirect calls to global files - like /home - to the version directly
on the root.  However, if it attempts to redirect calls to the client-specific
versions of things with the above layout, it wil be recursive.  It needs access
to the files "below" it another way.  Note that mount points are not
(automatically) recursive.  Thus, this can be acheived by mirroring the clients
via a bind-mount and only mounting brf on one copy.  The filesystem will end up
looking more like ("layout1")

/
/bedrock
/bedrock/clients/
/bedrock/xclients/ <which is a bind-mount to /bedrock/clients>
/bedrock/clients/arch <which then has brf mounted onto it>
/bedrock/clients/debian <which then has brf mounted onto it>
/bedrock/clients/bedrock <which is bind-mounted to the root>
/bedrock/xclients/arch <which is the same as /bedrock/clients/arch but without any mounts on it>
/bedrock/xclients/debian <which is the same as /bedrock/clients/debian but without any mounts on it>
/bedrock/xclients/bedrock <which is also bind-mounted to the root maybe?>

This is potentially confusing as there are to copies of clients right next to
each other.  An alternative layout ("layout2")

/
/bedrock
/bedrock/clients/
/bedrock/clients/arch <which then has brf mounted onto it>
/bedrock/clients/debian <which then has brf mounted onto it>
/bedrock/clients/bedrock <which is bind-mounted to the root>
/bedrock/clients/arch/bedrock <which is bind-mounted from root's /bedrock on top of brf>
/bedrock/clients/arch/bedrock/clients/arch/
/bedrock/clients/arch/bedrock/clients/debian/
/bedrock/clients/arch/bedrock/clients/bedrock/ <bind-mounted to root>
/bedrock/clients/debian/bedrock <which is bind-mounted from root's /bedrock on top of brf>
/bedrock/clients/debian/bedrock/clients/arch/
/bedrock/clients/debian/bedrock/clients/debian/
/bedrock/clients/debian/bedrock/clients/bedrock/ <bind-mounted to root>
/bedrock/clients/bedrock/bedrock <which is bind-mounted from root's /bedrock>
/bedrock/clients/bedrock/bedrock/clients/arch/
/bedrock/clients/bedrock/bedrock/clients/debian/
/bedrock/clients/bedrock/bedrock/clients/bedrock/ <bind-mounted to root>

This is much more similar to what is currently being done in Bosco.  With this,
brf can redirect to either the root for global versions or the
/bedrock/clients/<client>/bedrock/clients/<client> version of things for
client-specific versions.  This is advantageous over the layout1 in that the
user can usually just go to /bedrock/clients/ to see (one copy of) all of the
clients.  It will look that way irrelevant of which process from which client
views it.  The disadvantage of this compared to layout1 are: (1) brf will be
dependent on another filesystem being mounted on top of it.  If this is not
mounted there it will become recursive and probably lock up, and (2) this
requires more point points per client than layout1.

Which leaves the following questions:
- Anything I'm missing that would make either of these not work?
- Maybe this will work but is a bad idea for another reason?
  - Maybe performance overhead will never be acceptable...
  - in which case we can stick with brf for just /etc and bind-mount everything else like bosco
- Are there other obvious layouts I'm missing?
- Is layout1 or layout2 (or another layout) preferable?
- For layout1, what should "clients" and "xclients" directories be called?  For
  layout2 this is a non-issue, "clients" wil lsuffice.
- Any better names for "brf"?
- Bedrock-as-a-client will not need brf, since it will "automatically" have the
  real copy; however, it will have to be bind-mouned to root.  For layout1,
  should both client directories have a bind-mount to root?

The most likely scenario is this will prove to slow and will be limited to
/etc, and the rest of the filesystem will very much resemble alpha3/Bosco for
alpha4/Flopsie's release, but it is still worth discussion just in case.  If
this idea proves popular, perhaps we could make a kernel module to do this with
much better performance and simply use fuse as a fall-back in case there is a
kernel update and the user doesn't recompile the module.

A clarification on the second design if you would: would the number of writes go up under this design or would they stay the same?