development environment needs

todo.md

## TODO - `butler bounce` command - generates csi static stuff in like `daft/templates` - generates the running .yaml config - reloads the server - watch file list - something that watches the yaml config files (there's now 3 of them) - `butler test` command -- essentially does what the [`ut`](https://github.com/siq/util/blob/master/bin/ut) script does now - `butler optimize` command -- same as `serve optimize`

wishlist.md

here's the stuff that i think we need (at least for the development environment):

• single-command switching between branches (master/release-W.X.Y.Z/etc)\
  • should be parallelized
  • whenever the current branch is switched, it should install python modules automatically to the virtualenv
• single-command git/svn pull/update across everything, not just the current app (serve pull in daft wont pull glad, for example)
  • also should be parallelized
• single-command switching between back-ends
  • like i'd like to be able to run something like serve backend ralph to point to ralph's stack
  • the developer would need to define what the 'ralph' backend consists of though, presumably along w/ like a 'master' and 'release-branch' or something
• better control over what servers are kicked-off/reloaded. right now there's essentially 4 classes of servers:
  • the front-end app server, i.e. glad and daft, which is usually a uwsgi instance
  • the bastion security server which we need in order to be able to log in to a locally-running app server
    • one of these could serve multiple app servers
  • the mock servers used for unit testing, i.e. the things we spin up so that we can run the url routing unit test, even though that's not executed in the jenkins client-side unit testing job
  • the unit test server itself, which is entirely node.js based
• some sort of 'create dev environment'
  • substeps:
    • clone all pertinent repos
    • create necessary virtualenvs and install the correct versions of 3rd-party python modules
    • link-install all of our repos that are csi components
    • possibly csi install the components as well?
  • this is important b/c right now we're running a bunch of these initialization-type commands every time we start the dev servers, and it's really effin slow, and it makes bootstrapping more complicated
• better control when restarting such that i'm not running npm install every friggin time
• good integration w/ local topic branches, like maybe if a repo is on a topic branch, and you run the global command to switch everthing to master, it should be smart enough to leave the one repo on the topic branch
• the app servers still need that uwsgi_logging_sanity.py b/c it's really useful to see what's getting cached

opinions

i think all of the repos should be in one big flat directory, so like ~/work/{daft,glad,enamel,etc.} such that each app is using the same library repositories (like enamel, spire, etc.). the next question is whether there should still be a separate .virtualenv directory for each app. i guess i dont think there should.

like i'd like to be able to run something like serve backend ralph to point to ralph's stack

+1

the developer would need to define what the 'ralph' backend consists of though, presumably along w/ like a 'master' and 'release-branch' or something

Not sure if I understand this. I would expect to only modify the local .yaml to point to the users gateway and appstack.

better control when restarting such that i'm not running npm install every friggin time

+1

I really like the idea of parallelizing some of this stuff. I've got my own bash script to do a global pull wish I would have thought of that. Pulling everything can take more time that it needs too.