Support

# Support

## Authors

#### Prometsource

Michelle Krejci, Greg Pamier, Will Milton, Doug Dobrzynski, Matthew Gilboy

#### Credits

Thanks to:

* Scott Massey, Pantheon - for his dedication to audits and inspiring some folks at Prometsource

## Client Paths

There are three paths client can follow to reach support:

* New support client with audit
* New support client from an internally developed project
* Grandfathered support client without audit

All three of the above types of client will require on-boarding. A workflow for on-boarding all three will be addressed. This is to be established after requirements because it is subordinate to determination of a set of requirements. 

The support process and offering differs from the existing QA, Project Management, and Development workflows. These workflows should inform support work and support work should inform development. The support on-boarding process is to accommodate this.

## Requirements

Below are the requirements that clients must meet before we provide support or begin a project. This is also the minimal state that clients can expect to be in after we build their site.

> Previous versions of these requirements have been laid out in the Support Wiki: https://projects.prometsupport.com/projects/updatesfortradesupportsjobt/wiki . Key contributors to that document have been Michelle Krejci, Greg Palmier.

The following requirements are not exhaustive but represent an initial minimum of compliance that can be tested against any support clients.

#### Audit

Support exists on a spectrum in contradistinction to development; both are defined by the praxis of client specificity. The purpose of an audit is to determine the extent to which the designation "support task" can be applied to a unit of work. In practice, this identifies the extent to which non-singular client directed work describes the scope. For reference, a reverse measure would identify units of work that can specifically only ever be construed as applying to a singular client, by their direction.

* A site audit should be performed for all new clients that identifies, prior to support obligations, the gap between the state of the site as it is, and the steps necessary to meet our threshold.
 * Checklist for Audits: https://projects.prometsupport.com/projects/sysadmins/wiki/Site_Audit
* Client buy-in on on-boarding the site. This includes:
 * Slaughtering of site
 * Push strategy documented
 * Featurizing of site
 * Disabling of all possible module UIs in production and  negotiation of features that will need to be ignored in features reverts and pushes

#### Environment / System Administration

* All code in git on Github
* A dev-staging-prod environment
 * A Promet Standard file structure (i.e., modules in /custom and /contrib, no files or directories in root, theme files properly labeled, no .bak or tar.gz files, etc.)
 * No other CMS or other websites in general inside of the Drupal Webroot (/dev, /staging); any landing pages will either require a fully documented and agreed upon gitignore policy, or will need to be excluded from liability
 * Root access to the server(s) provided
* PHP Tuning and Minimum Performance/Security Tasks completed or agreed upon as highest priority backlog items

#### Drupal Code and Config

Promet Source is a development shop that specializes in the delivery of Drupal applications.

* General compliance with current LTS Slaughterhouse methodology
* All content types, taxonomies, views, roles, permissions, and any other appropriate settings exported into features
* Generate drush.make file and place in git repo outside of root and in wiki of project 
* Get User Stories to use for QA purposes (so we all know what we mean when we say the site has passed QA) from client
* Updates applied to core and contrib

#### Client Expectations

It is essential that clients operate based on the same understanding as the support organization, and that support organization has a transparent and coherent understanding internally of these expectations. These expectations need to be standardized across all client types, engagements, and system users to ensure that costs for support are reduced by streamlining client deliverables.

The tension created here must be alleviated through fluidity of a given project interfacing simultaneously with both support and development engagements.

Mandatory Requirements:

* Agreement upon definitions of types of work that can be done by the support organization:
 * A clearly defined set of criteria for a hot fix
 * An understanding of how frequently backups are performed and what they consist of
 * An understanding that no manner of support task requires work on production
* Agreement upon continued engagement and education
  * Clients will be required to provide user stories, acceptance criteria, and collaborate in the maintenance of testing details
  * Clients will be required to attend occasional training on topics deemed necessary to the maintenance of their application, required features, and as applicable to their skill level
* Agreement upon definitions of the application
 * A complete assessment of what a 'good' state is and what a rollback means
 * A tentative assessment of deliverable features to the application that would be consider designated as support
 * A complete assessment of non-deliverable features to the application that we would provide consulting or development work for, but not support
 * These three assessments would need to remain subject to audit

The intent of the above is to further engage all stakeholders in having a coherent understanding of what is a 'support task'.

Generally, the support organization must be capable of explaining and soliciting client buy-in with the mission of support: to maximize common goods applied across all projects, while ensuring the individual survival of their site.

Clients will be subject to, and in some cases responsible for, establishing that their site remains in compliance with supportability requirements. It is possible that a site will become compromised, occasionally through an outdated architecture, external hacking, or client misuse. In such cases, it is likely that the support organization will have to decline developing short term fixes for the site in favor of a full-fledged development effort.

Once the designation that a site cannot be supported has been reached, the support organization will need to work with the client and operations to identify a path forward be it through a development remedy or cessation of site support, be it wholesale or by segment.

## Remedies

All of the above requirements are remediable tasks. The should be guided by a singular intent: make the work boring. Essentially we are selling a promise. The promise is that we will keep the clients site up. If it breaks we will put it back.

The remedying of these deficiencies is what can be construed as on-boarding. Any client that purchases a support package must understand that their site will be subject to, and benefit from, the requirements above. Further, as noted above, remediation is not permanent and sites may be subject to additional development work.

The stated ideal of support is to provide a stable platform that permits for branches of short-term engagements as well as large-scale iterative development. Both of these involve subjecting the supportable code base to flux which then requires testing to confirm that elements of the site remain supportable.

#### There are three paths a client can follow to reach support:

To repeat, there are three scenarios for on-boarding: new support from audit, from internal development, and grandfathered in clients.

On-boarding workflows:

* New support client with audit
 * audit reveals findings
 * audit provides recommendations to meet requirements above
 * post-audit follow-up:
  * client pays for development work to be executed outside of support contract
  * client pays to execute complete list of recommendations
 * until post-audit follow-up is complete, client is deemed 'unsupportable'
* New support client from internal development project: In certain instances, only certain portions of a project are performed by internal teams. 
 * Untenable assertions - Additionally it cannot be the insistence by support that <Rephrase, Remove ASAP>:
  * all development will be 100% supportable; obversely, 
  * all development will be 100% unsupportable
 * What is tenable - It must be the insistence that delivery and its nonfunctional aspects are:
  * clearly documented
  * clearly agreed upon
  * clearly delivered
 * Standardized development deliverables
  * fully documented and testable push strategy
  * a backlog of future functionality requested and added to the parking lot during the development process
  * a prioritized list of technical debt with fully storied tasks
  * a backlog of nice-to-haves and other recommended architectural/backend improvements (i.e. items for upsale)
 * Requirements for [grandfathered]
  * ability for the support manager to make the decision to deny supporting a site until requirements are met
this affects binding agreement with subcontractors

> Cebu & Availability

* Grandfathered support client without audit

## Purpose

The purpose of support is to streamline all possible tasks. The support department is not a short run shop. Its primary objective is to perform the same tasks over and over and over for a stable of sites rather than one-off exciting work for a handful of favorites and friends and complainers. This includes applying updates, rolling back items the client broke to the previous known 'good' state.

Support is a retainer service. Support lives by maximizing the same output across multiple clients and this is only achievable when we have a stable platform to deploy against.

Support specializes in two ways: engineering and development. The former works on clients as a platform. The latter works with clients on a directed project basis. Directed projects are very limited engagements and are very limited in scope and scale. The intent of directed projects is to provide a secondary backlog for support projects without disrupting the supportability of a client or causing them to need to go through the on-boarding process.

This specialization is to improve the fluidity of project transfer. This is shown to be good when a client can have an expedient membrane transfer between support engineering, support development, and large-scale iterative development.

#### Escape Velocity for Support Clients to Development Projects

For projects of sufficient scale or scope, the support department will have no choice than to reject work in progress.  unless performed within the agile methodology, with milestones incrementally delivered in minimum viable projects, and with thorough documentation and a bevy of automated and manual tests. 

In all likelihood, clients will find transitioning away from support prohibitive because the site will require on-boarding again. The cost will be reduced but will not be trivial.

Support will do well to work in tandem with QA and Development to ensure that the site is regularly audited throughout development. Support infrastructure will need to be able to address both functional and non-functional specifications; clients exceeding these specifications will need to be turned down or management will need to provide a viability study to address an expansion.

Further requests for documentation include:

* A checklist for determining when and how a project will be handed off from the support platform to the development workflow
* A checklist for determining the viability of a development project being incorporated into the support platform
* A matrix for determining complexity, scope, scale, and other elements that contribute to pushing a site off support

#### Motivations

On a per client basis:

* Management time is kept to a minimum and not billed
* Support time is kept to a minimum and not billed
* Every ounce of effort that is put into any form of development is billed

On a division level:

* Management time is intended to identify accounts that are lagging behind in compliance and require work
* Support time is meant to:
 * deploy changes across the entire ecosystem
 * develop tests to assist in on-going audit and expanding the scope of auditable, testable features
 * improve internal deployment and delivery tools
 * find ways to identify and minimize unique and interesting tasks

#### Metrics for Success in Support

When support tasks are able to accommodate more changes to the code base while simultaneously maintaining stability we are able to provide value while minimizing risk. This means that we are able to provide support to more features of the application and facilitate more autonomy of application users.

Other measures:

* Degree to which QA and continuous testing and auditing can be included in support and development workflows
* Eventually all tasks divide into Experience (prescriptive) or Delivery (descriptive); the success of a support organization depends on the
 * complete specialization in Delivery
 * efficient transacting with prescriptive development
* The support organization can define internal tasks that:
 * cross-train 

This assumes that we begin with a certain portion of the code base that is 'supportable'. As the platform for support grows, this portion also. For example, if we 'support' 5% of the code base, and make an improvement to the 5% for client A, then the improvement will be applied across clients B-Z as well.

An edge case of this would be Drupal Core updates. Ideally, we might have such command over the code base that we would be able to roll out updates to development environments for UAT across all support sites, simultaneously.

Support needs to determine or access to:

* What is considered to be an "expensive" task and client
* Historical data of the most successful support clients to date




# Change Log

2014
1/30 - Initial commit
1/31.1 - Updates of engagement scope
1/31.2 - Include of change log, escape velocity, metrics section
2/1.1 - Added detail to a variety of sections
2/1.2 - Added extensive detail to expand on the question of 'what is support?', credits; added a ton of redundancies, the <remove> tag

L14: Confusing because "Internal Development" generally means technical work on applications that are not meant for consumption by customers
L17: Surely we have at least one client with no audit.
L33: this may read better if you define "support task" first. All of this could come down to "agree on the changes we will make and how to make them". This way, we are encouraged to document how demanding the procedures and tasks are for a given client and why they are so, eg: "Client has decided it's not efficacious to proceed with an onboarding engagement to fix dependence on deprecated projects by removing some functionality, therefore core and contrib updates are error-prone and unpredictable". We have had little success getting our clients to agree to full onboarding, and it's made worse by the fact that the standards to which we wish to onboard them are still in flux. Documenting the difference between the ideal and the current state may be more useful, especially if we can trace it back to a client decision.

General note: if we want support to be just the execution of support tasks, it would be best not to maintain definitions of development procedures in the support documents. This may be a sign that we need to write up several standards and keep those in the development wiki. For example, Promet Source Development Standard 1 could be just like this:

PSDS1:

• All unmodified contrib modules will be in sites/all/modules/contrib
  ** 'unmodified' means that the .info file references a version of the module that is exactly the same as the files present.
• All modified (hacked) contrib modules will be in sites/all/modules/modified
• All custom (not available on Drupal.org) modules will be in sites/all/modules/custom
• All Features Modules will be in sites/all/modules/features

and maybe PSDS2 comes along and says "in addition to PSDS2 all features modules must export exactly one primary component and any configuration (ie variables) that pertain only to that component and no others". Now you can say in the support docs "all support projects must adhere to PSDS1" or "all support projects must document which PSDS they adhere to".

I'd encourage you to come up with a draft of this document that does not specify anything Drupal specific and instead focus on ways to make visible the feasibility or difficulty of performing the desired support tasks. Ideally, support policy would be flexible enough that we just have a list of technical tasks we are expected to perform, an idea of how hard they are and how we can elevate that constraint.

Updates:

• @14 - "New support client from an internally developed project"
• @17 : Exposited intent - "All three of the above types of client will require on-boarding. A workflow for on-boarding all three will be addressed. This is to be established after requirements because it is subordinate to determination of a set of requirements."
• @33 - "Support exists on a spectrum in contradistinction to development; both are defined by the praxis of client specificity. The purpose of an audit is to determine the extent to which the designation "support task" can be applied to a unit of work. In practice, this identifies the extent to which non-singular client directed work describes the scope. For reference, a reverse measure would identify units of work that can specifically only ever be construed as applying to a singular client, by their direction." (currently working to resolve some poorly written redundancies added in addressing this)