Skip to content

Instantly share code, notes, and snippets.

@totten
Last active Mar 28, 2019
Embed
What would you like to do?
  • Plan A1:

    • Synopsis: Update the Mosaico UI/extension to include A/B testing. Update other layers as necessary (in accord with general vision).
    • User Experience: To send an A/B test in Mosaico, create a regular Mosaico mailing -- and add multiple subjects.
    • Data Model: Each A/B test is stored as one record in civicrm_mailing.
    • Major Tasks:
      • (civicrm-core) Update CiviMail data-model to allow:
        • Storing optional variants of a mailing, as defined by overrides in the civicrm_mailing (e.g. variants.a.subject, variants.b.subject, variants.a.body_text)
        • Storing optional AB sizing (i.e. deliver 100 copies of variant A)
        • Storing optional variant ID for each delivered message
      • (civicrm-core) Update CiviMail API to support these new fields
      • (flexmailer) Update FlexMailer's ComposeBatch pipeline to:
        • Assimilate overrides during composition
        • For applicable mailings, assign a variant ID to each recipient
      • (mosaico) Update Mosaico Extension UI to:
        • Accept/display multiple subject lines, storing each as an override
        • Accept/display AB delivery params (number of recipients per test)
      • (civicrm-core) Update CiviMail reports to:
        • Allow comparison of stats for each variant
    • Strengths:
      • Addresses existing debt about experimental correctness (delivery ordering/timing)
      • Lined-up well to support conditional blocks (per-variant) in Mosaico mailings
    • Weaknesses / Debts / Potential Stretch Topics:
      • For a user, the navigation-path is confusing/ambiguous -- you have to understand more subtlties among "New Mailing (Mosaico)", "New Mailing (Traditional)", and "New A/B Test"
      • If someone tries to send a mailing with variants with BAO Delivery Engine (aka v1 delivery system), then that's not supported. Report some kind of error/warnings.
  • Plan A2: (As above, but with data-mapping/decomposition on submit)

    • Synopsis: Update the Mosaico UI/extension to include A/B testing. Map one mailing to autogenerated AB record.
    • User Experience: To send an A/B test in Mosaico, create a regular Mosaico mailing -- and add multiple subjects.
    • Data Model: Mosaico UI initially generates one civicrm_mailing (with AB options under template_options); however, after submission, it is materialized into a few distinct mailings.
    • Major Tasks:
      • (mosaico) Update Mosaico Extension UI to:
        • Accept/display multiple subject lines, storing each in template_options
        • Accept/display AB delivery params (number of recipients per test), strong in template_options
      • (mosaico) Hook into Mailing.submit. Examine template_options and conditionally generate AB and related mailings. Maybe some API or hook updates in civicrm-core to facilitate this.
    • Strengths:
      • Less need to change core reports and data model
      • Less ambiguity/conflict in the navigation than Plan A1. (Still some, but less.)
    • Weaknesses / Debts / Potential Stretch Topics:
      • Still using less-correct experimental protocol.
      • Strange-loop in the model (mailing=>AB=>3x mailing) might look odd to others
      • Does not address existing debt about experimental correctness (delivery ordering/timing)
  • Plan B:

    • Synopsis: Update the core CiviMail UI ("New Mailing (Traditional)" and "New A/B Test") to make ckeditor and Mosaico more interchangeable
    • User Experience: To send an A/B test in Mosaico, create an A/B test by "Subject" -- and choose to compose in Mosaico (rather than CKeditor).
    • Data Model: Each A/B test is stored as one record in civicrm_mailing_abtest with multiple relations to civicrm_mailing.
    • Major Tasks:
      • (civicrm-core) Expand or replace the API for richtext email (ckeditor/tinymce) to allow loading/storing Mosaico templates (extra metadata)
      • (civicrm-core) Define new UX/API for browsing/selecting templates (in which ckeditor-style temlates and Mosaico-style templates are interchangeable)
      • (civicrm-core) Implement that new UX
      • (mosaico) Integrate with the expanded/replaced API for richtext email
      • (mosaico) Deprecate the standalone composition screen
    • Needful But Not Fully Resolved Requirements:
      • Potentially sets up conflict over pageflow/layout in the composition screen -- existing users have conflicting expectations in "New Mailing (Mosaico)" and "New Mailing (Traditional)".
    • Strengths:
      • Addresses existing debt in navigation-path (confusion/ambiguity about "New Mailing (Mosaico)" and "New Mailing (Traditional)")
      • Reduces future maintenance in keeping "New Mailing (Mosaico)" and "New Mailing (Traditional)" in sync
    • Weaknesses / Debts / Potential Stretch Topics:
      • Lined up poorly for conditional (per-variant) blocks in Mosaico.
      • Still using less-correct experimental protocol.
      • Likely to change existing user-experiences in significant ways, so it seems more important to engage other stakeholders. This, in turn, can make it more expensive to settle the design and/or get through review.
  • Other Streth Topics:

    • FlexMailer Phase-in
      • (Development) Cleanup/expose option to deliver messages for traditional mailings
      • (Discussion) Bundle with core
      • (Admin) Bundle with core
      • (CI) Run Flexmailer tests with core PRs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment