automationtesting.md

Test Automation

• PPA
  • This decision should be isolated from what has already been done for CSS
  • CSS is a different beast and has terminal emulation mixed in (although I'm sure you could use node-pty)
  • Very long term goal with PPA is to replace CSS and massively simplify our entire product / tech stack
  • Plan is to keep the project as a mono repository so we aren't going to be spread out across a bunch of repos like CSS where it makes sense for the automation tests to be in yet another repo
• Stack complexity
  • Business is mandating MS tech stack and reducing stack complexity
  • Adding an additional language / framework to the mix is not in line with that mandate
  • We already need JavaScript developers to work on the app (can't build a web app without them)
  • Tests should be run as part of the CI pipeline
    • Using Ruby would mean we'd need to install a 3rd language runtime on our builds
• Continuous Integration
  • Tests should live with the code and be branched with the code which forces them to stay in sync and be updated with code changes
  • Tests should be run as part of the CI process, if broken tests don't break the build then tests are way more likely to get out of sync and become tech debt
  • UI automation tests are not our only kind of tests, we have C# unit tests, C# integration tests that test the complete application is working correctly by calling the APIs, JS unit tests, as well as UI automation tests
    • These things should compliment each other and not have massive duplication, use the level of test that makes sense
  • When the project gets more mature, we will feature branch and part of a check list for that feature will be to have matching automation tests for the feature that only exists in that branch
  • Forces dev and QA teams to work closely together, we really should be acting as a single team and not throwing things over the wall when they are done
    • ie. Update data attributes and selectors as we go along to make things more testable
• Cross browser support
  • Ferrum seems to only support Chrome
  • Can't imagine we are ok with only testing on Chrome
• Popularity
  • Playwright is the most popular automation testing framework besides puppeteer (which is much older and starting to fade out)
  • Ferrum 1.5k stars, Playwright 53.5k and maintained by Microsoft
  • JavaScript test automation frameworks dominate the space: https://ossinsight.io/collections/testing-tools/trends/
  • JavaScript is the most popular language on the planet, easy to hire for especially junior devs
  • Arguments about