system testing.md

Build Tools and Test

Build Manager

Goal- centralized UI, and scheduler, for managing automated builds
- Terraform Enterprise - HashiCorp

• Jenkins - Cloudbees
  
• TravisCI
  
• CircleCI
  
• Bamboo - Atlassian
  
• KitchenCI - Chef

Build Runner

Goal - automated build runner
- Linux: Bash, Makefile
- Java: Gradle, Maven, Ant
- Python: tox, fabric, molecule
- Ruby: rake
- Node.js: Gulp, Grunt

Build Tasks

Goal - automatically call build runners
https://githooks.com/
https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks


 

Static Analysis

Dependency Security

Goal - everyone's code should use secure components.
OWASP Dependency Check (java, python, ruby, php, node.js, Swift)
Snyk (java, python, ruby, node.js)
Gemnasium (java, python, ruby, node.js)
OSSIndex (java, python, ruby, node.js)
Java - dependency-check-maven
Python - dependency-check, pyup
Ruby - bundle-audit, Hakiri
Javascript - RetireJS

Code Format

Goal - everyone's code should look the same, automatically.
Java - Google Java Format
Python - autopep8, black
Ruby - rubocop, rufo
Javascript - prettier

Code Linting

Goal - everyone's code should look the same, and be easy to maintain.
Java - checkstyle, findbugs, Google error-prone
Python - pylint, flake8
Ruby - rubocop
Javascript - eslint, jslint, jshint

Code Complexity

Goal - everyone's code should not be complicated, no conjugations in descriptions, and easy to maintain
McCabe Cyclomatic Complexity
SonarCube (Java, Python, Ruby, Node.js, Javascript)
PMD (Java, Python, Ruby, Javascript)
Java - ???
Python - radon
Ruby - Ruby Critic
Javascript - eslint, jshint

Code Documentation

Goal: everyone's code should be easy to understand, and easy to maintain
Java - javadoc, checkstyle
Python - pydocstyle
Ruby - rubocop
Javascript - jsdoc

Unit Testing w/ Mocks

Goal - code should function as expected, in isolation.

Integration Testings, Server-side, w/ Mocks

Goal - code should function as expected, with immediate dependencies

Integration Testing, Client-side, w/ mocks

Goal - code should behave consistently, only on the browser / mobile app, offline
Javascript - Mocha
jQuery - QUnit
Angularjs - Jasmine + angularjs-mocks
https://docs.angularjs.org/guide/unit-testing
React - React-unit, Jest, Enzyme
https://reactjs.org/community/testing.html

Code Security

Goal - everyone's code should be written securely
SonarCube (Java, Python, Ruby, Node.js, Javascript)
PMD (Java, Python, Ruby, Javascript)
Java - spotbugs
Python - bandit
Ruby - Brakeman
Javascript - mustache-security


 

Deploy

Build

Goal - build local VMs, Linux Containers, or Cloud VMs.
Packer - Virtualbox, VMWare, Hyper-V, AWS AMI, Azure VM, Google Image

Local VM

Goal - automatically deploy local VMs.
Vagrant - Virtualbox, VMWare, Hyper-V

Local Linux Container

Goal - deploy a linux container.
Docker
LXC

Local / Remote Containers

Goal - orchestrate container deployment.
Kubernetes
OpenShift
LXD

Container Security

Goal - ensure containers have secure components.
Artifactory JFrog XRay
anchore-cli
coreos claire
dagda
cilium - container network security
sysdig falco - Kubernetes
https://docs.docker.com/v17.12/docker-cloud/builds/image-scan/
https://techbeacon.com/security/10-top-open-source-tools-docker-security
https://sysdig.com/blog/20-docker-security-tools/

Cloud

Goal - deploy remote cloud resources.
AWS Cloudformation
Azure Resource Manager
Google Deployment Manager
Terraform (AWS, Azure, GCE)

Provision Runner

Goal - automate running provision in multiple environments
Ansible - Molecule
Chef - Kitchen

Provision

Goal - install and configure services.
Ansible
Puppet
Chef
Saltstack

Infrastructure Testing

Goal - environments should be installed consistently.
Ruby - serverspec
Python - testinfra, goss
Ruby - inspec (Chef)

BDD Testing, Infrastructure (ops devs)

Goal: environment should behave consistently.
Python - infrataster (HTTP, pgsql, redis)
Terraform - Terragrunt

 

Dynamic Analysis

Integration Testing, Server-side

Goal - code should behave consistently, only on the server.

BDD Testing, Integration, Server-side (server devs)

Ruby - cucumber, rspec
https://www.martinfowler.com/articles/rake.html
Python - behave
Java - jbehave

Integration Testing, Client-side, headless

Goal - code should behave consistently, only in the browser / mobile app, quickly.
Javascript - CasperJS + PhantomJS
AngularJS - Karma, Protractor

BDD Testing, Integration, Client-side (ui devs)

Goal - systems should behave consistently, on the client-side
Javascript - cucumber.js
jQuery - ???
Angularjs - Jasmine
React - react-cucumber

BDD Testing, e2e in a Browser (project managers)

Goal - systems should behave consistently, end-to-end, w/ Selenium
Java - Cucumber-jvm + Cucumber-junit
Python - Lettuce

Testing, Performance

Goal - systems should be performant, and reliable.
JMeter
Blazemeter - Hosted JMeter
Gatling Locust The Grinder

BDD Testing, Security (security)

Goal - systems should behave securely, end-to-end
Ruby - Gauntlt
Python - Mittn
Java - BDD-Security


 

References

https://erik.doernenburg.com/2008/11/how-toxic-is-your-code/

https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis

https://github.com/collections/clean-code-linters

https://realpython.com/python-code-quality/

https://www.aspectsecurity.com/uploads/downloads/2012/03/aspect-security-the-unfortunate-reality-of-insecure-libraries.pdf

http://softwaretestingfundamentals.com/integration-testing/

https://martinfowler.com/bliki/IntegrationTest.html

https://www.youtube.com/watch?v=hQyXgKENDtg&t=2321s

https://developer.ibm.com/tutorials/d-bbd-guide-iac/

https://www.softwaretestinghelp.com/behavior-driven-development-bdd-tools/

https://docs.angularjs.org/guide/unit-testing

https://scotch.io/tutorials/testing-angularjs-with-jasmine-and-karma-part-1

https://reactjs.org/community/testing.html

https://www.toolsqa.com/selenium-cucumber-framework/convert-selenium-test-into-cucumber-bdd-style-test/

https://www.guru99.com/what-is-security-testing.html