Notes about continuous and delivery and my thoughts.
- for Docker use Alpine linux image (5MB)
- for Docker use not specific cmds like apt ...
- for Docker use .Dockerignore file to small down context
- if needed to reuse more images in one image use Docker multi stage builds
- simple ci /cd jobs / processes
- little number of inputs
- system inputs not as inputs (hide to system)
- not so variable inputs to properties file or similar
- dont copy jobs / processes / try reuse them.
- minimize boilerplate code (with librries, frameworks, bytecode manipulations ...)
- config files as yaml
- kubernetes yaml ConfigMaps
- Spring boot knows yaml too
- there is way to create kubernetes yaml from properties files.
- ConfigMaps and Secrets to have less error prone ad more portable solution.
env:
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: database-secret-config
key: dbname- "no business logic" to code that runs runtime annotations (validations, checks, logging)
- dependency architecture should follow Dependency_inversion_principle
- old parts of software should go with old technological stack
- old software should start to pick up new strategies and code (testing, architecture, best practises)
- new software stack
- Vue.js
- Vannila.js
- Spring Boot (reactive)
- AWS and Kubernetes EKS https://aws.amazon.com/eks/
- Artifactory, +Helm
- In future look at OpenJDK Fibers 1, 2, 3 (as opposite to heavy Threads)
- Junit 5 over Junit 4
- Additional layer represented by Spring Boot RESTs. (for simplification completely from scratch)
- old as @Controller (no web server) / new as @RestController (web server)
- old and new as @RestController (web server)
- all acceptance tests should go to this layer (new)
- just Edge services, no other service layer and additional inter-communication needed.
- layer will be foundation for simplier and safer migration to new stack if needed in future.
- layer dockerized prepared for Kubernetes.
- layer documented with Swagger / JsonDoc.
- layer will have simple run interface (generated). No code require.
- VPS UI has extended UI logic therefore not easy to go with simple gnerated one (JsonDoc, Swagger)
- SpringBoot REST as foundation for new applications (client) that will communicate with REST calls (server).
- Avoiding VPS-UI - Control-M communication with SpringBoot REST communication mentioned before ( 1 app per 1 product )
- Kubernetes Job One per product. Multithreading is done in java implementation. No need for job parallelization.
- Migration (1L) :
- Properties to new layer
- Runtime from new layer.
- Migration (2L) :
- All existing product to newer stack. (not planned, time consuming, error prone.) -> stop developing in old way imidiatelly as first step.
There was just one permanent branch master. That was not enough. Development and relese and bug fixing cannot be done on just one branch without any code or submit (merge) freeze.
- avoid any freeze
- keep still Git flow simple as possible
Current:
master
|
|->|
|<-| |
| |<-|
|->|
|
|
|
- one permanent branch master
- several local branches (not long term)
New:
master (develop) release (master)
| |
| | ->|
|<-| | | bug fix, hot fox
impl. | |<...........| <-| TAG 1
| | |
|->|...........>| TAG 2
| |
| |
| |
| |
- two permanent branches
- https://medium.com/goodtogoat/simplified-git-flow-5dc37ba76ea8
- standard git flow is too complicated for team purposes http://blog.qxconsultant.com/2015/10/software-development-git-flow-overview.html
Formatter (Eclipse Formatter) was used manually by each developer in IDE.
- usage of formatter-maven-plugin with examples
- can be in separate maven profile.
- triggered after each commit (Pre commit builds)
- config xml file should go to separate git repo, or be on server, or in kuberbetes available for all builds (as url ...).
- TODO can configFile work with http ? if not scm plugin etc.
simple code quality check was missing
- after each commit (Pre commit builds) shoud be simple code quality check with Sonar
- if commit not pass all checks pre commit build FAIL
- Git + Sonar
TODO Sonarcube, PMD, CheckStyle, FindBugs
- pre-submit : simple Sonar checks
- method length
- line length
- class methods count
- implementation has at least test
- java conventions
- post-submit : all checks (Sonar, Checkstyle, PMD, ...)
- compile
- analysis
- code coverage
- test coverage
- static code analysis
- dynamic code analysis
- tests
- junit + integration
- little inspiration
Exist external documentation (maybe not up to date, very granular)., code documentation weak.
- Swagger
- Check Javadoc if is missing, triggered after each commit (Pre commit builds)
- Create simple small Annotation library to make lightweight documentation for old code parts at least.
@Job(name = “alpha”, inputNames = [ “a”, “b”, “c‘ ], outputs = “ppp”)
@JobStep(name = “alpha-step-name1”, inputNames = [ “a”, “b”, “c‘ ], outputs = “ppp”, readsToExt=[“”, “”], writesToExt= [“”, “”])- Annotation as markers for start but they can be extended to do somethign usefull in future.
- make documentation for new pipelines. TODO : maybe Gitlab can generate something usefull. Otherwise use something like PlantUML.
This was present also in previous version.
- Release Notes Confluence
- Release Tracking Wiki Confluence
- Jira Release Ticket
- Confluence API https://developer.atlassian.com/server/confluence/confluence-rest-api-examples/
- JIRA API https://developer.atlassian.com/server/jira/platform/rest-apis/
TODO : How it was in previous versions of CI ? , TODO : What client need ? Exist plugin for that ?
- SNAPSHOT to snapshot Artifactory repository, RELEASE (no SNAPSHOT) to release Artifactory repository
- also SNAPSHOT and RELEASE artifacts stay in repository to work with them
- multi-version software, doftware is not multi-module !
- if top pom version is changed to RELEASE all calls in dependencies section should be changed is same way ? (We want to have newest versions automatically ? ) => Developer should never change SNAPSHOT to RELEASE any more !!
- if so (and no plugin exists than -> ) update-child-modules but it should work also for non multi module project.
mvn dependency:list | grep -e <artifactID-or-groupID> -e <artifactID-or-groupID>
git clone ssh://<USERNAME>@gerrit.wikimedia.org:29418/mediawiki/core.git
# for each pom ->
# change them [(not parsing!)](https://stackoverflow.com/questions/4838591/is-there-a-library-for-reading-maven2-3-pom-xml-files) use for each maven version or maven release plugin.
# commit changes
# push changes-
to have CICD - there is approach to change version righ after master merge but ... - no all changed new version software will be released. It is possible to that will crash before release. - master is releasable that is the rule
-
we used approach that after release, the vesion were increased - multiple master merges have same version !! - each version change is release - need to have additional version change for master merge but more if version needs to be reverted (?)
TODO : Describe manual release steps and make them automatic
- release == deployment to production
- see also version change section
TODO: Staging ? Reason ? It is very slow and human prone.
- Staging should be presented as staging environment with automatic tests running on our side.
- We should definitely test software with external resources (Integration testing, SAT ...)
- Existing Smoke tests :
- avoid or automate them
- dockerize them
- kubernetes : container in pod with application
- SpringBoot layer should contain all acceptance tests and other higher tests.
- do not deploy end packages which are not dependencies for others like ( REST services, UI apps ... ) to repository (artifactory).
- deploy just dependencies.
- after master merge build there should be a propagation of builded package to the next steps no other builds required !
- so you are building your app just once.