ConcourseNotes.txt

not much resource managment as everything is containers
decleratives in nature

What is Concourse:

DEF: 

Concourse “pipelines” are YAML files that declare resources to use, e.g. Git repos or Docker images, and contain a set of jobs to execute. In turn, jobs are sub-divided into tasks and each task runs in a container. Some readers may be surprised to learn that Docker and containerisation aren’t synonymous. Concourse uses Garden containers and when a task needs Docker, e.g. to build a Docker image, Docker runs in a Garden container!

Why Concourse:

1. Completely container based. --> Create once use many times, resourece management overhead is not there
2. Pipelines are declerative. 
3. No Plugin systems --> it depends on something called resources ->eg: git resource https://github.com/telia-oss/github-pr-resource
-----------------------------------------------------------------------------


To install fly.
Download the fly for linux and execute below command.
sudo install ./fly_linux_amd64 /usr/local/bin/fly
docker-compose up -d
fly --target tutorial login --concourse-url http://<ip>:8080 -u admin -p admin
fly -t tutorial workers
fly -t tutorial pipelines
fly -t tutorial builds



install the docker and docker compose:
try installing the docker compose using the docker documentiontation. if fails, download the dockercompose ile from the github and run the command
eg: I have downloaded the file "docker-compose-Linux-x86_64" from git hub https://github.com/docker/compose/releases.
once downloaded run the below command
sudo mv docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
Test the installation by running the below command 
docker-compose -version
after running you should see the version info.

After this follow the tutorial from the URL : https://concoursetutorial.com/ [fly install instructions are also provided]

To access the concourse UI from outside the host machine use the param CONCOURSE_EXTERNAL_URL=http://<Docker Host IP>:8080

fly commands:

fly -h : help

fly sync : eg fly --target tutorial sync

fly login : eg fly --target example login --team-name my-team --concourse-url https://ci.example.com

fly execute: eg fly -t tutorial execute -c task_hello_world.yml [ to execute a task file, -t is a shortform or --target, e is a shortform for execute]
			 eg fly -t tutorial e -c task_hello_world.yml

fly targets : To see what targets are currently known to fly, run: this will show each target's name, URL, and when its token expires.

fly  status : to see the status of the target : eg: fly -t tutorial status


t : target
e : execute
sp : set-pipeline
c : config

options -n in fly set-pipeline command is for non-interactive. eg: fly -t targetname set-pipeline -p tutorial-pipeline -c pipeline.yml -n

to set a pipeline: 
fly -t targetname set-pipeline -p hello-world -c pipeline.yml

to unpause the pipeline:

fly -t tutorial up -p publishing-outputs

to trigger a job
fly -t tutorial trigger-job -j hello-world/job-hello-world

to watch a job posts trigger
fly -t tutorial watch -j hello-world/job-hello-world

to trigger a job and watch
fly -t tutorial trigger-job -j hello-world/job-hello-world -w

for concourse to use local registry 

https://github.com/pivotalservices/concourse-pipeline-samples/tree/master/concourse-pipeline-hacks/private-docker-registry

https://github.com/pivotalservices/concourse-pipeline-samples


fly --target tutorial login --concourse-url http://<ip>:8080 -u admin -p admin
fly --target tutorial sync
fly -t tutorial set-pipeline -p buildtest -c sample.yml -l credentials.yml
fly -t tutorial up -p buildtest
fly -t tutorial check-resource -r buildtest/prometheus-main
fly -t tutorial trigger-job -j buildtest/job-clone-check -w
fly -t tutorial watch -j buildtest/job-clone-check
fly -t tutorial destroy-pipeline -p buildtest
fly -t tutorial intercept -j buildtest/job-clone-check
fly -t tutorial containers



useful-resources:

https://web.archive.org/web/20171205105324/http://concourse.ci:80/versioned-s3-artifacts.html
https://github.com/18F/s3-resource-simple
https://github.com/concourse/s3-resource
https://github.com/pivotalservices/concourse-pipeline-samples
https://concoursetutorial.com/
http://concoursetutorial.com/miscellaneous/versions-and-buildnumbers/
https://github.com/cloudfoundry/infrastructure-ci
https://specify.io/how-tos/concourse-ci-continious-integration-and-delivery-of-microservices
https://github.com/EngineerBetter/concourse-up
https://github.com/concourse/cf-resource
https://cloud.spring.io/spring-cloud-pipelines/multi/multi_concourse-pipeline-cf.html
https://github.com/spring-cloud/spring-cloud-pipelines
https://github.com/EngineerBetter/concourse-up
https://github.com/cloudfoundry/bosh-deployment-resource

https://medium.com/@kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd

https://github.com/pivotalservices/concourse-pipeline-samples/wiki/Concourse-Samples-and-Recipes
http://engineering.pivotal.io/post/compiled-releases-for-pipelines/

https://github.com/EngineerBetter/control-tower/blob/master/ci/pipeline.yml

http://www.mikeball.info/blog/concourse-git-resource/
https://github.com/EngineerBetter/weavescope-boshrelease

https://blog.betomorrow.com/yaml-tips-and-tricks-for-concourse-ci-35a3b577a239

https://github.com/EngineerBetter/stopover
https://itnext.io/writing-a-custom-resource-for-concourse-detecting-pull-request-close-merge-events-e40468eb2a81
https://github.com/EngineerBetter/fly-github-auth-task
https://github.com/telia-oss/github-pr-resource