rasheedamir/Jenkins CI on EC2.md

## Jenkins CI on EC2.md

      
    Raw
  

              Jenkins CI on EC2.md
            
          
    The practice of automated continuous deployment ensures that the latest checked in code is deployed, running, and accessible to various roles within an organization.
Jenkins is a popular continuous integration server that is easy to install and configure
Continuous Delivery is about automation of the software development process. It combines four core disciplines:

continuous integration,
continuous testing,
continuous deployment and
continuous inspection

Continuous integration has become part of the software development process in most companies. Apart from building software artefacts it also contains version control and dependency management. Continuous testing has become more and more popular as a result of better integration in CI servers like Jenkins, CruiseControl and Bamboo. Continuous deployment on the other hand has become a bottleneck for releasing software since development and operation teams are historically separated within most organizations.
How do we measure code quality?
Measuring quality is about applying the concept of Continuous Inspection. Just like continuous integration makes sure your project builds and all tests pass, continuous inspection is about looking at your code and notify you when the quality decreases. Both Jenkins and Sonar have the ability to give a trend of what is going on over time.
Continuous Integration with Jenkins on Amazon EC2

Follow these tutorials:

Install Jenkins

Tools


Jenkins
Apache Tomcat Server
MySQL
Apache HTTP Server
Java
Maven
Nodejs & NPM
Bower
Git
Grunt

Connect Server


Given that OS is Ubuntu AMI [Amazon Machine Image] (these are tested on Ubuntu Server 14.04 LTS)
Download and save key pair (amazon server key pair i.e. pem file) to some location on your computer.
Change permission so, that the pem file is not accessible to others run: chmod 700 <file-name>.pem
Make a SSH script file (create a text file with extension .sh - you can name it e.g. connect--.sh). And copy following content in it.
ssh -i <path-to-your-pem-file> ubuntu@<server-name> (e.g. ssh -i /home/myproject/myproject-dev.pem ubuntu@myproject.com)
Make the script executable. Run this command in the terminal: chmod +x connect-<project-name>-<environment-name>.sh
Open the terminal and run the script: ./connect-<project-name>-<environment-name>.sh
If everything goes well then you should be able to connect the server!

Installing Jenkins

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
echo "deb http://pkg.jenkins-ci.org/debian binary/" | sudo tee -a /etc/apt/sources.list.d/jenkins.list
sudo apt-get update
sudo apt-get install jenkins
Verify jenkins is running: ps -ef | grep jenkins
Installing and Configuring Apache

Installing Apache

sudo apt-get install apache2
sudo a2enmod proxy
sudo a2enmod proxy_http


Try to access the dashboard of jenkins; simply put the server name in the browser and see what you get

Configuring Jenkins

Update Jenkins Configuration

Edit sudo vim /etc/apache2/sites-available/jenkins.conf and add following:
<VirtualHost *:80>
	ServerName HOSTNAME
	ProxyRequests Off
	<Proxy *>
		Order deny,allow
		Allow from all
	</Proxy>
	ProxyPreserveHost on
	ProxyPass / http://localhost:8080/
</VirtualHost>
Enable Jenkins Configuration

sudo a2ensite jenkins
sudo service apache2 reload
Enable Jenkins Security

Manage Jenkins --> Configure Global Security

Click on Enable Security
** Jenkin's own user database
*** (un-select allow users to signup)
Logged in users can do anything
Click Save

This will take us to login page! Jenkins does allow to signup first user; click on Jenkins and it will show you signup page. It only allow's to sign up first time and never afterwards!
Installing MySQL

To install MySQL, run the following command from a terminal prompt:

sudo apt-get install mysql-server

During the installation process you will be prompted to enter a password for the MySQL root user.

username / password : root / root

Once the installation is complete, the MySQL server should be started automatically. You can run the following command from a terminal prompt to check whether the MySQL server is running:

sudo netstat -tap | grep mysql

When you run this command, you should see the following line or something similar:

tcp        0      0 localhost:mysql         *:*                     LISTEN      2759/mysqld

Create an empty schema and a fmu user. Grant this fmu user permissions to create, update and delete objects for this schema. The charset of the database has to be set to "UTF-8" and the language (database and user) to "English".

Here is a MYSQL Script
To run the script: sudo mysql -u root -p < create_database.sql
To verify the user has been created: SELECT User FROM mysql.user; you should see user with name fmu
To verify the database has been created: show databases; you should see database with name fmu

Installing Tomcat


Create directory "tools" in home directory: sudo mkdir tools


Move to tools directory: cd tools


Run this command to download tomcat as tar (zip) file:


`sudo wget http://www.us.apache.org/dist/tomcat/tomcat-7/v7.0.55/bin/apache-tomcat-7.0.55.tar.gz`

or

`sudo wget http://apache.mivzakim.net/tomcat/tomcat-7/v7.0.57/bin/apache-tomcat-7.0.57.tar.gz`


Then unzip the file: sudo tar xzf apache-tomcat-7.0.55.tar.gz


Move to tomcat directory: cd apache-tomcat-7.0.55


Change tomcat default port from 8080 to 8181: cd conf then sudo nano server.xml then change <Connector port="8080" protocol... to <Connector port="8181" protocol...  then save it by pressing CTRL+X. ENSURE! that port 8181 is open on the server; confirm from the guy who had setup the instance


Increase permgem space. Change directory to cd apache-tomcat-7.0.55\bin then create a new file named setenv.sh in the same directory using the following command: sudo nano setenv.sh then the file would be created and opened in terminal. Paste the following piece of code export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xms256m -Xmx1024m -XX:PermSize=512m -XX:MaxPermSize=512m" then close the file by pressing Ctrl + x and then press Y and Enter


To start the tomcat execute: sudo ./startup.sh


To verify tomcat is running then execute: ps -ef | grep tomcat you should see an output like


root     20537     1 49 21:43 pts/0    00:00:05 /usr/bin/java -Djava.util.logging.config.file=/home/ubuntu/tools/apache-tomcat-7.0.55/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dfile.encoding=UTF-8 -Xms256m -Xmx1024m -XX:PermSize=512m -XX:MaxPermSize=512m -Djava.endorsed.dirs=/home/ubuntu/tools/apache-tomcat-7.0.55/endorsed -classpath /home/ubuntu/tools/apache-tomcat-7.0.55/bin/bootstrap.jar:/home/ubuntu/tools/apache-tomcat-7.0.55/bin/tomcat-juli.jar -Dcatalina.base=/home/ubuntu/tools/apache-tomcat-7.0.55 -Dcatalina.home=/home/ubuntu/tools/apache-tomcat-7.0.55 -Djava.io.tmpdir=/home/ubuntu/tools/apache-tomcat-7.0.55/temp org.apache.catalina.startup.Bootstrap start


To stop the tomcat execute: sudo ./shutdown.sh 


Verify! that you can access the tomcat home page... you can find more details here: https://gist.github.com/rasheedamir/d6be4e42155d200a0b94


Installing Java, Maven, Git, Nodejs & NPM, Bower, Grunt

Installation

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer maven git-core

sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

sudo npm install -g bower
sudo npm install -g grunt-cli
Verification

Verify!

javac -version
mvn -version
git --version
npm --version
node --version
bower --version
grunt --version
Configuring Jenkins

Install NodeJS Plugin

From the Jenkins dashboard, navigate to 'Manage Jenkins' and then 'Manage Plugins'.

NodeJS Plugin - This will install Node, Bower & Grunt

Add JDK Information


From the Jenkins dashboard, navigate to Manage Jenkins and then Configure System.
Click Add JDK and enter Name and JAVA_HOME.
Click Apply

Get path to javac (go to terminal and run following):
readlink -f /usr/bin/javac
or
java -version
Add Maven Information


From the Jenkins dashboard, navigate to Manage Jenkins and then Configure System.
Click Add Maven and enter Name and MAVEN_HOME.
Click Apply

Get path to maven (go to terminal and run following):
readlink -f /usr/bin/mvn
or
mvn -version
Add Git Information


From the Jenkins dashboard, navigate to Manage Jenkins and then Manage Plugins.
Click on the Available tab and search using the browser (CTRL+F) for git plugin.
Check Git Plugin and click Download now and install after restart. The install can take a while.
From the Jenkins dashboard, navigate to Manage Jenkins and then Configure System.
Add Git information Name & Path to git executable
Add Git user and email info under section Git Plugin.

Get path to git:
???
Configure a Jenkins Job


From the Jenkins dashboard, navigate to New Job.
Enter a 'Job Name' and select Build a maven2/3 project. Click OK.
Enter a Description.
Select Discard Old Builds.
Enter an amount by Max # of builds to keep.
Select Git and enter a Repository URL. If you wanted to track a git repo on Github, this is where you would enter the git URL.
Check Poll SCM and add * * * * * to Schedule. * * * * * is a cron expression which means every minute.
Under Build add a Maven goal in Goals and options.

???

Click Save.

Helpful Jenkins Plugins


Cloudbees Folders Plugin : This plugin allows users to create "folders" to organize jobs. Users can define custom taxonomies (like by project type, organization type etc). Folders are nestable and you can define views within folders.

Junk!

Manage Jenkins -> Configure System

JDK
Git
Maven
E-mail Notification ??

Continuous Delivery in the Cloud
New Item


Provide name
Select git
Open terminal
'sudo su - jenkins'
'ssh-keygen -t dsa' (Accept defaults...)
cd .ssh
cat id_dsa.pub
git ls-remote -h git@github.com:rasheedamir/fmu.git HEAD

Deployment with Apache Tomcat

The Tomcat manager application allows remote deployment to an instance of Tomcat. In this case, you need to configure Tomcat to allow access to the manager application through the plain text interface. This is accomplished by assigning the manager-script role to the credentials that will be performing deployments. This configuration depends on which Realm implementation you are using. The following example is utilizing the default MemoryRealm which reads an XML formatted file stored at $CATALINA_BASE/conf/tomcat-users.xml.
<tomcat-users>
	<user password="tomcat" roles=" manager-script" username="tomcat">
</user></tomcat-users>
Next, you need to configure your job in Jenkins to utilize the manager interface in order to deploy your recently built application. This is configured using the Post Build task plugin.
You can use wget, curl, or a similar tool to call the manager HTTP interface. It is necessary to first undeploy the application before deploying the new build. This avoids issues with deploying to an existing context path. The location of the newly built application war file can be determined by examining the output from a previous Jenkins job execution for that job.
wget "http://tomcat:password@localhost:8080/manager/text/undeploy?path=/spring-travel" -O - -q
wget "http://tomcat:password@localhost:8080/manager/text/deploy?path=/spring-travel&war=file:<path to="" war="">" -O - -q </path>
Easy Installation

Jenkins comes as a WAR file (there are also a wide variety of Linux packages available and a Windows installer) that you can drop into your favourite JEE container (Tomcat, Jetty, Glassfish etc.) or you can start the WAR file directly with 'java -jar jenkins.war'. That’s it! Now that it is up and running, you can start configuring jobs and installing plugins. It will use a folder in your $HOME to persist the configuration and store the actual jobs. No database, no special installation, no special registration. Clean and easy.