DevOps Tools/Integration/Jenkins Gitlab

Overview | Continuous Integration (CI) | Source Control Management (SCM) | Containerization | Configuration | Integration

Jenkins with Gitlab

referencereference#2
Make sure that you have Jenkins and Gitlab installed.

Configure Gitlab

You will need to have at least (1) project setup, though there doesn't need to be anything in it.

• Log-in to the Gitlab web portal > [click] Projects > [click] New project > give it:

• Project Name
• Description
• Visibility level (i've chosen private)

Next you'll need to add a deploy key, which is found by:

• [click] on the project name > go to Settings > [click] Repository > [click] Expand next to Deploy Keys > and define the following:

• Title - pick a easy description that will help identify it as the public ssh key from your jenkins server
• Key - Paste the public ssh key from your jenkins server.
• Write access allowed - make sure to check the box to allow.

Finally you will need to create a Jenkins user on Gitlab and then create a API token to provide Jenkins.

• First create a new user, in this case I called it Jenkins, and then assign them as a maintainer to the previously created project.
• Log into Gitlab as the new user
• Next go to the user Settings > [click] Access Tokens > and define:

• Name - define a name to easily remember what user this is for.
• Scopes - I opted to just check api

When you create the access token, it will be displayed on the webpage. NOTE: the api key cannot be recovered after you leave this page. Make sure to copy the key and potentially put it somewhere secure, like a keepass database.

Configure Jenkins

First thing we need to do is give Jenkins a copy of the Private SSH key that was created for the jenkins user on the jenkins server.

• Log-in to the Jenkins web portal > [click] credentials > [click] system > [click] Global credentials > [click] Add credentials

Obviously you can create a domain for a greater level of security instead of using the unrestricted global credentials group

• Kind - select SSH Username with private key
• Scope - select the appropriate scope, in my instance i selected global so that all project have access.
• Username - input the Username of the user created on Gitlab which you obtained an access token from and that has access to the right project
• Private Key - I only had the option to enter directly as this was the Jenkins Master. Paste the private key you created from the Jenkins user on the Jenkins server.
• Description - Enter in something that can help you easily identify it in the future.

Now we want to give Jenkins the access token from gitlab.

• Log-in to the Jenkins web portal > [click] credentials > [click] system > [click] Global credentials > [click] Add credentials

• Kind - select GitLab API token
• API token - paste the access token from gitlab
• Description - you know what to do

We need to configure the connection between Jenkins and Gitlab.

• Log-in to the Jenkins web portal > [click] Manage Jenkins > [click] Configure System
• Under Gitlab, input the following:

• Connection name - define a name, like gitlab
• Gitlab host URL - define the gitlab URL, mine was http://gitlab01.r00tedvw.com/gitlab
• Credentials - select the Gitlab API Token that you created in Jenkins earlier.

And now you should be able to successfully Test Connection
Make sure to [click] Save at the bottom of the page (footer frame).

NOTE: I was having problems getting Jenkins to connect to Gitlab and it turned out to be because I set the hostnames and URLs for both Jenkins and Gitlab to localhost (both were running as vms within virtualbox with port forwards). After rebuilding both from scratch and defining FQDNs with host file entries for DNS, the gitlab connection was successful.

Webhook

Now we want to setup a really basic webhook that will mark commits in gitlab as successful. This is just an example, its not a very useful webhook.

Jenkins

Jenkins > [click] New Item > Specify item name > [click] Freestyle project General

• GitLab Connection

[select] Gitlab from the drop down

• GitLab Repository Name

enter in your repository name. Mine was project01

Source Code Management

• Git

[select]

• Repository URL

the URL of the remote repo. Same syntax as used with git clone. Mine was [email protected]:user/project01.git

• Credentials

the credentials of the Jenkins user that you added earlier.

• (Advanced) Name

the ID of your repo. same name as you would use in git remote. Mine was origin

• (Advanced) Refspec

I have no idea what this stuff does yet. Mine was +refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*

• Branches to build

specify the branch you'd like to track. In my instance, I let Gitlab pass the branch name to jenkins as a variable, so I specified that instead. Mine was origin/${gitlabSourceBranch}

Build Triggers

• [select] Build when a change is pused to GitLab. GitLab webhook URL:

Make sure to copy the GitLab webhook URL, you'll need it later.

• Push Events

[select] check box

• Opened Merge Request Events

[select] check box

• Approved Merge Requests (EE-only)

[select] check box

• Comments

[select] check box

• (Advanced) Secret Token

[click] Generate and copy Jenkins Secret Token somewhere you can use it later.

Post-Build Actions

• [click] Add post-build-action > [select] Publish build status to Gitlab

Troubleshooting

While I was specifying the Git Repo URL in the Source Code Management section, I kept getting errors citing access failures to local resources on the Jenkins server. Checking SELinux, I found it was blocking git and tomcat from accessing resources, so I made the following changes:

SELinux Policy

I had to add an extra policy to allow git and tomcat access to resources.
I wanted to restrict changes to SELinux to only what was necessary, so I opted to repeat these steps until all

~$ sudo ausearch -m AVC,USER_AVC,SELINUX_ERR -ts recent    
     #look at recent SELinux events and find the ID of the denial.
~$ sudo grep 1547056915.268:2946 /var/log/audit/audit.log | audit2allow -a -M local-git2     
     #with the denial ID, use audit2allow to create a new set of policy files (one TE for human readable and one PP for an encoded version that can be inserted).  You can create a fresh one for each restriction you find and then manually combine them together.
~$ cat local-git2.te
     #Look at the TE file and (if you have an existing one already), compare it to see what needs to be added.
~$ vim local-git.te
     #Manually merge the changes you saw in the previous TE file.
~$ sudo checkmodule -M -m -o local-git.mod local-git.te
     #create the module from the TE file
~$ sudo semodule_package -o local-git.pp -m local-git.mod
     #create the PP file from the Module
~$ sudo semodule -i local-git.pp
     #Install the new policy file.

The resulting policy file was:

~$ cat local-git.te

module local-git 1.0;

require {
	type tomcat_t;
	type ssh_exec_t;
	type tomcat_cache_t;
	type tomcat_var_lib_t;
	class file execute;
	class file execute_no_trans;
	class file getattr;
	class file read;
	class file open;
}

#============= tomcat_t ==============
allow tomcat_t ssh_exec_t:file execute;
allow tomcat_t tomcat_cache_t:file execute;
allow tomcat_t tomcat_cache_t:file execute_no_trans;
allow tomcat_t ssh_exec_t:file getattr;
allow tomcat_t ssh_exec_t:file read;
allow tomcat_t ssh_exec_t:file open;
allow tomcat_t ssh_exec_t:file execute_no_trans;
allow tomcat_t tomcat_var_lib_t:file execute;
allow tomcat_t tomcat_var_lib_t:file execute_no_trans;

SELinux Boolean

I also had to enable a boolean

~$ sudo getsebool -a | grep nis_enabled
~$ sudo setsebool -P nis_enabled on
~$ sudo getsebool -a | grep nis_enabled

GitLab

Now we need to add a webhook to gitlab so that gitlab can trigger the Jenkins Job.
Gitlab > [select] the Project > [select] Integrations

• URL

[insert] GitLab webhook URL (which you should have copied earlier).

• Secret Token

[insert] Jenkins Secret Token (which you should have copied earlier).

• Trigger

  Push Events

[select] check box

• Merge request events

[select] check box

• SSL verification

[select] check box (Enable SSL verification) if Jenkins is not using SSL.

Troubleshooting

I encountered an error where GitLab could not connect to Jenkins because it was within the local network. In Gitlab, you have to specifically allow requests to the local network from hooks and services.

• GitLab > [click] More > [click] Admin Area > [select] Settings > [click] Network > [select] Expand next to Outbound requests > [select] check box for Allow requests to the local network from hooks and services