DevOps Tools/Configuration/Ansible
(→Install base packages, independent on package manager) |
(→Installation) |
||
(15 intermediate revisions by one user not shown) | |||
Line 1: | Line 1: | ||
− | [[DevOps_Tools/Configuration|Configuration]]<br | + | [[DevOps_Tools/Overview|Overview]] | [[DevOps_Tools/CI|Continuous Integration (CI)]] | [[DevOps_Tools/SCM|Source Control Management (SCM)]] | [[DevOps_Tools/Containerization|Containerization]] | [[DevOps_Tools/Configuration|Configuration]] | [[DevOps_Tools/Integration|Integration]]<br> |
− | [[DevOps_Tools/Configuration/Ansible|Ansible]] | + | [[DevOps_Tools/Configuration/Ansible|Ansible]] | [[DevOps_Tools/Configuration/Ansible/Playbook_Examples|Playbook Examples]] |
− | =Installation= | + | =[[DevOps_Tools/Configuration/Ansible|Ansible]]= |
+ | ==Installation== | ||
Ansible does not require any server component, no daemon needs to be running. Ansible runs over SSH.<br/> | Ansible does not require any server component, no daemon needs to be running. Ansible runs over SSH.<br/> | ||
Where ever you are going to run your playbooks from, needs to have ansible installed, the clients do not. <br/> | Where ever you are going to run your playbooks from, needs to have ansible installed, the clients do not. <br/> | ||
To get ansible 2.0+, at the time of this writing, use the epel repo | To get ansible 2.0+, at the time of this writing, use the epel repo | ||
− | <nowiki>~$ sudo yum install epel-release | + | <nowiki>~$ sudo yum install -y epel-release |
− | ~$ sudo yum install git python python-devel python-pip openssl ansible | + | ~$ sudo yum install -y git python python-devel python-pip openssl ansible |
~$ ansible --version | ~$ ansible --version | ||
ansible 2.7.0 | ansible 2.7.0 | ||
config file = /etc/ansible/ansible.cfg | config file = /etc/ansible/ansible.cfg | ||
− | configured module search path = [u'/home/ | + | configured module search path = [u'/home/r00t/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] |
ansible python module location = /usr/lib/python2.7/site-packages/ansible | ansible python module location = /usr/lib/python2.7/site-packages/ansible | ||
executable location = /usr/bin/ansible | executable location = /usr/bin/ansible | ||
python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]</nowiki> | python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]</nowiki> | ||
− | =Configuration= | + | |
+ | ==Configuration== | ||
Basic configuration, define the inventory path & specify the sudo user | Basic configuration, define the inventory path & specify the sudo user | ||
<nowiki>~$ sudo vim /etc/ansible/ansible.cfg | <nowiki>~$ sudo vim /etc/ansible/ansible.cfg | ||
Line 21: | Line 23: | ||
inventory = /etc/ansible/hosts | inventory = /etc/ansible/hosts | ||
... | ... | ||
− | + | [privilege_escalation] | |
+ | become=True | ||
+ | become_method=sudo | ||
+ | become_user=root | ||
+ | become_ask_pass=False</nowiki> | ||
Nodes are determined by their names. These live in the inventory hosts file: <code>/etc/ansible/hosts</code>. By default there is a file there that can be used as an example. | Nodes are determined by their names. These live in the inventory hosts file: <code>/etc/ansible/hosts</code>. By default there is a file there that can be used as an example. | ||
<nowiki>~$ sudo vim /etc/ansible/hosts | <nowiki>~$ sudo vim /etc/ansible/hosts | ||
Line 34: | Line 40: | ||
<br/> | <br/> | ||
− | ==Ansible User== | + | ===Ansible User=== |
Ansible needs to run as a non-privileged user with sudo rights. It also needs to be able to run commands without specifying a password as the playbooks will fail with password prompts.<br/> | Ansible needs to run as a non-privileged user with sudo rights. It also needs to be able to run commands without specifying a password as the playbooks will fail with password prompts.<br/> | ||
'''NOTE:''' This will need to be done on ''EACH Node''. | '''NOTE:''' This will need to be done on ''EACH Node''. | ||
Line 45: | Line 51: | ||
ansible ALL=(ALL) NOPASSWD: ALL</nowiki> | ansible ALL=(ALL) NOPASSWD: ALL</nowiki> | ||
− | ==SSH key exchange== | + | ===SSH key exchange=== |
You can opt to either: | You can opt to either: | ||
*define the SSH password for ansible in the playbook | *define the SSH password for ansible in the playbook | ||
Line 57: | Line 63: | ||
~$ ssh-copy-id [email protected]</nowiki> | ~$ ssh-copy-id [email protected]</nowiki> | ||
− | =Basic Commands= | + | ==Basic Commands== |
You can specify the '''group''' or '''all''' to apply the ansible command to. | You can specify the '''group''' or '''all''' to apply the ansible command to. | ||
<nowiki>~$ ansible all -s -a "ls -la /etc/ansible"</nowiki> | <nowiki>~$ ansible all -s -a "ls -la /etc/ansible"</nowiki> | ||
Line 64: | Line 70: | ||
;-a | ;-a | ||
:this flag runs a specified command defined after it. | :this flag runs a specified command defined after it. | ||
− | ==copy file to node== | + | ===copy file to node=== |
<nowiki>~$ ansible group1 -m copy -a "src=/home/ansible/test.txt dest=/tmp/test.txt"</nowiki> | <nowiki>~$ ansible group1 -m copy -a "src=/home/ansible/test.txt dest=/tmp/test.txt"</nowiki> | ||
− | ==install & remove package== | + | ===install & remove package=== |
<nowiki>~$ ansible group1,group2 -m yum -s -a "name=lynx state=latest" | <nowiki>~$ ansible group1,group2 -m yum -s -a "name=lynx state=latest" | ||
~$ ansible group1,group2 -m yum -s -a "name=lynx state=absent"</nowiki> | ~$ ansible group1,group2 -m yum -s -a "name=lynx state=absent"</nowiki> | ||
− | ==run local command across nodes== | + | ===run local command across nodes=== |
<nowiki>~$ ansible env -m shell -s -a "firewall-cmd --get-active-zone"</nowiki> | <nowiki>~$ ansible env -m shell -s -a "firewall-cmd --get-active-zone"</nowiki> | ||
− | ==add firewalld rule== | + | ===add firewalld rule=== |
<nowiki>~$ ansible env -m firewalld -s -a "zone=public service=nfs permanent=true state=enabled" -vvv</nowiki> | <nowiki>~$ ansible env -m firewalld -s -a "zone=public service=nfs permanent=true state=enabled" -vvv</nowiki> | ||
− | ==modify service== | + | ===modify service=== |
<nowiki>~$ ansible env -m service -s -a "name=firewalld state=reloaded enabled=yes"</nowiki> | <nowiki>~$ ansible env -m service -s -a "name=firewalld state=reloaded enabled=yes"</nowiki> | ||
− | ==add new firewalld service== | + | ===add new firewalld service=== |
<nowiki>~$ ansible env -m shell -s -a "sudo firewall-cmd --permanent --new-service-from-file=/usr/lib/firewalld/services/nfs3.xml --name=nfs4" | <nowiki>~$ ansible env -m shell -s -a "sudo firewall-cmd --permanent --new-service-from-file=/usr/lib/firewalld/services/nfs3.xml --name=nfs4" | ||
~$ ansible env -m shell -s -a "sudo firewall-cmd --permanent --service=nfs4 --add-port=111/tcp" | ~$ ansible env -m shell -s -a "sudo firewall-cmd --permanent --service=nfs4 --add-port=111/tcp" | ||
Line 87: | Line 93: | ||
~$ ansible env -m service -s -a "name=firewalld state=reloaded enabled=yes"</nowiki> | ~$ ansible env -m service -s -a "name=firewalld state=reloaded enabled=yes"</nowiki> | ||
− | ==add nfs share to nodes== | + | ===add nfs share to nodes=== |
<nowiki>~$ ansible env -m shell -s -a "sudo mkdir /mnt/nfsshare" | <nowiki>~$ ansible env -m shell -s -a "sudo mkdir /mnt/nfsshare" | ||
~$ ansible env -m mount -s -a "state=mounted fstype=nfs path=/mnt/nfsshare src=ansible1.r00tedvw.local:/nfsshare"</nowiki> | ~$ ansible env -m mount -s -a "state=mounted fstype=nfs path=/mnt/nfsshare src=ansible1.r00tedvw.local:/nfsshare"</nowiki> | ||
− | =Playbooks= | + | ==Playbooks== |
+ | See what hosts the playbook would apply to:<br> | ||
+ | <code>~$ ansible-playbook {playbook.yml} --list-hosts</code> | ||
+ | <nowiki>$ ansible-playbook /etc/ansible/playbooks/centos_base_packages.yml --list-hosts | ||
+ | |||
+ | playbook: /etc/ansible/playbooks/centos_base_packages.yml | ||
+ | |||
+ | play #1 (centos): centos TAGS: [] | ||
+ | pattern: [u'centos'] | ||
+ | hosts (3): | ||
+ | jenkins01.ncvw.org | ||
+ | ansible01.ncvw.org | ||
+ | gitlab01.ncvw.org</nowiki><br> | ||
+ | |||
+ | Test Playbook | ||
+ | <nowiki>~$ ansible-playbook general.yml --syntax-check</nowiki> | ||
+ | Limit Host application | ||
+ | <nowiki>~$ ansible-playbook general.yml --limit ncwv-linux01.r00tedvw.local</nowiki> | ||
+ | Example: Running an ansible playbook against a specific host, specifying the host file location, providing root credentials, and the sudo password. (this also requires ansible.cfg to be located within the inventory folder and contain host_key_checking = false to avoid issues with SSH key authentication) | ||
+ | <nowiki>~$ ansible-playbook general.yml -i /home/ansible/ansible/hosts.yml --extra-vars "ansible_user=root ansible_password=Password ansible_sudo_pass=Password" --limit ncwv-linux01.r00tedvw.local</nowiki> | ||
+ | Prompt for sudo password | ||
+ | <nowiki>~$ ansible-playbook general.yml -i /home/ansible/ansible/hosts.yml --limit ncwv-linux01.r00tedvw.local --ask-become-pass</nowiki> | ||
+ | |||
Basic example: | Basic example: | ||
− | <nowiki>--- | + | <nowiki>--- |
− | - | + | - name: haproxy |
− | + | hosts: haproxy | |
− | + | become: yes | |
− | + | become_method: sudo | |
− | + | tasks: | |
− | + | - name: Install HAProxy on CentOS | |
− | + | yum: | |
− | + | name: haproxy | |
− | + | state: latest | |
− | + | - name: enable service haproxy and ensure it is not masked | |
− | + | systemd: | |
− | + | name: haproxy | |
− | + | enabled: yes | |
− | + | masked: no | |
− | - | + | - name: update haproxy conf |
− | + | blockinfile: | |
− | + | path: /etc/haproxy/haproxy.cfg | |
− | service: | + | block: | |
− | + | # [HTTP Site Configuration] | |
− | + | listen http_web 0.0.0.0:80 | |
− | ==Install base packages, independent on package manager== | + | mode http |
+ | balance leastconn # Load Balancing algorithm | ||
+ | option httpchk | ||
+ | option forwardfor | ||
+ | server apache01 ncrv-apache01:80 weight 1 maxconn 512 check | ||
+ | server apache02 ncrv-apache01:80 weight 1 maxconn 512 check | ||
+ | - name: open http | ||
+ | firewalld: | ||
+ | zone: public | ||
+ | service: http | ||
+ | permanent: yes | ||
+ | immediate: yes | ||
+ | state: enabled | ||
+ | - name: restart haproxy service | ||
+ | service: | ||
+ | name: haproxy | ||
+ | state: restarted</nowiki> | ||
+ | ===Install base packages, independent on package manager=== | ||
The package module can be used to install packages across distributions, but it has some serious limitations when compared to the yum and apt modules, primarily concerning the options to define state, version, etc. | The package module can be used to install packages across distributions, but it has some serious limitations when compared to the yum and apt modules, primarily concerning the options to define state, version, etc. | ||
<nowiki>- hosts: all | <nowiki>- hosts: all | ||
Line 140: | Line 185: | ||
name: nfs-utils | name: nfs-utils | ||
state: present</nowiki> | state: present</nowiki> | ||
+ | |||
+ | ==Troubleshooting== | ||
+ | ===urllib3 or chardet doesn't match supported version=== | ||
+ | Overview: When running <code>ansible-playbook</code>, I would get the below error.<br> | ||
+ | Error: | ||
+ | <nowiki>/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25.10) or chardet (2.2.1) doesn't match a supported version!</nowiki> | ||
+ | Resolution:<br> | ||
+ | <code>__init__.py</code> calls for the following package versions: | ||
+ | <nowiki>urllib3 >= 1.21.1, <= 1.25 | ||
+ | chardet >= 3.0.2, < 3.1.0</nowiki> | ||
+ | When I checked, I had the following installed: | ||
+ | <nowiki>~$ sudo pip list | ||
+ | ... | ||
+ | chardet (3.0.4) | ||
+ | urllib3 (1.25.10) | ||
+ | ...</nowiki> | ||
+ | First I downgraded <code>urllib3</code> to be equal or below 1.25 | ||
+ | <nowiki>~$ sudo pip uninstall urllib3 | ||
+ | ~$ sudo pip install --upgrade urllib3==1.25</nowiki> | ||
+ | This gave me an error citing that: | ||
+ | <nowiki>requests 2.24.0 requires urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you'll have urllib3 1.25 which is incompatible.</nowiki> | ||
+ | So after uninstalling and reinstalling different versions of <code>requests</code>, I found that <code>requests 2.21.0</code> works with versions of <code>urllib3</code> below 1.25.0 | ||
+ | <nowiki>~$ sudo pip uninstall requests | ||
+ | ~$ sudo pip install --upgrade requests==2.21.0</nowiki> | ||
+ | <code>requests 2.21.0</code> installed <code>urllib3 1.24.3</code>, which is fine.<br> | ||
+ | Lastly, <code>ansible-playbook</code> was reporting I had <code>chardet 2.2.1</code> when I really had <code>chardet 3.0.4</code>. I resolved this by removing it and installing the last version. | ||
+ | <nowiki>~$ sudo pip uninstall chardet | ||
+ | ~$ sudo pip install --upgrade chardet==3.0.2</nowiki> |
Latest revision as of 14:38, 8 May 2021
Overview | Continuous Integration (CI) | Source Control Management (SCM) | Containerization | Configuration | Integration
Ansible | Playbook Examples
Contents |
[edit] Ansible
[edit] Installation
Ansible does not require any server component, no daemon needs to be running. Ansible runs over SSH.
Where ever you are going to run your playbooks from, needs to have ansible installed, the clients do not.
To get ansible 2.0+, at the time of this writing, use the epel repo
~$ sudo yum install -y epel-release ~$ sudo yum install -y git python python-devel python-pip openssl ansible ~$ ansible --version ansible 2.7.0 config file = /etc/ansible/ansible.cfg configured module search path = [u'/home/r00t/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
[edit] Configuration
Basic configuration, define the inventory path & specify the sudo user
~$ sudo vim /etc/ansible/ansible.cfg uncomment ... inventory = /etc/ansible/hosts ... [privilege_escalation] become=True become_method=sudo become_user=root become_ask_pass=False
Nodes are determined by their names. These live in the inventory hosts file: /etc/ansible/hosts
. By default there is a file there that can be used as an example.
~$ sudo vim /etc/ansible/hosts [local] localhost [group1] ansible2.r00tedvw.local [group2] ansible3.r00tedvw.local
[edit] Ansible User
Ansible needs to run as a non-privileged user with sudo rights. It also needs to be able to run commands without specifying a password as the playbooks will fail with password prompts.
NOTE: This will need to be done on EACH Node.
~$ sudo adduser ansible ~$ sudo passwd ansible ~$ visudo ... ## Allow root to run any commands anywhere root ALL=(ALL) ALL ansible ALL=(ALL) NOPASSWD: ALL
[edit] SSH key exchange
You can opt to either:
- define the SSH password for ansible in the playbook
- setup ssh key pairs so that no password is needed for login/authentication.
Below i'll quickly go over setting up ssh key exchange since this is the obvious choice.
On your control server, setup the key exchange so ansible can get to all the nodes without a password.
~$ sudo su ansible - ~$ ssh-keygen ~$ ssh-copy-id [email protected] ~$ ssh-copy-id [email protected] ~$ ssh-copy-id [email protected]
[edit] Basic Commands
You can specify the group or all to apply the ansible command to.
~$ ansible all -s -a "ls -la /etc/ansible"
- -s
- this flag runs the command as sudo on the node
- -a
- this flag runs a specified command defined after it.
[edit] copy file to node
~$ ansible group1 -m copy -a "src=/home/ansible/test.txt dest=/tmp/test.txt"
[edit] install & remove package
~$ ansible group1,group2 -m yum -s -a "name=lynx state=latest" ~$ ansible group1,group2 -m yum -s -a "name=lynx state=absent"
[edit] run local command across nodes
~$ ansible env -m shell -s -a "firewall-cmd --get-active-zone"
[edit] add firewalld rule
~$ ansible env -m firewalld -s -a "zone=public service=nfs permanent=true state=enabled" -vvv
[edit] modify service
~$ ansible env -m service -s -a "name=firewalld state=reloaded enabled=yes"
[edit] add new firewalld service
~$ ansible env -m shell -s -a "sudo firewall-cmd --permanent --new-service-from-file=/usr/lib/firewalld/services/nfs3.xml --name=nfs4" ~$ ansible env -m shell -s -a "sudo firewall-cmd --permanent --service=nfs4 --add-port=111/tcp" ~$ ansible env -m shell -s -a "sudo firewall-cmd --permanent --service=nfs4 --add-port=111/udp" ~$ ansible env -m firewalld -s -a "zone=public service=nfs4 permanent=true state=enabled" ~$ ansible env -m service -s -a "name=firewalld state=reloaded enabled=yes"
[edit]
~$ ansible env -m shell -s -a "sudo mkdir /mnt/nfsshare" ~$ ansible env -m mount -s -a "state=mounted fstype=nfs path=/mnt/nfsshare src=ansible1.r00tedvw.local:/nfsshare"
[edit] Playbooks
See what hosts the playbook would apply to:
~$ ansible-playbook {playbook.yml} --list-hosts
$ ansible-playbook /etc/ansible/playbooks/centos_base_packages.yml --list-hosts playbook: /etc/ansible/playbooks/centos_base_packages.yml play #1 (centos): centos TAGS: [] pattern: [u'centos'] hosts (3): jenkins01.ncvw.org ansible01.ncvw.org gitlab01.ncvw.org
Test Playbook
~$ ansible-playbook general.yml --syntax-check
Limit Host application
~$ ansible-playbook general.yml --limit ncwv-linux01.r00tedvw.local
Example: Running an ansible playbook against a specific host, specifying the host file location, providing root credentials, and the sudo password. (this also requires ansible.cfg to be located within the inventory folder and contain host_key_checking = false to avoid issues with SSH key authentication)
~$ ansible-playbook general.yml -i /home/ansible/ansible/hosts.yml --extra-vars "ansible_user=root ansible_password=Password ansible_sudo_pass=Password" --limit ncwv-linux01.r00tedvw.local
Prompt for sudo password
~$ ansible-playbook general.yml -i /home/ansible/ansible/hosts.yml --limit ncwv-linux01.r00tedvw.local --ask-become-pass
Basic example:
--- - name: haproxy hosts: haproxy become: yes become_method: sudo tasks: - name: Install HAProxy on CentOS yum: name: haproxy state: latest - name: enable service haproxy and ensure it is not masked systemd: name: haproxy enabled: yes masked: no - name: update haproxy conf blockinfile: path: /etc/haproxy/haproxy.cfg block: | # [HTTP Site Configuration] listen http_web 0.0.0.0:80 mode http balance leastconn # Load Balancing algorithm option httpchk option forwardfor server apache01 ncrv-apache01:80 weight 1 maxconn 512 check server apache02 ncrv-apache01:80 weight 1 maxconn 512 check - name: open http firewalld: zone: public service: http permanent: yes immediate: yes state: enabled - name: restart haproxy service service: name: haproxy state: restarted
[edit] Install base packages, independent on package manager
The package module can be used to install packages across distributions, but it has some serious limitations when compared to the yum and apt modules, primarily concerning the options to define state, version, etc.
- hosts: all tasks: - name: Ensure base packages are installed package: name: telnet state: present package: name: net-tools state: present package: name: tcpdump state: present package: name: bind-utils state: present package: name: redhat-lsb-core state: present package: name: wget state: present package: name: nfs-utils state: present
[edit] Troubleshooting
[edit] urllib3 or chardet doesn't match supported version
Overview: When running ansible-playbook
, I would get the below error.
Error:
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25.10) or chardet (2.2.1) doesn't match a supported version!
Resolution:
__init__.py
calls for the following package versions:
urllib3 >= 1.21.1, <= 1.25 chardet >= 3.0.2, < 3.1.0
When I checked, I had the following installed:
~$ sudo pip list ... chardet (3.0.4) urllib3 (1.25.10) ...
First I downgraded urllib3
to be equal or below 1.25
~$ sudo pip uninstall urllib3 ~$ sudo pip install --upgrade urllib3==1.25
This gave me an error citing that:
requests 2.24.0 requires urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you'll have urllib3 1.25 which is incompatible.
So after uninstalling and reinstalling different versions of requests
, I found that requests 2.21.0
works with versions of urllib3
below 1.25.0
~$ sudo pip uninstall requests ~$ sudo pip install --upgrade requests==2.21.0
requests 2.21.0
installed urllib3 1.24.3
, which is fine.
Lastly, ansible-playbook
was reporting I had chardet 2.2.1
when I really had chardet 3.0.4
. I resolved this by removing it and installing the last version.
~$ sudo pip uninstall chardet ~$ sudo pip install --upgrade chardet==3.0.2