🤖 Week 8: Ansible Lab Guide

• Understand the concepts and usage of Ansible .

  • Integrate Ansible to the existing CI/CD pipeline to improve the continuous delivery of our application. Thus, our CI/CD pipeline will be robust and smooth for continuous delivery even for the complex setup of multiple target systems.

So far, we have used Jenkins as a Continuous Integration and Continuous Deployment tool.

However, Jenkins works best as a Continuous Integration only as it has problems dealing with more complex setups with multiple target environments of servers.

➜ So we need to introduce Ansible , which is a Configuration Management tool , which is the best practice of handling updates and changes of target system requirements.

So here is the setup for our CI/CD pipeline this week:

Note: To proceed with the lab, we should have already set up our Jenkins Server and Docker Server in the previous lab. If you do not have the Jenkins Server or Docker Server (because you have already deleted those EC2 instances) then please try to set them up again by following the tutorial instructions in previous weeks.

Let's create a new EC2 server to be our Ansible Server.

The outline for this section:

• Setup a Linux EC2 instance “ Ansible_Server ".

  • Set the hostname for convenience (as we have many servers to manage now).

  • Create ansibleadmin user.

  • Add the user to sudoers file.

  • Generate SSH keys to a quick authenticate (for the Ansible server to communicate/control Docker Server).

  • Enable password-based login for this guided setup.

  • Install ansible

sudo su -

nano /etc/hostname

reboot

sudo su -

useradd ansibleadmin

passwd ansibleadmin

nano /etc/sudoers

ansibleadmin ALL=(ALL) NOPASSWD: ALL

nano /etc/ssh/sshd_config

service sshd reload

ssh-keygen

sudo su -

yum install ansible

ansible --version

The outline what you need to do:

Connection checkpoint: Ansible is agentless. What must still be true about SSH connectivity, the remote user, and privilege escalation before a playbook can manage the Docker server?

• On the docker server:

  • Create ansibleadmin

  • Add ansibleadmin to sudoers files

  • Enable password login

• On the ansible server:

  • Add to hosts file.

  • Copy ssh the public key from ansible server to docker server.

  • Test the connection from ansible server to docker server.

Let's go!

• Let's SSH into the Docker Server:

• If you have not changed the hostname for the docker server, then please change the hostname of this server to be “ docker-server ", reboot and ssh again:

• Alright, let’s login as a root user and change our current directory to /root:

sudo su -

• Let's create a new user ansibleadmin:

useradd ansibleadmin

• Let's add a password for that new user ansibleadmin:

passwd ansibleadmin

• Let’s add the ansibleadmin to be in the docker group:

  • This is necessary to run Docker commands as the ansibleadmin user without typing sudo every time.

sudo usermod -aG docker ansibleadmin

• We can double check if the ansibleadmin is now in the docker group:

id ansibleadmin

• Alright, to keep things simple, I want the ansibleadmin user to be able to execute any command without password so let's modify Sudoers file and save the file:

nano /etc/sudoers

• By default, EC2 server will not allow users to login with the password so we need to enable that:

nano /etc/ssh/sshd_config

• Enable Password Authentication like so, then save and exit from nano:

• Alright, let’s restart the ssh service for the change to take effect:

service sshd reload

• Nice, let's check the IP address of our Docker Server so we can add the IP address of Docker Server to our Ansible Inventory in the Ansible Server:

  • Keep in mind that, the highlighted is the private IP address so only EC2 within the same network will be able to communicate with it.

ifconfig

• [Switch to Ansible Server Terminal] Now, let's switch to the terminal that controls Ansible server.

• The default location for the inventory file ( /etc/ansible/hosts ) may vary based on how Ansible was installed. To confirm the location of the default inventory file, check the Ansible configuration:

ansible-config dump | grep DEFAULT_HOST_LIST

The screenshot confirms that the default location of the ansible hosts is indeed at /etc/ansible/hosts

• Let's modify the Ansible host file (Ansible Inventory):

  • This file is the "address book" of Ansible . Every machine you want Ansible to manage needs to be listed here.

nano /etc/ansible/hosts

• That's good! Let's switch back to the ansibleadmin user on the Ansible Server:

sudo su - ansibleadmin

• Let's change the current working directory to .ssh to see our public and private keys:

cd .ssh
ls -la

• We can inspect further the public key of Ansible Server by printing it out:

cat id_rsa.pub

• Let's copy the public key of ansibleadmin on the Ansible Server to the Docker Server:

  • So the ansibleadmin user can connect without a password. This sets up passwordless SSH, allowing Ansible to run commands and playbooks on the Docker Server automatically without needing manual password input every time as this is the core of automation.

ssh-copy-id [the-ip-address-of-docker-server]

• [Switch back to Docker Server Terminal] Let's switch back to terminal of Docker Server and check if the public key has been copied over:

• First, let's change from root to ansibleadmin:

sudo su - ansibleadmin

• Then, need to change current working directory to .ssh :

cd .ssh

• Print out the authorized_keys :

  • You should see the same content of the public key of ansibleadmin on the Ansible Server that we copied over in Docker Server.

cat authorized_keys

Now, it's time to test the connection between Ansible Server and Docker Server:

• [Switch back to Ansible Server Terminal] In the Ansible Server , we can use Ansible to ping the target server in the inventory like so:

ansible all -m ping

• Similarly, we can use ansible from Ansible Server to run the bash command like “uptime" on Docker Server like so:

ansible all -m command -a uptime

This is like how Ansible Server (master) can control everything on the Docker Server (slave) with any credentials or password authentication. Image, you can have ten different target servers in the Ansible Inventory file so one Ansible server can control all target servers easily.

Extra Note: you might wonder that while the manual setup of each server (e.g., creating a user, enabling SSH password authentication, etc.) works for a small number of servers, it quickly becomes cumbersome and error-prone when scaling to manage 10, 100, or more target servers. However, in the challenge exercise, you can use Ansible itself to automate the onboarding of new target servers (slaves) to make the process repeatable and scalable.

• Let's SSH into the Jenkins Server:

• In case you have not changed the hostname of Jenkins server, let’s change the hostname of this server to be “ jenkins-server ", reboot and ssh again:

• Alright, let’s login as a root user and change our current directory to /root:

sudo su -

• Let’s start Jenkins service if you need to:

service jenkins start

• Let’s go to the configured system:

• Let’s add new SSH Server information for our Ansible Server!

  • Jenkins needs Ansible Server SSH config because Jenkins is acting as the "orchestrator," but it delegates configuration and deployment to Ansible. Without this step, Jenkins can only build code, not deploy it

• Make sure you take note of the IP address of Ansible Server using “ifconfig” command:

• After configuring these settings, let’s test the connection:

  • If you see “Success” then it can connect to that server using the username and password provided!

• Then we can save the configuration.

• Let’s create a new Jenkins job:

  • To save time, you can copy the configuration settings from the “BuildAndDeployOnDockerContainer” job from last week.

• We keep everything the same (Git, GitHub, Maven, … ) apart from the SSH publishers. We want to change from docker-server to ansible-server :

• Let’s save the job for now! And run the job to test if the job can run correctly and copy the artifacts over to the Ansible Server!

• The result looks good! It should show build success and successfully connect to the Ansible Server to copy the artifact WAR file over that server.

  • Note that the WAR file is sent to Ansible instead of directly to Docker because Ansible is becoming the "deployment manager" in the pipeline.

• [Switch to the Ansible Server Terminal] In the Ansible Server, you can check if the WAR artifact file has been transferred from Jenkins server to Ansible Server like so:

  • Keep in mind, you might need to run cd ~ and change the working directory to the home directory of ansibleadmin user first before listing the files inside it.

Alright, so far so good! Now, let’s install docker on the Ansible server so we can push the docker image of our web application to Docker Hub !

We can sudo into the root user or in this case, we can stay as ansibleadmin user and add sudo at the beginning of every command to execute them as a root user like so:

sudo yum install docker -y

• Let’s add the ansibleadmin to be in the docker group:

  • This is necessary to run Docker commands as the ansibleadmin user without typing sudo every time.

sudo usermod -aG docker ansibleadmin

• For this to take effect, please logout and login in as ansibleadmin again:

exit
sudo su - ansibleadmin

• We can double check if the ansibleadmin is now in the docker group:

id ansibleadmin

• Let’s start the docker service:

sudo service docker start

• Check the Docker service status:

sudo service docker status

• Let’s create a Dockerfile to build an image from war artifact file:

nano Dockerfile

FROM tomcat:latest
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
COPY ./*.war /usr/local/tomcat/webapps

• Let’s test if you can build a docker image from this Dockerfile:

docker build -t myapp:v1 .

• Let’s list all docker images to check if our new image is there:

docker images

• We can even test further by creating a container from this image and access this image via the browser:

  • The website should be available on the browser via the Ansible Public IP address with the port 8081 .

docker run -d --name myapp-container -p 8081:8080 myapp:v1

It means that our Docker image is healthy!

sudo nano /etc/ansible/hosts

[dockerserver]

private-ip-address-of-docker-server
[ansibleserver]
private-ip-address-of-ansible-server

ssh-copy-id 172.31.94.133

ansible all -a uptime

nano buildAndPush.yml

---
- hosts: ansibleserver
  tasks:
  - name: create docker image

command: docker build -t myapp:latest .

ansible-playbook buildAndPush.yml

docker images

docker tag [image-id] [new-image-repo-name]

docker login

docker push tomhuynhsg/myapp:latest

nano buildAndPush.yml

---
- hosts: ansibleserver
  tasks:
    - name: Build the Docker image
      command: docker build -t myapp:latest .
    - name: Tag the image for Docker Hub
      command: docker tag myapp:latest tomhuynhsg/myapp:latest
    - name: Push the image to Docker Hub
      command: docker push tomhuynhsg/myapp:latest

ansible-playbook buildAndPush.yml

• Let’s create a new Ansible Playbooks to create a container by pulling it from Docker Hub :

nano pullToCreateContainer.yml

---
- hosts: dockerserver
  tasks:
    - name: create a container from the Docker Hub image
      command: docker run -d --name mywebapp-container -p 8082:8080 tomhuynhsg/myapp:latest

• [Switch to Docker Server Terminal] Before running this playbook on Ansible Server, make sure Docker service is running on Docker Server :

sudo service docker start

• [Switch to Ansible Server Terminal] Let’s run this Playbooks on Ansible Server:

  • The website should be available to access on the browser on the public IP address of the Docker server with the port 8082 .

ansible-playbook pullToCreateContainer.yml

• The first run works, but rerunning the playbook requires us to handle resources that may already exist. We need to:

  Idempotency checkpoint: On a second playbook run, which tasks should report changed and which should report ok? What would repeated destructive recreation reveal about the playbook design?

  • Remove existing container

  • Remove existing image

  • Create new container

• Let’s modify the playbook:

nano pullToCreateContainer.yml

---
- hosts: dockerserver
  tasks:
    - name: stop the existing container
      command: docker stop mywebapp-container
      ignore_errors: yes
    - name: remove the existing container
      command: docker rm mywebapp-container
      ignore_errors: yes
    - name: remove the existing image
      command: docker rmi tomhuynhsg/myapp:latest
      ignore_errors: yes
    - name: create a container from the Docker Hub image
      command: docker run -d --name mywebapp-container -p 8082:8080 tomhuynhsg/myapp:latest

[Explanation] We use ignore_errors so the cleanup tasks continue when a container or image does not exist. This makes the exercise rerunnable, but it is not true idempotence because unrelated command failures are also ignored. Production playbooks should prefer state-aware Ansible modules or narrowly define acceptable failures.

• Let’s run this playbook on the Ansible server again:

  • You should see that the playbook runs on the ansible server, gathers system facts, stops and removes the existing container, removes the old image, and then creates a new Docker container from the image on Docker Hub , completing successfully with no errors.

ansible-playbook pullToCreateContainer.yml

Alright, it’s perfect! Now, both of our Ansible Playbooks are ready for Jenkins to use!

• Open the configuration page of the Jenkins job again to add this script as “Exec command” to the ansible server section:

  • First playbook → Builds a new container image and pushes it to the Docker Hub repo registry.

  • Sleep → Pushing images to Docker Hub isn’t instant, so this wait briefly (15 seconds) ensures Docker Hub has the new image available before trying to pull it.

  • Second playbook → Pulls the image from the Docker Hub repo registry and creates (or updates) the running container.

Note : this separation (build & push vs pull & run) mirrors real-world pipelines as it builds artifacts in one stage and deploys in another.

Release checkpoint: If deployment fails after an image is pushed, which immutable artifact lets you roll back reliably? Why is reusing only the latest tag risky?

ansible-playbook /home/ansibleadmin/buildAndPush.yml
sleep 15
ansible-playbook /home/ansibleadmin/pullToCreateContainer.yml

• Try to run the Jenkins job to test if our configuration is all good!

• Please confirm that the website is available to access on the browser via the public IP address of Docker Server via the port 8082 (as defined in the pullToCreateContainer.yml playbook so feel free to change the port if you want to)

• Also, if you want to do further testing, you could try to push some changes to trigger the Jenkins job to see if the Ansible playbooks execute correctly to show the website with the new changes or not!

Awesome, you have completed this CI/CD Pipeline for this week! Please continue to test this pipeline further by making a change to the website and pushing it to GitHub to see the full automation in action!

Congratulations! You have successfully created this CI/CD Pipeline with Ansible Playbooks! Now, by adding more IP addresses/domain names into the Ansible Inventory, you can easily manage to deploy this web application to hundreds of different target servers!

🏆 You are now ready to receive a DevOps champion award in automation !

Using AWS EC2 User Data , you can automate the initial setup of an Ansible Slaves EC2 instance when it is launched. This allows you to configure servers without manual intervention with the benefits:

• Repeatability : Every instance launched with the script has a consistent configuration.

• Scalability : The same script can initialize dozens or hundreds of servers.

• Automation : Reduces manual setup, saving time and minimizing errors.

Here's a step-by-step guide:

• Launch an Ansible Slaves EC2 Instance with User Data

  • OS: default Amazon Linux 2023 AMI (Free-tier)

  • Instance Type: t3.micro (Free-tier)

  • Key pair: select and reuse your old key in other labs ( devops_project_key )

  • Storage: Default 8 GB is plenty enough!

  • Security Group:

    • Optional Name: Jenkins_Slave_Security_Group

    • Source Type: Anywhere

    • Open ports:

      • 22 for SSH

      • Open other necessary ports (e.g., 8080 for Docker ).

  • Add User data

    • Click on Advanced details

• Upload or paste in the User data:

• Note: remember that the public key of the Ansible master server should be at /home/ansibleadmin/.ssh/id_rsa.pub

  • After setting up the User data, you can launch the Ansible slave instance.

  • Wait until the instance is ready:

• [Optional] To test whether the user-data script worked correctly, you can choose to ssh to the Ansible slave to run these commands:

• Check the hostname is correct:

  • Test Docker: docker --version

    • Test Ansible user login: su - ansibleadmin

• Confirm SSH keys to match the public key in User data: cat /home/ansibleadmin/.ssh/authorized_keys

• [Back to the Ansible Master] Add the Ansible Slave instance to the Ansible inventory in the Ansible master ( /etc/ansible/hosts )

• Test if the master can connect to the slave

ansible ansibleslave -m ping

• Test if the master can run commands on the slave:

ansible ansibleslaves -a uptime

Success! Using EC2 User Data is very efficient to quickly set up multiple Ansible slave EC2 instances for Ansible master to manage them.

[Extra thoughts]

If you would like to be more efficient to launch EC2 instances without GUI, you might consider:

• You can launch an EC2 instance directly from the AWS CLI by passing the User Data script.

  • Save the User Data script to a file, e.g., user-data.sh .

  • Use the following CLI command to launch the instance:

This approach aligns with real-world industry practices, enabling efficient and scalable infrastructure management.

Now that you have a solid understanding of AWS CLI, EC2 User Data, and Ansible , here are some advanced tasks to further enhance your skills:

1. Scale Up with Multiple Ansible Groups :

   • Use the AWS CLI to quickly launch multiple EC2 instances, for example:

     • A group for test-servers (staging environment).

     • A group for production-servers .

   • Configure your / etc/ansible/hosts file to manage these groups separately:

     • Note : your groups can have more ip addresses for more EC2 instances if you want to.

[test-servers]
192.168.x.x
[production-servers]
192.168.y.y

• This separation enables precise control and testing before deploying changes to production.

1. Deploy Applications with Jenkins and Ansible :

   • Use Jenkins to integrate your CI/CD pipeline with Ansible's inventory groups:

     • First, deploy your application to test-servers .

     • Run some basic front-end tests using tools like curl to ensure the deployment is functional.

     • Once tests pass, automate the deployment to production-servers .

2. Automate Testing :

   • Implement Ansible tasks or playbooks to:

     • Verify that the application is running on test servers.

     • Use simple health checks ( curl or HTTP status checks) to confirm that the deployed app responds as expected.

3. Leverage AWS CLI for Instance Management :

   • Automate instance creation with the following AWS CLI command:

aws ec2 run-instances --image-id ami-xxxxxx --count 3 --instance-type t3.micro --key-name devops_project_key --security-group-ids sg-xxxxxx --user-data file://user-data.sh

• This ensures repeatable infrastructure setup and avoids reliance on manual GUI interactions.

1. End-to-End Pipeline Simulation :

   • Push a new version of your web app to GitHub.

   • Trigger a Jenkins pipeline that:

     • Builds and pushes a Docker image.

     • Deploys the image to test-servers via Ansible.

     • Runs tests on test-servers .

     • Upon successful tests, deploy to production-servers .

2. [Optional - Thoughts] Experiment with Advanced Tools to Simulate Real-World Scenarios :

   • Practice a blue-green deployment strategy :

     • Deploy the new version of the app to a group (e.g., test-servers or a subset of production-servers ).

     • Perform tests before switching traffic over to the updated servers with AWS Elastic Load Balancer (ELB) which we will learn later in later weeks.

By practicing these advanced challenges, you'll gain deeper insights into DevOps workflows and be well-prepared for real-world scenarios involving scalable infrastructure and efficient automation. Keep pushing boundaries, and you'll be ready to tackle increasingly complex projects with 🏆 confidence!