Managing a multi-datacenter VM based or an Edge server executing Java applications is not a simple task and can become even more challenging as the complexity of the hosted app grows. If a pristine installation runs without a hitch, after a while, recurring problems appear and what is called nowadays “Day One” operations take the front seat. Obviously, those are the struggles facing any modern IT infrastructure, and while the target could not be more clear and defined (a functional system with zero or minimum downtime), yet the crucible, how to achieve it, may not be evident.
Regardless of the actual implementation, the key here is automation. Especially in the context of large and complex Java deployments which often requires the capability to set up software, configure them properly and maintain up to date the underlying infrastructure (operating systems, databases, and so on), but also to be able to effectively interact with the Java Virtual Machines (JVM) executing the applications.
This is the broad and intriguing topic we are going to discuss in this article. How to build an efficient, and modern infrastructure for Java-based systems, in a fully automated fashion, yet also designed to keep running on its own, and thus imbued with the capacity to react to changes and autocorrect, without manual (or minimum) intervention.
Ansible to the rescue?
Ansible in a few words
As we are going to demonstrate in our article, Ansible, an Open Source, Python-based product for automation, is an appropriate fit to address all the concerns and requirements we have just mentioned. It might feel like an odd pick, a non-Java-based ware to manage Java deployments, yet, as we’ll see, the software flexibility, features and extended ecosystem makes it a perfect solution.
Before going any further, it’s important to note that Ansible does not mandate Python skills and is, as Java, cross-platform. Also, Ansible has an agentless architecture, which means that, apart from the controller (the machine where Ansible is executed) no other software needs to be installed and managed. And, if Ansible Automation Platform is used (see the last section of the article), the appropriate Python environment is provided and administered by the product.
How does Ansible fit the bill?
As we are going to demonstrate in our article, Ansible, an Open Source, Python-based product for automation, is an appropriate fit to address all the concerns and requirements we have just mentioned. It might feel like an odd pick, a non-Java-based ware to manage Java deployments, yet, as we’ll see, the software flexibility, features and extended ecosystem makes it a perfect solution.
Before going any further, it’s important to note that Ansible does not mandate Python skills and is, as Java, cross-platform. Also, Ansible has an agentless architecture, which means that, apart from the controller (the machine where Ansible is executed) no other software needs to be installed and managed. And, if Ansible Automation Platform is used (see the last section of the article), the appropriate Python environment is provided and administered by the product.
Vocabulary note: With Ansible, the system executing the tool is referred to as the controller, which is often opposed to the targets, the hosts that Ansible manages. Keep in mind this distinction for the rest of the articles. Also note that the targets can be bare metal, VMs or even run in a private or public cloud.
Contrary to other options employing agents, and thus requiring it to run on the target’s system, Ansible simply leverages SSH (or a similar authentication mechanism) to communicate with the target hosts. No Python code is ever executed on any system managed by Ansible, nor is any kind of Python setup necessary on the targets (at least, most of the time).
Ansible playbook
To define the automation needed, Ansible uses YAML descriptors, not code. Those files, called playbooks, are not scripts per se, and utilize a syntax specific to Ansible, to depict the state the target systems are expected to be in along with the steps required to achieve it.
Below is a simple example of a playbook designed to ensure that nginx, a popular Open Source HTTP server, is installed and running on the targets:
---
- name: "Ensure nginx is installed and running"
hosts: webservers
tasks:
- name: "Ensure nginx package is installed"
ansible.builtin.package:
name: nginx
state: present
notify: "Restart Nginx"
- name: "Ensure nginx is properly configured"
ansible.builtin.template:
src: nginx.conf.j2
dest: /etc/nginx/nginx.conf
notify: "Restart Nginx"
handlers:
- name: "Restart Nginx"
ansible.builtin.service:
name: nginx
state: restarted
As shown by the playbook above, even without knowing the syntax used by Ansible, it’s easy to comprehend. The header of the playbook specifies its name, the targets (designated by a group name) and then the tasks to be performed. Each required operation is specified by using an Ansible module (such as ansible.builtin.package). The purpose of each of them is easy to understand: package ensures the required software packages are available, service manages the state of the system’s services and template deploys a configuration (based on a template, we’ll discuss this in a bit more detail below).
Notification and Handlers
The only item that requires some explanation is the use of the attribute notify and the section called handlers. In short, Ansible has an internal notification system allowing one module to notify a handler. This is mostly used when a change triggers other operations on the target system. Most of the time this mechanism is utilized to ensure that the service associated with the software being deployed is started (or restarted to take a config change into account).
In our example, both the use of package and template triggers such notification. Indeed, if the package has to be installed, nginx is certainly not running on the system, so it has to be started. If its configuration file needs to be updated, then the service also needs to be restarted (or at least reloaded) so that the changes are applied.
Overall this playbook ensure that the state of the target is as follow:
- The nginx package has been installed (prerequisite)
- The appropriate configuration is deployed (configuration)
- The nginx service is running (state)
Most of the time, the organization of a playbook will follow the same three high-level steps. Each might be more complex depending on the solutions to manage, but in the end, most playbooks ensure that prerequisites are in place, that configuration is properly deployed and that the appropriate software is running.
Templating with Jinja2
Note that, for the second step, the module used is ansible.builtin.template. Indeed, Ansible utilizes a template engine (named Jinja2) that allows his playbook to generate the correct configuration file, based on the specificity of the target system.
Below is an extract of the template employed in the example above:
…
server {
listen {{ ansible_all_ipv4_addresses }}:80;
server_name {{ ansible_nodename }};
root /usr/share/nginx/html;
…
A Jinja2 template employs a few simple primitives to replace the dynamic parts of the file by the content of variables managed by Ansible. Here again the syntax is pretty straightforward. In the example above nginx’s bind address will be the one of the target (by default, Ansible has such information on any target). Same approach applies for the server name, it is set to match the name used by Ansible for this particular host (by default, it is its hostname).
There is obviously more to understand about playbooks and their capabilities, but this brief overview is enough for the reader, new to Ansible, to comprehend the rest of this article.
Before we look at why Ansible’s features and capacities make it an excellent tool to deploy and maintain Java software, let’s talk about its extension mechanism. Like any automation technology, Ansible functionalities can be expanded by adding specific plugins. In Ansible’s parlance, those extensions are called collections. We’ll see how to install and use such plugins in the next section.
How can Ansible manage my Java deployment?
Extending Ansible capacities using Collection
All of the above points made Ansible a compelling solution for automation, yet it does not illustrate how it is a particularly good fit to handle Java products. To demonstrate that, we’ll now show how to implement a similar playbook to the one we just did, but this time, with a Java Product: Red Hat JBoss Web Server.
As we just mentioned, Ansible capacities can be extended by adding collections to the ones used by its controller. It’s very much akin to introducing a dependency in pom.xml of a Java project built by Maven. And like Maven, just one command line does the job:
# ansible-galaxy collection install redhat.jws
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Downloading https://console.redhat.com/api/automation-hub/v3/plugin/ansible/content/published/collections/artifacts/redhat-jws-2.1.1.tar.gz to /root/.ansible/tmp/ansible-local-17405g5h0ey9/tmpp71vcsnf/redhat-jws-2.1.1-0l_n4fph
Installing 'redhat.jws:2.1.1' to '/root/.ansible/collections/ansible_collections/redhat/jws'
Downloading https://console.redhat.com/api/automation-hub/v3/plugin/ansible/content/published/collections/artifacts/redhat-runtimes_common-1.2.2.tar.gz to /root/.ansible/tmp/ansible-local-17405g5h0ey9/tmpp71vcsnf/redhat-runtimes_common-1.2.2-b2lbzxjg
redhat.jws:2.1.1 was installed successfully
Downloading https://console.redhat.com/api/automation-hub/v3/plugin/ansible/content/published/collections/artifacts/redhat-rhel_system_roles-1.88.9.tar.gz to /root/.ansible/tmp/ansible-local-17405g5h0ey9/tmpp71vcsnf/redhat-rhel_system_roles-1.88.9-7nbobi3i
Installing 'redhat.runtimes_common:1.2.2' to '/root/.ansible/collections/ansible_collections/redhat/runtimes_common'
redhat.runtimes_common:1.2.2 was installed successfully
Installing 'redhat.rhel_system_roles:1.88.9' to '/root/.ansible/collections/ansible_collections/redhat/rhel_system_roles'
Downloading https://console.redhat.com/api/automation-hub/v3/plugin/ansible/content/published/collections/artifacts/ansible-posix-2.0.0.tar.gz to /root/.ansible/tmp/ansible-local-17405g5h0ey9/tmpp71vcsnf/ansible-posix-2.0.0-i06rpbpn
redhat.rhel_system_roles:1.88.9 was installed successfully
Installing 'ansible.posix:2.0.0' to '/root/.ansible/collections/ansible_collections/ansible/posix'
ansible.posix:2.0.0 was installed successfully
Again, in a similar fashion to Maven, the ansible-galaxy tool ensures that not only the collection for JWS is installed, but also its dependencies.
Setting Up JWS with Ansible
Now that we have the collection installed, let’s look at the implementation. Our goals are the same as with nginx. We want to set up, configure and run as JWS. As a configuration change, we’ll have a specific version of Java and, like before, bind the server against the default external IP of the target system.
Because the collection comes with a ready-to-use playbook, we don’t even need to write up one. We can simply run the one provided, overriding a few default variables to achieve our goals:
$ ansible-playbook -i inventory redhat.jws.playbook -e @service_account.yml -e jws_java_version=21 -e jws_listen_http_bind_address="{{ ansible_all_ipv4_addresses[0] }}"
The use of @ in front of a filename allows passing all the variables defined in it to Ansible. In this case, we are employing one named service_account.yml to provide the tool the credentials required to download the software (JWS) from the Red Hat Customer Portal. Note that, by default, it’s always the latest version available (latest version of JWS 6 as of the writing of this article).
Check the results
At the end of this playbook execution, we can check that JWS is installed and running, simply by leveraging the systemctl command on its service on the target. That can be performed directly by Ansible using a feature called adhoc command:
# ansible -i inventory webservers -a "/usr/bin/systemctl status jws6-tomcat.service"
localhost | CHANGED | rc=0 >>
● jws6-tomcat.service - Jboss Web Server
Loaded: loaded (/usr/lib/systemd/system/jws6-tomcat.service; enabled; preset: disabled)
Active: active (running) since Wed 2025-03-05 11:17:45 UTC; 38min ago
Process: 2697 ExecStart=/opt/jws-6.0/tomcat/bin/systemd-service.sh start (code=exited, status=0/SUCCESS)
Main PID: 2705 (java)
CPU: 5.128s
CGroup: /system.slice/jws6-tomcat.service
└─2705 /etc/alternatives/jre_21/bin/java -Djava.util.logging.config.file=/opt/jws-6.0/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED -classpath /opt/jws-6.0/tomcat/bin/bootstrap.jar:/opt/jws-6.0/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/jws-6.0/tomcat -Dcatalina.home=/opt/jws-6.0/tomcat -Djava.io.tmpdir=/opt/jws-6.0/tomcat/temp org.apache.catalina.startup.Bootstrap start
Mar 05 11:17:45 e9ba2ac1f2b9 systemd[1]: Starting Jboss Web Server...
Mar 05 11:17:45 e9ba2ac1f2b9 systemd-service.sh[2698]: Tomcat started.
Mar 05 11:17:45 e9ba2ac1f2b9 systemd-service.sh[2697]: Tomcat runs with PID: 2705
Mar 05 11:17:45 e9ba2ac1f2b9 systemd[1]: Started Jboss Web Server.
We can use the same feature to get more information on the version of JWS installed, and thus confirmed that the collection did fetch the latest available version:
# ansible -i inventory webservers -a "cat /opt/jws-6.0/version.txt"
localhost | CHANGED | rc=0 >>
Red Hat JBoss Web Server - Version 6.0 GA
Java Components:
Apache Tomcat 10.1.8-4.redhat_00011.1.el8jws
Tomcat Vault 1.1.9-2.Final_redhat_00002.1.el8jws
JBoss mod_cluster 2.0.3-4.Final_redhat_00001.1.el8jws
Native Components:
Apache Tomcat Native 1.2.36-1.redhat_1.el8jws
Notes regarding the redhat.jws Ansible collection:
The collection we just installed is a certified Ansible content provided by Red Hat for its runtime product Red Hat JBoss Web Server, a servlet engine based on Apache Tomcat, fully supported by the company. To obtain such an extension, and benefit from the software vendor support, one needs a valid subscription for both Ansible Automation Platform and JBoss Web Server. However, as always with Red Hat products, there is an upstream and community version, not supported, that can be installed and used without a subscription:
$ ansible-galaxy collection install middleware_automation.jws
Content-wise, these two collections are almost identical. One uses by default the community product (Apache Tomcat), the other one the downstream version supported by Red Hat (JWS). Their functionalities differ only where the underlying products do. For instance, the downstream version can update JWS by applying a patch provided by Red Hat. This feature has no equivalent with Apache Tomcat.
What about automatic remediation?
Event-Driven Ansible
What we have demonstrated so far is the capability of Ansible to work smoothly with java solutions such as Red Hat JBoss Web Server to ensure that the target systems are in the expected state and from there maintaining this set-up over time (by running the playbook regularly). However, we have yet to tackle the infrastructure’s reactiveness we described at the beginning of this article. We want to have automation also able to cope with incidents and execute, consequently, appropriate remediation strategies.
Ansible has a powerful feature, recently introduced, to deal with such use cases. It’s called Event-Driven Ansible (EDA). This functionality run playbook if triggered by the detection of some specific events. Under the hood, EDA utilizes an Open Source Java software named Drools. It’s a Business Rules Management System (BRMS) including a core Business Rules Engine (BRE), which enables EDA to define rules based on events. Yet another proof that Ansible works smoothly with the Java ecosystem.
EDA in Action
Let’s see a concrete example of how to automate a remediation strategy in the context of Java software deployment. For this, we’ll describe a simple use case, but also utilize a more elaborated Java solution, the Red Hat JBoss Enterprise Application Platform (EAP) for our demonstration. Here again, to install and run this software we’ll leverage the Ansible Collection for EAP provided by the vendor.
# ansible-galaxy collection install redhat.eap
Starting galaxy collection install process
[WARNING]: Collection redhat.jws does not support Ansible version 2.15.12
Process install dependency map
Starting collection install process
Downloading https://console.redhat.com/api/automation-hub/v3/plugin/ansible/content/published/collections/artifacts/redhat-eap-1.5.7.tar.gz to /root/.ansible/tmp/ansible-local-3004po7xx5f4/tmp3jum19hl/redhat-eap-1.5.7-moqnbw2b
Installing 'redhat.eap:1.5.7' to '/root/.ansible/collections/ansible_collections/redhat/eap'
redhat.eap:1.5.7 was installed successfully
'ansible.posix:2.0.0' is already installed, skipping.
'redhat.runtimes_common:1.2.2' is already installed, skipping.
'redhat.rhel_system_roles:1.88.9' is already installed, skipping.
Do note that we have installed this new collection on the same Ansible controller, so the dependencies shared with the redhat.jws already available have not been fetched again.
Setting up the cluster
Now, let’s discuss our use case. Our infrastructure manages with Ansible a large JBoss EAP deployment. Each instance hosts several applications, which synchronize their states thanks to the Java/Jakarta server’s feature of clustering. To emulate such an environment, we are going to provision EAP with the standalone-full-ha.xml configuration.
Again, the playbook supplied with the Ansible Collection for EAP does the heavy lifting, we just need to specify the configuration we wish to set up all the machines targeted to be part of the cluster:
$ ansible-playbook -i inventory redhat.eap.playbook -e eap_bind_addr="{{ ansible_all_ipv4_addresses[0] }}" -e eap_config_base=standalone-full-ha.xml -e @service_account.yml
Now that we have a functioning array of EAP instances and an active cluster, let’s discuss our use case for EDA. This cluster is dedicated to a back end application utilizing a data source. The app has been in production for years, but since the last update, there is a recurring issue. Some connections in the database pool are not being properly returned and remain unusable, in an idle state.
Implementing the remediation with EDA
The developers managed to create a ReST endpoint to report the problem, but have yet to find a fix. A simple workaround, however, is to leverage JBoss CLI to flush EAP’s idle connections. The following rulebook automate this remediation strategy:
---
- name: EDA lab
hosts: all
sources:
- ansible.eda.url_check:
urls:
- http://backend.example.com:8080/app/rest/db_healthcheck
delay: 10
rules:
- name: Restart Wildfly if url is not reachable
condition: event.url_check.status_code == 500
action:
run_playbook:
name: playbooks/flush_idle_connections.yml
Here again the syntax employed by Ansible is easy to understand. The rulebook is basically divided in two sections. The first one, sources, as it names implies, lists event sources consumed by EDA. In our case, we are utilizing a built-in primitive, url_check, which allows EDA to poll a URL and use the response as an event.
The second portion, titled rules, defines how to react to specific events. In our example, if the rest point returns an internal server error, it means that the number of unemployed connections in the database pool is becoming dangerously high. If this is happening, EDA will then trigger a playbook called flush_idle_connections.yml responsible for flushing the idle connections of all the instances of the cluster:
---
- name: "Flush idle connections of {{ ds_name }}"
hosts: all
vars:
ds_name: "BackendDS"
collections:
- redhat.eap
tasks:
- name: "Flush idle connection {{ ds_name }}"
ansible.builtin.include_role:
name: eap_utils
tasks_from: jboss_cli.yml
vars:
jboss_cli_query: "/subsystem=datasources/data-source={{ ds_name }}:flush-idle-connection-in-pool()"
The playbook above leverages the jboss_cli.yml provided by the Ansible Collection for EAP. It takes care of all the required configuration to connect to the remote instance and execute a JBoss CLI query.
Note: A nice benefit of using Ansible is that the remote instance does not need to expose the EAP management port (used by JBoss CLI). Indeed, Ansible will instead execute the query directly on the host (so the EAP server address will be localhost).
Ansible Automation Platform
An Automation Control Center
To define processes and automation, the high-level abstraction provided by Ansible’s s is more than appropriate. It’s a clean design that works well with a source control system (such as Git), which makes it a perfect tool for DevOps purposes.
However, these, on their own, are not able to give a real overview of a massive and complex infrastructure. And this is where Red Hat Ansible Automation Platform (AAP) comes into the picture. This product is conceived to be the control center of the automation and also provides much-needed features for enterprise or large organizations (such as SSO integration or ACL).
So, what is exactly AAP? It’s web UI designed to manage the high-level components of Ansible: organizations, projects (containing playbooks or rulebooks), inventories (dynamic or not) and scheduled jobs (execution of a playbook). It gives a visual overview of the infrastructure, but also enables fine-grained and role-based authorized access to the automation to individuals inside the organization.
For instance, using AAP, a developer may be allowed to trigger a job (a playbook run) only on a specific group of machines, however, they might not be authorized to amend the playbook or the job itself. A contractor working on the database deployment may be permitted to alter the playbooks (by pushing changes into the repository) and execute them, yet only in dry mode (without performing modifications on the target system). And so on.
A little tour of AAP
Login into AAP
Let’s give a quick tour of AAP with a few screenshots to get a better sense of the overview provided by the product. First, we’ll begin with the login page:
As shown above, AAP can, of course, integrate with SSO solutions such as Keycloak, and thus the authentication process and the users and roles can be entirely externalized from the tool. If this is needed, AAP can also administer local users and groups.
Landing page
Once logged in, the landing page provides a graphical overview of the infrastructure managed:
Obviously, the screenshot above displays the very low activity of a demo environment. It is not representative of a productive system.
Organizations
As many large groups or enterprises may use the same instance of AAP for different infrastructure, the tool has a notion of “organization.” Anything related to one can be hidden and isolated from the others. This allows AAP to manage, together, completely distinct projects and deployments.
Inventories
For AAP to know which systems it needs to manage, an inventory must be provided:
Provisioning the inventory can be done statically or dynamically. In any case, once this is done, the hosts tied to it and listed:
Ansible’s playbooks naturally fit into any kind of source control. Therefore, AAP has no real internal mechanism to manage them. Instead it is handling projects checked out from independent source control systems. All that’s needed is to provide the connection information to the external repository:
Job template
Once a project is defined, a job template can be created to run any of the playbooks it contains on the systems of the attached inventory. From there, AAP can directly fire a job (based on the template) or schedule its execution on a regular basis.
Event-Driven Ansible
Inside AAP, EDA follows a similar general approach. Rulesbooks live in projects that are also attached to organizations and can be associated with inventories. Note that if AAP has two different projects (one for the automation, one for the event-driven remediation), both playbooks and rulebooks can be stored and versioned in the same external repository, if it’s more convenient.
Once a rulebook has been properly integrated, AAP will run it on a regular basis and triggers the appropriate job template if needed:
Note that the rulebook needs to be modified to use the job_template primitive rather than the run_playbook one to work properly in the context of AAP.
Summary
Ansible itself is a powerful tool for automation that works well with Java software and requires no extra skills, short of learning the grammar and syntax of its playbooks, rulebooks and templates. With Event-Driven Ansible, it can go even further than automation and management, it can be enhanced to implement remediation of all sorts based on events triggered by incidents.
Adding Ansible Automation Platform on top of these two technologies enables its users to safely oversee massive infrastructure while ensuring security concerns are fully addressed (access and authorization, ACL, audit trails, …). Last, but not the least, Ansible has a rich ecosystem and can be easily extended through the utilization of collections, as shown by the ones provided by Red Hat for its (Java) Runtimes products.
All in all, Ansible is an excellent solution to manage one’s Java deployment, but also its infrastructure at large and other (non-Java) projects.
