Automating JVM Thread Dump Analysis with AI: Practical Observability for Java on Amazon ECS and EKS

Imagine the following scenario: a Java service that ran flawlessly yesterday suddenly starts consuming 90% CPU, barely responding to user requests. Users encounter timeouts, and the operations team is under immediate pressure to triage the incident. In these situations — whether the application appears stuck or CPU saturation has occurred — one of the most powerful diagnostic assets available is the thread dump. A thread dump captures the state of all JVM threads at a specific point in time, showing execution states, stack traces, and lock contention — the raw material needed to understand what the runtime is actually doing.

However, conventional thread-dump based troubleshooting is not without pain. Interpreting these snapshots requires deep JVM knowledge — expertise that typically only a few engineers in a team possess. Manual analysis is slow and error-prone, especially when hundreds of threads are involved, and teams often only trigger thread capture after latency and throughput have already degraded — meaning insights arrive too late and customer impact has already materialized.

Generative AI can make this process faster and more accessible. In this article, we show how to build an automated analysis pipeline that combines traditional observability components such as Prometheus and Grafana with an AI layer powered by Amazon Bedrock. The system continuously monitors Java workloads on Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (Amazon EKS), collects thread dumps when abnormal thread activity is detected, and automatically produces structured diagnostic reports within seconds.

The Challenge of Troubleshooting Java Inside Containers

Every Java developer knows the power and complexity of the JVM. It abstracts memory management, threading, and I/O across platforms, but this abstraction also makes performance issues harder to diagnose. In container environments, the situation becomes even more complicated. Containers are short-lived, IP addresses change dynamically, and horizontal scaling creates multiple JVM instances whose behavior can differ slightly under load. By the time an engineer connects to a pod or task to gather diagnostic data, the container may already have been restarted.
Manually inspecting each dump also does not scale. Some enterprise systems generate hundreds of threads; others run many applications across several clusters. In both cases, the human bottleneck is clear: teams need a solution that automates thread dump capture and interpretation while retaining the deep JVM insight that experts rely on.

Overview of the Automated Analysis Architecture

The solution described here is deployed as Infrastructure as Code (IaC) using AWS CloudFormation. It brings together three layers of functionality. Prometheus and Grafana form the monitoring and alerting layer, collecting and visualizing JVM metrics from Spring Boot applications. AWS Lambda provides the orchestration layer that captures and processes thread dumps on demand. Finally, Amazon Bedrock supplies the AI analysis engine that transforms raw thread dumps into actionable findings.

The monitoring infrastructure continuously scrapes JVM metrics (in our case jvm_threads_live_threads ) from each containerized application. When thread counts exceed a predefined threshold, Grafana triggers an alert through a webhook. The alert invokes a Lambda function that automatically determines whether the affected service runs on Amazon ECS or Amazon EKS. Depending on the environment, it executes the appropriate collection method. The system stores thread dumps and analysis reports in Amazon Simple Storage Service (Amazon S3) for later inspection or trend analysis.
The solution code and deployment instructions are available through Java on AWS Immersion Day Workshop and on GitHub.

Deploying the Base Infrastructure

To illustrate the setup, we use a simple Java application called UnicornStore, built with Spring Boot and exposing standard actuator endpoints. The monitoring environment can be deployed directly from AWS CloudShell with a few commands.

To follow this walkthrough, you need:

• An AWS account with permissions to create ECS/EKS clusters, Lambda functions, and S3 buckets
• AWS CLI configured with appropriate credentials or AWS CloudShell
• Basic familiarity with Java Spring Boot applications
• Understanding of container orchestration concepts

The following snippet creates an S3 bucket for deployment artifacts and provisions the entire infrastructure through CloudFormation.

curl https://raw.githubusercontent.com/aws-samples/java-on-aws/main/infrastructure/cfn/unicornstore-stack.yaml > unicornstore-stack.yaml
CFN_S3=cfn-$(uuidgen | tr -d - | tr '[:upper:]' '[:lower:]')
aws s3 mb s3://$CFN_S3
aws cloudformation deploy --stack-name unicornstore-stack \
  --template-file ./unicornstore-stack.yaml \
  --s3-bucket $CFN_S3 \
  --capabilities CAPABILITY_NAMED_IAM

After the stack completes, the CloudFormation output provides the URL and password for the integrated development environment used in the workshop. At this point, the basic application, monitoring stack, and IAM permissions are ready for further configuration.

Configuring Observability in a Java Application

Our sample application uses Spring Boot Actuator and Micrometer to expose detailed metrics from the JVM. In application.config, enabling JMX and Prometheus integration looks as follows:

spring.jmx.enabled=true
management.endpoints.web.exposure.include=threaddump,prometheus,health,info
management.endpoint.health.probes.enabled=true
management.endpoint.prometheus.access=unrestricted
management.endpoint.health.group.liveness.include=livenessState
management.endpoint.health.group.readiness.include=readinessState

management.metrics.enable.all=true
management.metrics.tags.application=unicorn-store-jmx

The Prometheus registry collects data such as heap usage, garbage collection pauses, and live thread counts. Within the application, the MonitoringConfig class enriches these metrics with contextual tags identifying the cluster, container name, and network information. These tags help the Lambda function later decide how to capture the appropriate dump:

registry.config().commonTags(
    "cluster", cluster,
    "cluster_type", clusterType,
    "container_name", containerName,
    "task_pod_id", taskOrPodId,
    "instance", ipAddress,
    "container_ip", ipAddress
);

This configuration ensures that each metric can be traced back to its source container and orchestration system.

Collecting Thread Dumps Automatically

Once the monitoring infrastructure is active, Grafana continuously evaluates metrics against defined thresholds. When the number of live threads exceeds the limit, the alerting rule triggers a webhook to the Lambda function. The function parses the incoming payload and extracts metadata such as the cluster type, namespace, container name, and IP address. Based on this information, it uses different strategies for ECS and EKS.
For Amazon ECS, the function directly calls the Spring Boot Actuator endpoint:

response = requests.get(f"http://{container_ip}:8080/actuator/threaddump", timeout=10)
response.raise_for_status()

For Amazon EKS, it uses the Kubernetes API to execute native JVM diagnostic commands within the affected pod:

# Execute command to capture JVM thread dump from a running container
exec_command = [
    '/bin/sh',
    '-c',
    ('if command -v jcmd >/dev/null 2>&1; then '
        'PID=$(jcmd | grep -v jcmd | cut -d" " -f1); '
        'jcmd $PID Thread.print; '
        'elif command -v jstack >/dev/null 2>&1; then '
        'PID=$(ps -ef | grep java | grep -v grep | awk \'{print $2}\'); '
        'jstack $PID; '
        'else echo "Neither jcmd nor jstack found"; '
        'fi')
]

logger.info(f"Executing thread dump command in pod {pod_name}, container {container_name}")

resp = stream(
    self.core_v1_api.connect_get_namespaced_pod_exec,
    pod_name,
    namespace,
    container=container_name,
    command=exec_command,
    stderr=True,
    stdin=False,
    stdout=True,
    tty=False
)

The captured dump is then uploaded to Amazon S3 together with the relevant metadata. From there, the function triggers the AI analysis stage.

AI-Powered Analysis of Thread Dumps

Traditionally, analyzing a large thread dump requires extensive JVM expertise. Engineers must distinguish between blocked, waiting, runnable, and timed-waiting threads, understand which locks are contended, and detect possible deadlocks. The integration with Amazon Bedrock automates much of this reasoning. The Lambda function sends the dump content to Bedrock with a structured prompt that instructs the model to produce a multi-section report including an executive summary, identified issues, and optimization recommendations.

**Thread Dump Input**:
{thread_dump}
"""
    payload = json.dumps({
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 8192,
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0.7
    })

    # Implement exponential backoff with jitter to handle
    # Bedrock throttling during high-volume scenarios
    # This is critical when multiple alerts trigger simultaneously

    max_attempts = 5
    base_delay = 10

    # Cap backoff to avoid over-waiting
    max_delay = 30

    for attempt in range(1, max_attempts + 1):
        logger.info(f"Invoking Bedrock with payload ... ")
        try:
            response = bedrock.invoke_model(
                modelId="global.anthropic.claude-sonnet-4-20250514-v1:0",
                body=payload
            )
            body = json.loads(response.get("body").read())
            return body.get("content")[0].get("text")

The response returned by Bedrock includes detailed commentary such as which threads are blocked, which pools are saturated, and where synchronization overhead occurs. Instead of manually searching for “BLOCKED” entries or waiting for a senior JVM specialist, developers now receive structured guidance within seconds of an alert.

Example workflow and analysis

Once the environment is deployed, it is easy to reproduce a scenario that triggers the automation. Using Grafana’s “Explore” view, query the metric jvm_threads_live_threads for the UnicornStore application.

The dashboard will display the number of live threads along with metadata such as cluster name, namespace, and task or pod ID.
To generate load, create 500 threads in the application by sending a POST request to its REST endpoint. For Amazon ECS, obtain the application endpoint via the load balancer:

SVC_URL=http://$(aws elbv2 describe-load-balancers --names unicorn-store-spring --query "LoadBalancers[0].DNSName" --output text)

For Amazon EKS, use the Kubernetes ingress hostname:

SVC_URL=http://$(kubectl get ingress unicorn-store-spring -n unicorn-store-spring -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Start the threads and check the count:

curl --location --request POST $SVC_URL'/api/threads/start' --header 'Content-Type: application/json'
sleep 2
curl --location --request GET $SVC_URL'/api/threads/count' --header 'Content-Type: application/json'

The corresponding API endpoint is backed by the ThreadGeneratorService, a dedicated Spring component whose sole purpose is to generate synthetic CPU pressure. For this demo we let the service spawn a large number of worker threads that execute meaningless computation loops — effectively simulating a heavy CPU workload. In our scenario, we deliberately create 500 of these threads. Once the load is active, we can switch over to the dashboards in Grafana — Dashboards → Unicorn Store Dashboards → JVM Metrics – EKS & ECS — and observe how the JVM behaves under stress in real time.

When the Grafana alert triggers, it invokes our thread-dump Lambda function via a webhook — with a 60-second delay configured after the alert fires.

Roughly a minute later, we can inspect our Amazon S3 bucket: it now contains both the captured thread dumps and the corresponding analysis files. Below you can see an example output from the Amazon Bedrock inference step — it highlights relevant JVM state observations and includes concrete improvement suggestions for the application.

Cleaning Up the Environment

To remove all resources created during this walkthrough, execute the cleanup scripts provided in the workshop repository. The commands below delete the EKS add-ons, S3 buckets, and CloudFormation stacks created earlier:

~/java-on-aws/infrastructure/scripts/cleanup/eks.sh
aws cloudformation delete-stack --stack-name eksctl-unicorn-store-addon-iamserviceaccount-kube-system-s3-csi-driver-sa
aws s3 rm s3://$S3PROFILING --recursive
aws s3 rb s3://$S3PROFILING
aws cloudformation delete-stack --stack-name unicornstore-stack

Always verify that no residual resources remain, especially S3 buckets used for deployment artifacts or dump storage.

Conclusion

Automating thread dump analysis brings significant efficiency to Java performance troubleshooting, reducing diagnostic time from hours to minutes. Instead of relying on manual inspection and deep JVM expertise for every incident, developers can now receive diagnostic insights automatically within moments of an alert. The integration of Prometheus, Grafana, AWS Lambda, and Amazon Bedrock transforms thread dump interpretation from a reactive, time-consuming process into an automated, repeatable workflow.
Beyond accelerating root-cause analysis, the stored thread dumps and their corresponding AI reports in Amazon S3 create a valuable historical record. Over time, this data reveals recurring patterns such as thread leaks or periodic contention in specific code paths, enabling proactive optimization before issues impact users. The same architecture can easily be extended to other diagnostic artifacts such as heap dumps or application logs, or integrated with incident management systems like PagerDuty and OpsGenie for automated ticket enrichment.
For Java teams operating in containerized environments, this approach closes the gap between observability and understanding. By combining traditional JVM metrics with AI-driven interpretation, engineers gain deeper visibility into their applications’ behavior and can act on performance issues long before they affect users. Deploy the complete solution from the Java on AWS GitHub repository and start building your diagnostic intelligence today.

Interested in Learning More?
Sascha Möllering is a speaker at JCON. This article explores how AI can automate JVM thread dump analysis and improve observability in modern Java environments – and his JCON session takes the next step by demonstrating a complete AI-powered performance diagnostics and optimization workflow, from profiling and monitoring to root cause analysis and automated fixes. If you can’t attend live, the session video will be available after the conference – it’s worth checking out!

This article is part of the JAVAPRO magazine issue:

Engineering Intelligence – Building Systems in the AI Age

Explore what engineering means in the AI age — beyond code generation and automation.
Gain insights into performance-critical Java systems, evolving architectures, and the responsibilities that come with building intelligent software.

Discover the edition →