Logging is the art of making sense of a system.
Software writes lines of information into log files, and developers sift through them, hoping to find what they need. The problem? Systems rarely log the right things. Yet without those files, we’d be flying blind, unaware that anything even went wrong.
This is the story of Apache Log4j—one of the oldest pieces of Java code still in use today. A tool that gave developers a way to listen to their systems.
Log4j is a European project
It all started in the mid-nineties when the European Union (EU) was in its early days. The SEMPER project was an early EU initiative to build a “secure toolkit for electronic commerce.” It started in September 1995, ended three years later, and was funded with around €10,000,000. At that time, it was a huge investment–not just because of the amount but also because many doubted the internet itself would survive, let alone e-commerce. Prototypes were built and tested, and in the end, SEMPER became history. People often complain today that the EU is inflexible. But in 1995, SEMPER was proof that the EU could innovate.
In 1996, Ceki Gülcü, N. Asokan, and Michael Steiner introduced the idea of “hierarchical categories“. This was a groundbreaking idea at the time. Like all good programming jokes, logging was strictly binary: “on” or “off.” There was no way to log just database-related events or track user activity.
Hierarchies changed everything. Developers could now enable logging at the package level, with child packages automatically inheriting their parents’ settings. The SEMPER project funded the first version of Log4j, recognizing the importance of logging for system observability.
Transition to Open Source
Once SEMPER ended, Mr Gülcü brought the package to the IBM Zürich Research Laboratory, where he continued development. By 1998 or 1999, Log4j had found a new home at IBM Zürich. The project could have faded, but IBM Zürich kept it alive. Early records show that Log4j was first distributed through IBM’s alphaWorks initiative. IBM used alphaWorks to foster emerging technologies, and Log4j aligned perfectly with that vision.
Wait, wait, emerging technologies?
Let’s not forget Java launched as a beta version only a few years earlier (in 1995) and hit 1.0 just one year later. Log4j emerged alongside J2SE 1.2 (1998) and 1.3 (2000). Java 1.2 brought us the Collections framework, and 1.3 followed with JNDI. In the era of Lambdas, working with Java 1.2 would send most of us running to woodworking. These upgrades made Java development massively easier.
As the millennium turned, many watched the skies in fear, wondering if Y2K would send planes plummeting. Meanwhile, Log4j’s future was taking shape. Mr Gülcü moved Log4j to SourceForge, taking advantage of its public infrastructure, and registered log4j.org. Don’t go there now–it leads to some weird places today.
Community over code
Open Source means something different today than it did in the past. Today, Open Source is often seen as a business model. We worry about funding, abandoned projects, and companies reclaiming their once-open code. Open Source takes many forms today, but back then, it was simple.
Money wasn’t the driving force. Open Source was like punk—part political statement, part rebellion, a movement for software freedom.
When I was much younger, I met the CEO of a company. He told me how much he hated free software. In his opinion, he’d lose a lot of money because people would distribute their work for free. I was perplexed. How would a small company build Linux? How would they provide support for database servers and web servers? They’d charge, he told me. They would also charge African hospitals, small businesses, and charities. Without open source, only big corporations would exist. Open Source makes the world a better, fairer place.
Even today, the fight for software freedom continues, though in different forms.
But freedom also comes with challenges—challenges that shaped where Log4j would end up.
The Free Software Foundation (FSF) was one of the strongest voices in early Open Source. Many Open Source projects found a home at the FSF, and Log4j could have been one of them. But there were a few reasons why this never happened:
The biggest obstacle was licensing: FSF’s GPL worked well for C and C++ but clashed with Java. Because Java loads libraries dynamically, it conflicted with the GPL’s strict linking rules.
Another likely reason was an early commit of an Ant build file to Log4j. At some point, Log4j’s maintainer reached out to the Apache Software Foundation (ASF), a rising community embracing Java projects. ASF’s permissive license encouraged broad adoption.
So why wouldn’t Log4j join ASF, where Ant was already at home?
Log4j joined the ASF and became part of the Apache Jakarta project. Yes, Apache Jakarta—the same name now known from the Eclipse Foundation, which develops Java Enterprise standards.
By 2000, Apache Jakarta had become an umbrella project, hosting five major Java-related projects, including Apache Ant and Apache Tomcat.
Meanwhile, in 2001, Apache Commons Logging was proposed. Its goal was to provide a simple API compatible with both JDK logging and Log4j. It also introduced a pluggable API, allowing other logging frameworks to integrate seamlessly.
Commons Logging aimed to unify logging under a single interface, positioning itself as the standard.
Log4j’s time in Jakarta was brief. In 2003, it was promoted to a top-level project under the name “Logging Services.” By then, Log4j had built a small community of six contributors.
Growing from a solo project to a true community is rarely straightforward. As Log4j evolved, the ASF board tasked the project with integrating its clones, such as Log4net and Log4php.
The future looked bright—at least, for a while.
However, in 2005, technical disputes and personal conflicts brought development to a halt. Momentum faded, and the community struggled to hold itself together. Eventually, Log4j’s founder walked away and created Logback and Slf4j in 2006.
Code without community
You’d think the split would bring peace and stability to Log4j. But long disputes don’t heal fast, and Log4j had no clear vision. The community was exhausted, and progress ground to a halt. Releases slowed to a crawl, taking years.
Version 1.2.15 came out in 2007. The next bugfix, 1.2.16, wouldn’t arrive for another three years. Even sister projects like Log4php wound down and eventually stopped.
It was a dark time, with no sign of change. Log4j faded into obscurity, at risk of becoming dormant.
Comeback
By 2009, I stepped into the project at a crucial moment. For work, I needed Log4php—but it was barely functional. So, I started patching it—the first real work on the project in years.
Soon after, Ralph Goers jumped in to revive Log4j 2. By chance, our efforts aligned—sparking new life into the project. The community came back to life. Enthusiasm grew, and new contributors joined.
New contributors brought a fresh, unbiased perspective to the aging Log4j 1 codebase. While Logback positioned itself as a successor, Log4j 2 was built to learn from past mistakes.
Log4j 2 was a complete rewrite, built on lessons learned from Log4j 1. Apache Logging Services might not have survived without breaking free from the old codebase.
Developers worldwide contributed to Log4j 2, shaping it into a community-driven project. Its open, community-driven approach encouraged developers to contribute patches and feedback.
Over time, it became clear that Log4j 2 was the future of Java logging and the days of Log4j 1 were numbered.
In 2015, Log4j 1 officially reached its end of life. Development ceased as the focus shifted to Log4j 2. The Log4j 2 team was confident that it was production-ready, and it delivered.
Positive feedback fueled adoption, drawing in more developers. Significant contributions included asynchronous logging and the LMAX disruptor, making Log4j 2 insanely fast.
Things looked bright again.
But not everyone welcomed the new logging landscape. Developers now had JUL, Slf4j, Logback, and Commons Logging to choose from. On top of that, both Log4j 1 and its successor, Log4j 2, coexisted.
Developers had to choose a framework and configure bridges to connect different APIs. Log4j 2 wasn’t just one product—it was two, packaged together. Like Slf4j and Logback, it combined an API and an implementation. It just never got the same attention.
Dark days: Log4shell
In July 2013, about a year before Log4j 2’s first stable release, a developer contributed feature 313—enabling JNDI support. This feature integrated JNDI into Log4j 2.
It was accepted without hesitation. Since JNDI is a core Java feature, adding it seemed like a logical step.
Log4j 2 was still in beta, so few questioned the change—or even noticed it.
Of course, we were wrong.
Log4j 2 shipped with feature 313—one of the most severe security flaws ever discovered.
The warning signs were there—we might have caught them at Black Hat 2016, a major security conference. But we didn’t.
I hadn’t even heard of Black Hat until Log4Shell went public. That year, security researchers at Black Hat detailed exactly how JNDI could be exploited.
And then came November 2021.
We communicate through e-mail lists, and every project has its own private channel. These lists are usually low-traffic, reserved for occasional personal matters or undisclosed security discussions.
But in November 2021, the mailing list traffic suddenly spiked—something that almost never happened. It didn’t take long to spot the words: “remote code execution.”
My first instinct? Close the e-mail client. I told myself I couldn’t deal with this right now.
An hour later, I knew I had no choice—even though I wasn’t actively developing. Fortunately, it wasn’t my part of the code—others took the lead.
Say what you will about the issue—Log4j’s team acted fast and handled it responsibly.
The first report came in on November 24, 2021.
A developer in China discovered the vulnerability. Within a day, we acknowledged the issue and realized it affected major projects like Apache Struts, Druid, and Flink—and likely countless others. After a few days of discussion, we fixed it by December 5. As usual, we initiated a three-day release vote. Despite the severity of the issue, the team remained confident—and relieved when everything worked out.
But just as the release vote wrapped up, two things happened.
First, security groups started discussing the Log4j issue openly—on social media and beyond.
Then came the real shock: an email from the original reporter.
The fix didn’t work.
That’s when everything changed. Suddenly, I was in the middle of it—explaining the issue and fielding calls from companies. Panic set in, and we rushed to push out a new release. On December 9, we built a new release—cutting the usual three-day wait period.
Log4j 2.15 was released.
But the nightmare wasn’t over.
More flaws emerged, and we patched them—one by one.
Amid the chaos, some of us received e-mails filled with anger and blame. On forums, frustration exploded. Looking back, a handful of dedicated people worked tirelessly to fix this.
None of us were paid for this work. Some even took days off, sacrificing their own vacation time.
This was our issue, no doubt. But we didn’t abandon the sinking ship—we fought to keep it afloat.
I often look back on this time and what it taught me. If there’s one thing I learned, it’s that this project was never just one or two people—it was a team.
With a strong team and dedicated people, we pulled through.
Aftermath
After the fix, Open Source was under a new kind of scrutiny. Governments and major newspapers picked up the story, fueling a deeper conversation about Open Source sustainability.
For decades, developers worked for free, contributing their work for the public good—while organizations profited from it.
And then, the foundation of organizational code blew up.
And once again, the same few people stepped up to fix it.
Unpaid.
Twenty years ago, programming and software played a very different role in our lives. Back then, smartphones didn’t even exist. But today, we use them to make payments. Your car is now more computer than machine. Almost every device in your home is now a tiny computer—even your water boiler connects to Wi-Fi.
Thankfully, things are starting to change. It’s far from perfect, but organizations like the Sovereign Tech Agency (STA) are stepping in to support Open Source sustainability.
In 2023 and 2024, the Log4j team received funding—and we gladly accepted. Well, part of the team, at least.
I quickly learned that funding developers isn’t as simple as it sounds. Some couldn’t accept due to job contract restrictions. Others faced tax complications. And some didn’t have time for a major project with limited funding.
Three people worked full-time on Log4j for 1.5 years. We added security features such as SBOM, provided cleanups, and improved documentation and the website. We also made it possible to use Log4j with Graal VM and allowed it to run on Android. We learned a lot, and Log4j grew despite the crisis it faced.
Takeaways
What if there were only two of us?
What if we were on vacation at that time?
What if we didn’t care?
Log4j could have been abandoned many times, but it survived. Sometimes, it was luck, and sometimes, it was the right people at the right time.
What did I learn?
Software engineers must take security more seriously. Security will be one of the biggest challenges in software’s future. AI will help attackers discover vulnerabilities faster, and we must stay vigilant. As threats evolve, we must stay one step ahead.
Think about this: Log4Shell was introduced in 2013—and remained undetected until 2021. And we still don’t know if it was exploited before then.
Community is the backbone of software development. High-impact projects can’t be maintained alone—we need a reliable team.
Open Source has a sustainability problem. Tech giants profit from Open Source, yet without it, innovation would stall.
Open Source is everywhere. Let’s make it sustainable.
Future
So where does that leave Log4j today?
Because of Log4Shell, no other logging framework has been tested as thoroughly. The team has proven its responsibility and commitment to security. Yes, it is “safe.” Nobody can predict the next Log4Shell, but we’ve strengthened our security and built a cleaner, more robust codebase.
Collaboration is everything. Some of us hope to one day work under a shared logging API, hosted within the Eclipse Jakarta project.
Until that day comes, we look forward to another 20 years of logging. And who knows? Maybe in twenty years, I’ll write another article like this one. History is still written every day.
You can be part of it. Leave your mark–we are just one commit away.
