The Java ecosystem is entering a new phase. Generative AI is no longer experimental; it is becoming a foundational capability in enterprise systems. Yet while frameworks for inference and orchestration have matured, one critical bottleneck continues to slow down adoption: data access for semantic retrieval.
At the heart of modern GenAI systems lies Retrieval-Augmented Generation (RAG). This paradigm depends on fast access to embeddings – high-dimensional vectors that represent meaning. For Java developers, integrating this capability has historically meant introducing an external vector database into the architecture. What appears to be a straightforward design decision quickly becomes a source of complexity, inefficiency, and architectural friction.
EclipseStore 4 introduces a fundamentally different approach. By integrating the JVector library directly into a Java-native object graph persistence engine, it eliminates the boundary between application memory and vector search. The result is not just another database integration, but a shift in how GenAI systems can be built in Java.
This article explores that shift, from the challenges of external vector systems to the mechanics of embedded vector search, and ultimately to the architectural implications of bringing vector intelligence directly into the JVM.
Data Wall in Java-Based GenAI Architectures
Java developers building GenAI applications today typically rely on external vector databases such as Pinecone, Milvus, or Weaviate. These systems introduce a structural mismatch with Java’s object-oriented programming model. The fundamental issue lies in translation. Domain objects in Java must be flattened into vectors or serialized formats before being sent across the network. The vector database operates independently, returning identifiers or payloads that must then be rehydrated into Java objects. This constant transformation introduces what can be described as a serialization tax.
The cost of this tax is not just computational. It manifests in multiple dimensions. Latency increases due to network round-trips. System complexity grows as additional infrastructure must be provisioned and maintained. Consistency becomes harder to guarantee because the vector store and the primary data store evolve independently. More importantly, this architecture breaks one of Java’s core strengths: working with rich, interconnected object graphs. Instead of navigating relationships directly in memory, developers must orchestrate distributed queries across heterogeneous systems. This is the “Data Wall” – a boundary between application logic and semantic data that prevents truly efficient GenAI implementations in Java.
EclipseStore: Native Object Graph Persistence
EclipseStore approaches persistence from a fundamentally different perspective. Rather than mapping objects to tables or JSON documents, it stores entire Java object graphs directly in binary form. The persistence layer is not an external system but an extension of the application itself.
When an application starts, the object graph can be loaded into memory, either fully or partially. From the developer’s perspective, working with persistent data feels identical to working with in-memory objects. There is no impedance mismatch, no ORM, and no need for query languages.
This design has profound implications. It allows applications to maintain complex relationships without translation. It enables direct navigation through object graphs. And it shifts the performance model from network-bound operations to memory-bound operations. With EclipseStore 4, this model is now extended into the domain of vector search.
JVector: Embedded Vector Search for Java
JVector is a high-performance vector search library implemented entirely in Java. It provides efficient similarity search capabilities for high-dimensional embeddings using modern approximate nearest neighbor algorithms such as HNSW (Hierarchical Navigable Small World). Unlike external vector databases, JVector is designed to run inside the same JVM as the application. This eliminates network overhead and allows direct interaction with application data structures.
The integration of JVector into EclipseStore creates something new: an embedded, Java-native vector search engine. Vector indices become part of the object graph. Similarity queries operate directly on in-memory data. The distinction between “database” and “application” begins to dissolve.
Vector Search in Practice
At its core, vector search is about measuring similarity. Unstructured data—text, images, or signals—is transformed into numerical vectors using embedding models. Each vector represents a point in a high-dimensional space. The distance between two vectors indicates how similar they are. Common distance metrics include cosine similarity and Euclidean distance. A query vector is compared against a collection of stored vectors, and the closest matches are returned. In traditional systems, this process requires specialized infrastructure. In EclipseStore 4, it becomes a native operation.
public class Document {
private String id;
private String content;
private float[] embedding;
}
Once stored in EclipseStore, these objects can be indexed using JVector. A similarity search becomes a simple in-process operation:
List<Document> results = store.search(
VectorQuery.builder()
.index("documents")
.vector(queryEmbedding)
.limit(5)
.build()
);
There is no network call, no serialization step, and no transformation layer. The query operates directly on the in-memory representation of the data.
From Persistence Engine to Embedded Vector Database
The integration of JVector effectively transforms EclipseStore into an embedded vector database. However, it is important to understand that this transformation does not follow the traditional database model. EclipseStore does not introduce a query language or a separate runtime. Instead, it extends the capabilities of the native Java object graph. Vector indices are attached to objects, and search operations are exposed as APIs. This approach preserves the strengths of Java development. Developers continue to work with familiar constructs – classes, collections, and references – while gaining access to advanced semantic search capabilities.
Indexing Concepts in EclipseStore
Indexing in EclipseStore is explicit and developer-controlled. Unlike traditional databases, where indexing is often abstracted away, EclipseStore encourages developers to align index structures with application requirements.
Vector indices are used for semantic similarity. They are built on top of JVector and optimized for high-dimensional search. In addition to vector indices, EclipseStore supports other index types tailored to specific use cases. Standard object indices provide efficient lookups for exact matches or range queries. Specialized indices, such as geospatial indices, enable efficient spatial queries. The key principle is alignment. Each index type is designed for a specific class of problems. Using the correct index ensures optimal performance, while misuse can lead to inefficiencies.
GigaMap: Scaling Beyond Memory
One of the most significant challenges in GenAI systems is scale. While vector indices benefit from being in memory, the underlying data can grow far beyond the capacity of RAM. EclipseStore addresses this challenge with the GigaMap concept. GigaMap introduces a separation between hot and cold data. Hot data, such as vector indices, remains in memory for fast access. Cold data, including large object graphs, is persisted to storage.
When a query identifies relevant vectors, the associated objects are loaded into memory on demand. This lazy loading mechanism ensures that only the necessary data is brought into RAM, minimizing memory usage while preserving performance. The result is a system that can handle massive datasets while maintaining the responsiveness of in-memory operations.
Persistence Mechanics and Performance Considerations
Persistence in EclipseStore is based on writing binary representations of object graphs to storage. This process is highly efficient, but its performance depends on how it is used. One of the most important principles is batching. Writing data in large blocks significantly improves throughput. Small, frequent writes introduce overhead and reduce performance.
The underlying reason lies in the nature of I/O operations. Writing to disk is orders of magnitude slower than CPU operations. Maximizing throughput requires minimizing the number of I/O roundtrips and maximizing the amount of data written per operation. Empirical measurements show that write performance increases dramatically with larger batch sizes, approaching hardware limits when data is written in sufficiently large blocks. This has direct implications for application design. Developers should structure their persistence logic to collect and store data in bulk. Avoiding redundant writes and minimizing fragmentation are critical for achieving optimal performance.
Keeping Hot Indices in Memory
A central advantage of EclipseStore 4 is the ability to keep vector indices in memory while persisting the rest of the data. This hybrid approach combines the speed of in-memory computation with the scalability of persistent storage. Vector search operations are executed entirely in RAM, ensuring minimal latency. Once relevant results are identified, the associated data can be loaded from storage. This design aligns with the access patterns of GenAI applications. Most queries require fast similarity search followed by retrieval of a relatively small subset of data. By optimizing for this pattern, EclipseStore achieves both performance and scalability.
Open Source and Community
EclipseStore is an Eclipse open-source project published under the Eclipse Public License (EPL) 2.0, ensuring that it remains open and accessible to the Java community. The license allows using the code in commercial software. Developers can explore the source code, contribute to the project, and build custom extensions. The project’s documentation provides comprehensive guidance on usage and architecture. The source code is available on GitHub, enabling transparency and collaboration. The project is gaining great momentum and attracting more and more Java developers.
Rethinking GenAI Architectures in Java
EclipseStore 4 challenges the prevailing assumption that vector search must be externalized. By embedding vector capabilities directly into the JVM, it simplifies architecture, reduces latency, and restores the integrity of the object model. This shift has broader implications. It enables new classes of applications that were previously impractical due to latency or complexity constraints. It allows developers to build systems where data and logic coexist seamlessly, without artificial boundaries.
Practical Use Cases
EclipseStore 4 opens the door to a wide range of applications. Local-first AI systems can perform semantic search over private data without relying on external services. Enterprise knowledge systems can integrate structured and unstructured data into a single object graph. Real-time decision systems can combine vector similarity with traditional business logic in a single process. Applications in finance, healthcare, manufacturing, and edge computing can all benefit from this approach. Any scenario that requires fast, context-aware retrieval can leverage embedded vector search.
Getting Started
Developers interested in exploring EclipseStore 4 can access the project through its official documentation and GitHub repository. The documentation provides detailed guides, examples, and best practices for building applications with embedded vector search. Code examples and sample projects demonstrate how to integrate vector indexing, perform similarity queries, and optimize persistence strategies. The combination of EclipseStore and JVector represents a new paradigm for Java-based GenAI development. By bringing vector intelligence into the JVM, it removes long-standing barriers and enables a more natural, efficient way to build intelligent applications.
Distributed Vector Database Apps with Eclipse Data Grid
EclipseStore is built for single-node apps. With Eclipse Data Grid, you can run your EclipseStore app as a distributed application very easily. It begins where EclipseStore ends. Eclipse Data Grid is not a server, but a software-as-code project. It provides the code that creates the entire cluster infrastructure you need to run EclipseStore across multiple cluster nodes. The cluster provides you with replication, high availability, elastic scaling, scale-to-zero, and backup features. It takes over the entire cluster control, including when new nodes are started up, or nodes are shut down, and what happens when nodes crash. Eclipse Data Grid is also an Eclipse open-source project under the EPL 2.0 license.
Links:
EclipseStore project: www.eclipsestore.io
EclipseStore code on GitHub: https://github.com/eclipse-store/store
EclipseStore Documentation: https://docs.eclipsestore.io/manual/storage/getting-started.html
Eclipse Data Grid code on GitHub: https://github.com/eclipse-datagrid/
Eclipse Public License 2.0: https://www.eclipse.org/legal/epl-2.0/
JVector on GitHub: https://github.com/datastax/jvector
Want to Dive Deeper?
Richard Fichtner is a speaker at JCON! In this article, he shows how to implement a high-performance vector data layer directly in the JVM. In Markus Kett’s keynote “Scaling Data in a Sovereign AI Platform”, he explains how to build a sovereign AI platform at scale together with Johann Strauss (CTO AI Solutions, Dell Technologies).
For a deeper technical perspective, join the session & workshop “Building Powerful GenAI Apps with Pure Java” (Florian H.) Christinan K. shows in “Caching and Beyond – Smarter Data Processing with Java” how similar principles can be applied to reduce complexity, improve performance, & consolidate data systems using pure Java.
If you can’t attend live, the session videos will be available after the conference – it’s worth checking out!
