13 minute read

How to Write Your Own DbUnit

5. August 2026

Intro

You need to reach that (at least) 80% test coverage demanded either by your outer customer or your inner perfectionist. To achieve that noble goal you need a lot of tests — which usually means that you need a lot of test data.

Intro
The first step to dealing with a problem is admitting that you have a problem. — Jase Robertson
Is there a better way?
How come I have never heard of it before?
Refresher: test pyramid vs. test trophy, testcontainers.
What about X?
Problem formulation and initial design
Quick introduction to etcd and its Java client jetcd
Implementation
Down the rabbit hole
Conclusion

Moment after moment, sprint follows sprint, and now you are suddenly left alone in the dark (mode UI) debugging the inner depths of some hundreds or thousands LOCs of the dreadful TestData class, trying to understand the dependencies between helper and builder methods all trying to simplify creation of complex interwoven data objects.

Or worse, you need to dive into some persistence mock layer someone thought would be a good idea to introduce a while ago (shortly before leaving the project, obviously).

Or you notice that your tests were accidentally working simply because all the database interaction in the test was running in the same transaction and on production systems nothing actually was persisted.

Or you are caught in the urgent need to exactly order the test method executions due to reuse of residue data.

This list of horror scenarios can be continued forever, and I am almost certain each and every developer faced above challenges in one or another form.

So — are we doomed or is there a way out of these torment circles?

The first step to dealing with a problem is admitting that you have a problem. — Jase Robertson

How do you spot the problem? Some of the indicators were already mentioned above, but here is a list of what I have already seen in the wild:

TestUtil#createTestData(Connection) Testdata#createX()

EntityManager#unwrap(Session.class)

@BeforeEach void setUp() {
    datasource.getConnection()
              .execute("src/test/resources/some-test.sql"); 
}

not using Testcontainers/using a different in-memory database as a replacement for the real one

H2
embedded versions of MySQL/Redis/etc.

Is there a better way?

Yes!

The core problem stated above is either
a) lack of separation between test logic and test data, or
b) cumbersome boilerplate to fill data into the database before test execution/ to clean up the database afterwards

Let’s briefly look into DbUnit and Database Rider (starting with Database Rider) and check their offering to see how they can help us.

In a nutshell, Database Rider offers you the possibility to easily preseed your database before the test and assert on the database after the test by means of (mostly) only 3 annotations:

@DBRider ~ to tell the test that it’s going to use
@DataSet ~ to preseed the database
@ExpectedDataSet ~ to assert on the database

So instead of using State + Action ⇒ Reaction, asserting on Reaction and hoping the whatever is right was indeed persisted, you now do State + Action ⇒ Reaction + NewState, also verifying the persistence layer itself.

A typical test could look like (https://github.com/database-rider/database-rider/blob/master/rider-examples/quarkus-dbunit-sample/src/test/java/com/github/quarkus/sample/QuarkusDBUnitTest.java)

@QuarkusTest
@DBRider
public class QuarkusDBUnitTest {
    @Test
    @DataSet("book-empty.yml")
    @ExpectedDataSet("book-expected.yml")
    public void shouldCreateBookViaRestApi() {
        final Book book = new Book("Joshua Bloch", "Effective Java (2nd Edition)", 2001, "Tech", " 978-0-3213-5668-0");

        given()
                .body(book)
                .contentType(MediaType.APPLICATION_JSON)
                .when()
                .post("/api/books")
                .then()
                .statusCode(CREATED.getStatusCode());
    }
}

with book-empty.yml

book:

and book-expected.yml

book:
  - id: "regex:[0-9]{1,3}" #any number with 1 to 3 digits
    author: "Joshua Bloch"
    genre: "Tech"
    isbn: "regex:^[0-9]{4}-[0-9]{1,3}-TEST$" #ex: 1234-291-TEST or 1234-29-TEST or 1234-2-TEST
    title: "Effective Java (2nd Edition)"
    year: 2001

respectively.

And in the background, Database Rider would use DbUnit — defining some abstraction layer to compare datasets and to read/ write datasets from/to databases — all utilizing standard JDBC (+ database-specific extensions) features.

How come I have never heard of it before?

Surprisingly (at least from my experience), neither DbUnit nor Database Rider seem to be widely known in the developer community — are those libraries mature enough to be used in production, is there support and/or community?

DbUnit was first released to public (https://www.dbunit.org/) in 2002 to assist in writing better quality tests covering the persistence layer (quick reminder: JDBC was released in 1997, see https://en.wikipedia.org/wiki/Java_Database_Connectivity). It offered a framework to seed the database before the tests and be able to compare the contents of the database against an expected dataset. Back then stored procedures still played a huge role in implementing the business logic, so being able to also validate all the side effects was very precious.

Still it was very verbose, so Arquillian, which was already simplifying application server developers’ lives added Arquillian Persistence Extension (https://arquillian.org/arquillian-extension-persistence/) to ease developers pains and actually encourage them write more tests, separate test logic from test data and de-boilerplate code.

Well as much as we all love our application servers (cough!) with introduction of Spring and later SpringBoot a similar simplified integration was needed and founded — SpringTest DbUnit (https://github.com/springtestdbunit/spring-test-dbunit).

Also — and that’s what I love about open-source — DbUnit Rules project (https://github.com/rmpestano/dbunit-rules) was started based on Arquillian Persistence Extension, DbUnit Rules would later become Database Rider (https://github.com/database-rider/database-rider) and my go-to database testing weapon of choice in almost every project involving relational databases as persistence layer.

Refresher: test pyramid vs. test trophy, testcontainers.

Who has not been asked at least once about the test pyramid (https://martinfowler.com/articles/practical-test-pyramid.html) in the job interviews during the course of the career? Historically the differentiation between unit and integration/component tests made a lot of sense:

unit test are faster and require less external dependencies compared to integration tests
database instances are limited and subject to strict licensing and hardware requirements

So the databases would be either mocked or replaced by in-memory ones, thus reducing the reliability of automated tests and reducing functionality coverable by tests (surprise-surprise, all those stored procedures and feature/syntax differences between the databases are still present!).

But with Docker (or other container runtimes) and Testcontainers to support their usage programmatically, many databases (even the “enterprisier” ones) allow for quickly spinning up short-lived database instances for CI purposes, thus reducing the need to fiddle with the persistence layer and allowing for more robust and reliable testing also of side effects happening directly in the database.

The industry wizards have long recognized the shift (see https://x.com/kentcdodds/status/960723172591992832, https://martinfowler.com/articles/2021-test-shapes.html) in the ratio between different test types and so should we! Assisted with modern tooling, testing becomes a breeze!

What about X?

Thankfully, we’ll always have DbUnit for relational databases (until maybe someday license changes do us part). We also have neat integrations (Database Rider has grown to support not only Spring/SpringBoot, but also Quarkus, Micronaut and many others). But what about non-relational databases? Mongo, Redis, Cassandra, Neo4j? More esoteric and rarely used like ExistDB or maybe even Eclipsestore?

Although there are several projects, that try to close the gap — e.g. NoSqlUnit (https://github.com/lordofthejars/nosql-unit/), CassandraUnit (https://github.com/jsevellec/cassandra-unit) or MongoUnit (https://mongounit.org/, https://github.com/mongounit/mongounit) — they mostly look either stale or abandoned.

So how hard is it really to write (and support) a framework like DbUnit or extension like Database Rider? Let’s try to find out.

Problem formulation and initial design

We need a database to implement support for — we take etcd (https://etcd.io/), which is a distributed key-value store. Selection seems to be appropriate due to its similarity to other non-relational databases/ stores (Redis, Zookeeper), no frameworks to support it (yet) and manageable amount of features.

It also comes with a Java client out-of-the-box, which is a great plus for us.

Now to what we want to achieve in our EtcdUnit. We want a Database Rider similar setup with one class-level annotation to indicate the use of EtcdUnit — let’s name it @EtcdUnit. We also need @EtcdDataset and @EtcdExpectedDataset to preseed the database before the test method execution and assert once the test method completes. We only plan for support of JUnit 5 — so we’ll use JUnit 5 Extension. Last but not least, we want to support YAML datasets only.

Note: we will use words “database”, “storage” and “store” interchangeably when addressing etcd.

Quick introduction to etcd and its Java client jetcd

Etcd (https://etcd.io/) is a strongly consistent, distributed key-value store. Most of us know it from Kubernetes, where it is used for all API server data (https://kubernetes.io/docs/concepts/overview/components/).

Although it offers several core features — watches, leases (also supported by the jetcd client) — we will concentrate our efforts of the key-value store part of it.

To get ourselves familiarized with the API let us first write a small test not relying on using future @EtcdUnit.

First, some helper methods

public class BsUtils {
  public static ByteSequence s2bs(String s) {
    return ByteSequence.from(s, StandardCharsets.UTF_8);
  }

  public static String bs2s(ByteSequence bs) {
    return bs.toString(StandardCharsets.UTF_8);
  }
}

Now the test itself

public class ClientTest {

  @Test
  void valueUpdateIsSuccessful() throws ExecutionException, InterruptedException {
    try (EtcdCluster cluster = Etcd.builder().withDebug(true).withNodes(1).build()) {
      cluster.start();
      Client client = Client.builder().endpoints(cluster.clientEndpoints()).build();

      // setup
      ByteSequence key1 = s2bs("key1");
      ByteSequence value1 = s2bs("value1");
      client.getKVClient().put(key1, value1).get();

      // business logic
      ByteSequence value2 = s2bs("value2");
      client.getKVClient().put(key1, value2).get();

      // assertions
      assertThat(bs2s(client.getKVClient().get(key1).get().getKvs().getFirst().getValue()))
          .isEqualTo("value2");
      assertThat(DatasetReader.from(client).getRows()).hasSize(1);

      client.close();
      cluster.stop();
    }
  }

  @Test
  void deleteAllIsSuccessful() throws ExecutionException, InterruptedException {
    try (EtcdCluster cluster = Etcd.builder().withDebug(true).withNodes(1).build()) {
      cluster.start();
      Client client = Client.builder().endpoints(cluster.clientEndpoints()).build();

      // setup
      ByteSequence key1 = s2bs("key1");
      ByteSequence value1 = s2bs("value1");
      client.getKVClient().put(key1, value1).get();

      // business logic
      ByteSequence key = s2bs("\0");
      client.getKVClient().delete(key, DeleteOption.builder().withRange(key).build()).get();

      // assertions
      assertThat(DatasetReader.from(client).getRows()).isEmpty();

      client.close();
      cluster.stop();
    }
  }
}

Notice, how clumsy and bloated the test looks, how much boilerplate is there and how it’s masking the actual test! And all that already in the simplest possible scenario of key-value store, where we don’t have to create any complex linked structures!

There is definitely room for improvement, so let’s proceed.

Implementation

Note: we omit import statements for brevity, full source code can be found at https://github.com/coiouhkc/article-javapro-dbunit

Let’s start with defining the extension skeleton first.

public class EtcdUnitExtension
    implements BeforeTestExecutionCallback,
        AfterTestExecutionCallback,
        BeforeEachCallback,
        AfterEachCallback,
        BeforeAllCallback,
        AfterAllCallback {

  @Override
  public void afterAll(ExtensionContext context) throws Exception {
    System.out.println("afterAll");
  }

  @Override
  public void afterEach(ExtensionContext context) throws Exception {
    System.out.println("afterEach");
  }

  @Override
  public void afterTestExecution(ExtensionContext context) throws Exception {
    System.out.println("afterTestExecution");
  }

  @Override
  public void beforeAll(ExtensionContext context) throws Exception {
    System.out.println("beforeAll");
  }

  @Override
  public void beforeEach(ExtensionContext context) throws Exception {
    System.out.println("beforeEach");
  }

  @Override
  public void beforeTestExecution(ExtensionContext context) throws Exception {
    System.out.println("beforeTestExecution");
  }
}

Then we go over to the annotations.

@Target({ ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
@ExtendWith(EtcdUnitExtension.class) // <<-- magic happens here, automatically including @ExtendWith whenever @EtcdUnit is used.
public @interface EtcdUnit {
}

@Target({ ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
public @interface EtcdDataset {
  String value() default ""; // <<-- path to the dataset file
}

@Target({ ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
public @interface EtcdExpectedDataset {
  String value() default ""; // <<-- path to the expected dataset file
}

Now let’s create the first tests (that will inevitably fail at the moment). You’ve already seen the setUp and tearDown parts of bootstrapping the etcd cluster and stopping it before and the three tests simply test the happy path of different dataset with expected dataset combinations.

@EtcdUnit
public class ExtensionTest {
  private static EtcdCluster cluster;
  private static Client client;

  @BeforeAll
  static void setUp() {
    ExtensionTest.cluster = Etcd.builder().withDebug(true).withNodes(1).build();
    cluster.start();

    ExtensionTest.client = Client.builder().endpoints(cluster.clientEndpoints()).build();
  }

  @AfterAll
  static void tearDown() {
    client.close();
    cluster.stop();
  }

  @EtcdExpectedDataset(value = "datasets/expected_put.yml")
  @Test
  void withEmptyPutSuccessful() {
    client
        .getKVClient()
        .put(
            ByteSequence.from("hello".getBytes(StandardCharsets.UTF_8)),
            ByteSequence.from("world".getBytes(StandardCharsets.UTF_8)));
  }

  @EtcdDataset(value = "datasets/single.yml")
  @Test
  void withPreseedGetSuccessful()
      throws ExecutionException, InterruptedException, TimeoutException {
    GetResponse getResponse =
        client
            .getKVClient()
            .get(ByteSequence.from("hello".getBytes(StandardCharsets.UTF_8)))
            .get(1L, TimeUnit.SECONDS);
    assertThat(getResponse).isNotNull();
    assertThat(getResponse.getKvs()).hasSize(1);
    assertThat(getResponse.getKvs().getFirst()).isNotNull();
    assertThat(getResponse.getKvs().getFirst().getValue().toString(StandardCharsets.UTF_8))
        .isEqualTo("world");
  }

  @EtcdDataset(value = "datasets/single.yml")
  @EtcdExpectedDataset(value = "datasets/expected_overwritten.yml")
  @Test
  void withPreseedOverwriteSuccessful()
      throws ExecutionException, InterruptedException, TimeoutException {
    client
        .getKVClient()
        .put(
            ByteSequence.from("hello".getBytes(StandardCharsets.UTF_8)),
            ByteSequence.from("javapro".getBytes(StandardCharsets.UTF_8)));

    GetResponse getResponse =
        client
            .getKVClient()
            .get(ByteSequence.from("hello".getBytes(StandardCharsets.UTF_8)))
            .get(1L, TimeUnit.SECONDS);
    assertThat(getResponse).isNotNull();
    assertThat(getResponse.getKvs()).hasSize(1);
    assertThat(getResponse.getKvs().getFirst()).isNotNull();
    assertThat(getResponse.getKvs().getFirst().getValue().toString(StandardCharsets.UTF_8))
        .isEqualTo("javapro");
  }

Now the real work begins, let’s roll up the sleeves.

For starters we definitely need to configure an instance of jetcd Client in our JUnit 5 extension to read from and write to the database, but how are we going to do it? There are multiple options:

passing the client endpoints via the annotation directly
system properties
environment variables
some kind of client connection holder
class lookup
(framework-dependent, for later considerations) cdi, bean injection

Since we spin up the etcd cluster dynamically at runtime (using Testcontainers under the hood), let’s use the quick-and-dirty passing of client endpoints via system properties:

public class EtcdUnitExtension
    implements BeforeTestExecutionCallback,
        AfterTestExecutionCallback,
        BeforeEachCallback,
        AfterEachCallback,
        BeforeAllCallback,
        AfterAllCallback {
  public static String PROP_CLIENT_ENDPOINTS = "etcd-unit-extension.client-endpoints";

  ...

  private Client newClient() {
    return Client.builder()
        .endpoints(System.getProperty(PROP_CLIENT_ENDPOINTS).split(","))
        .connectTimeout(Duration.ofSeconds(1L))
        .build();
  }
}

and simultaneously update our EtcdExtensionTest

@EtcdUnit
public class ExtensionTest {
  
  ...

  @BeforeAll
  static void setUp() {
    ...

    System.setProperty(
        EtcdUnitExtension.PROP_CLIENT_ENDPOINTS,
        cluster.clientEndpoints().stream().map(URI::toString).collect(Collectors.joining(",")));
  }

  ...
}

Now some helpers to read and write datasets to/from file/store (note that we purposefully only include key and value, omitting all the advanced features, e.g. version, revision, lease, etc.)

public class Data {
  private String key;
  private String value;

  public Data() {}

  public Data(String key, String value) {
    this.key = key;
    this.value = value;
  }

  public String getKey() {
    return key;
  }

  public void setKey(String key) {
    this.key = key;
  }

  public String getValue() {
    return value;
  }

  public void setValue(String value) {
    this.value = value;
  }
}

Note: usage of records might be complicated by choice of de-/serialization library and/or Java version.

public class Dataset {
  private List<Data> rows;

  public Dataset() {}

  public Dataset(List<Data> rows) {
    this.rows = rows;
  }

  public List<Data> getRows() {
    return rows;
  }

  public void setRows(List<Data> rows) {
    this.rows = rows;
  }
}

public class DatasetReader {
  public static Dataset from(String url) {
    Yaml yaml = new Yaml(new Constructor(Dataset.class, new LoaderOptions()));
    InputStream inputStream = DatasetReader.class.getClassLoader().getResourceAsStream(url);
    return yaml.load(inputStream);
  }

  public static Dataset from(Client client) throws ExecutionException, InterruptedException {
    // see https://github.com/etcd-io/jetcd/issues/266
    ByteSequence key = ByteSequence.from("\0", StandardCharsets.UTF_8);
    return new Dataset(
        client
            .getKVClient()
            .get(
                key,
                GetOption.builder()
                    .withSortField(GetOption.SortTarget.KEY)
                    .withSortOrder(GetOption.SortOrder.DESCEND)
                    .withRange(key)
                    .build())
            .get()
            .getKvs()
            .stream()
            .map(
                keyValue ->
                    new Data(
                        keyValue.getKey().toString(StandardCharsets.UTF_8),
                        keyValue.getValue().toString(StandardCharsets.UTF_8)))
            .toList());
  }
}

public class DatasetWriter {
  public static void to(Client client, Dataset dataset) {
    if (dataset == null || dataset.getRows() == null) {
      return;
    }

    dataset
        .getRows()
        .forEach(
            data -> {
              try {
                client
                    .getKVClient()
                    .put(
                        ByteSequence.from(data.getKey(), StandardCharsets.UTF_8),
                        ByteSequence.from(data.getValue(), StandardCharsets.UTF_8))
                    .get();
              } catch (InterruptedException | ExecutionException e) {
                throw new RuntimeException(e);
              }
            });
  }
}

And we ready to return to implementing our extension.

As agreed in the design phase, we want to preseed the database before the test method execution iff it’s annotated with @EtcdDataset:

public class EtcdUnitExtension
    implements BeforeTestExecutionCallback,
        AfterTestExecutionCallback,
        BeforeEachCallback,
        AfterEachCallback,
        BeforeAllCallback,
        AfterAllCallback {
  ...

  @Override
  public void beforeTestExecution(ExtensionContext context) throws Exception {
    EtcdDataset datasetAnnotation =
            context.getElement().get().getAnnotation(EtcdDataset.class);
    if (datasetAnnotation != null) {
      String datasetUri = datasetAnnotation.value();
      Dataset dataset = DatasetReader.from(datasetUri);
      try (Client client = newClient()) {
        DatasetWriter.to(client, dataset);
      }
    }
  }
  
  ...
}

Similarly, once test method execution completes iff the test method is annotated with @EtcdExpectedDataset we want to check that the content of the database is exactly the same as expected and we want to prepare the next test method execution by wiping the database clean.

public class EtcdUnitExtension
    implements BeforeTestExecutionCallback,
        AfterTestExecutionCallback,
        BeforeEachCallback,
        AfterEachCallback,
        BeforeAllCallback,
        AfterAllCallback {
  ...

  @Override
  public void afterTestExecution(ExtensionContext context) throws Exception {
    EtcdExpectedDataset expectedDatasetAnnotation =
        context.getElement().get().getAnnotation(EtcdExpectedDataset.class);
    if (expectedDatasetAnnotation != null) {
      // compare content
      String expectedDatasetUri = expectedDatasetAnnotation.value();
      Dataset expectedDataset = DatasetReader.from(expectedDatasetUri);
      try (Client client = newClient()) {
        Dataset actualDataset = DatasetReader.from(client);
        DataComparator.compare(actualDataset, expectedDataset);
      }

      // cleanup all
      try (Client client = newClient()) {
        ByteSequence key = ByteSequence.from("\0", StandardCharsets.UTF_8);
        client.getKVClient().delete(key, DeleteOption.builder().withRange(key).build()).get();
      }
    }
  }
  
  ...
}

Note the shortcut in DataComparator again — we are keeping it extremely simple and simply throw a RuntimeException with minimal information about the cause. Definitely something to improve on if you intend to use in production!

public class DataComparator {
  public static void compare(Dataset x, Dataset y) {
    if (x == null && y == null) {
      return;
    }

    if (x == null || y == null) {
      throw new DataComparisonException("One of datasets is <null>.");
    }

    if (x.getRows() == null && y.getRows() == null) {
      return;
    }

    if (x.getRows() == null || y.getRows() == null) {
      throw new DataComparisonException("One of datasets has <null> rows.");
    }

    if (x.getRows().size() != y.getRows().size()) {
      throw new DataComparisonException("Different row size.");
    }

    boolean xInY =
        x.getRows().stream()
            .map(
                dataInX ->
                    y.getRows().stream()
                        .anyMatch(
                            dataInY ->
                                dataInY.getKey().equals(dataInX.getKey())
                                    && dataInY.getValue().equals(dataInX.getValue())))
            .reduce(Boolean.TRUE, (b1, b2) -> b1 & b2);

    boolean yInX =
        y.getRows().stream()
            .map(
                dataInY ->
                    x.getRows().stream()
                        .anyMatch(
                            dataInX ->
                                dataInX.getKey().equals(dataInY.getKey())
                                    && dataInX.getValue().equals(dataInY.getValue())))
            .reduce(Boolean.TRUE, (b1, b2) -> b1 & b2);

    if (!xInY || !yInX) {
      throw new DataComparisonException("Different datasets.");
    }
  }
}

public class DataComparisonException extends RuntimeException {
  public DataComparisonException(String message) {
    super(message);
  }
}

And just like that we are now ready to execute our test again!

Down the rabbit hole

Are we done yet? Do we fell lucky? Well, it depends… During the implementation we have definitely taken some shortcuts and now it’s exactly the right time to revise them and sketch the roadmap for next iterations.

So if you feel adventurous and up for the challenge, here is a thing or two to add, based on Database Rider features:

Better diff messages

Obviously, nobody is going to like the Different datasets. exception message, longing for more — it’s up to you to define a suitable format. Some kind of 3-way diff (left-only, both, right-only) should be appropriate.

Multiple schemas and multiple databases

Usually, one (micro)service is supposed to operate on a single database, but the enterprise reality is often way harsher. The necessity to operate on multiple database of same kind could be caused be migration, company policies, write vs. read replicas and many more.
Same reasoning could be applied to use of different schemas within the same database (if supported), though (potentially, if fully qualified table names could be used) with slightly different consequences.

Although Database Rider does offer some support for multiple schemas and connections out of the box using the annotation-based approach above, the recommendation (particularly if trying to combine multiple datasources in the same test) is using RiderDSL (see https://github.com/database-rider/database-rider?tab=readme-ov-file#riderdsl and https://github.com/database-rider/database-rider/issues/248)

Connection getConnFromDs1() {
  ...
}
Connection getConnFromDs2() {
  ...
}
@Test
void useMultipleDatasources() throws SQLException, DatabaseUnitException {
  withConnection(getConnFromDs1())
          .withDataSetConfig(new DataSetConfig("dataset-ds1.yml"))
          .createDataSet();
  withConnection(getConnFromDs2())
          .withDataSetConfig(new DataSetConfig("dataset-ds2.yml"))
          .createDataSet();
  
  // ... test logic ...
  
  withConnection(getConnFromDs1())
          .withDataSetConfig(new DataSetConfig("expected-ds1.yml"))
          .expectDataSet();
  withConnection(getConnFromDs2())
          .withDataSetConfig(new DataSetConfig("expected-ds2.yml"))
          .expectDataSet();
}

Repeatable annotations, dataset merging

Sometimes (in particularly in case of larger databases with many tables) your datasets contain both fixed and variable parts:

fixed being entries that you copy over and over again between your datasets, e.g. some basic configuration, list of predefined users with predefined roles, VATs, units, etc.
variable being entries you need specifically for the one test, e.g. a user without roles or a posts with comments nested over newly introduced maximum nesting level.
In these cases it makes sense to split the dataset and re-use the fixed part over and over again, thus embodying the DRY principle.
To enable it we could make value in EtcdDataset be an array (also see https://github.com/database-rider/database-rider/blob/master/rider-core/src/main/java/com/github/database/rider/core/api/dataset/DataSet.java)

@Target({ ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
public @interface EtcdDataset {

  String[] value() default "";
}

Define the annotation to be @Repeatable, define a “container” annotation (e.g. @EtcdDatasets); define the overwrite/merge rules and we”re good to go (still relatively easy due to locality of dataset configuration).

Those who want to go the extra mile, might consider merging @EtcdDataset annotations defined on class (@Target({ ElementType.TYPE })) level with the @EtcdDatasets on @BeforeAll, @BeforeEach and test method levels as well.

Partial expectations

For now, we have only implemented a very strict dataset comparison strategy — both the actual and expected dataset must match exactly, all attributes of all rows. But what if we only want to assert on parts of data, what if simply cannot know all the exact values (think e.g. now() as default value and auto increment or nextval() for ids)?

Applied to our case, what if would only like to test for a specific key being stored in the database?

Well, in these cases we would need the ability to specify a comparison strategy. In case of Database Rider those are EQUALS (default), CONTAINS and PROLOG (more on it below).

Smarter datasets

Going further, once you have familiarized yourself with the basic features, you will most probably want to add some dynamics to your datasets, e.g. value generation (e.g. “set last logged in date to yesterday” or “include contents of a large file here”). If you would want to skip creating those datasets programmatically, it” where “dataset replacers” and “scriptable dataset” features (as in Database Rider, see https://github.com/database-rider/database-rider?tab=readme-ov-file#dataset-replacers and https://github.com/database-rider/database-rider?tab=readme-ov-file#scriptable-datasets) could come in very handy.

rows:
  - key: "[DAY,YESTERDAY]"
    value: "Weather was OK"

rows:
  - key: "groovy:java.util.UUID.randomUUID()"
    value: "This could be your personal identifier"

Obviously, defining the set of available replacers and possibilities to extend them with custom implementations; as well as defining the list of supported scripting languages might be quite time-consuming.

Even smarter assertions

As you might have noticed, most of the improvements above were enhancing out ability to do something with single row of a dataset. But what if would need to check the dependencies between the rows, e.g. the relations between dynamically created tree nodes, where we do not know the exact ids of children and parent nodes?
Specifically for this case, Database Rider offers PROLOG (https://en.wikipedia.org/wiki/Prolog) kind assertions. By using specific placeholders, formulating the state of the database as “theory” and our expectations as “query” we could assert on the dataset as a whole!

DatabaseRider:

USER:
  - ID: $$x$$
    NAME: "@realpestano"
TWEET:
  - ID: abcdef12345
    CONTENT: "dbunit rules!"
    USER_ID: $$x$$

etcd-unit:

rows:
  - key: "$$r$$"
    value: '{"name": "root"}'
  - key: "$$n_1$$"
    value: '{"name": "root", "parent": "$$r$$"}'

Multiple data format support

We all know the “competing standards” comic (https://xkcd.com/927/), so why limit the options? Initial choice of YAML is obviously very opinionated, to conquer the audience you might need to support TXT, CSV, XML, TOML, even EXCEL!

And many more

data types and custom data types (think enums, arrays, json, geometric, etc.) which can be very different across different databases
data cleanup strategies (e.g. clean all tables, clean only dataset tables)
wildcard/regexp support (e.g. skipping data cleanup for all QUARTZ_* tables)

Conclusion

You use relational database in your project — give Database Rider a try, if you are stuck — the database rider community is there for you!

You use a non-relational database — try one of the tools listed or try creating your own.

And if you want to work on EtcdUnit — drop me a line or a PR at https://github.com/coiouhkc/article-javapro-dbunit!

This article is part of the JAVAPRO magazine issue:

Autonomous Java

Explore how modern Java is evolving beyond frameworks and APIs toward intelligent, highly automated software systems. Discover why sustainable engineering, maintainable architectures, automation, semantic search, AI agents, and runtime optimization are becoming the foundation for the next generation of enterprise applications.

Discover the edition →