Vasily Sizov. Funky life.

Blog of IT manager, Java EE architect and developer more...

Tuesday, January 26, 2010

Step-by-step guide to JPA-enabled unit testing with TestNG, Unitils, DBUnit, EasyMock and in-memory RDBMS HSQLDB

The term «unit testing» means testing of some atomic units of code (usually some beans, some classes) in isolation from other parts of a system. My experience shows that a lot of time the code is just interaction with a database. That's why it is a good idea to have an ability of unit testing with a live persistence context.

This post contains a complete testing solution based on top of Unitils and included following products:

  • EasyMock — mocking framework which allows to create mock objects easily and use them within our tests. We use it to isolate our tested object from other parts of a system.
  • DBUnit — framework which allows us to set defined state of a database between tests. We use it when we perform database-related testing.
  • DBMaintain — utility which allows to control schema of a database including: generating, editing, migrating from version to version, etc. In our case we use it to prepare database schema for using with DBUnit.
  • TestNG — framework which provides basic architecture for out tests. You can easily switch to JUnit if you like.
  • Unitils — framework which integrates all of the enumerated technologies and provides tools to simplify testing.
  • HSQLDB — fast in-memory database to serve our database-related tests.

To grasp the concept let's take a look at the diagrams.

This first one displays the live environment of The Bean we are going to test.

The second one displays the testing environment.

Please note, you can still use your production DBMS within the test environment if your code depends on its proprietary features. All you need is to create another Persistence Unit and to define it within test. Greater details follow.

How to setup the test environment

I assume that you use Maven to build your project. First of all we should configure our project’s .pom properly.


<build>
    <testResources>
        <testResource>
            <directory>src/test/resources</directory>
        </testResource>

        <testResource>
            <directory>${basedir}/src/test/java/</directory>
            <excludes>
                <exclude>**/*.java</exclude>
            </excludes>
        </testResource>
    </testResources>
     
    …

    <plugins>          
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.5</version>
        </plugin>
       
        …

    </plugins>
</build> 

In this part of .pom we specify that Maven should place various resource files to target test directory (by default it places only .class files ignoring .xml, etc). We have to do it because DBUnit uses .xml files to determine data fixture. As you can see we also specify the latest version of Surefire plug-in to run TestNG/JUnit tests. Earlier versions of Surefire contain various bugs related to TestNG tests running.

What's next? We specify needed dependencies in Dependency Management. Then we should append dependencies to concrete module’s .pom.


<dependencyManagement>
    <dependencies>                                         
        <dependency>
            <groupId>org.unitils</groupId>
            <artifactId>unitils-dbunit</artifactId>
            <version>3.1</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>org.unitils</groupId>
            <artifactId>unitils-orm</artifactId>
            <version>3.1</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>org.unitils</groupId>
            <artifactId>unitils-dbmaintainer</artifactId>
            <version>3.1</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>org.unitils</groupId>
            <artifactId>unitils-easymock</artifactId>
            <version>3.1</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>org.unitils</groupId>
            <artifactId>unitils-testng</artifactId>
            <version>3.1</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>org.unitils</groupId>
            <artifactId>unitils-inject</artifactId>
            <version>3.1</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>5.10</version>
            <scope>test</scope>
            <classifier>jdk15</classifier>
        </dependency>
        
        <dependency>
            <groupId>hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
            <version>1.8.0.10</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>oswego-concurrent</groupId>
            <artifactId>concurrent</artifactId>
            <version>1.3.4</version>
            <scope>test</scope>
        </dependency>       
    
        ...    
        
    </dependencies>
</dependencyManagement>

Note. At this moment there is a little problem with resolving of required jta-1.0.1B.jar dependency from main Maven repository. To solve it you have to download JTA 1.0.1B classes zip file from Sun’s site. Then install it to the Maven’s local repository with a command: mvn install:install-file -Dfile=./jta-1_0_1B-classes.zip -DgroupId=javax.transaction -DartifactId=jta -Dversion=1.0.1B -Dpackaging=jar
That’s it!

Now let’s configure persistence.xml to make our tests JPA-enabled. We don’t need to install HSQLDB separately -- It runs automatically during a testing phase. As mentioned earlier you can define another Persistence Unit or edit this one to use another data base.
Please notice, you have to list your entity classes manually.


<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
  version="1.0">
    <persistence-unit name="testPersistenceUnit" transaction-type="RESOURCE_LOCAL">
        <provider>org.hibernate.ejb.HibernatePersistence</provider>
        <class>com.sizovpoint.entity.Tag</class>
        <class>com.sizovpoint.entity.Post</class>

        <exclude-unlisted-classes>true</exclude-unlisted-classes>
        <properties>
            <property name="hibernate.connection.provider_class" value="org.hibernate.connection.DriverManagerConnectionProvider" />
            <property name="hibernate.connection.url" value="jdbc:hsqldb:mem:unit-testing-jpa"/>
            <property name="hibernate.connection.driver_class" value="org.hsqldb.jdbcDriver"/>
            <property name="hibernate.dialect" value="org.hibernate.dialect.HSQLDialect"/>
            <property name="hibernate.connection.username" value="sa"/>
            <property name="hibernate.connection.password" value=""/>
        </properties>
    </persistence-unit>
</persistence>

Also we should place some configuration files to test resources directory:

unitils.properties file to configure Unitils:


# Defaults and other keys with explanations can be found there: http://unitils.org/unitils-default.properties
unitils.module.database.enabled=true
unitils.module.dbunit.enabled=true
unitils.module.hibernate.enabled=false
unitils.module.mock.enabled=true
unitils.module.easymock.enabled=true
unitils.module.inject.enabled=true
unitils.module.spring.enabled=false
unitils.module.jpa.enabled=true

database.driverClassName=org.hsqldb.jdbcDriver
database.url=jdbc:hsqldb:mem:unit-testing-jpa

database.dialect=hsqldb
database.userName=sa
database.password=
database.schemaNames=PUBLIC

org.unitils.core.dbsupport.DbSupport.implClassName.hsqldb=org.unitils.core.dbsupport.HsqldbDbSupport
org.dbunit.dataset.datatype.IDataTypeFactory.implClassName.hsqldb=org.dbunit.ext.hsqldb.HsqldbDataTypeFactory

dbMaintainer.script.locations=src/test/resources
dataSetStructureGenerator.xsd.dirName=target/xsd
dbMaintainer.autoCreateExecutedScriptsTable=true
dbMaintainer.fromScratchEnabled=true
updateDataBaseSchema.enabled=true

jpa.persistenceProvider=hibernate

The next one is database_schema.sql file which contains DDL SQL for database generation with DBMaintainer. For our example it contains the following:


create table Post (
    id bigint generated by default as identity (start with 1),
    title varchar(255) not null,
    content varchar(2000) not null,
    creationDate timestamp,
    primary key (id));

create table Tag (
    id bigint generated by default as identity (start with 1),
    name varchar(255) not null,
    primary key (id),
    unique (name));

create table PostsToTagsLink (
    id bigint generated by default as identity (start with 1),
    post_id bigint not null,
    tag_id bigint not null,
    primary key (id));

alter table PostsToTagsLink add constraint FKC1 foreign key (post_id) references Post;

alter table PostsToTagsLink add constraint FKC2 foreign key (tag_id) references Tag;

That's it! Let’s move on.

Example application to test

Let’s assume we have four classes:

CreatePost EJB is an entrance for post creating. Invoking public void create(String title, String content, String tagsNames); method with some title, content and string like «apple, red» creates and persists a new instance of the Post entity with related Tags. For working with Tags there is TagManager which responsible for persisting of new tags and resolving of already persisted ones. TagManager is used by CreatePost.

Summarizing the code looks like:


@Entity
public class Post {
    Long id;
    List<Tag> tags; // many-to-many association
    String content;
    String title;
    Date creationDate;
   
    … // getters & setters
}

@Entity
@Table(uniqueConstraints = {@UniqueConstraint(columnNames={"name"})})
public class Tag {
    Long id;
    String name;

    … // getters & setters
}

@Name("createPost")
@Stateless
public class CreatePostBean implements CreatePost {
    @In TagManager tagManager;
    @In EntityManager entityManager;

    public void create(String title, String content, String tagsNames) {
        List<Tag> persistedTags = tagManager.findOrPersistTags(tagsNames);

        Post post = new Post();
        post.setTags(persistedTags);
        post.setContent(content);
        post.setTitle(title);
        post.setCreationDate(new Date());

        entityManager.persist(post);
    }
}


@AutoCreate
@Name("tagManager")
@Stateless
public class TagManagerBean implements TagManager {
    @In EntityManager entityManager;

    public List<Tag> findOrPersistTags(String tagsNames) {
        List<Tag> tags = new ArrayList<Tag>();

        List<String> distinctTagsNames = getDistinctTagsNamesList(tagsNames);
        for (String tagName : distinctTagsNames) {
            tags.add(findOrPersistTag(tagName));
        }

        return tags;
    }

    public List<String> getDistinctTagsNamesList(String tagsNames) {
        List<String> names = new ArrayList<String>();
        StringTokenizer tokenizer = new StringTokenizer(tagsNames, ",");

        while (tokenizer.hasMoreElements()) {
            String foundTagName = tokenizer.nextToken().trim();
            if (!names.contains(foundTagName)) names.add(foundTagName);
        }

        return names;
    }

    public Tag findOrPersistTag(String tagName) {
        Tag tag;
        try {
            tag = (Tag) entityManager
              .createQuery("select t from Tag as t where lower(t.name) = lower(:tagName)")
              .setParameter("tagName", tagName)
              .getSingleResult();
        } catch (NoResultException e) {
            tag = new Tag(tagName);
            entityManager.persist(tag);
        }

        return tag;
    }
}

Please notice that I provide the code before tests only for better understanding. Good practice is to follow Test-Driven Development principles.

Testing

Let's begin with testing of all 3 public methods of our TagManager.

First of all we should prepare some fixture for the database. This is DBUnit feature. Fixture is written in XML. Let’s call it with some meaningful name. I prefer to give it the same name as for the test class – TagManagerTest.xml:


<?xml version='1.0' encoding='UTF-8'?>
<dataset>
    <tag id="1" name="Apple" />
    <tag id="2" name="Dog" />
    <tag id="3" name="Horse" />
</dataset>

Done with fixture. So far so good. Let's write tests. Please take a look at @JpaEntityManagerFactory annotation. You can use it to define which Persistence Unit you would like to use within your tests.


@DataSet("TagManagerTest.xml")
@JpaEntityManagerFactory(persistenceUnit = "testPersistenceUnit")
public class TagManagerTest extends UnitilsTestNG {
    @TestedObject
    TagManagerBean tagManager;

    @InjectIntoByType
    @PersistenceContext
    EntityManager entityManager;

    @Test
    public void getPersistedTagTestForNewTag() {
        // There is no newTag "Mouse" in the DB, so we expect that
        // it will be created and persisted
        Tag newTag = tagManager.findOrPersistTag("Mouse");

        assert newTag.getName().equals("Mouse");
        assert findTag("Mouse") != null;
    }

    @Test
    public void getPersistedTagTestForExistingTag() {
        // Get it manually from the DB and compare with a result of the tested method
        Tag existingTag = findTag("Apple");
        Tag tag = tagManager.findOrPersistTag("Apple");

        assert existingTag.equals(tag);
    }

    @Test
    public void getDistinctTagsNamesListTest(){
        List<String> result = tagManager.getDistinctTagsNamesList("Horse, Apple, Mouse");

        // Unitils provides some extremely useful methods like this
        ReflectionAssert.assertReflectionEquals(Arrays.asList("Horse", "Apple", "Mouse"), result);
    }


    @Test
    public void getPersistedTagsTest() {
        List<Tag> tags = tagManager.findOrPersistTags("Horse, Apple, Mouse, Dog, Orange");

        // Expecting that Mouse and Orange are became persisted
        assert findTag("Mouse") != null;
        assert findTag("Orange") != null;

        // Checks equality of field "name" of each Tag with provided values
        ReflectionAssert.assertPropertyLenientEquals("name", Arrays.asList("Horse", "Apple", "Mouse", "Dog", "Orange"), tags);
    }

    private Tag findTag(String name) {
        try {
            return (Tag) entityManager
              .createQuery("select t from Tag t where t.name = :name")
              .setParameter("name", name)
              .getSingleResult();
        } catch (NoResultException e) {
            return null;
        }
    }
}

Now let’s test CreatePostBean class.
The fixture (UploadPicture.xml):


<?xml version='1.0' encoding='UTF-8'?>
<dataset>
    <tag id="1" name="Apple" />
    <tag id="2" name="Dog" />
    <tag id="3" name="Horse" />
    <tag id="4" name="Cat" />
    <tag id="5" name="Bird" />
</dataset>

And the test class:


@DataSet("CreatePostTest.xml")
@JpaEntityManagerFactory(persistenceUnit = "testPersistenceUnit")
public class CreatePostTest extends UnitilsTestNG {
    @TestedObject
    CreatePostBean createPost;

    @InjectIntoByType
    @PersistenceContext
    EntityManager entityManager;

    @Mock
    @InjectIntoByType
    TagManagerBean tagManager;


    @Test
    public void createTest() {
        // Prepare state for testing
        String tagsNames = "Horse, Apple, Cat, Dog, Bird";
        String title = "Test title";
        String content = "Test content";

        // Prepare data for tagManager mock to return
        List<Tag> tagsListForMocking = (List<Tag>) entityManager.createQuery("select t from Tag t").getResultList();

        // Set expectations
        EasyMock
          .expect(tagManager.findOrPersistTags(tagsNames))
          .andReturn(tagsListForMocking);

        // Set replay mode and execute the code
        EasyMockUnitils.replay();

        createPost.create(title, content, tagsNames);

        // Done! Let's check what's happened

        Post post = (Post) entityManager.createQuery("select p from Post p").getSingleResult();

        assert post.getTitle().equals(title);
        assert post.getContent().equals(content);

        ReflectionAssert.assertPropertyLenientEquals("name", Arrays.asList("Horse", "Apple", "Cat", "Dog", "Bird"), post.getTags());

        // If you are familiar with EasyMock you notice that there's no invocation of verify() method to check
        // expectations for our mock. Thanks to Unitils -- it's invoked automatically.
    }
}    

That's all for today. You can download the project here. Please feel free to contact me if you have any questions.

8 comments:

  1. Really goog Article

    ReplyDelete
  2. Hi,

    Can you get away with not using database_schema.sql?
    Can you let the code figure out your schema from the EJB3 annotations?

    Thanks,
    J

    ReplyDelete
  3. Hi Juan,

    You are not the first who ask me about it :)

    I'm afraid, it doesn't work out of the box. At least in the version I used here.
    I think it's possible, but requires some research to do it.

    I remember that I read some kind of discussions about enabling Hibernate's hbm2ddl plug-in for schema generation,
    but there were some problems in lifecycle and it didn't work properly.

    Maybe it's already solved.

    Hope it helps.


    Best regards,
    Vasily

    ReplyDelete
  4. I have a simpe testproject that verifies autocreate for three different jpa-providers. They all work well and are tested in the project they are located in.

    Hibernate:
    http://code.google.com/p/injection-extensions/source/browse/trunk/transactions/src/test/resources/META-INF/persistence.xml

    OpenJPA
    http://code.google.com/p/injection-extensions/source/browse/trunk/transactions/src/test/openjparesources/META-INF/persistence.xml

    EclipseLink
    http://code.google.com/p/injection-extensions/source/browse/trunk/transactions/src/test/eclipselinkresources/META-INF/persistence.xml

    ReplyDelete
  5. Hi Robert,
    Thank you very much for your contribution!

    ReplyDelete
  6. recently i faced the problem of testing against db. since i don't have "per developer test db" i had no choice but to use embedded db.
    i used your article as my starting point and made some changes:
    1) let hibernate to figure out schema and create it for you
    2) since dbunit had some boolean conversion problems which i couldn't resolve after hour or so and remember (from previous projects) unreadable mid-size dbunit files and all the configuration files and few other reasons i thought: would it be great to have one simple tool which would execute ddl statements (update schema if needed) and move the db to a state required by test in a object oriented way? Because of all those reasons i've written my own tool "jpa-unit" :) which allow me to define entities, execute statements and manage schema definition files if needed.

    So, here it is: JPA-Unit (http://code.google.com/p/jpa-unit/)
    I hope you will find it useful.

    ReplyDelete
  7. Hi Vasily,


    I read your post. I'm trying to get working Unitils and DBMaintainer self, but I still have no success. I post my problem to Stackoverflow (http://stackoverflow.com/questions/6679370/unitils-db-maintainer-in-memory-db-and-non-existing-table), but nobody knows, what happens. You might know how to crack this problem.

    Thanks

    ReplyDelete
  8. Business Phone Service is ideal for small business and entrepreneurs. It gives a professional image and make sure that you don't miss a single call. Few months back, I decided to buy Hosted pbx from Telcan. You can get more information here: Business Phone Service

    ReplyDelete