Automatic DB migrations for Spring Boot with Liquibase

Introduction

I have lately written a short tutorial on Building REST services with Spring Boot, JPA and MySQL, with Part 1 and Part 2.

I decided to add an essential part to any serious project with an SQL store: management of Database Migrations.

In a real world where requirements will change, or when schemas cannot be fully designed up front, you will be facing a real problem sooner than later: how to manage changes in Database schema, once the application or web service is running?

I wrote an article some time ago, which still seems to be quite valid. You can read there the details of a recommended development workflow to cope with Database migrations in all phases of development.

In this article, I will apply the same ideas from my previous article to a more up-to-date app: a Spring Boot Rest Web service with MySQL.

Adding the Liquibase plug-in for Maven

Let's add the Liquibase plug-in.
What we want to achieve at this first step is to be able to evolve our model and keep working as before adding Liquibase: the Hibernate Maven plugin will take care of recreating the schema up-to-date every time we run tests or lunch the application.

We add the dependency:



Our initial Liquibase changeset will be empty.

We need to add the Database Changelog File for liquibase: the file where all changes managed by Liquibase are registered. It will initially be empty.
We add the file src/main/resources/db/db.changelog.xml:



We add the liquibase.properties file:



We finally add the plug-in with relevant configuration:



With this configuration, we are telling Liquibase not to try any database migrations. This is our normal workflow of updating model with JPA annotations and running tests. Hibernate plugin takes care of dropping and re-creating the database.

This is what we see if we run mvn clean test

Generating DB diff automatically with Liquibase: First migration

We have finished evolving our model and adding the additional logic and tests.
We are now happy and ready to commit a change.
This point could even be the very first version of your DB schema!

Let's generate the DB diff with Liquibase.
The generated diff file will be incorporated to the registered Liquibase DB schema updates. Additionally, when running our app, Liquibase will take care of migrating the DB schema to the latest version registered in our codebase.

To make all this magic happen, let's add a profile in our pom file so we can generate the DB diff anytime.


Let's generate our first Liquibase migration with mvn clean process-test-resources -Pdb-diff:



Liquibase has generated for us the file src/main/resources/db/db-20170709_144112.changelog.xml with this contents:




Great! We can now add this filename to our global DB changelog file:

Subsequent migrations with Liquibase

 To check our migration mechanism works well, let's update our model with a version field and generate again a DB diff via mvn process-test-resources -Pdb-diff.

Liquibase generates this file:



This seems like magic!

Automatic DB migration embedded in the app

Having added liquibase in our dependencies has also included a Liquibase Spring Bean to our app. This bean runs at application startup, checks the registered changesets against the app DB and brings the DB schema up-to-date automatically by applying any needed migrations.

It would be good to see this in action at development time, so we can test it.

Let's add another profile to our maven project for this.



This profiles skips any schema generation by the Hibernate plugin and drops the database. This way, when the app starts, the Liquibase Spring bean will enter into action and be forced to run all registered migrations.

If we now run the app via mvn clean spring-boot:run -Pdb-test:



Works as expected!

Source Code and Additional Info

Source code for this tutorial here.
Additional Liquibase maven plugin info: mvn liquibase:help
More info about Liquibase here.
Some more info in my previous article about DB migration.

Building REST services with Spring Boot, JPA and MySql: Part 2

Introduction

In the first part of this tutorial we saw how to build a skeleton java app from scratch based on the Spring framework and implemented the persistence to MySql database.


In this second part, we will implement a REST web service with the Spring framework.


I'll be using maven 3, version 3.0.5 and Java 8 SDK. Google around for installation of these in your environment.

Step 2: Implement a REST endpoint with Spring

In order to use the Spring framework as the basis for our REST endpoint, we need to add the necessary dependencies to our existing pom.xml:



We already have a model persisted to MySql and now we will add a controller with a method index that retrieves all persisted instances of our model.

We will annotate this method so that it is published as a REST endpoint when running our app within a Servlet container.



The Spring annotations added to our code are:

• @RestController This declares our class as a controller returning domain objects instead of views. Spring will take care of the JSON serialization automatically via Jackson serializer
  
  
•  @RequestMapping(value="/games", method = RequestMethod.GET) This maps GET requests for the path /games to our controller method.

We can now add a test for our new REST endpoint.



In our test, instead of running our controller within an external application Server, we use the Spring class MockMvc, which will direct requests to our controller, making our test faster.

If we now run mvn clean test:

Running our REST endpoint

We are now ready to package our app and run it.

If we run mvn clean package:



We now have a jar and we can just run it. Yes!!! That's right: we can just run it directly!
Spring has generated an uber jar: a jar with all needed dependencies to run our app, including an embedded servlet container: by default Tomcat, but you can  easily change it for Jetty or any other of your preference.

If we launch the command
 
java -jar target/spring-boot-mysql-0.0.1-SNAPSHOT.jar


We can see on the console a Tomcat has started and is listening on 8080 for requests!

Source Code

Source code on GitHub

Building REST services with Spring Boot, JPA and MySql: Part 1

Introduction

In this tutorial, we will see how to build a skeleton Java app from scratch based on the Spring framework and capable of having an evolving model persisted on MySql and a related REST web service.

As requirements are changing continuously, we will be handling updates to our model, which in the end translate as updates to our underlying database schema, with Liquibase: a database migration tool

For an overview of how you can manage Database migrations in your development lifecycle, have a look at one of my previous articles: Automatic DB migration for Java web apps with Liquibase

I'll be using maven 3, version 3.0.5 and Java 8 SDK. Google around for installation of these in your environment.

Step 1: Persist a model with JPA and Hibernate

Let's start with what Spring gives us in Spring Initializr for  a maven project with dependencies JPA and MySql.

Here's the generated POM.



In order to have a non-failing maven project, we need to add the details of the database schema to our project.

The resulting properties section in the POM:


And we add these properties to application.properties:


If we launch  mvn clean package  we have now a successful build.

For details on how to create and assign user permissions on MySql, Google is your friend :-)

Adding our Model and Repository

Let's add a sample Model class to our app.



And a Repository interface to access the persisted data. Spring Data will automatically generate the implementation for us.



We can now add a test to load all instances from our repository and verify it is working correctly.



In order to populate our Database for tests, we have the option of using Spring annotations directly in our Java unit test source code.

In this case, we will be using the maven-db-unit plugin instead.

Our updated pom.xml:



And our src/test/resources/sample-data.xml for the unit tests.




If we now run our test with mvn clean test, we have a build failure: we have no tables in our MySql schema and dbunit cannot insert the test data.



At this point, we need to generate a DDL script for our schema.

There are a number of options. You could opt for a Spring solution.

We will apply a more generic solution from a third party which works on Spring and non-Spring frameworks: Hibernate Maven Plugin from juplo.de. This is a completely new implementation of the Hibernate Maven plugin updated to Hibernate 5.

We need to add these lines to our pom.xml:



And the file src/test/resources/hibernate.properties needed by the hibernate-maven-plugin:


Notice in the updated pom.xml:

• The hibernate-maven-plugin must appear before the dbunig-maven-plugin: the database tables will be created before the dbunit sample data is inserted.

• Additionally, the file src/test/resources/hibernate.properties needs to be filtered by the standard maven resources plugin.

If we run mvn clean test, our test is finally passing after creating the database tables and populating them with unit test data:


We leave for a future part publishing a REST web service for our model and handling automatic Database migration with Liquibase.

Source code: GitHub

Check Part 2 of this tutorial here.