2013/04/07

Adding Signup email confirmation to AppFuse

Introduction


In todays SaaS world, user signup usually requires email confirmation because of continuous events that trigger new email messages to users.

AppFuse is a RAD Java web app framework, supporting a multitude of web frameworks: Spring MVC, Struts2, JSF, Tapestry, and there are more to come.

Let's implement a user signup email confirmation service for AppFuse. This can be adapted to any other Java web app framework.

Initial Web App

From the AppFuse quickstart page, I copy the maven command to generate an initial Struts 2 app from AppFuse archetypes:

mvn archetype:generate -B 
 -DarchetypeGroupId=org.appfuse.archetypes 
 -DarchetypeArtifactId=appfuse-basic-struts-archetype 
 -DarchetypeVersion=2.2.1 
 -DgroupId=com.operatornew 
 -DartifactId=signup 
 -DarchetypeRepository=http://oss.sonatype.org/content/repositories/appfuse

As we will be updating the User services, we generate the full source code from the newly created signup directory:

mvn appfuse:full-source

User email verification

 How can we verify a new user's email?

1. A new user signs up and fills in an email address which we want to verify as valid.

2. After the user submits their data, we'll generate a unique and difficult to guess token for each that signs up. The new user won't be able to log in until they complete their email verification process.

3. We'll send an email with a URL from our app which will include this generated token. As AppFuse supports multi-language, we'll generate the email in the active locale.

4. When the new user receives the confirmation email, they will be able to visit the included URL with unique token to say "Hey, it's me. I've received your difficult-to-guess token at the email address I gave you". We will mark then this user as confirmed.

User Signup Confirmation: Service and Model Layers

Ok. Let's add a new Java interface for our new Signup confirmation service. The service will be responsible for starting a user's data confirmation process and confirming the user's data. We will apply it to email verification, but it could be applied to mobile phone number verification as well.

public interface SignupConfirmService {

    User doConfirm(User user);

    /**
     * Starts a user Confirmation procedure
     *
     * @param user the User whose data needs to be confirmed. The user's signupDate will be updated accordingly
     * @param context the Web App context from which the locale and URL will be obtained
     * @return
     */
    User startConfirm(User user, WebAppContext context);

}

Hold on. What is this WebAppContext type? We'll need to include our web app URL in the generated email. As we'll implement the confirmation in the service level, we'll avoid to add an unnecessary dependency to servlet classes. After all, we're working at the service layer.


The WebAppContext type will be an abstract class:

public abstract class WebAppContext {
    private Locale locale;

    public WebAppContext(Locale locale) {
        this.locale = locale;
    }

    public Locale getLocale() {
        return locale;
    }

    abstract public String getAppUrl();
}

Updating our Model

We'll add to our User class three pieces of information:
  • a signup date
  • an email confirmation token
  • a confirmation date
We add the dates for administration purposes.

public class User extends BaseObject implements Serializable, UserDetails {

    ...

    private Date signupDate;
    private String emailConfirmToken;
    private Date confirmDate;

    ...

    @Column(name="signup_date", nullable=false)
    @Temporal(TemporalType.TIMESTAMP)
    @Field
    public Date getSignupDate() {
        return signupDate;
    }

    public void setSignupDate(Date signupDate) {
        this.signupDate = signupDate;
    }

    @Column(name="confirm_token", length=32)
    public String getEmailConfirmToken() {
        return emailConfirmToken;
    }

    public void setEmailConfirmToken(String confirm) {
        this.emailConfirmToken = confirm;
    }

    @Column(name="confirm_date")
    @Temporal(TemporalType.TIMESTAMP)
    @Field
    public Date getConfirmDate() {
        return confirmDate;
    }

    public void setConfirmDate(Date confirmDate) {
        this.confirmDate = confirmDate;
    }
}

Implementing our new Service

We can now write an implementation for our new SignupConfirmService to verify a new user's email address.
Our implementation will use AppFuse MailEngine service to send email, a ResourceBundle for mail subject i18n, and the Java SecureRandom class to generate a unique and difficult to guess token for each new user.

public class SignupConfirmEmail implements SignupConfirmService {

    // For the secure random generation of tokens
    private SecureRandom random = new SecureRandom();

    // AppFuse email service, injected by Spring
    private MailEngine mailEngine;
    // AppFuse email prototype, injected by Spring
    private SimpleMailMessage mailMessage;

    // i18n for the subject's email. Configurable in spring bean, with a default bundle name
    private String resourceBundleName = "MailResources";
    private ResourceBundle rb;


    .... // spring bean setters omitted

    @Override
    public User startConfirm(User user, WebAppContext context) {
        String code = new BigInteger(130, random).toString(32);
        user.setEmailConfirmToken(code);
        mailMessage.setTo(user.getFullName() + "<" + user.getEmail() + ">");
        mailMessage.setSubject(getFromResourceBundle("email.signup.subject", context.getLocale()));
        Map<String, Object> model = new HashMap<String, Object>();
        model.put("userFirstName", user.getFirstName());
        model.put("appURL", context.getAppUrl());
        model.put("signupConfirmURL", context.getAppUrl() + "/account/confirm?confirm="+user.getEmailConfirmToken());
        mailEngine.sendMessage(mailMessage, "signupConfirm.vm", model);
        return user;
    }

    @Override
    public User doConfirm(User user) {
        if (!user.isEnabled()) {
            // has not been confirmed yet:
            user.setEnabled(true);
            user.setConfirmDate(new Date());
        }
        return user;
    }

    private String getFromResourceBundle(String key, Locale locale) {
        if (rb == null) {
            try {
                rb = ResourceBundle.getBundle(resourceBundleName, locale);
            } catch (MissingResourceException ex) {
                rb = ResourceBundle.getBundle("com.operatornew." + resourceBundleName, locale);
            }
        }
        String str = rb.getString(key);
        return str;
    }

We can now define the SignupConfirmService implementation as a Spring bean in our applicationContext-service.xml file:

<bean id="signupConfirmService" class="com.operatornew.service.impl.SignupConfirmEmail">
    <property name="mailEngine" ref="mailEngine" />
    <property name="mailMessage" ref="mailMessage" />
</bean>

Next, we'll modify AppFuse's UserManager as follows:
  • add a signup() method for user signup, which will check if there is a signup confirmation service in place
  • we can make the signup confirmation service globally optional with a configuration parameter
  • add a confirmSignup() method for user signup confirmation

The implementation could be this one:

public User signup(User user, WebAppContext app) throws UserExistsException {
    // First save user to check there is no conflict with unique fields
    user = saveUser(user);

    if (signupConfirmService != null) {
        signupConfirmService.startConfirm(user, app);
        if (enableAccountAfterSignupConfirm) {
            user.setEnabled(false);
        }
    }

    // save updated fields
    return save(user);
}

public User confirmSignup(String token) throws UserSignupTokenNotFoundException {
    User user = userDao.getUserFromSignupToken(token);
    if (signupConfirmService != null) {
        user = signupConfirmService.doConfirm(user);
        return userDao.saveUser(user);
    } else {
        throw new RuntimeException("signupConfirmService null while trying to confirm signUp for token " + token);
    }
}

For confirmSignup, we need to add a method to userDao to retrieve a user by their signup token:

public User getUserFromSignupToken(String token) throws UserSignupTokenNotFoundException {
    List users = getSession().createCriteria(User.class).add(Restrictions.eq("emailConfirmToken", token)).list();
    if (users == null || users.isEmpty()) {
        throw new UserSignupTokenNotFoundException("user token'" + token + "' not found...");
    } else {
        return (User) users.get(0);
    }
}

Testing the new Service

If we run our tests now:

mvn test

We see we have broken something:

...
Results :

Failed tests:
  testSave(com.operatornew.webapp.action.SignupActionTest)

Tests in error:
  testAddAndRemoveUser(com.operatornew.service.UserManagerTest)

Tests run: 71, Failures: 1, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

The surefire junit report for one of the failing tests looks like this:
...
Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.584 sec <<< FAILURE!
testAddAndRemoveUser(com.operatornew.service.UserManagerTest)  Time elapsed: 0.082 sec  <<< ERROR!
com.operatornew.service.UserExistsException: User 'john' already exists!
 at com.operatornew.service.impl.UserManagerImpl.saveUser(UserManagerImpl.java:143)
...

However, it does not make sense that a user already exists after our changes. Upon inspection of the test log in the console, I can spot the error:

...
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'signup_date' cannot be null
We fix this by setting a signupDate if it is null in saveUser() method:
...
if (user.getSignupDate() == null) {
    user.setSignupDate(new Date());
}

We rerun our tests:

...
Results :

Tests in error:
  testSave(com.operatornew.webapp.action.SignupActionTest)

Tests run: 71, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

After inspecting the surefire report, we can see this:

-------------------------------------------------------------------------------
Test set: com.operatornew.webapp.action.SignupActionTest
-------------------------------------------------------------------------------
Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.168 sec <<< FAILURE!
testSave(com.operatornew.webapp.action.SignupActionTest)  Time elapsed: 0.064 sec  <<< ERROR!
org.springframework.dao.EmptyResultDataAccessException: Incorrect result size: expected 1, actual 0
 at org.springframework.dao.support.DataAccessUtils.requiredSingleResult(DataAccessUtils.java:71)
 at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:730)
 at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:749)
 at com.operatornew.dao.hibernate.UserDaoHibernate.getUserPassword(UserDaoHibernate.java:109)
...
 at com.operatornew.service.impl.UserManagerImpl.saveUser(UserManagerImpl.java:123)
 at com.operatornew.service.impl.UserManagerImpl.signup(UserManagerImpl.java:88)
...

Our implementation is failing in the new UserManagerImpl signup() method, when saveUser() is called the second time to save fields updated by the SignupConfirmationService implementation. The saveUser() method eventually calls UserDao's getUserPasword() to check if the password needs to be re-encrypted. But getUserPassword() is annotated not to support propagation of transactions, whichs makes it fail because our new user is created within a transaction that has not finished yet:

public interface UserDao extends GenericDao<User, Long> {
    ...
    @Transactional(propagation = Propagation.NOT_SUPPORTED)
    String getUserPassword(Long userId);
}

The second saveUser method call in our new signup method does not need to re-encrypt user password clearly. We can fix it with a private method that just saves the user instance without checking for password encoding, and a bit of refactoring to avoid duplicated code:

public User signup(User user, WebAppContext app) throws UserExistsException {
    // First save user to check there is no conflict with unique fields
    user = saveUser(user);

    if (signupConfirmService != null) {
        signupConfirmService.startConfirm(user, app);
        if (enableAccountAfterSignupConfirm) {
            user.setEnabled(false);
        }
    }
    // save updated fields
    return saveUserNoPwdEncoding(user);
}

public User saveUser(User user) throws UserExistsException {
    ...
    return saveUserNoPwdEncoding(user);
}

private User saveUserNoPwdEncoding(User user) throws UserExistsException {
    if (user.getSignupDate() == null) {
        user.setSignupDate(new Date());
    }

    try {
        return userDao.saveUser(user);
    } catch (Exception e) {
        e.printStackTrace();
        log.warn(e.getMessage());
        throw new UserExistsException("User '" + user.getUsername() + "' already exists!");
    }
}

All our tests now pass.

We can now write a new test for our SignupConfirmService class. We'll use Wiser to mock smtp server, and verify that startConfirm() sets a value on user's emailConfirmToken, that an email is sent and that the email's body contains the generated token:

public class SignupConfirmServiceTest extends BaseManagerTestCase {
    private Log log = LogFactory.getLog(SignupConfirmServiceTest.class);
    @Autowired
    MailEngine mailEngine;
    JavaMailSenderImpl mailSender = new JavaMailSenderImpl();

    @Autowired
    private SignupConfirmService signupConfirmService;
    private User user;

    @Before
    public void setUp() {
        mailSender.setHost("localhost");
        mailEngine.setMailSender(mailSender);
    }

    @After
    public void tearDown() {
        mailEngine.setMailSender(null);
    }

    @Test
    public void testStartConfirm() throws Exception {
        // mock smtp server
        Wiser wiser = new Wiser();
        // set the port to a random value so there's no conflicts between tests
        int port = 2525 + (int)(Math.random() * 100);
        mailSender.setPort(port);
        wiser.setPort(port);
        wiser.start();

        user = new User();

        // call populate method in super class to populate test data
        // from a properties file matching this class name
        user = (User) populate(user);
        assertNull(user.getEmailConfirmToken());

        user = signupConfirmService.startConfirm(user, new WebAppContext(new Locale("en")) {
            public String getAppUrl() {
                return "http://localhost:8080";
            }
        });
        wiser.stop();
        assertNotNull(user.getEmailConfirmToken());

        assertTrue(wiser.getMessages().size() == 1);
        WiserMessage wm = wiser.getMessages().get(0);
        assertThat((String)wm.getMimeMessage().getContent(), containsString(user.getEmailConfirmToken()));
    }
}

It will initially fail as we need to add resources for email subject i18n and the Velocity templates. In MailResources.properties:

email.signup.subject=Confirm your email Address

In signupConfirm.vm Velocity template:

Hello ${userFirstName},

To enable your account, please click in the following link or copy it onto the address bar of your favourite browser.

${signupConfirmURL}

After that, the test passes.

User Signup Confirmation: Web Layer

After implementing the service layer, we can now implement the web layer.

When a new user clicks on the sing up link, a page with a fill-in form is showed and the user enters their info.

After completion of the form, the user will press the signup button and, if configured to confirm email before the account is enabled, a page will be displayed informing the user that an email has been sent to their email address and they need to confirm their account by following email instructions.

In AppFuse with Struts2, the SignupAction class implements the Struts2 action for Signup. The save() method currently saves the new signed up user, sends them an email of signup welcome and logs the new user in.

We will change the implementation by calling the new UserManager.signup() method, eliminate the welcome email and only will log in the new user if the app is NOT configured to confirm email before the account is enabled.

public String save() throws Exception {
    // Set the default user role on this new user
    user.addRole(roleManager.getRole(Constants.USER_ROLE));

    try {
        // call signup, passing current locale and a new anonymous inner class returning the web app URL
        user = userManager.signup(user, new WebAppContext(this.getLocale()) {
            public String getAppUrl() {
                return RequestUtil.getAppURL(getRequest());
            }
        });

    } catch (AccessDeniedException ade) {
        // thrown by UserSecurityAdvice configured in aop:advisor userManagerSecurity
        log.warn(ade.getMessage());
        getResponse().sendError(HttpServletResponse.SC_FORBIDDEN);
        return null;
    } catch (UserExistsException e) {
        log.warn(e.getMessage());
        List<Object> args = new ArrayList<Object>();
        args.add(user.getUsername());
        args.add(user.getEmail());
        addActionError(getText("errors.existing.user", args));

        // redisplay the unencrypted passwords
        user.setPassword(user.getConfirmPassword());
        return INPUT;
    }

    if (userManager.isEnableAccountAfterSignupConfirm()) {
        // user signup needs confirmation to be enabled
        getSession().setAttribute(Constants.REGISTERED, user.getEmail());
        return "confirm";
    } else {
        saveMessage(getText("user.registered"));
        getSession().setAttribute(Constants.REGISTERED, Boolean.TRUE);

        // log user in automatically
        UsernamePasswordAuthenticationToken auth = new UsernamePasswordAuthenticationToken(
                user.getUsername(), user.getConfirmPassword(), user.getAuthorities());
        auth.setDetails(user);
        SecurityContextHolder.getContext().setAuthentication(auth);

        return SUCCESS;
    }
}

We will add two new struts actions:
  • account/needconfirm will inform the user that they need to confirm their account by following email instructions
  • account/confirm will try to confirm a user account with the supplied confirmation token. The result can either be successful or unsuccessful, informing the user accordingly
 The relevant Struts 2 xml configuration file for the new Struts 2 actions:

<action name="saveSignup" class="signupAction" method="save">
    <result name="cancel" type="redirect">/home</result>
    <result name="input">/WEB-INF/pages/signup.jsp</result>
    <result name="success" type="redirectAction">home</result>
    <result name="confirm" type="redirectAction">account/needconfirm</result>
</action>

<action name="account/needconfirm">
    <result>/WEB-INF/pages/signupNeedConfirm.jsp</result>
</action>

<action name="account/confirm" class="signupAction" method="confirm">
    <result name="success">/WEB-INF/pages/signupConfirmed.jsp</result>
    <result name="error">/WEB-INF/pages/signupConfirmError.jsp</result>
</action>

The page to inform the user that they need to confirm their account by following the email instructions can be like this signupNeedConfirm.jsp:

<%@ include file="/common/taglibs.jsp" %>
<%@page import="com.operatornew.Constants" %>
<% String email = (String)session.getAttribute(Constants.REGISTERED);
%>

<head>
    <title><fmt:message key="signupNeedConfirm.title"/></title>
</head>

<body class="signup-confirm"/>

<div class="span2">
</div>

<div class="span8 dialog-info">
<div class="alert alert-block alert-info">
        <h2><fmt:message key="signupNeedConfirm.heading"/></h2>
        <fmt:message key="signupNeedConfirm.message">
            <fmt:param><%= email %></fmt:param>f
        </fmt:message>
    </div>
</div>

<div class="span2"/>

It displays a message informing to which email has been sent the message explaining the user what they need to perform in order to complete the signup process.

The other two new jsp pages are very similar to this one.

Not least important is making the new action URL's publicly accessible to any user. For this, we update Spring's security config file security.xml like this:

<http auto-config="true">
    <intercept-url pattern="/account/confirm*" access="ROLE_ANONYMOUS,ROLE_ADMIN,ROLE_USER"/>
    <intercept-url pattern="/account/needconfirm*" access="ROLE_ANONYMOUS,ROLE_ADMIN,ROLE_USER"/>
    ...
</http>


Screenshots

Here are some screenshots of the added functionality.

Page requesting the user to confirm their email:
Email received by the user:



Page showing successful confirmation and account activation:


Page showing wrong account activation:

Sources


Sources can be found here.

2012/11/30

Automatic DB migration for Java web apps with Liquibase

Introduction


In an scenario of agile development, new versions are frequently released and deployed, and continuous changes in your database schema are frequent.

To deal with these database changes, a mechanism should be in place. In Ruby on Rails, you have it out-of-the-box and it works great. But in Java web apps, you have to find a solution and plug it in your own projects.

We will implement an automatic database update mechanism for Java web apps trying to meet the following goals:
  • It should not interfere during development
  • It should be easy to generate database updates during development
  • It should be easy to test database updates during development
  • At production, database updates should be performed automatically
Liquibase is a good tool to deal with database migrations:
  • Open-source
  • Database agnostic: can update most popular SQL databases
  • Integration: available as command line tool, maven plugin, ant task
  • Flexibility: deals SQL schema updates, custom updates via a Java class, even system commands updates
  • Automatic: can be integrated for automatic updates as spring bean or as servlet listener

Our App Before Liquibase


If we use hibernate and the hibernate3-maven-plugin,  during development our database schema is automatically kept up-to-date: hibernte3-maven-plugin extracts schema info from JPA annotations and hibernate configuration.

We will use a project based on the AppFuse framework, with flavours for Spring MVC, Struts2, Tapestry, JSF, but this is applicable to any java web app based on any framework.

From the AppFuse quickstart page, I copy the maven command to generate an initial Spring MVC app from AppFuse archetypes:

mvn archetype:generate -B
 -DarchetypeGroupId=org.appfuse.archetypes
 -DarchetypeArtifactId=appfuse-basic-spring-archetype
 -DarchetypeVersion=2.2-SNAPSHOT
 -DgroupId=com.mycompany
 -DartifactId=migration
 -DarchetypeRepository=http://oss.sonatype.org/content/repositories/appfuse

By inspecting the project's pom.xml, we can see the database schema is kept up-to-date during development by generating drop and create DDL commands, extracted from the JPA annotations in our model classes.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>hibernate3-maven-plugin</artifactId>
    <version>2.2</version>
    <configuration>
        <components>
            <component>
                <name>hbm2ddl</name>
                <implementation>annotationconfiguration</implementation>
                <!-- Use 'jpaconfiguration' if you're using JPA. -->
                <!--<implementation>jpaconfiguration</implementation>-->
            </component>
        </components>
        <componentProperties>
            <drop>true</drop>
            <jdk5>true</jdk5>
            <propertyfile>target/classes/jdbc.properties</propertyfile>
            <skip>${skipTests}</skip>
        </componentProperties>
    </configuration>
    <executions>
        <execution>
            <phase>process-test-resources</phase>
            <goals>
                <goal>hbm2ddl</goal>
            </goals>
        </execution>
    </executions>
    <dependencies>
    ... <!-- jdbc driver here -->
    </dependencies>
</plugin>

This is fine during development, because we do not need to worry about database migrations as yet. By running maven process-test-resources or maven jetty:run, we can see the tables are dropped and recreated each time we run our web app with jetty:

> mvn jetty:run
...
[INFO] [hibernate3:hbm2ddl {execution: default}]
[INFO] Configuration XML file loaded: file:.../migration/src/main/resources/hibernate.cfg.xml
[INFO] Configuration XML file loaded: file:.../migration/src/main/resources/hibernate.cfg.xml
[INFO] Configuration Properties file loaded: ...\migration\target\classes\jdbc.properties
alter table user_role drop foreign key FK143BF46A9B523CC9;
alter table user_role drop foreign key FK143BF46A407D00A9;
drop table if exists app_user;
drop table if exists role;
drop table if exists user_role;
create table app_user (id bigint not null auto_increment, account_expired bit not null, account_locked bit not null, address varchar(150), city varchar(50), country varchar(100), postal_code varchar(15), province varchar(100), credentials_expired bit not null, email varchar(255) not null unique, account_enabled bit, first_name varchar(50) not null, last_name varchar(50) not null, password varchar(255) not null, password_hint varchar(255), phone_number varchar(255), signup_date date, username varchar(50) not null unique, version integer, website varchar(255), primary key (id)) ENGINE=InnoDB;
create table role (id bigint not null auto_increment, description varchar(64), name varchar(20), primary key (id)) ENGINE=InnoDB;
create table user_role (user_id bigint not null, role_id bigint not null, primary key (user_id, role_id)) ENGINE=InnoDB;
alter table user_role add index FK143BF46A9B523CC9 (role_id), add constraint FK143BF46A9B523CC9 foreign key (role_id) references role (id);
alter table user_role add index FK143BF46A407D00A9 (user_id), add constraint FK143BF46A407D00A9 foreign key (user_id) references app_user (id);
...

Managing db updates in our project


Our development workflow could be like this one:


We will have two main Liquibase usage scenarios:
  • Liquibase at build-time: will generate all db changelogs
  • Liquibase at run-time: will automatically update the server schema as needed on deployment, including generation of the first database version for an empty schema

Liquibase at build-time

We will be evolving our app and when we have something to commit and push to our project's global repo, we can then generate the database migrations, if any.

We will add to our maven project the liquibase plugin and needed executions in order to:
  • generate database diff changelogs at any time when we want to consolidate our model updates
  • generate database production data dumps as changelogs at any time to consolidate app preloaded db data
  • exercise liquibase db migrations at any time for rapid testing (jetty:run)

Liquibase at runtime

The first time we deploy the app, it will contain the initial db changelog for an empty schema. Liquibase will generate all the db tables and populate with initial database data (default users, user roles, any lookup tables...). On subsequent deployments, our app will contain an additional db changelog to bring the server database schema up-to-date.


Integrating the db update at app startup


Liquibase can perform automatic db update at runtime by looking at the registered change sets in a changelog file and checking if they are applied against a table in our schema called DATABASECHANGELOG. It will create it automatically if it does not exist.

We can implement the automatic db update either with a Spring bean or with a Servlet listener.

We add the liquibase lib as dependency:
<dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-core</artifactId>
    <version>2.0.5</version>
</dependency>

And configure the Liquibase Spring bean in spring's applicationContext-resources.xml configuration file:

<bean id="liquibase" class="liquibase.integration.spring.SpringLiquibase">
    <property name="dataSource" ref="dataSource" />
    <property name="changeLog" value="classpath:db/db.changelog.xml" />
    <property name="defaultSchema" value="${db.name}" />
</bean>


Working on our app during development: evolving our model


During development, we will possible be making many changes to our app. We do not want to spend time for now on db migrations. Just evolve our model, annotate it with JPA and when doing rapid testing with jetty:run, generate the db from scratch.

Liquibase maintains a table with applied change sets to our database. Based on the contents of that table, on startup our app will run liquibase to apply any missing migration to our db.

In order to avoid:
  • liquibase trying to create db tables already created by hibernate3 maven plugin
  • liquibase changeset version conflicts (because of changes applied to existing changelogs, for instance)
we will need that maven performs these tasks:
  • Drop all tables from our schema, so DATABASECHANGELOG is deleted
  • Let Hibernate generate our db up-to-date based on our annotations
  • Make liquibase mark the db as up-to-date by updating DATABASECHANGELOG, without applying any db migrations
We can accomplish this by adding to our pom.xml:

<plugin>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-maven-plugin</artifactId>
    <version>2.0.5</version>
        <configuration>
            <skip>${skipTests}</skip>
            <propertyFile>target/classes/liquibase.properties</propertyFile>
            <changeLogFile>target/classes/db/db.changelog.xml</changeLogFile>
        </configuration>
    <executions>
        <!-- drop db before generating schema with hbm2ddl to avoid any 
            inconsistencies between changelog files and DATABASECHANGELOG table -->
        <execution>
            <id>drop-db</id>
            <phase>process-resources</phase>
            <goals>
                <goal>dropAll</goal>
            </goals>
            <configuration>
                <propertyFile>target/classes/liquibase.properties</propertyFile>
                <changeLogFile>target/classes/db/db.changelog.xml</changeLogFile>
            </configuration>
        </execution>
        <!-- mark db up-to-date in the  DATABASECHANGELOG table after generating 
            schema with hbm2ddl so that no migration is executed -->
        <execution>
            <id>mark-db-up-to-date</id>
            <phase>test-compile</phase>
            <goals>
                <goal>changelogSync</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Our liquibase.properties config file:

driver=${jdbc.driverClassName}
url=${jdbc.url}
username=${jdbc.username}
password=${jdbc.password}

Our initial empty liquibase changelog file (db.changelog.xml):

<databaseChangeLog>
</databaseChangeLog>


Generating db diffs to consolidate our model


And now we want to generate our db migration so we can consolidate our updated model. We will want our app ready to run a db update when deployed.

To generate our db diff, this is what we'll do:
  • Generate a mydb_prev schema, based on the current liquibase registered changelogs.
  • Generate a mydb schema based on our JPA annotations
  • Compute the db diff between these schemas and generate the db changelogs
We'll do this in a specific maven profile (db-diff), so we can activate it at any time to generate the db changelogs.

<plugin>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-maven-plugin</artifactId>
    <version>${liquibase.version}</version>
    <configuration>
        <propertyFile>target/classes/liquibase-diff.properties</propertyFile>
        <changeLogFile>target/classes/db/db.changelog.xml</changeLogFile>
        <diffChangeLogFile>src/main/resources/db/db-${timestamp}.changelog.xml</diffChangeLogFile>
        <logging>info</logging>
    </configuration>
    <executions>
        <execution>
            <id>generate-db-prev</id>
            <phase>process-resources</phase>
            <goals>
                <goal>update</goal>
            </goals>
            <configuration>
                <dropFirst>true</dropFirst>
            </configuration>
        </execution>
        <execution>
            <id>generate-db-diff</id>
            <phase>process-test-resources</phase>
            <goals>
                <goal>diff</goal>
            </goals>
        </execution>
    </executions>
    <dependencies>
        <dependency>
            <!-- jdbc driver here -->
        </dependency>
    </dependencies>
</plugin>

I omit the buildnumber-maven-plugin config in our pom.xml at validate phase so that we can generate unique changelog filenames based on current timestamp.

Our liquibase-diff.properties config file:

driver=${jdbc.driverClassName}
url=${jdbc.url.prev}
username=${jdbc.username}
password=${jdbc.password}
referenceDriver=${jdbc.driverClassName}
referenceUrl=${jdbc.url}
referenceUsername=${jdbc.username}
referencePassword=${jdbc.password}

We can now run maven like this:
mvn process-test-resources -Pdb-diff

and voilà. Liquibase generates for us the changelog file for the initial version of our db, as we have run the diff against an empty changelog file:

<databaseChangeLog>
    <changeSet author="jgarcia (generated)" id="1354207484885-1">
        <createTable tableName="app_user">
            <column autoIncrement="true" name="id" type="BIGINT">
                <constraints nullable="false" primaryKey="true"/>
            </column>
            <column name="account_expired" type="BIT">
                <constraints nullable="false"/>
            </column>
            <column name="account_locked" type="BIT">
                <constraints nullable="false"/>
            </column>
            <column name="address" type="VARCHAR(150)"/>
            <column name="city" type="VARCHAR(50)"/>
            <column name="country" type="VARCHAR(100)"/>
            ...
        </createTable>
    </changeSet>
    <changeSet author="jgarcia (generated)" id="1354207484885-2">
        <createTable tableName="role">
    ...

We can now include this file in our initially empty db.changelog.xml file like this:

<databaseChangeLog>
    <include file="db/db-20121120_120949.changelog.xml" />
</databaseChangeLog>


Generating preloaded db data if any


Many web apps will have a set of data preloaded in the db: initial set of internal user accounts, available user roles, list of applicable taxes, ... whatever.

These can also be defined as a changesets so that Liquibase can update the database for us when running our app.

To generate data changesets, the maven liquibase plugin won't be of much help, as it does not include a goal for this. Instead, as it is also included in the plugin, we'll call directly the liquibase main java class as if we were using it from the command line. We'll do it with the exec-maven-plugin in a db-data maven profile so that we can generate the preloaded db data at any time:

<plugin>
    <groupId>org.codehaus.mojo</groupId>  
    <artifactId>exec-maven-plugin</artifactId>  
    <version>1.2.1</version>
    <executions>
        <execution>
            <phase>process-resources</phase>
            <goals>
                <goal>java</goal>
            </goals>
            <configuration>
                <mainClass>liquibase.integration.commandline.Main</mainClass>
                <includePluginDependencies>true</includePluginDependencies>
                <arguments>  
                    <argument>--driver=${jdbc.driverClassName}</argument>
                    <argument>--changeLogFile=src/main/resources/db/db-data-${timestamp}.changelog.xml</argument>
                    <argument>--url=${jdbc.url}</argument>
                    <argument>--username=${jdbc.username}</argument>
                    <argument>--password=${jdbc.password}</argument>
                    <argument>--diffTypes=data</argument>
                    <argument>--logLevel=info</argument>
                    <argument>generateChangeLog</argument>
                </arguments>
            </configuration>
        </execution>
    </executions>
    <dependencies>
        ...
        <!-- jdbdc driver -->
        <!-- liquibase plugin -->
        ...
    </dependencies>
</plugin>
...
<properties>
    <!-- avoid generating db schema + inserting db-unit -->
    <skipTests>true</skipTests>
</properties>

In AppFuse, the profile prod feeds the database with production data instead of test data. We use this profile to regenerate the db and populate it with production data.
After this, we can use the db-data profile to generate our changelog for the initial db data.

After running the maven commands:
mvn test-compile -Pprod
mvn process-resources -Pdb-data

we obtain this file from liquibase:

<databaseChangeLog>
    <changeSet author="jgarcia (generated)" id="1354214520109-1">
        <insert tableName="user_role">
            <column name="user_id" valueNumeric="2"/>
            <column name="role_id" valueNumeric="1"/>
        </insert>
        ...
    </changeSet>
    <changeSet author="jgarcia (generated)" id="1354214520109-2">
        <insert tableName="role">
            <column name="id" valueNumeric="1"/>
            <column name="description" value="Administrator role (can edit Users)"/>
            <column name="name" value="ROLE_ADMIN"/>
        </insert>
        ...
    </changeSet>
    <changeSet author="jgarcia (generated)" id="1354214520109-3">
        <insert tableName="app_user">
            <column name="id" valueNumeric="1"/>
            <column name="account_expired" valueBoolean="false"/>
            <column name="country" value="US"/>
            <column name="postal_code" value="80210"/>
            ...

This file needs to be updated, as liquibase:
  • dumps all data. You will have to keep only the added data to your db from the previous db version. No diff here performed by Liquibase.
  • data is not properly ordered regarding referential integrity

Once this file has been cleaned-up, we can include it as well in our main db.changelog.xml file:

<databaseChangeLog>
    <include file="db/db-20121120_120949.changelog.xml" />
    <include file="db/db-data-20121128_170043.changelog.xml" />
</databaseChangeLog>


Exercising the automatic db migration during rapid testing


Our app is now ready to perform all registered db migrations when deployed in a server. However, it would be nice too to exercise this during development when we launch a jetty:run for rapid testing of our unpackaged app.

For this purpose, we add a profile that performs these steps:
  • drops all tables from our schema
  • skips db schema generation from our JPA annotations
  • skips feeding db with unit test data
We can add these in a db-test profile:
<plugin>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-maven-plugin</artifactId>
    <version>2.0.5</version>
    <executions>
        <execution>
            <id>drop-db</id>
            <phase>process-resources</phase>
            <goals>
                <goal>dropAll</goal>
            </goals>
            <configuration>
                <propertyFile>target/classes/liquibase.properties</propertyFile>
                <skip>false</skip>
            </configuration>
        </execution>
    </executions>
</plugin>
...
<properties>
    <skipTests>true</skipTests>
</properties>

We can now exercise our db migration to check it works fine:

> mvn jetty:run -Pdb-test
...
[INFO] [liquibase:dropAll {execution: drop-db}]
...
[INFO] [hibernate3:hbm2ddl {execution: default}]
[INFO] skipping hibernate3 execution
...
[INFO] Started Jetty Server
...
2012-11-30 13:25:19.341:INFO:/:Initializing Spring root WebApplicationContext
INFO 30/11/12 13:25:liquibase: Successfully acquired change log lock
INFO 30/11/12 13:25:liquibase: Reading from `migration`.`DATABASECHANGELOG`
INFO 30/11/12 13:25:liquibase: Reading from `migration`.`DATABASECHANGELOG`
INFO 30/11/12 13:25:liquibase: ChangeSet db/db-20121120_120949.changelog.xml::20121120_120949::jgarcia (generated) ran successfully in 921ms
INFO 30/11/12 13:25:liquibase: ChangeSet db/db-data-20121128_170043.changelog.xml::20121128_170043-data::jgarcia (generated) ran successfully in 37ms
INFO 30/11/12 13:25:liquibase: Successfully released change log lock

The logs show Liquibase has updated our db successfully.


Liquibase pitfalls


Your db schema will usually be specified in your jdbc connection parameters.
Liquibase automatically inserts schema references in the generated changelogs, for indexes, lke this:
  • baseTableSchemaName="migration"
  • referencedTableSchemaName="migration"
You better erase these or you will run into trouble if you set a different schema name in your jdbc configuration file.

For a full list of the maven liquibase plugin goals and params, you can run this command:

mvn liquibase:help

It is up-to-date, as opposed to the liquibase site documentation.

Liquibase validates db change sets by comparing some attributes of the change sets present in the changelog file against those registered in the DATABASECHANGELOG table as applied change sets. It compares:
  • the full path filename that contains the change set
  • the MD5 checksum of the change sets
If any of these is different, it will try to re-apply the changeset. If you want to avoid this, you can clear the corresponding fields in the db table and liquibase will refill them with the actual values.

To avoid differences in filename because of different path (relative vs absolute path, path updtes, etc), you can set the logicalFilePath attribute of the in each liquibase file. There is no parameter to omit the path of changelog files in an applied changeset.


Liquibase best practices


It is cleaner to have a single db.changelog.xml file that includes the generated db changelog files, so changelogs are grouped together:

<databaseChangeLog>
    <include file="db/db-20121120_120949.changelog.xml" />
    <include file="db/db-data-20121128_170043.changelog.xml" />
    <include file="db/db-20121129_093229.changelog.xml" />
</databaseChangeLog>

I also like to consolidate a set of related changesets from a changelog file in a single changeset. Instead of having many changesets, each one creating a table, creating an index, creating a referential integrity, etc, I tend to group many of these updates as a single changeset.

Liquibase autogenerates a numeric id to identify each changeset. I prefer to assign it a timestamp, as it gives more info and they still appear ordered.

Test, test, test.


Sources


Sources can be found here.

2012/11/07

About Me

I have been working as developer for more than 20 years. I enjoy software development.

In these years I have worked with a variety of technologies: C, C++, ObjectStore and Poet at the beginning of my professional career. Later on, Java and the incipient servlets, Oracle, MySql. Then Struts, Hibernate, Lucene ... And lately, I am working with Java, Spring, Spring Security, Hibernate Search, Struts 2, Apache CXF, Bootstrap and jQuery, to name a few.

Same goes for Engineering practices: from cascade lifecycle to spiral to agile.

And the tools: make, ant, maven, jenkins, ... CVS, visual source safe, subversion, mercurial, git ...

Lately I am into experimenting with Ruby on Rails, Grails, MongoDB. There are some great online courses around about these!

I am interested in technologies that allow to build better applications: meeting user expectations, and with beautiful and maintainable code.

I am a committer of AppFuse: an open-source java web framework. Among other things, I have contributed with:
  • better i18n support
  • upgrade to Hibernate 4
  • re-implement the full-text search service with Hibernate Search + Lucene

BitBucket GitHub

2012/10/24

Adding web resource fingerprinting to AppFuse with wro4j

As a Java web developer, web resource optimization is something that is usually left until you have time to optimize things. Usually, you don't have that time, really.

However, when you have a SaaS on the cloud, you don't want to call or mail your users to ask them clear their browser's cache after a css or js update or bug fix! Do you?

There are a few Java apis for Web Resource Optimization: jawr and wro4j are two of the available Java apis for web resource optimization and fingerprinting.

I decided to integrate wro4j into AppFuse for static resource optimization with the following goals:
  • optimization would be performed at build time
  • static resources should be automatically versioned (fingerprinting)
Let's work through the steps to integrate wro4j into the latest and greatest AppFuse version: the one in development. That is: 2.1.1-SNAPSHOT.

From the AppFuse quickstart page, we can generate a basic web app to start working. I choose Struts2 as the framework.

mvn archetype:generate -B 
  -DarchetypeGroupId=org.appfuse.archetypes 
  -DarchetypeArtifactId=appfuse-basic-struts-archetype 
  -DarchetypeVersion=2.1.1-SNAPSHOT 
  -DgroupId=es.jogaco 
  -DartifactId=wro 
  -DarchetypeRepository=http://oss.sonatype.org/content/repositories/appfuse

After adding a Document test class to the model with a date field, and pushing the app to CloudFoundry, the result is here.
@Entity
@Table(name="document")
public class Document {

    private Long id;
    private String name;
    private Integer copies;
    private Double price;
    private Date releaseDate;
    private Date updated;

    // getters, setters, etc...
 }

Ok. Let's integrate wro4j to do the resource optimization at build-time.


Step 1: defining the resources that will be grouped and processed by wro4j

I first do a bit of resource file reorganization so that all css files are under webapp/styles/** and all js files are under webapp/scripts/**.
In an AppFuse Struts2 project, the js files that are used are these:

scripts/login.js
scripts/script.js
scripts/lib/bootstrap.min.js
scripts/lib/plugins/jquery.cookie.js
scripts/lib/jquery-1.7.2.min.js
scripts/datepicker/bootstrap-datepicker.js
scripts/datepicker/locales/bootstrap-datepicker.br.js
scripts/datepicker/locales/bootstrap-datepicker.da.js
scripts/datepicker/locales/bootstrap-datepicker.de.js
scripts/datepicker/locales/bootstrap-datepicker.es.js
... the rest of locale files for datepicker

The used css files:
styles/style.css
styles/lib/bootstrap.min.css
styles/datepicker/datepicker.css

Then, the resulting wro.xml file is:

<groups xmlns="http://www.isdc.ro/wro">
  <group name='all'>
    <css>/styles/lib/bootstrap.min.css</css>
    <css>/styles/datepicker/datepicker.css</css>
    <css>/styles/style.css</css>
    <js>/scripts/lib/jquery-1.7.2.min.js</js>
    <js>/scripts/lib/bootstrap.min.js</js>
    <js>/scripts/datepicker/bootstrap-datepicker.js</js>
    <js>/scripts/datepicker/locales/*.js</js>
    <js>/scripts/lib/plugins/jquery.cookie.js</js>
    <js>/scripts/script.js</js>
  </group>
</groups>

The order of the files in wro.xml is important, by the way.

Step 2: add wro4j maven plugin to the pom.xml

Of the maven lifecycle phases, the generate-sources seems the most appropriate.

<plugin>
    <groupId>ro.isdc.wro4j</groupId>
    <artifactId>wro4j-maven-plugin</artifactId>
    <version>1.4.9</version>
    <executions>
        <execution>
            <phase>generate-sources</phase>
            <goals>
                <goal>run</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <targetGroups>all</targetGroups>
        <cssDestinationFolder>src/main/wro-target/styles/</cssDestinationFolder>
        <jsDestinationFolder>src/main/wro-target/scripts/</jsDestinationFolder>
        <contextFolder>src/main/webapp/</contextFolder>
        <extraConfigFile>src/main/wro.properties</extraConfigFile>
        <ignoreMissingResources>true</ignoreMissingResources>
        <groupNameMappingFile>src/main/wro-target/wromapping.properties</groupNameMappingFile>
        <wroManagerFactory>ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory</wroManagerFactory>
    </configuration>
</plugin>



By adding this maven plugin, wro4j will generate the bundled resources at build time as defined in our configuration files. Wro4j will generate:

  • all.css
  • all.js

I have chosen to generate files in a separate src/main/wro-target folder so I don't pollute my source folders with generated sources. Other options are fine too.


Step 3: defining the processors for the resources

As the wro4j documentation explains, the pre and post processors of css and js files need to be defined in a wro.properties file, referenced in the maven plugin configuration at src/main/wro.properties.

Starting initially with the example given in wro4j documentation, except for the namingStrategy and hashStrategy which we'll use later on for fingerprinting:
managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory
preProcessors=cssUrlRewriting,cssImport,semicolonAppender,cssMin
gzipResources=true
encoding=UTF-8
postProcessors=cssVariables,jsMin
uriLocators=servletContext,uri,classpath

With this configuration, I run now maven:

Okay. Let's run maven.
> mvn clean generate-sources
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building AppFuse Struts 2 Application
[INFO]    task-segment: [generate-sources]
[INFO] ------------------------------------------------------------------------
[INFO] [wro4j:run {execution: default}]
[INFO] Executing the mojo:
[INFO] Wro4j Model path: ...\appfuse-test\wro\src\main\webapp\WEB-INF\wro.xml
[INFO] targetGroups: all
[INFO] minimize: true
[INFO] ignoreMissingResources: true
[INFO] destinationFolder: ...\appfuse-test\wro\target\wro
[INFO] jsDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] cssDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] groupNameMappingFile: src/main/wro-target/wromapping.properties
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] processing group: all.css
[INFO] wroManagerFactory class: ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory
[INFO] file size: all.css -> 104702 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\styles\all.css (104702 bytes)
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] processing group: all.js
[INFO] file size: all.js -> 143589 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\scripts\all.js (143589 bytes)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Fri Oct 19 19:16:11 CEST 2012
[INFO] Final Memory: 35M/553M
[INFO] ------------------------------------------------------------------------

wro4j has generated for us the two optimized resources. Now it's time we use these in our jsp pages.

Step 4: integrate generated resources in our web app

In AppFuse, we have the default.jsp which composes the final web page with sitemesh.
There we have a few css and javascript referenced. In our documentForm.jsp, there will be a few extra resources to manage the date as well: the datepicker component.
These are our original resources referenced in our jsp pages:

<link rel="stylesheet" type="text/css" media="all" 
    href="<c:url value='/styles/lib/bootstrap.min.css'/>" />
<link rel="stylesheet" type="text/css" media="all" 
    href="<c:url value='/styles/style.css'/>" />
<script type="text/javascript" 
    src="<c:url value='/scripts/lib/jquery-1.7.2.min.js'/>"></script>
<script type="text/javascript" 
    src="<c:url value='/scripts/lib/bootstrap.js'/>"></script>
<script type="text/javascript" 
    src="<c:url value='/scripts/lib/plugins/jquery.cookie.js'/>"></script>
<script type="text/javascript" 
    src="<c:url value='/scripts/script.js'/>"></script>
<link rel="stylesheet" type="text/css" media="all" href="<c:url value='/scripts/datepicker/css/datepicker.css'/>" />
<script type="text/javascript" src="<c:url value='/scripts/datepicker/js/bootstrap-datepicker.js'/>"></script>
<script type="text/javascript" 
   src="<c:url value='/scripts/datepicker/js/locales/bootstrap-datepicker.${pageContext.request.locale.language}.js'/>">
</script>


What we should have to use now is something like this:
<link rel="stylesheet" type="text/css" media="all" href="<c:url value='/styles/all.css'/>" />
<script type="text/javascript" src="<c:url value='/scripts/all.js'/>"></script>


We update the webapp/decorators/default.jsp accordingly, and can safely remove datepicker scripts/css from documentForm.jsp, as these will now be integrated in all.js/all.css.

We must now tell jetty where to find these additional resources, as they're out of webapp.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>8.1.3.v20120416</version>
    <configuration>
        <webApp>
            <contextPath>/${artifactId}</contextPath>
            <baseResource implementation="org.eclipse.jetty.util.resource.ResourceCollection">
                <resourcesAsCSV>src/main/webapp,src/main/wro-target</resourcesAsCSV>
            </baseResource>
        </webApp>
    </configuration>
</plugin>

Lets launch jetty and check the results:





The resulting css is clearly mixed-up, with the navigation top bar placed incorrectly, and the bootstrap glyphs not showing properly in the buttons.


After a bit of trial and error, I arrive at the following configuration for resource processing:

managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory
preProcessors=semicolonAppender,cssMinJawr
gzipResources=true
encoding=UTF-8
postProcessors=cssVariables,jsMin
uriLocators=servletContext,uri,classpath

This configuration works smoothly. Here we can see AppFuse with the top navigation bar, and the bootstrap datepicker in action.


We now have the resource optimization that groups together resources files and minimizes size and server requests. But we are missing the fingerprinting of these resources.

Step 5: add fingerprinting to the optimized web resources

Resource fingerprinting is generating a different name for different versions of the same resource. When its contents change, by changing its name we will be sure the client browser will always get the last and correct version.

Wro4j has a feature by which it generates a hash based on the contents of the generated resource, and appends this hash to the filename of the generated resource.

This option is configured in wro.properties, in the last two lines:

managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory
preProcessors=semicolonAppender,cssMinJawr
gzipResources=true
encoding=UTF-8
postProcessors=cssVariables,jsMin
uriLocators=servletContext,uri,classpath
# Configurable options available since 1.4.7 (not mandatory)
hashStrategy=MD5
namingStrategy=hashEncoder-CRC32


Let's run maven again.
> mvn clean generate-sources
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building AppFuse Struts 2 Application
[INFO]    task-segment: [generate-sources]
[INFO] ------------------------------------------------------------------------
[INFO] [wro4j:run {execution: default}]
[INFO] Executing the mojo:
[INFO] Wro4j Model path: ...\appfuse-test\wro\src\main\webapp\WEB-INF\wro.xml
[INFO] targetGroups: all
[INFO] minimize: true
[INFO] ignoreMissingResources: true
[INFO] destinationFolder: ...\appfuse-test\wro\target\wro
[INFO] jsDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] cssDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] groupNameMappingFile: src/main/wro-target/wromapping.properties
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] processing group: all.css
[INFO] wroManagerFactory class: ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory
[INFO] file size: all-ca26d802.css -> 104702 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\styles\all-ca26d802.css (104702 bytes)
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] processing group: all.js
[INFO] file size: all-4601e0db.js -> 143589 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\scripts\all-4601e0db.js (143589 bytes)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Fri Oct 19 19:16:11 CEST 2012
[INFO] Final Memory: 35M/553M
[INFO] ------------------------------------------------------------------------

Now, wro4j has generated for us the two fingerprinted resources, and its corresponding wromapping.properties file:
all.css=all-ca26d802.css
all.js=all-4601e0db.js

But wait a minute. Our resources are now fingerprinted. Their names will be different each time we change a single comma. How can we refer to them in our jsp's?

To get around these, we'll write a simple appfuse:wro tag to obtain the fingerprinted name from the mapping file that wro4j generates for us.

Step 5.1: the appfuse:wro tag


We'll write this tag to obtain the real fingerprinted resource path from the group name, like this:

<script type="text/javascript" 
  src="<appfuse:wro path='/scripts' resource='all.js'/>">
</script>

The source code for our wro tag:

public class WroResourceTag extends TagSupport {
    private static Properties wroMapping;
    private String resource;
    private String path;

    // ... setters
    
    @Override
    public int doStartTag() throws JspException {
        String ctx = pageContext.getServletContext().getContextPath();
        StringBuilder sb = new StringBuilder(ctx);
        sb.append(path);
        if (!path.endsWith("/")) {
            sb.append('/');
        }
        try {
            sb.append(getWroMapping().getProperty(resource, resource));
            pageContext.getOut().write(sb.toString());
        } catch (IOException io) {
            throw new JspException(io);
        }
        return super.doStartTag();
    }

    private Properties getWroMapping() throws IOException {
        if (wroMapping == null) {
            wroMapping = new Properties();
            String wroMappingFile = "wromapping.properties";
            InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream(wroMappingFile);
            if (inputStream == null) {
                throw new FileNotFoundException("property file '" + wroMappingFile
                        + "' not found in the classpath");
            }
            wroMapping.load(inputStream);
        }
        return wroMapping;
    }
}

Step 5.2: our final jsp fragment


By using this new tag, we end-up with something like this in our webapp/decorators/default.jsp:

<link rel="stylesheet" type="text/css" media="all" 
      href="<appfuse:wro path='/styles' resource='all.css'/>" />
<script type="text/javascript" 
        src="<appfuse:wro path='/scripts' resource='all.js'/>"></script>

After re-launching our app, we can see it is correctly referring to the fingerprinted resources:



Results


Now that we have the two versions of this AppFuse app, we can compare their web resource optimization with YSlow.

The AppFuse with wro4j optimization is deployed in CloudFoundry here.

YSlow before web resource optimization.


YSlow after web resource optimization:


There is still room for improvement: add expiry date headers to fingerprinted resources, apply a gzip filter...

Sources


Sources can be downloaded from BitBucket here.