2012/10/24

Adding web resource fingerprinting to AppFuse with wro4j

As a Java web developer, web resource optimization is something that is usually left until you have time to optimize things. Usually, you don't have that time, really.

However, when you have a SaaS on the cloud, you don't want to call or mail your users to ask them clear their browser's cache after a css or js update or bug fix! Do you?

There are a few Java apis for Web Resource Optimization: jawr and wro4j are two of the available Java apis for web resource optimization and fingerprinting.

I decided to integrate wro4j into AppFuse for static resource optimization with the following goals:
  • optimization would be performed at build time
  • static resources should be automatically versioned (fingerprinting)
Let's work through the steps to integrate wro4j into the latest and greatest AppFuse version: the one in development. That is: 2.1.1-SNAPSHOT.

From the AppFuse quickstart page, we can generate a basic web app to start working. I choose Struts2 as the framework.

mvn archetype:generate -B 
  -DarchetypeGroupId=org.appfuse.archetypes 
  -DarchetypeArtifactId=appfuse-basic-struts-archetype 
  -DarchetypeVersion=2.1.1-SNAPSHOT 
  -DgroupId=es.jogaco 
  -DartifactId=wro 
  -DarchetypeRepository=http://oss.sonatype.org/content/repositories/appfuse

After adding a Document test class to the model with a date field, and pushing the app to CloudFoundry, the result is here.
@Entity
@Table(name="document")
public class Document {

    private Long id;
    private String name;
    private Integer copies;
    private Double price;
    private Date releaseDate;
    private Date updated;

    // getters, setters, etc...
 }

Ok. Let's integrate wro4j to do the resource optimization at build-time.


Step 1: defining the resources that will be grouped and processed by wro4j

I first do a bit of resource file reorganization so that all css files are under webapp/styles/** and all js files are under webapp/scripts/**.
In an AppFuse Struts2 project, the js files that are used are these:

scripts/login.js
scripts/script.js
scripts/lib/bootstrap.min.js
scripts/lib/plugins/jquery.cookie.js
scripts/lib/jquery-1.7.2.min.js
scripts/datepicker/bootstrap-datepicker.js
scripts/datepicker/locales/bootstrap-datepicker.br.js
scripts/datepicker/locales/bootstrap-datepicker.da.js
scripts/datepicker/locales/bootstrap-datepicker.de.js
scripts/datepicker/locales/bootstrap-datepicker.es.js
... the rest of locale files for datepicker

The used css files:
styles/style.css
styles/lib/bootstrap.min.css
styles/datepicker/datepicker.css

Then, the resulting wro.xml file is:

<groups xmlns="http://www.isdc.ro/wro">
  <group name='all'>
    <css>/styles/lib/bootstrap.min.css</css>
    <css>/styles/datepicker/datepicker.css</css>
    <css>/styles/style.css</css>
    <js>/scripts/lib/jquery-1.7.2.min.js</js>
    <js>/scripts/lib/bootstrap.min.js</js>
    <js>/scripts/datepicker/bootstrap-datepicker.js</js>
    <js>/scripts/datepicker/locales/*.js</js>
    <js>/scripts/lib/plugins/jquery.cookie.js</js>
    <js>/scripts/script.js</js>
  </group>
</groups>

The order of the files in wro.xml is important, by the way.

Step 2: add wro4j maven plugin to the pom.xml

Of the maven lifecycle phases, the generate-sources seems the most appropriate.

<plugin>
    <groupId>ro.isdc.wro4j</groupId>
    <artifactId>wro4j-maven-plugin</artifactId>
    <version>1.4.9</version>
    <executions>
        <execution>
            <phase>generate-sources</phase>
            <goals>
                <goal>run</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <targetGroups>all</targetGroups>
        <cssDestinationFolder>src/main/wro-target/styles/</cssDestinationFolder>
        <jsDestinationFolder>src/main/wro-target/scripts/</jsDestinationFolder>
        <contextFolder>src/main/webapp/</contextFolder>
        <extraConfigFile>src/main/wro.properties</extraConfigFile>
        <ignoreMissingResources>true</ignoreMissingResources>
        <groupNameMappingFile>src/main/wro-target/wromapping.properties</groupNameMappingFile>
        <wroManagerFactory>ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory</wroManagerFactory>
    </configuration>
</plugin>



By adding this maven plugin, wro4j will generate the bundled resources at build time as defined in our configuration files. Wro4j will generate:

  • all.css
  • all.js

I have chosen to generate files in a separate src/main/wro-target folder so I don't pollute my source folders with generated sources. Other options are fine too.


Step 3: defining the processors for the resources

As the wro4j documentation explains, the pre and post processors of css and js files need to be defined in a wro.properties file, referenced in the maven plugin configuration at src/main/wro.properties.

Starting initially with the example given in wro4j documentation, except for the namingStrategy and hashStrategy which we'll use later on for fingerprinting:
managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory
preProcessors=cssUrlRewriting,cssImport,semicolonAppender,cssMin
gzipResources=true
encoding=UTF-8
postProcessors=cssVariables,jsMin
uriLocators=servletContext,uri,classpath

With this configuration, I run now maven:

Okay. Let's run maven.
> mvn clean generate-sources
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building AppFuse Struts 2 Application
[INFO]    task-segment: [generate-sources]
[INFO] ------------------------------------------------------------------------
[INFO] [wro4j:run {execution: default}]
[INFO] Executing the mojo:
[INFO] Wro4j Model path: ...\appfuse-test\wro\src\main\webapp\WEB-INF\wro.xml
[INFO] targetGroups: all
[INFO] minimize: true
[INFO] ignoreMissingResources: true
[INFO] destinationFolder: ...\appfuse-test\wro\target\wro
[INFO] jsDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] cssDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] groupNameMappingFile: src/main/wro-target/wromapping.properties
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] processing group: all.css
[INFO] wroManagerFactory class: ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory
[INFO] file size: all.css -> 104702 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\styles\all.css (104702 bytes)
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] processing group: all.js
[INFO] file size: all.js -> 143589 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\scripts\all.js (143589 bytes)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Fri Oct 19 19:16:11 CEST 2012
[INFO] Final Memory: 35M/553M
[INFO] ------------------------------------------------------------------------

wro4j has generated for us the two optimized resources. Now it's time we use these in our jsp pages.

Step 4: integrate generated resources in our web app

In AppFuse, we have the default.jsp which composes the final web page with sitemesh.
There we have a few css and javascript referenced. In our documentForm.jsp, there will be a few extra resources to manage the date as well: the datepicker component.
These are our original resources referenced in our jsp pages:

<link rel="stylesheet" type="text/css" media="all" 
    href="<c:url value='/styles/lib/bootstrap.min.css'/>" />
<link rel="stylesheet" type="text/css" media="all" 
    href="<c:url value='/styles/style.css'/>" />
<script type="text/javascript" 
    src="<c:url value='/scripts/lib/jquery-1.7.2.min.js'/>"></script>
<script type="text/javascript" 
    src="<c:url value='/scripts/lib/bootstrap.js'/>"></script>
<script type="text/javascript" 
    src="<c:url value='/scripts/lib/plugins/jquery.cookie.js'/>"></script>
<script type="text/javascript" 
    src="<c:url value='/scripts/script.js'/>"></script>
<link rel="stylesheet" type="text/css" media="all" href="<c:url value='/scripts/datepicker/css/datepicker.css'/>" />
<script type="text/javascript" src="<c:url value='/scripts/datepicker/js/bootstrap-datepicker.js'/>"></script>
<script type="text/javascript" 
   src="<c:url value='/scripts/datepicker/js/locales/bootstrap-datepicker.${pageContext.request.locale.language}.js'/>">
</script>


What we should have to use now is something like this:
<link rel="stylesheet" type="text/css" media="all" href="<c:url value='/styles/all.css'/>" />
<script type="text/javascript" src="<c:url value='/scripts/all.js'/>"></script>


We update the webapp/decorators/default.jsp accordingly, and can safely remove datepicker scripts/css from documentForm.jsp, as these will now be integrated in all.js/all.css.

We must now tell jetty where to find these additional resources, as they're out of webapp.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>8.1.3.v20120416</version>
    <configuration>
        <webApp>
            <contextPath>/${artifactId}</contextPath>
            <baseResource implementation="org.eclipse.jetty.util.resource.ResourceCollection">
                <resourcesAsCSV>src/main/webapp,src/main/wro-target</resourcesAsCSV>
            </baseResource>
        </webApp>
    </configuration>
</plugin>

Lets launch jetty and check the results:





The resulting css is clearly mixed-up, with the navigation top bar placed incorrectly, and the bootstrap glyphs not showing properly in the buttons.


After a bit of trial and error, I arrive at the following configuration for resource processing:

managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory
preProcessors=semicolonAppender,cssMinJawr
gzipResources=true
encoding=UTF-8
postProcessors=cssVariables,jsMin
uriLocators=servletContext,uri,classpath

This configuration works smoothly. Here we can see AppFuse with the top navigation bar, and the bootstrap datepicker in action.


We now have the resource optimization that groups together resources files and minimizes size and server requests. But we are missing the fingerprinting of these resources.

Step 5: add fingerprinting to the optimized web resources

Resource fingerprinting is generating a different name for different versions of the same resource. When its contents change, by changing its name we will be sure the client browser will always get the last and correct version.

Wro4j has a feature by which it generates a hash based on the contents of the generated resource, and appends this hash to the filename of the generated resource.

This option is configured in wro.properties, in the last two lines:

managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory
preProcessors=semicolonAppender,cssMinJawr
gzipResources=true
encoding=UTF-8
postProcessors=cssVariables,jsMin
uriLocators=servletContext,uri,classpath
# Configurable options available since 1.4.7 (not mandatory)
hashStrategy=MD5
namingStrategy=hashEncoder-CRC32


Let's run maven again.
> mvn clean generate-sources
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building AppFuse Struts 2 Application
[INFO]    task-segment: [generate-sources]
[INFO] ------------------------------------------------------------------------
[INFO] [wro4j:run {execution: default}]
[INFO] Executing the mojo:
[INFO] Wro4j Model path: ...\appfuse-test\wro\src\main\webapp\WEB-INF\wro.xml
[INFO] targetGroups: all
[INFO] minimize: true
[INFO] ignoreMissingResources: true
[INFO] destinationFolder: ...\appfuse-test\wro\target\wro
[INFO] jsDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] cssDestinationFolder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] groupNameMappingFile: src/main/wro-target/wromapping.properties
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\styles
[INFO] processing group: all.css
[INFO] wroManagerFactory class: ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory
[INFO] file size: all-ca26d802.css -> 104702 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\styles\all-ca26d802.css (104702 bytes)
[INFO] folder: ...\appfuse-test\wro\src\main\wro-target\scripts
[INFO] processing group: all.js
[INFO] file size: all-4601e0db.js -> 143589 bytes
[INFO] ...\appfuse-test\wro\src\main\wro-target\scripts\all-4601e0db.js (143589 bytes)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Fri Oct 19 19:16:11 CEST 2012
[INFO] Final Memory: 35M/553M
[INFO] ------------------------------------------------------------------------

Now, wro4j has generated for us the two fingerprinted resources, and its corresponding wromapping.properties file:
all.css=all-ca26d802.css
all.js=all-4601e0db.js

But wait a minute. Our resources are now fingerprinted. Their names will be different each time we change a single comma. How can we refer to them in our jsp's?

To get around these, we'll write a simple appfuse:wro tag to obtain the fingerprinted name from the mapping file that wro4j generates for us.

Step 5.1: the appfuse:wro tag


We'll write this tag to obtain the real fingerprinted resource path from the group name, like this:

<script type="text/javascript" 
  src="<appfuse:wro path='/scripts' resource='all.js'/>">
</script>

The source code for our wro tag:

public class WroResourceTag extends TagSupport {
    private static Properties wroMapping;
    private String resource;
    private String path;

    // ... setters
    
    @Override
    public int doStartTag() throws JspException {
        String ctx = pageContext.getServletContext().getContextPath();
        StringBuilder sb = new StringBuilder(ctx);
        sb.append(path);
        if (!path.endsWith("/")) {
            sb.append('/');
        }
        try {
            sb.append(getWroMapping().getProperty(resource, resource));
            pageContext.getOut().write(sb.toString());
        } catch (IOException io) {
            throw new JspException(io);
        }
        return super.doStartTag();
    }

    private Properties getWroMapping() throws IOException {
        if (wroMapping == null) {
            wroMapping = new Properties();
            String wroMappingFile = "wromapping.properties";
            InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream(wroMappingFile);
            if (inputStream == null) {
                throw new FileNotFoundException("property file '" + wroMappingFile
                        + "' not found in the classpath");
            }
            wroMapping.load(inputStream);
        }
        return wroMapping;
    }
}

Step 5.2: our final jsp fragment


By using this new tag, we end-up with something like this in our webapp/decorators/default.jsp:

<link rel="stylesheet" type="text/css" media="all" 
      href="<appfuse:wro path='/styles' resource='all.css'/>" />
<script type="text/javascript" 
        src="<appfuse:wro path='/scripts' resource='all.js'/>"></script>

After re-launching our app, we can see it is correctly referring to the fingerprinted resources:



Results


Now that we have the two versions of this AppFuse app, we can compare their web resource optimization with YSlow.

The AppFuse with wro4j optimization is deployed in CloudFoundry here.

YSlow before web resource optimization.


YSlow after web resource optimization:


There is still room for improvement: add expiry date headers to fingerprinted resources, apply a gzip filter...

Sources


Sources can be downloaded from BitBucket here.

3 comments:

  1. Thanks.
    If you try to upgrade to newer versions of the wro4j maven plugin, with maven 2 you will get an error:

    [INFO] Internal error in the plugin manager executing goal 'ro.isdc.wro4j:wro4j-maven-plugin:1.5.0:run':
    Unable to load the mojo 'ro.isdc.wro4j:wro4j-maven-plugin:1.5.0:run' in the plugin 'ro.isdc.wro4j:wro4j-maven-plugin'.
    A required class is missing: org/codehaus/plexus/util/Scanner
    org.codehaus.plexus.util.Scanner

    You can get arount this by upgrading to maven 3 or by including a dependency to the wro4j plugin:


    org.jvnet.hudson.plugins.m2release
    nexus
    0.0.3

    ReplyDelete
  2. <dependency>
    <groupId>org.jvnet.hudson.plugins.m2release</groupId>
    <artifactId>nexus</artifactId>
    <version>0.0.3</version>
    </dependency>

    ReplyDelete