Showing posts with label grails. Show all posts
Showing posts with label grails. Show all posts

Sunday, March 6, 2016

Grails 2.5.4 Controller Action Arguments

After upgrading to Grails 2.5.4, one of our projects failed with an exception in the 'canonicalization' phase of the build, due to a NullPointerException in Grails' ControllerActionTransformer:

Caused by: java.lang.NullPointerException
    at org.codehaus.groovy.grails.compiler.web.ControllerActionTransformer.getCodeToInitializeCommandObjects(ControllerActionTransformer.java:424)
    at org.codehaus.groovy.grails.compiler.web.ControllerActionTransformer.addMethodToInvokeClosure(ControllerActionTransformer.java:493)
    at org.codehaus.groovy.grails.compiler.web.ControllerActionTransformer.processClosures(ControllerActionTransformer.java:479)
    at org.codehaus.groovy.grails.compiler.web.ControllerActionTransformer.performInjectionOnAnnotatedClass(ControllerActionTransformer.java:206)
    at org.codehaus.groovy.grails.compiler.web.ControllerActionTransformer.performInjection(ControllerActionTransformer.java:197)
    at org.codehaus.groovy.grails.compiler.injection.GrailsAwareInjectionOperation.call(GrailsAwareInjectionOperation.java:154)
    at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1055)
    ... 517 more
| Error Fatal error during compilation org.apache.tools.ant.BuildException: BUG! exception in phase 'canonicalization' in source unit '/home/justin/projects/apps/grails-app/controllers/com/pitchstone/apps/controller/LandingController.groovy' unexpected NullpointerException

Turns out that was because one of our controller actions, which used the old-style "closure" action definitions, had an empty argument list, like this:

def someAction = { ->
    forward action: 'anotherAction'
}

I think once upon a time that action had a command object, and when the action was updated to just forward somewhere else, the command object was deleted from the action's argument list, leaving it with no arguments (instead of the usual single implicit argument, it). Like at one time it probably looked like this:

def someAction = { SomeCommand cmd ->
    [cmd: cmd]
}

Previous versions of Grails were fine with closure-style actions that had no arguments, but not Grails 2.5.4. Fortunately, simply removing the errant arrow (->) fixed the exception (and everything else with Grails 2.5.4 has gone smoothly):

def someAction = {
    forward action: 'anotherAction'
}

Thursday, September 26, 2013

Overriding toString() in Groovy Using Grails' ExtendedProxy

In Groovy, most of the time you can override the behavior of an object instance's method using the object's metaClass property, like to make the following code print "Yes" instead of "true":

def x = true
x.metaClass.toString { -> delegate ? 'Yes' : 'No' }
println x.toString()

But particularly with toString(), there are some cases (documented in GROOVY_2599) where this doesn't work; for example, the following code will still print "true":

def x = true
x.metaClass.toString { -> delegate ? 'Yes' : 'No' }
println "${x}"

To get around this issue for a project on which I was working recently, I used Grails' ExtendedProxy class to wrap other object instances for which I wanted to override the toString() method. The ExtendedProperty class delegates calls to get and set properties on the wrapped object, as well as method invocations. (It extends Groovy's Proxy class, which delegates method invocations only.)

This allowed me to apply some pretty formatting to a few standard Java objects (like to format Date objects with a US-style date format) without choosing between proxying every property/method explicitly or losing the other aspects of the wrapped objects' functionality. To maintain the functionality I wanted, the only other method (other than toString()) that I found I needed to proxy explicitly was asBoolean() (allowing for wrapped collections to behave as falsey when empty).

This was the wrapper class I ended up creating:

class PrettyToStringWrapper extends grails.util.ExtendedProxy {

    /** Wraps only if it makes a difference for the specified object. */
    static Object wrapMaybe(Object o) {
        (
            o instanceof Collection ||
            o instanceof Date ||
            o instanceof Boolean
        ) ? new PrettyToStringWrapper().wrap(o) : o
    }

    /** Proxies truthy and falsey. */
    boolean asBoolean() {
        getAdaptee().asBoolean()
    }

    /** Overrides toString() with pretty implementation. */
    String toString() {
        def wrapped = getAdaptee()

        if (wrapped instanceof Collection)
            return wrapped.toString().replaceAll(/^\[|\]$/, '')

        if (wrapped instanceof Date)
            return wrapped.format('MM/dd/yyyy')

        if (wrapped instanceof Boolean)
            return wrapped ? 'Yes' : 'No'

        return wrapped.toString()
    }

}

And used it like this:

def emptyList = PrettyToStringWrapper.wrapMaybe([])
println "${emptyList ? 'full' : 'empty'} list contains ${emptyList}"
// prints 'empty list contains '

def fullList = PrettyToStringWrapper.wrapMaybe([1, 2, 3])
println "${fullList ? 'full' : 'empty'} list contains ${fullList}"
// prints 'full list contains 1, 2, 3'

def date = PrettyToStringWrapper.wrapMaybe(new Date(0))
println "epoch begins on ${date}"
// prints 'epoch begins on 12/31/1969' (in US timezones)

def yup = PrettyToStringWrapper.wrapMaybe(true)
println "${yup} this is true"
// prints 'Yes this is true'

I added the static wrapMaybe() method to avoid wrapping objects needlessly — one caveat I found to using ExtendedProxy was that it doesn't proxy the dynamic properties of fancier classes which implement Groovy's special propertyMissing() method (propertyMissing() allows those classes to provide properties without declaring them anywhere).

And one other thing to watch out when using the ExtendedProperty class is that you must reference the wrapped object via the getAdaptee() method instead of simply accessing the adaptee property (the adaptee property is defined by the Proxy class). Using the adaptee property results in a call to the wrapper's getProperty() method for the adaptee property, and is delegated by ExtendedProperty to the wrapped object (raising an IllegalArgumentException as the wrapped object won't have an adaptee property); whereas getAdaptee() accesses the wrapper's adaptee property directly, without a call to getProperty().

Saturday, August 17, 2013

Archiva Repository Manager for Grails

Recently deployed Apache Archiva as a Maven repository manager, for use by our Grails projects (primarily as a local cache for remote artifacts). The default configuration for Archiva does this almost completely out-of-the-box — just needed a little extra configuration for Grails plugins. Here are the steps I took to install Archiva on Ubuntu 12.04 and configure our Grails projects to use it:

Install Archiva

There isn't yet an Ubuntu apt package for Archiva, so you have to download and install it manually. It's pretty straightforward, though:

# download archiva 1.3.6
wget http://download.nextag.com/apache/archiva/1.3.6/binaries/apache-archiva-1.3.6-bin.tar.gz

# extract and move to /opt/archiva
tar xf apache-archiva-1.3.6-bin.tar.gz
sudo mv apache-archiva-1.3.6 /opt/.
sudo ln -s /opt/apache-archiva-1.3.6 /opt/archiva

# delete wrapper-linux-x86-32 files (if you're using 64-bit linux -- otherwise keep them!)
sudo rm /opt/archiva/bin/wrapper-linux-x86-32
sudo rm /opt/archiva/lib/libwrapper-linux-x86-32.so

# create archiva working dir with the default conf files
sudo mkdir /srv/archiva
sudo cp -r /opt/archiva/conf /srv/archiva/.
sudo mkdir /srv/archiva/data
sudo mkdir /srv/archiva/logs

# add daemon user
sudo useradd -r archiva
sudo chown -R archiva:archiva /srv/archiva

# create daemon script
echo '#!/bin/sh -e
#
# /etc/init.d/archiva daemon control script
#
### BEGIN INIT INFO
# Provides:          archiva
# Required-Start:    $local_fs $remote_fs $network
# Required-Stop:     $local_fs $remote_fs $network
# Should-Start:      $named
# Should-Stop:       $named
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start Archiva
# Description:       Start/Stop Aparche Archiva at /opt/archiva.
### END INIT INFO

export ARCHIVA_BASE=/srv/archiva
export RUN_AS_USER=archiva

/opt/archiva/bin/archiva $@
' | sudo tee /etc/init.d/archiva
sudo update-rc.d archiva defaults 80 20

The above script will install Archiva 1.3.6 at /opt/archiva, create a working dir for it at /srv/archiva, create a new, unprivileged archiva user, and create an /etc/init.d/archiva script to run Archiva as a daemon. You can now start Archiva with the command sudo service archiva start, and it automatically will start whenever the machine boots.

Before I started it, however, I also configured Archiva to use a MySQL DB as its data store, since MySQL was already running on the same box (Archiva uses Apache Derby by default). To do so, I first created a database for Archiva and a database for Apache Redback (which Archiva uses for its user store):

echo "create archiva mysql db as root..."
mysql -uroot -p - e'
    CREATE DATABASE archiva DEFAULT CHARACTER SET ascii;
    GRANT ALL ON archiva.* TO "archiva" IDENTIFIED BY "secret-archiva-password";

    CREATE DATABASE redback DEFAULT CHARACTER SET ascii;
    GRANT ALL ON redback.* TO "redback" IDENTIFIED BY "secret-redback-password";
'

And configured Archiva to use MySQL by altering its /srv/archiva/conf/jetty.xml configuration file to use MySQL settings in place of Derby:

  <!-- Archiva Database -->

  <New id="archiva" class="org.mortbay.jetty.plus.naming.Resource">
    <Arg>jdbc/archiva</Arg>
    <Arg>
      <New class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
        <Set name="serverName">localhost</Set>
        <Set name="databaseName">archiva</Set>
        <Set name="user">archiva</Set>
        <Set name="password">archiva-secret-password</Set>
      </New>
    </Arg>
  </New>

  <New id="archivaShutdown" class="org.mortbay.jetty.plus.naming.Resource">
    <Arg>jdbc/archivaShutdown</Arg>
    <Arg>
      <New class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
        <Set name="serverName">localhost</Set>
        <Set name="databaseName">archiva</Set>
        <Set name="user">archiva</Set>
        <Set name="password">archiva-secret-password</Set>
      </New>
    </Arg>
  </New>

  <!-- Users / Security Database -->

  <New id="users" class="org.mortbay.jetty.plus.naming.Resource">
    <Arg>jdbc/users</Arg>
    <Arg>
      <New class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
        <Set name="serverName">localhost</Set>
        <Set name="databaseName">redback</Set>
        <Set name="user">redback</Set>
        <Set name="password">redback-secret-password</Set>
      </New>
    </Arg>
  </New>

  <New id="usersShutdown" class="org.mortbay.jetty.plus.naming.Resource">
    <Arg>jdbc/usersShutdown</Arg>
    <Arg>
      <New class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
        <Set name="serverName">localhost</Set>
        <Set name="databaseName">redback</Set>
        <Set name="user">redback</Set>
        <Set name="password">redback-secret-password</Set>
      </New>
    </Arg>
  </New>

And finally, linked the MySQL java driver into Archiva's lib directory:

sudo ln -s /usr/share/java/mysql.jar /opt/archiva/lib/.

Proxy Archiva Thru Apache

Archiva runs on port 8080 by default. To avoid conflicts with other services, I changed it to port 6161:

sudo perl -pli -e 's/(name="jetty.port" default=")\d+/\16161/' /srv/archiva/conf/jetty.xml

(Restart Archiva after making this change, if you've already started it.) Then I added an Apache (web server) virtual host for it at /etc/apache2/sites-available/archiva, to proxy it from port 80 (running on a server with a DNS entry of archiva.example.com):

echo '
<VirtualHost *:80>
    ServerName archiva.example.com

    ProxyPreserveHost On
    RewriteEngine On

    # redirect / to /archiva
    RewriteRule ^/$ /archiva [L,R=301]

    # forward all archiva requests to archiva servlet
    RewriteRule (.*) http://localhost:6161$1 [P]
</VirtualHost>
' | sudo tee /etc/apache2/sites-available/archiva

sudo a2enmod proxy proxy_http rewrite
sudo a2ensite archiva
sudo service apache2 restart

Now you should be able to access Archiva simply by navigating to http://archiva.example.com/ (which will redirect to http://archiva.example.com/archiva). The first time you access it, you'll be prompted to create a new admin user. Do that so you can configure a few more things.

Add Proxy Connector for Grails Plugins

Once Archiva is up and running, and you've logged in as admin, navigate to the "Administration" > "Repositories" section of Archiva by using the leftnav. Click the "Add" link on the right side of the "Remote Repositories" section of the page, and enter the following settings:

Identifier: grails-plugins
Name: Grails Plugins
URL: http://repo.grails.org/grails/plugins/
Username:
Password:
Timeout in seconds: 60
Type: Maven 2.x Repository

Click the "Add Repository" button to save the new remote repo. Then navigate to the "Administration" > "Proxy Connectors" section using the leftnav. Click the "Add" link at the top-right of the page, and enter the following settings:

Network Proxy: (direct connection)
Managed Repository: internal
Remote Repository: grails-plugins
Return error when: always
Cache failures: yes
Releases: once
On remote error: stop
Checksum: fix
Snapshots: hourly

Click the "Save Proxy Connector" button to save the new proxy connector. The Archiva server should now be acting as a proxy for the Grails Plugins repo. It already comes configured as a proxy for the Maven Central repo, so you should be ready to use it with Grails.

Update BuildConfig.groovy

You can now comment out all the other default repos in the repositories section of the conf/BuildConfig.groovy files of your various Grails projects, and add a repo entry for your new Archiva server:

    repositories {
        mavenRepo 'http://archiva.example.com/archiva/repository/internal/'

        //grailsPlugins()
        //grailsHome()
        //grailsCentral()
        //mavenCentral()
        //mavenLocal()
        //...
    }

After updating your BuildConfig.groovy file, test out your changes by deleting your ivy2 cache folder (~/.ivy2/cache), and running a clean grails build (which will re-download all the dependencies for the Grails project through your new Archiva server).

Sunday, June 3, 2012

Using /etc/mime.types for Grails Files Downloads

I find it sorely disappointing that ServletContext#getMimeType() doesn't just use apache's standard /etc/mime.types automatically. Instead, it seems to use some smaller subset of extension mappings included in some servlet jar somewhere. Fortunately, it's easy enough to use spring's ConfigurableMimeFileTypeMap bean to get the mappings from the mime.types files of your choosing.

The way I like to set it up is to add a custom configuration property in my grails-app/conf/Config.groovy, and specify a comma-separated list of file paths to use (that way I can use the standard /etc/mime.types file, but also add in my own custom mime.types file if I need some mappings not in the standard one):

grails-app/conf/Config.groovy
mime.types = '/etc/mime.types'

Then, in grails-app/conf/spring/resources.groovy, I define a ConfigurableMimeFileTypeMap bean (I call it fileTypeMap, but you can call it whatever):

grails-app/conf/resources.groovy
fileTypeMap(org.springframework.mail.javamail.ConfigurableMimeFileTypeMap) { application.config.mime.types.split(/,/).each { mappingLocation = new org.springframework.core.io.FileSystemResource(it) } }

Finally, in the controller actions where I want to stream out a file, I use the ConfigurableMimeFileTypeMap bean's getContentType() method to calculate the file's content type from its name:

grails-app/controllers/MyController.groovy
def fileTypeMap def download = { try { // some logic here to locate the appropriate file to stream def file = new File('/tmp/foo.txt') // use /etc/mime.types to determine the file's content type from its extension def contentType = fileTypeMap.getContentType(file.name.toLowerCase()) // must special-case text/html; see http://jira.grails.org/browse/GRAILS-1223 if (contentType == 'text/html') return render(contentType: contentType, text: file.text) // set up standard (inline) file-download headers response.contentType = contentType response.contentLength = file.length() as Integer response.setHeader 'Content-Disposition', "inline; filename=\"${file.name}\"" // stream the file file.inputStream.withStream { response.outputStream << it } } catch (FileNotFoundException e) { response.sendError 404 } }

Sunday, September 11, 2011

Grails Foreign ID Generator

I haven't found a complete example on the web of using a foreign id generator in grails, so here's one: Say you have two domains, Primary and Secondary, with a one-to-one relationship (Primary hasOne Secondary and Secondary belongsTo Primary). Primary and Secondary basically represent the same entity, but Secondary has a bunch of data about the entity you rarely use. You map Primary to the DB table named primary, and Secondary to the DB table secondary; and since you've got a one-to-one relationship between Primary and Secondary, you just want the same column in the secondary table to be used as both its primary key and its foreign key to the primary table.

So you define Primary and Secondary like this:

class Primary { int oftUsedInfo int moreOftUsedInfo static hasOne = [ secondary: Secondary ] static mapping = { secondary cascade: 'all-delete-orphan' } } class Secondary { String littleUsedInfo String moreLittleUsedInfo static belongsTo = [ primary: Primary ] static mapping = { id column: 'primary_id', generator: 'foreign', params: [ property: 'primary' ] primary insertable: false, updateable: false } }

With that mapping, hibernate will create tables for you like the following (when using the MySQL InnoDB dialect):

CREATE TABLE `primary` ( `id` BIGINT(20) NOT NULL AUTO_INCREMENT, `version` BIGINT(20) NOT NULL, `oft_used_info` INT(11) NOT NULL, `more_oft_used_info` INT(11) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; CREATE TABLE `secondary` ( `primary_id` BIGINT(20) NOT NULL, `version` BIGINT(20) NOT NULL, `little_used_info` VARCHAR(255) NOT NULL, `more_little_used_info` VARCHAR(255) NOT NULL, PRIMARY KEY (`primary_id`), KEY `FK12344567ABCDEF` (`primary_id`), CONSTRAINT `FK12344567ABCDEF` FOREIGN KEY (`primary_id`) REFERENCES `primary` (`id`) ) ENGINE=InnoDB;

Instead of the secondary table having its own separate AUTO_INCREMENT id column, it just re-uses the primary_id column (referencing the primary table) as its primary key.

When you create a new Primary and Secondary instances programmatically, you'd do it like this:

new Primary( oftUsedInfo: 1, moreOftUsedInfo: 2, secondary: new Secondary( littleUsedInfo: 'foo', moreLittleUsedInfo: 'bar', ), ).save()

Or, if you want to do it property by property:

def primary = new Primary() primary.oftUsedInfo = 1 primary.moreOftUsedInfo = 2 primary.secondary = new Secondary() primary.secondary.littleUsedInfo = 'foo' primary.secondary.moreLittleUsedInfo = 'bar' primary.save()

And when you delete, you only have to delete the Primary domain (because of the all-delete-orphan cascade setting):

Primary.findAllByOftUsedInfo(1).each { it.delete() }

One more thing to note: using the assigned generator like this seems to generate the same database schema:

class Secondary { String littleUsedInfo String moreLittleUsedInfo static belongsTo = [ primary: Primary ] static mapping = { id column: 'primary_id', generator: 'assigned' primary insertable: false, updateable: false } }

Not sure if the behavior is exactly the same, however.

Wednesday, August 24, 2011

Handling HTML5 "Multiple" File-Inputs With Grails

HTML5 includes a multiple attribute on file inputs; when set, a user can select multiple files to upload for that input with a single Browse... dialog. Grails makes handling uploads from non-multiple file-inputs really easy. For example, here's an action that echoes the content of a file uploaded via an input named myfile:

class MyController { def echoSimple = { def charset = (params.myfile?.contentType =~ /charset=([^;]+)/). collect { it[1].trim() }.join('') ?: 'ISO-8859-1' def content = new String(params.myfile?.bytes ?: ''.bytes, charset) render contentType: params.myfile?.contentType ?: 'text/plain', text: content } }

Handling multiple file-inputs is still pretty easy, but instead of accessing file objects via the controller's params map, you use the multiFileMap property of the controller's request property. File objects will be instances of spring's MultipartFile interface; for multipart form posts, the request property will be an instance of spring's MultipartRequest interface. The multiFileMap property is a map of file-input names to the list of MultipartFiles uploaded by that input. So if you have a form like the following:

<form action="${g.createLink(controller:'my', action:'echoMultiple')}" method="post" enctype="multipart/form-data"> <input type="file" name="myfile" multiple> <button type="submit">Submit</button> </form>

You can echo the content of all the files selected by the user with this action:

class MyController { def echoMultiple = { def content = request.multiFileMap?.myfile?.collect { file -> def charset = (file.contentType =~ /charset=([^;]+)/). collect { it[1].trim() }.join('') ?: 'ISO-8859-1' new String(file.bytes, charset) }?.join('\n') ?: '' render contentType: 'text/plain', text: content } }

(Note that while other browsers have supported the multiple attribute for some time, no version of IE currently supports it — although it should be supported in IE 10.)

Sunday, February 6, 2011

Custom Grails Constraints

There are already a couple of good tutorials on the web about creating custom grails constraints, but they include some extra cruft that you don't really need in their example classes. All you really need for your custom constraint class is to override the processValidate() method of the AbstractConstraint class to run your custom validation, and call rejectValue() if it fails:

package myapp import org.codehaus.groovy.grails.validation.AbstractConstraint import org.springframework.validation.Errors class LowerCaseConstraint extends AbstractConstraint { static NAME = 'lowerCase' boolean supports(Class type) { true } String getName() { NAME } protected void processValidate(Object target, Object value, Errors errors) { if (constraintParameter && value =~ /[A-Z]/) rejectValue target, errors, "default.invalid.${name}.message", "${name}.invalid", [constraintPropertyName, constraintOwningClass, value] as Object[] } }

The above example checks if the constraintParameter property is true (or at least truthy), and if so, rejects the value if it contains any uppercase ascii chars. The constraintParameter property is the value specified for this constraint in the domain class's constraints DSL. In the following example, it's true (from the lowerCase: true part):

static constraints = { myProperty size: 1..20, lowerCase: true }

The other thing you need to do is register your custom constraint; do this by adding the following line to your conf/Config.groovy (wrapped to fit the content area of this blog):

org.codehaus.groovy.grails.validation. ConstrainedProperty.registerNewConstraint myapp.LowerCaseConstraint.NAME, myapp.LowerCaseConstraint.class

And you're probably better off overriding and implementing a few more of the AbstractConstraint methods just to make sure that the constraint is used correctly (applied to an appropriate property type, and passed an appropriate constraint parameter). Also, it's probably better to segregate your validation logic into a separate method, for clarity and to make it easier to unit test:

package myapp import org.codehaus.groovy.grails.validation.AbstractConstraint import org.springframework.validation.Errors class LowerCaseConstraint extends AbstractConstraint { static NAME = 'lowerCase' // overridden Constraint methods /** Returns true if this constraint can be applied * to a domain property of the specified type. */ boolean supports(Class type) { type != null && String.class.isAssignableFrom(type) } /** Sets the constraintParameter value * (first checking if the constraint parameter is of the correct type). */ void setParameter(Object param) { if (!(param instanceof Boolean)) throw new IllegalArgumentException("Parameter for constraint [$name] of property [$constraintPropertyName] of class [$constraintOwningClass] must be a boolean") super.setParameter(param) } /** Returns the name of this constraint. */ String getName() { NAME } /** Adds an error to the specified errors object * if the specified property-value does not conform to this constraint * for the specified target object. */ protected void processValidate(Object target, Object value, Errors errors) { if (constraintParameter && !validate(target, value)) rejectValue target, errors, "default.invalid.${name}.message", "${name}.invalid", [constraintPropertyName, constraintOwningClass, value] as Object[] } // custom implementation methods /** Returns true if the specified value conforms to this constraint. */ def validate(target, value) { !(value =~ /[A-Z]/) } }

You then can easily write a unit test for your constraint:

package myapp import grails.test.GrailsUnitTestCase class LowerCaseConstraintTests extends GrailsUnitTestCase { def constraint protected void setUp() { super.setUp() constraint = new LowerCaseConstraint() } /** Asserts that the specified string passes this constraint. */ protected validateTrue(s) { assertTrue constraint.validate(null, s) } /** Asserts that the specified string fails this constraint. */ protected validateFalse(s) { assertFalse constraint.validate(null, s) } void testPassesWhenValueIsNull() { validateTrue null } void testPassesWhenValueIsEmpty() { validateTrue '' } void testPassesWhenValueIsLowerCaseString() { validateTrue 'foobar' } void testPassesWhenValueIsStringOfNumbers() { validateTrue '1234' } void testFailsWhenValueIsUpperCaseString() { validateFalse 'FOOBAR' } void testFailsWhenValueIsMixedCaseString() { validateFalse 'fooBar' } }

For a five-star tutorial on how to build your custom constraint as a plugin, check out Geoff Lane's Build a Custom Validator in Grails with a Plugin blog post.

Sunday, June 20, 2010

Problem Accessing Grails Layouts

Just a quick note for posterity: If you're running a grails app (or probably any webapp) on jetty, and you configure the app to use the root (/) contextPath, and you leave jetty's default root webapp (/usr/share/jetty/webapps/root) around, you're in for a world of hurt. With grails, you'll get an obtuse 404 error page with the following message:

Problem accessing /WEB-INF/grails-app/views/layouts/my-custom-layout.gsp

(And no info in the logs.) The fix is of course to delete that vestigial root webapp.

Sunday, June 13, 2010

Mocking Taglibs with Grails Units

Grails taglibs are a million times better than JSP taglibs (requiring way less busywork), but they still can be tricky to unit test. Here are some tricks I've learned so far:

Basic Unit Testing

The first trick is that there's basically no official documentation for how to write even basic taglib unit tests. There is, however, a nice TagLibUnitTestCase class provided with grails that's easy to use (once you figure it out). For a simple taglib like this:

package myapp class MyTagLib { // optionally use <my:tag> namespace instead of <g:tag> static namespace = 'my' def heading = { attrs, body -> def level = attrs.level ?: 1 out << "<h$level>" << body() << "</h$level>" } }

You can write a TagLibUnitTestCase like this:

package myapp import grails.test.TagLibUnitTestCase class MyTagLibUnitTests extends TagLibUnitTestCase { void testHeadingWithNoLevelAndNoContent() { tagLib.heading [:], {''} assertEquals '<h1></h1>', tagLib.out.toString() } void testHeadingWithLevelAndSimpleContent() { tagLib.heading level: '2', {'simple'} assertEquals '<h2>simple</h2>', tagLib.out.toString() } }

The TagLibUnitTestCase class automatically determines which taglib you're testing based on the name of your test class, and sets up a stub taglib as the tagLib member of the test class. Content written to the out variable in your taglib is written to the out (StringWriter) member of this stubbed taglib.

Using Codecs

If you use one of grails' encodeAs string methods (like encodeAsHTML()) in your taglib, you need to explicitly set up that codec in your unit test, using the loadCodec helper method, like this:

package myapp import grails.test.TagLibUnitTestCase import org.codehaus.groovy.grails.plugins.codecs.HTMLCodec class MyTagLibUnitTests extends TagLibUnitTestCase { protected void setUp() { super.setUp() loadCodec(HTMLCodec) } }

Another thing to note specifically about the HTMLCodec is that encodeAsHTML() escapes double-quotes, but not single-quotes — so when you output html from a taglib, make sure you always use double-quotes for attributes in your html. The reason for this is that if you print out a variable in a single-quoted attribute — or you change your code later to print out a variable — and that variable's value came from some content that your users can manipulate, you're vulnerable to xss — even though you used encodeAsHTML() to escape that variable's content. Here's a quick example of what not to do:

class MyTagLib { def foo = { attrs, body -> def bar = "' onmouseover='alert(\"xss\")'" out << "<a href='" << bar.encodeAsHTML() << "'>Bar</a>" } }

When the taglib is used, a single-quote in the variable content "breaks-out" of the attribute, giving the attacker full access to act as the user (presumably to do something nastier than simply display an alert):

<a href='' onmouseover='alert("xss")'>Bar</a>

Calling Taglibs from Taglibs

The first trick for calling taglibs from other taglibs is to make sure always to return an empty string from all your taglibs:

package myapp class MyTagLib { // optionally use <my:tag> namespace instead of <g:tag> static namespace = 'my' def linkToWikipedia = { attrs, body -> out << "<a href=\"http://en.wikipedia.org/wiki/${body().encodeAsHTML()}\">${body().encodeAsHTML()}</a>" '' } }

Otherwise you get duplicated text or other weird values in the stubbed output-stream when you try to unit-test the taglib. When you actually call one taglib from another taglib, the second trick is to append the result from the called taglib to the output of the caller:

package myapp class MyTagLib { // optionally use <my:tag> namespace instead of <g:tag> static namespace = 'my' def heading = { attrs, body -> def level = attrs.level ?: 1 out << "<h$level>" << (attrs.wikipedia ? linkToWikipedia([:], body) : body()) << "</h$level>" '' } def linkToWikipedia = { attrs, body -> out << "<a href=\"http://en.wikipedia.org/wiki/${body().encodeAsHTML()}\">${body().encodeAsHTML()}</a>" '' } }

Otherwise the output of the called taglib is discarded when you run your app in the full grails environment.

And the third trick is that when you're trying to simulate nested taglibs in a unit test, just call the nested taglibs in the container's closure, and don't do anything else — don't try to append their output to the stubbed output-stream or anything fancy like that; for example, when you're trying to simulate this GSP code:

<my:table in="mydata" var="data"> <my:tr><my:td>${data.id}</my:td><my:td>${data.name}</my:td></my:tr> </my:table>

Your unit test should look like this:

package myapp import grails.test.TagLibUnitTestCase class MyTagLibUnitTests extends TagLibUnitTestCase { void testTableWithDataAndTwoCells() { def mydata = [[id:'123', name:'Foo'], [id:'456', name:'Bar']] tagLib.table([ 'in': mydata, 'var': 'data' ], { m -> tagLib.tr([:], { tagLib.td([:], { "${m.data.id}" }) tagLib.td([:], { "${m.data.name}" }) '' }); '' }) assertEquals '<div class="data"><table><tbody><tr class="odd">' + '<td>123</td>' + '<td>Foo</td>' + '</tr><tr class="even">' + '<td>456</td>' + '<td>Bar</td>' + '</tr></tbody></table></div>', tagLib.out.toString() } }

Note that in the example test, the closures simulating the content of the my:table and my:tr taglibs both return an empty string (so you don't get extra junk in the output, as mentioned above); and although when you use (this made-up and hypothetical) my:table in GSP code you can reference the data variable that my:table sets directly (like ${data.name}), in the test you have to reference it as it is actually passed — as an entry in the map passed to the tag's body closure (like ${m.data.name}, where m is the first argument of the table body closure in the example test code).

Beyond Grails Stubs

There are some things that aren't covered by the stub tagLib object provided by the TagLibUnitTestCase. For example, you can't stub the request params map, and accessing the taglib's controllerName or actionName properties fails with a giant flaming fireball.

The simplest way to deal with that is with some groovy metaprogramming. For example, say you want to test a taglib that grabs some data from the request params map:

package myapp class MyTagLib { static namespace = 'my' def seeAlsoWikipedia = { attrs, body -> if (params.q) out << "See also <a href=\"http://en.wikipedia.org/wiki/${params.q.encodeAsHTML()}\">${params.q.encodeAsHTML()}</a> in wikipedia" } }

You can use the tagLib instance's metaClass property to stub out the params map:

package myapp import grails.test.TagLibUnitTestCase class MyTagLibUnitTests extends TagLibUnitTestCase { void testSeeAlsoWikipediaWithAQuery() { tagLib.metaClass.params = [q: 'foo'] tagLib.seeAlsoWikipedia [:], {''} assertEquals 'See also <a href="http://en.wikipedia.org/wiki/foo">foo</a> in wikipedia', tagLib.out.toString() } }

GMock and Expectations

There's a good deal of overlap between the GrailsUnitTestCase class (superclass of TagLibUnitTestCase) and the functionality provided by GMock (in the form of the GMockTestCase class), but GMock still can be useful when you need to stub functionality not already stubbed by TagLibUnitTestCase — and you'd like to do it just by setting up some simple expectations (ie "mocks" in TDD-correct terminology).

GMock has some pretty good documentation about how to include the GMock jar into Grails; to integrate GMock into a TagLibUnitTestCase, use the @WithGMock annotation:

package myapp import grails.test.TagLibUnitTestCase import org.gmock.WithGMock @WithGMock class MyTagLibUnitTests extends TagLibUnitTestCase { void testSeeAlsoWikipediaWithAQuery() { mock(tagLib) { params.returns([q: 'foo']) } play { tagLib.seeAlsoWikipedia [:], {''} assertEquals 'See also <a href="http://en.wikipedia.org/wiki/foo">foo</a> in wikipedia', tagLib.out.toString() } } }

Use GMock's mock() method to mock an object instance (in the above example, the tagLib object already stubbed initially by TagLibUnitTestCase); in the closure passed to the mock() method, set up your expectations. In the above example, the one and only expectation for the tagLib object is for its params property to be accessed; when accessed, GMock will make it return [q: 'foo']. After you've set up the mocks and expectations, everything in the play {} closure will be executed with those mocks.

The example above performs the same exact test as the example before — the only functional difference is that GMock also validates your expectation that params will be accessed.

GMock with Other Taglibs

One thing GMock is especially good for is when a taglib you want to test calls some other taglib. When unit testing, you don't really want to test that other taglib; GMock can help by replacing that other taglib with a mock which validates your expectation that the other taglib was called with the right attributes — but without doing anything other than returning some fixed content.

For example, say you've got a taglib that prints the label, content, and errors for a given property, rendering the errors with grails' standard g:hasErrors and g:renderErrors tags:

package myapp class MyTagLib { static namespace = 'my' def propertyLine = { attrs, body -> def beanAndField = attrs.findAll { ['bean','field'].contains(it) } out << '<div class="property">' out << "<label>${attrs.label}</label>" out << '<div class="content">' << body() << '</div>' out << g.hasErrors(beanAndField, { out << '<div class="errors">' << g.renderErrors(beanAndField, {''}) << '</div>' '' }) out << '</div>' '' } }

You can pretty much assume that g:hasErrors and g:renderErrors has already been tested thoroughly and is working — you just need to test that you're correctly passing the bean and field parameters to them. With the help of GMock, you can test it like this:

package myapp import grails.test.TagLibUnitTestCase import org.gmock.WithGMock @WithGMock class MyTagLibUnitTests extends TagLibUnitTestCase { void testPropertyLineWithBeanAndField() { def bean = new Expando() mock(tagLib.g) { hasErrors([bean:bean, field:'foo'], match{ it instanceof Closure }).returns('_errors_') } play { tagLib.errors [bean:bean, field:'foo'] {''} assertEquals '<div class="property">' + '<label>null</label>' + '<div class="content"></div>' + '_errors_' + '</div>', tagLib.out.toString() } } }

The GMock expectations in the above example validates that g:hasErrors is called with the right bean and field properties, and is passed a closure. It also makes g:hasErrors return '_errors_', so you can validate that the content g:hasErrors would normally generate is in the right place in the context of all the other output generated by your taglib.

Built-in Groovy Mocking

Groovy also has built-in stubbing and mocking functionality, via its StubFor and MockFor classes. These are most useful when you want to stub an object that your taglib creates itself, like the TokenGenerator class in this example:

package myapp class MyTagLib { static namespace = 'my' def onetimeToken = { attrs, body -> def onetime = new TokenGenerator() out << '<input type="hidden" value="' << onetime.generateToken() << '">' '' } }

You can mock the TokenGenerator class like this:

package myapp import grails.test.TagLibUnitTestCase class MyTagLibUnitTests extends TagLibUnitTestCase { void testOnetimeToken() { def mockTokenGenerator = new MockFor(TokenGenerator) mockTokenGenerator.demand.generateToken { '123' } mockTokenGenerator.use { tagLib.onetimeToken [:] {''} assertEquals '<input type="hidden" value="123">', tagLib.out.toString() } } }

The code inside the use closure will use your mocked TokenGenerator behavior in place of the regular TokenGenerator behavior (ie it will return '123' for the generateToken() method), and it will raise an exception if generateToken() wasn't called within the use closure.

Sunday, June 6, 2010

Grails Passwords, Salted

Getting authentication with Spring Security (s2) set up on Grails is nice and easy; getting your s2 passwords salted with a unique value, not so much. There's a real nice grails s2 plugin that Burt Beckwith maintains. It actually is quite well documented, but there's an awful lot of places where the code is still a ways ahead of the documentation.

So here's a quick tutorial to get your grails passwords salted:

1 Install S2

First install the spring-security plugin:

$ grails install-plugin spring-security-core

Then create your "user" and "role" domain objects. You can call the "user" and "role" classes whatever you want, and put them in whatever package you choose. I called mine cq.User and cq.Role:

$ grails s2-quickstart cq User Role

2 Basic Configuration

The s2-quickstart script will automatically add the following to your grails-app/conf/Config.groovy config file:

// Added by the Spring Security Core plugin: grails.plugins.springsecurity.userLookup.userDomainClassName = 'cq.User' grails.plugins.springsecurity.userLookup.authorityJoinClassName = 'cq.UserRole' grails.plugins.springsecurity.authority.className = 'cq.Role'

If you know that your usernames won't change, you can use them to salt the password. While not ideal as salts (an attacker can still build out rainbow tables of common username/password combinations pretty easily), they're a lot better than no salt at all.

To use the username as a salt, all you need to do is add one config setting to your Config.groovy. Unfortunately, the s2 manual had the wrong name for this setting; this is the right setting to add to Config.groovy:

grails.plugins.springsecurity.dao.reflectionSaltSourceProperty = 'username'

I'd also recommend turning on the setting that encodes the hashed passwords as base-64 strings (instead of strings of hex digits; it'll shave a dozen characters off the size of the hashed password):

grails.plugins.springsecurity.password.encodeHashAsBase64 = true

3 Basic Password Hashing

The other thing you need to do is make sure you hash the password with the salt whenever the password is saved. The s2 quickstart tutorial directs you to do this in your user controller. Don't; you should do this in the domain models, so you don't repeat yourself.

So update your "user" class to look like this:

package cq class User { def springSecurityService String username String password boolean enabled boolean accountExpired boolean accountLocked boolean passwordExpired static mapping = { // password is a keyword in some sql dialects, so quote with backticks // password is stored as 44-char base64 hashed value password column: '`password`', length: 44 } static constraints = { username blank: false, size: 1..50, unique: true password blank: false, size: 8..100 } def beforeInsert() { encodePassword() } def beforeUpdate() { if (isDirty('password')) encodePassword() } Set getAuthorities() { UserRole.findAllByUser(this).collect { it.role } as Set } protected encodePassword() { password = springSecurityService.encodePassword(password, username) } }

The main difference between the above and what s2-quickstart generates is the internal encodePassword() method. When a new user is saved, the beforeInsert() method will be called by the gorm framework, and our user class will hash the password, with the username as a salt. When an existing user is updated, the beforeUpdate() method will be called; it will check if the password has changed, and if it has, it will also hash the new password the same way.

This way you never have to hash a user's password in a controller or other code; just pass it on through to the domain model like any other property.

4 A Quick Test

At this point you've done enough to store passwords hashed with the username as a salt. Test it out by adding some test users in your bootstrap code, and a check for authenticated users on your home page.

In grails-app/conf/BootStrap.groovy, create and save a new test user:

class BootStrap { def init = { servletContext -> new cq.User(username: 'test', enabled: true, password: 'password').save(flush: true) } def destroy = { } }

And in grails-app/views/index.gsp, add this to the top of the body:

... <body> <sec:ifLoggedIn><h1>Hey, I know you; you're <sec:username/>!</h1></sec:ifLoggedIn> <sec:ifNotLoggedIn><h1>Who are you?</h1></sec:ifNotLoggedIn> ...

Now run your app (with clean, just to make sure everything gets rebuilt properly):

$ grails clean && grails run-app

Navigate to http://localhost:8080/cq/login (where cq is the name of your app), and login with a username of test and a password of password. Pretty sweet what you get (just about) out of the box, huh?

5 Adding a Unique Salt

Let's kick it up a notch. To create a unique salt for each user (making it impractical for an attacker to use rainbow tables to crack the hashed passwords), add a salt field to your "user" class, and override the getter for this field to initialize it with a unique salt:

package cq import java.security.SecureRandom; // add class User { def springSecurityService String username String password String salt // add boolean enabled boolean accountExpired boolean accountLocked boolean passwordExpired static mapping = { // password is a keyword in some sql dialects, so quote with backticks // password is stored as 44-char base64 hashed value password column: '`password`', length: 44 } static constraints = { username blank: false, size: 1..50, unique: true password blank: false, size: 8..100 // salt is stored as 64-char base64 value salt maxSize: 64 // add } def beforeInsert() { encodePassword() } def beforeUpdate() { if (isDirty('password')) encodePassword() } // add: String getSalt() { if (!this.salt) { def rnd = new byte[48]; new SecureRandom().nextBytes(rnd) this.salt = rnd.encodeBase64() } this.salt } Set getAuthorities() { UserRole.findAllByUser(this).collect { it.role } as Set } protected encodePassword() { password = springSecurityService.encodePassword(password, salt) // update } }

Don't forget to also update the encodePassword() method to hash the password with the salt field, instead of the username field.

6 Adding Custom UserDetails

Here's where it gets tricky. S2 maintains a user class for authentication called UserDetails that's completely separate from your "user" domain model. So you have to provide a custom UserDetailsService class that creates a custom UserDetails object given a username, as well as a custom SaltSource helper-class to extract the salt value from the custom UserDetails.

The good news is that you don't have to write a whole lot of code to do this. Create the following class as src/groovy/cq/MyUserDetailsService.groovy (or with whatever namespace and classname you like):

package cq import org.codehaus.groovy.grails.plugins.springsecurity.GormUserDetailsService import org.codehaus.groovy.grails.plugins.springsecurity.GrailsUser import org.springframework.security.core.GrantedAuthority import org.springframework.security.core.userdetails.UserDetails class MyUserDetailsService extends GormUserDetailsService { protected UserDetails createUserDetails(user, Collection authorities) { new MyUserDetails((GrailsUser) super.createUserDetails(user, authorities), user.salt ) } }

And create the following as src/groovy/cq/MyUserDetails.groovy:

package cq import org.codehaus.groovy.grails.plugins.springsecurity.GrailsUser class MyUserDetails extends GrailsUser { public final String salt MyUserDetails(GrailsUser base, String salt) { super(base.username, base.password, base.enabled, base.accountNonExpired, base.credentialsNonExpired, base.accountNonLocked, base.authorities, base.id) this.salt = salt; } }

And create the following as src/groovy/cq/MySaltSource.groovy:

package cq import org.springframework.security.authentication.dao.ReflectionSaltSource import org.springframework.security.core.userdetails.UserDetails class MySaltSource extends ReflectionSaltSource { Object getSalt(UserDetails user) { user[userPropertyToUse] } }

Note that if you use a java UserDetails implementation, instead of a groovy implementation, you can just use ReflectionSaltSource directly — you need to customize it only to do groovy "reflection" (it does java reflection just fine).

7 Configuring UserDetails

Finally, you can configure s2 to use your custom UserDetails class by adding the following to your grails-app/conf/spring/resources.groovy:

import org.codehaus.groovy.grails.commons.ConfigurationHolder as CH beans = { userDetailsService(cq.MyUserDetailsService) { sessionFactory = ref('sessionFactory') transactionManager = ref('transactionManager') } saltSource(cq.MySaltSource) { userPropertyToUse = CH.config.grails.plugins.springsecurity.dao.reflectionSaltSourceProperty } }

If you were to implement your UserDetails class in java you could omit the saltSource bean (since it comes configured out-of-the-box to do reflection on java classes). Otherwise, the one last piece of the puzzle is to go back and change the dao.reflectionSaltSourceProperty setting in your grails-app/conf/Config.groovy to your new salt field:

grails.plugins.springsecurity.dao.reflectionSaltSourceProperty = 'salt'

8 A Real Test

Now to verify that all this stuff is working (and will still work when you mess around with your user domain model in the future), you need some integration tests. First, let's tackle the password-hashing scheme; create a test/integration/cq/UsersTests.groovy class, and dump this in it:

package cq class UserTests extends GroovyTestCase { def springSecurityService void testPasswordIsEncodedWhenUserIsCreated() { def user = new User(username: 'testuser1', password: 'password').save(flush: true) assertEquals springSecurityService.encodePassword('password', user.salt), user.password } void testPasswordIsReEncodedWhenUserIsUpdatedWithNewPassword() { def user = new User(username: 'testuser1', password: 'password').save(flush: true) // update password user.password = 'password1' user.save(flush: true) assertEquals springSecurityService.encodePassword('password1', user.salt), user.password } void testPasswordIsNotReEncodedWhenUserIsUpdatedWithoutNewPassword() { def user = new User(username: 'testuser1', password: 'password').save(flush: true) // update user, but not password user.enabled = true user.save(flush: true) assertEquals springSecurityService.encodePassword('password', user.salt), user.password } void testPasswordIsNotReEncodedWhenUserIsReloaded() { new User(username: 'testuser1', password: 'password').save(flush: true) // reload user def user = User.findByUsername('testuser1') assertNotNull user assertEquals springSecurityService.encodePassword('password', user.salt), user.password } }

Now the authentication part. This is a bit awkward, because what we're really testing is that your custom UserDetails is implemented and configured correctly, but let's pretend that it's testing your login controller and stick it in your LoginControllerTests anyway. Create a test/integration/cq/LoginControllerTests.groovy class, and put this in it:

package cq import java.security.Principal import org.springframework.security.authentication.BadCredentialsException; import org.springframework.security.authentication.UsernamePasswordAuthenticationToken; class LoginControllerTests extends GroovyTestCase { def daoAuthenticationProvider void testAuthenticationFailsWithIncorrectPassword() { def user = new User( username: 'testuser1', password: 'password', enabled: true ).save(flush: true) def token = new UsernamePasswordAuthenticationToken( new TestPrincipal('testuser1'), 'password1' ) shouldFail(BadCredentialsException) { daoAuthenticationProvider.authenticate(token) } } void testAuthenticationSucceedsWithCorrectPassword() { def user = new User( username: 'testuser1', password: 'password', enabled: true ).save(flush: true) def token = new UsernamePasswordAuthenticationToken( new TestPrincipal('testuser1'), 'password' ) def result = daoAuthenticationProvider.authenticate(token) assertTrue result.authenticated } class TestPrincipal implements Principal { String name TestPrincipal(def name) { this.name = name } boolean equals(Object o) { if (name == null) return o == null return name.equals(o) } int hashCode() { return toString().hashCode() } String toString() { return String.valueOf(name) } } }

And now run your integration tests:

$ grails test-app integration:

The console outputs only a brief overview of the results. If something went wrong, you can find the details in the target/test-reports folder; enter the following in your browser address bar for the html version of the report (where $PROJECT_HOME is the full path to your project):

file://$PROJECT_HOME/target/test-reports/html/index.html

And hey presto, you've got salt.