Sunday, May 8, 2011

EC2 Sudo: Unable to Resolve Host

Sometimes after changing elastic-IP settings or stopping/starting instances on EC2, I get an irritating error like this when I execute a command with sudo:

sudo: unable to resolve host domU-12-34-ab-cd-56-78

The fix is to lookup the instance's private dns name (via ec2-describe-instances or the AWS console ui) and update the hostname on the instance with the first segment of that DNS name (which is something that looks like ip-12-34-56-78 or domU-12-34-ab-cd-56-78). On ubuntu, this is what you need to do (assuming ip-12-34-56-78 is the new hostname):

$ sudo hostname ip-12-34-56-78 $ echo 'ip-12-34-56-78' | sudo tee /etc/hostname

The first line will set the hostname until you reboot; and the second line will configure the hostname to use once you do reboot.

Saturday, March 19, 2011

Transparent Backgrounds in IE6

(And How to Duplicate TweakPNG's Tricks on Linux)

Reading the excellent HTML5 Boilerplate's Notes on Using PNGs for the first time, I found there's this windows-only program, TweakPNG, which lets you set the default background color for pngs. Normally, IE6 displays the alpha (transparency) channel of 32-bit pngs in a lovely blue-gray shade — unless you craft a css hack for each png (using the AlphaImageLoader filter), or do some magic with VML (like the impressive DD_belatedPNG library). The downside of both of these techniques is that they can really affect IE6's page-load time once you start using more than a couple pngs (although in fairness, just about anything other than plain-vanilla HTML hurts IE6's load time).

So if you're using transparent pngs in places where a solid background would be an acceptable alternative (like if the pngs usually are displayed against a plain-white background anyway), you can use TweakPNG to create pngs that IE6 will display with the background color you choose — instead of the default of blue-gray. And after a bunch of googling, I found that there's also a tool for this that linux (and other non-windows) users can use: The already super-useful pngcrush has an underdocumented feature (in the pngcrush man page, but not shown by its -h option) to change this same background-color setting.

For example, the following sets the default background of the new png to white (as well as performing pngcrush's default crushing magicks):

pngcrush -bkgd 255 255 255 original.png new.png

And the following instead sets its default background to dark red:

pngcrush -bkgd 127 0 0 original.png new.png

The new png will still retain the alpha-channel (and other pixel data) of the original — it just now will have a default setting for the background color, to be used by programs which don't support displaying the png with transparency (like IE6).

Monday, February 28, 2011

Getting Started with ELB

We recently started using Amazon's Elastic Load Balancing (ELB) service for our webapp (which is hosted in AWS). We're pretty happy with it so far; although we initially tried using Cherokee as our web server, that didn't seem to work quite right with ELB: About every few hours or so, we'd get a blank page back from the load balancer. I suspect it's a version of a similar issue between ELB and Apache that Amazon fixed back in May 2010. We switched over to using Apache for our web server, and that seemed to fix it, so we didn't dig any further.

One other little disappointing thing about ELB is that it can't serve "naked" domains (ie example.com instead of www.example.com), so we still have a single EC2 instance with a fixed ip address to which we point the DNS record for our naked domain name (and which our web server redirects to our "www" subdomain). Shlomo Swidler has a real nice blog post on why this is, and generally how ELB works (and how to test it).

Quick Start

The AWS docs already have a pretty quick, concise set of steps to setup an ELB for your web servers, so here's a little bit of a higher-level overview of what you have to do:

1 Create the Load Balancer

The only thing that's set in stone when you create the balancer is its name, which is the id you use to reference it, and also becomes the first segment of its dns name; and the region it's in. However, when you create the balancer, you also have to specify at least one port to listen on and at least one availability-zones among which to balance (although you can change both any time you want).

A load balancer can balance among all the availability zones of the region in which you create it. You do have to specifically enable each zone, however. If you don't register with the balancer any of your app's instances from any one particular availability-zone, the balancer will just ignore that zone (ie it won't try to balance to empty zones). You'll want to make sure you have an equivalent number of instances in each zone that you have any instances in, though, since the balancer doesn't adjust its balancing policy to account for the number of instances you have registered in a zone — it will just assume that it can balance to each zone equally.

If you're serving straight HTTP, you'll want to configure the balancer's listener to use the HTTP protocol (the other option is TCP, for everything else). You can configure the balancer to listen on one port and route to another port, if you want (like to listen on port 80, but route to port 8080 on your app instances).

2 Configure the Ping URL

Once you've create the balancer, the next step is to configure it with the URL of your app that the balancer should ping for each instance, to check if the instance is still healthy. You can actually specify a different port for the ping URL, so you could theoretically have the ELB ping a completely different application than the app is balancing (it just has to be some process listening on your instance). But you probably just want it to be some URL in your app that serves a stripped-down (or even empty) HTML page.

Also note that, as long as you specify the HTTP protocol for the ping URL, the ping URL must return an HTTP response status of 200 for the balancer to consider it to be OK. Anything else — including a 30x redirect to some other page — and the balancer will consider it to be unhealthy.

And when you configure the ping URL, you also configure the interval between pings (in seconds), and the max number of seconds in which your server must respond before the balancer times-out the ping. Also, you must set a threshold for the number of good pings before the instance is added to the group balanced by the ELB, and the number of bad pings before it is removed; AWS requires these threshold values to be at least 2. A pretty standard setting for this is to make the interval 30 seconds, the timeout 10 seconds, and the healthy and unhealthy thresholds 2 — so if an instance starts going sideways, the ELB will continue to balance to it for at most a minute and a half before it recognizes that it's gone bad.

3 Configure Sticky Sessions (or Not)

If you need the balancer to do "sticky" sessions (balance to the same instance over the life of your application's sesssion — for example because you have per-session state that's not shared among instances), you need to configure the ELB to use sticky sessions keyed off your application's session cookie. For example, if you're using a standard java servlet engine (like tomcat or jetty), you'd configure the ELB to use the JSESSIONID cookie to route all requests for the same session to the same instance. And note that with the ELB API, this is a two step process — the first step is to create a stickyness "policy", and the second step is to apply it (individually to each listener that needs to use the policy).

If you don't need sticky sessions, then you don't need to do anything special (ELB will do round-robin by default).

4 Register Web-Server Instances

The last step to get going is simply to add your app instances to the load-balancer. The balancer will balance only among instances in the availability zones you specified when you created the balancer (unless/until you separately enable/disable those zones). Note that you should add an equivalent number of instances to each zone to which you've added any instances, as the ELB will try to balance equally among all zones in which you have healthy instances.

5 Update Your DNS Records

The real last step, of course, is to point actual live traffic to the ELB. AWS gives you a unique (and unpredictable) DNS name for the balancer (like my-elb-name-123456789.us-east-1.elb.amazonaws.com; the DescribeLoadBalancers API response gives you it). The DNS name never changes over the life of the instance, though. So to direct your traffic through the ELB, you create/update CNAME records for all the subdomains you want the balancer to serve (ie www.example.com, www.example.org, etc), pointing each record to the balancer's AWS DNS name. As the DNS update propagates across the tubes, your users will start using the ELB.

Sunday, February 6, 2011

Custom Grails Constraints

There are already a couple of good tutorials on the web about creating custom grails constraints, but they include some extra cruft that you don't really need in their example classes. All you really need for your custom constraint class is to override the processValidate() method of the AbstractConstraint class to run your custom validation, and call rejectValue() if it fails:

package myapp import org.codehaus.groovy.grails.validation.AbstractConstraint import org.springframework.validation.Errors class LowerCaseConstraint extends AbstractConstraint { static NAME = 'lowerCase' boolean supports(Class type) { true } String getName() { NAME } protected void processValidate(Object target, Object value, Errors errors) { if (constraintParameter && value =~ /[A-Z]/) rejectValue target, errors, "default.invalid.${name}.message", "${name}.invalid", [constraintPropertyName, constraintOwningClass, value] as Object[] } }

The above example checks if the constraintParameter property is true (or at least truthy), and if so, rejects the value if it contains any uppercase ascii chars. The constraintParameter property is the value specified for this constraint in the domain class's constraints DSL. In the following example, it's true (from the lowerCase: true part):

static constraints = { myProperty size: 1..20, lowerCase: true }

The other thing you need to do is register your custom constraint; do this by adding the following line to your conf/Config.groovy (wrapped to fit the content area of this blog):

org.codehaus.groovy.grails.validation. ConstrainedProperty.registerNewConstraint myapp.LowerCaseConstraint.NAME, myapp.LowerCaseConstraint.class

And you're probably better off overriding and implementing a few more of the AbstractConstraint methods just to make sure that the constraint is used correctly (applied to an appropriate property type, and passed an appropriate constraint parameter). Also, it's probably better to segregate your validation logic into a separate method, for clarity and to make it easier to unit test:

package myapp import org.codehaus.groovy.grails.validation.AbstractConstraint import org.springframework.validation.Errors class LowerCaseConstraint extends AbstractConstraint { static NAME = 'lowerCase' // overridden Constraint methods /** Returns true if this constraint can be applied * to a domain property of the specified type. */ boolean supports(Class type) { type != null && String.class.isAssignableFrom(type) } /** Sets the constraintParameter value * (first checking if the constraint parameter is of the correct type). */ void setParameter(Object param) { if (!(param instanceof Boolean)) throw new IllegalArgumentException("Parameter for constraint [$name] of property [$constraintPropertyName] of class [$constraintOwningClass] must be a boolean") super.setParameter(param) } /** Returns the name of this constraint. */ String getName() { NAME } /** Adds an error to the specified errors object * if the specified property-value does not conform to this constraint * for the specified target object. */ protected void processValidate(Object target, Object value, Errors errors) { if (constraintParameter && !validate(target, value)) rejectValue target, errors, "default.invalid.${name}.message", "${name}.invalid", [constraintPropertyName, constraintOwningClass, value] as Object[] } // custom implementation methods /** Returns true if the specified value conforms to this constraint. */ def validate(target, value) { !(value =~ /[A-Z]/) } }

You then can easily write a unit test for your constraint:

package myapp import grails.test.GrailsUnitTestCase class LowerCaseConstraintTests extends GrailsUnitTestCase { def constraint protected void setUp() { super.setUp() constraint = new LowerCaseConstraint() } /** Asserts that the specified string passes this constraint. */ protected validateTrue(s) { assertTrue constraint.validate(null, s) } /** Asserts that the specified string fails this constraint. */ protected validateFalse(s) { assertFalse constraint.validate(null, s) } void testPassesWhenValueIsNull() { validateTrue null } void testPassesWhenValueIsEmpty() { validateTrue '' } void testPassesWhenValueIsLowerCaseString() { validateTrue 'foobar' } void testPassesWhenValueIsStringOfNumbers() { validateTrue '1234' } void testFailsWhenValueIsUpperCaseString() { validateFalse 'FOOBAR' } void testFailsWhenValueIsMixedCaseString() { validateFalse 'fooBar' } }

For a five-star tutorial on how to build your custom constraint as a plugin, check out Geoff Lane's Build a Custom Validator in Grails with a Plugin blog post.

Monday, January 31, 2011

Groovy HTTPBuilder Cookies

Groovy's super-convenient HTTPBuilder makes web-scraping a breeze. Surprisingly, it doesn't seem to have a built-in DSL for adding pre-configured cookies to your client's session. That's okay, because they're pretty easy to add (once you plow through the documentation). Here's a quick example:

#!/usr/bin/groovy import groovyx.net.http.HTTPBuilder import org.apache.http.impl.cookie.BasicClientCookie @Grapes([ @Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.5.1') ]) class MyScraper { static http = new HTTPBuilder('http://www.example.com/') static void main(String[] args) { try { run args } catch (e) { e.printStackTrace() } } static run(args) { // add some cookies addCookie domain: 'www.example.com', path: '/', name: 'mycookie', value: 'myvalue' addCookie domain: 'www.example.com', path: '/', name: 'anothercookie', value: 'anothervalue' // make some requests using the cookies def html = http.get(path: '/search', query: [q:'groovy']) // ... } // adds a cookie to the http client // with the specified name and value, and optional domain and path static addCookie(m) { // create the basic cookie object def cookie = new BasicClientCookie(m.name, m.value) // add optional cookie properties m.findAll { k,v -> !(k in ['name', 'value']) }. each { k,v -> cookie[k] = v } // add the new cookie to the client's cookie-store http.client.cookieStore.addCookie cookie } }

Thursday, January 27, 2011

Literal Tabs in Vi

My .vimrc, like any sane developer's, is configured to use spaces instead of tabs (with set expandtab). But sometimes you do actually need to insert a literal tab character (like when you're editing tab-delimited data). The web is pretty unhelpful on this topic — most of its advice is directed toward turning tabs into spaces, not vice-versa. Fortunately, vi's own help content has the answer (:help expandtab): in insert mode, just type [ctrl]-v [tab].

[Ctrl]-v is also useful when you need to insert other control characters, like a carriage return ([ctrl]-v [ctrl]-m), or a null ([ctrl]-v 0), etc. And you can also use it to insert characters via their unicode code-points (like μ [greek mu] with [ctrl]-v u03bc).

The other option for inserting literal tabs (useful if you need a bunch of them) is to temporarily turn off the expandtab setting, so that pressing [tab] in insert mode actually just inserts a literal tab. You can temporarily toggle off the expandtab setting with :set noexpandtab (or just :set noet), and then turn it back on again with :set expandtab (or :set et).

Sunday, January 16, 2011

MySQL to Groovy Gotchas

I've been running some raw sql in groovy recently (against a mysql db), and while most of the time the marshalling from sql result-sets to groovy objects is super convenient, there have been a couple things that caught me by surprise.

TINYINT Display Width

I had originally assumed that the "display widths" for integer types (like the (4) in INT(4)) were merely ornamental. But it turns out that, at least for TINYINTs, somewhere along the sql-to-groovy marshalling chain the display width is used to determine whether a TINYINT value should be marshalled as a boolean (for TINYINT(1)) or as a byte (for TINYINT(2)).

That's actually pretty clever, but not what I expected — maybe for BIT(1), where the values could only be 0 or 1— but not for TINYINT(1), where you'd expect the values might at least range from -9 to 9 (and still can actually be -128 to 127).

GROUP_CONCAT

I would have expected the result of this to be marshaled as a String (at least for TEXT fields) — but this seems to be instead marshalled as some sort of primitive array. When I dumped out its class name, it was [B. I think that might be like a primitive byte array; however, using groovy's join() method didn't work on it, so there must be something a little more complicated going on there. I didn't bother to check it out further — I just used the following to convert it to a string:

def sql = new Sql(dataSource) sql.eachRow('SELECT GROUP_CONCAT(my_field) AS concat FROM my_table GROUP BY other_field') { row -> println (row.concat as Character[]).join('') }