Certificate Signing-Request Cookbook

After reading up on the openssl documentation for creating ssl certs, I've put together some quick recipes for generating certificate-signing requests (CSR) and self-signed certificates from custom config files (avoiding the laborious interactive prompts and the extra intermediate steps):

CSR for a Single-Domain Certificate

If you just want to create a certificate-signing request for a single domain, create a new directory (call it 'single-csr'), create a new openssl config file, add the following to the config file:

$ mkdir single-csr && cd single-csr $ echo ' [ req ] default_bits = 2048 default_keyfile = secret.key distinguished_name = req_distinguished_name encrypt_key = no prompt = no [ req_distinguished_name ] C = US ST = Washington L = Seattle O = My Company, Inc OU = Research & Development CN = www.example.com ' > openssl.cnf

If you're not using an existing secret key, you can simply generate the CSR like this:

$ openssl req -new -config openssl.cnf -out request.csr

This will generate a new CSR as request.csr in the current directory, along with a new (2048-bit RSA) secret key as secret.key. The secret key is the secret component of the certificate, so you'll want to hang on to it and keep it safe (when you configure your server, you'll point the server's configuration to both the public certificate and this secret key).

If instead you already have a secret key, specify the path to the key when you generate the CSR:

$ openssl req -new -config openssl.cnf \ -key /path/to/existing-secret.key -out request.csr

A CSR is just a plain text file with some base64-encoded (binary) data in it. You can decode it to check out what you just created with the following command (which will output the CSR's contents to the terminal); do this to make sure everything looks kosher:

$ openssl req -text -noout -in request.csr

The output should look like this:

Certificate Request: Data: Version: 0 (0x0) Subject: C=US, ST=Washington, L=Seattle, O=My Company, Inc, OU=Research & Development, CN=www.example.com Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): 00:cc:13:7b:91:ee:9f:36:25:4a:d8:ad:ae:20:51: 1e:b1:3d:8e:9e:21:88:2c:78:b9:50:ee:ae:fc:60: 73:6c:b4:42:ad:fc:7c:c8:b2:78:33:84:74:87:9d: 23:07:6e:9b:14:5c:7c:9c:c6:75:05:b7:c7:cf:88: 1f:14:66:30:19:97:fc:f1:1d:6f:ee:16:3c:46:b7: ed:35:ce:a6:49:18:5f:2b:ea:89:69:a3:f1:99:fe: a0:95:9d:a5:d6:e8:a0:f5:38:07:c0:6d:98:2d:0b: 04:f6:a5:32:56:e4:12:ab:ee:34:6b:07:71:06:6f: 58:b6:e7:0d:26:75:6d:06:22:c2:d4:bb:1a:43:9d: a6:09:c1:0c:fe:cb:ad:40:c4:3e:62:2c:49:5d:20: 79:0b:c6:93:27:d2:e1:b9:bd:3b:2e:e4:88:71:c4: 5e:a1:ce:45:14:2c:15:99:a3:ea:fe:77:ea:14:e5: 71:8b:c0:01:57:f5:61:4e:a8:19:92:6d:23:6b:78: 02:fc:54:7f:2a:3c:95:6f:37:b2:63:09:6f:13:9d: 47:47:4f:39:7b:79:f6:60:83:c3:2f:e7:db:1b:58: 6b:1d:3d:d6:c4:be:6a:1a:0c:e1:08:a0:4b:30:aa: 27:a4:e0:4c:eb:ba:2a:64:96:75:fe:c0:01:0d:4c: d5:b3 Exponent: 65537 (0x10001) Attributes: a0:00 Signature Algorithm: sha1WithRSAEncryption 92:9f:6e:15:66:12:90:0f:62:6c:f6:ca:79:4b:04:88:35:0c: 10:7b:f5:5c:6d:b7:f5:19:a3:3b:5c:eb:b9:fa:d3:63:95:a0: 1b:7f:69:9a:ad:4d:23:03:7d:fc:83:2c:dd:76:6d:7f:a5:da: 8a:53:34:82:eb:10:12:8c:22:2f:7b:cd:94:3a:8a:7d:fd:33: f5:ca:21:23:37:96:cf:00:64:93:82:ac:41:95:01:74:dd:ed: 83:68:ec:4b:29:87:19:63:fe:72:bf:44:91:ef:ac:a1:50:d9: 63:06:e6:5b:00:42:61:ca:3b:86:01:f9:2e:21:3c:58:4f:a7: d4:97:3d:89:5a:0b:11:c6:0d:49:95:ee:20:80:31:eb:5b:1a: 3c:ef:66:88:5b:12:23:9f:6d:67:ed:eb:18:83:0a:69:e1:82: 2a:46:41:24:48:12:64:42:90:99:7c:8b:bd:6c:65:33:d4:2f: f8:c4:99:b8:95:f7:d6:c1:c0:fc:d7:d4:fd:b7:27:3d:4a:ab: 14:82:4c:17:25:b0:ec:3e:9d:97:ac:8e:f0:1f:e4:92:de:28: a2:36:59:cf:71:fc:81:ed:0a:2a:ba:16:63:35:03:65:17:a2: 7f:13:ac:2a:54:39:ec:f0:1b:9a:7e:c5:3b:d1:74:c5:df:9e: 2f:a9:3e:58

Once you've submitted the request to a certificate authority, you don't need to keep around the request.csr file.

CSR for a Wildcard Certificate

If you want to create a certificate-signing request for a wildcard domain (ie *.example.com), follow the same exact steps as above, except in your config file, use the wildcard domain for the CN value:

$ mkdir wildcard-csr && cd wildcard-csr $ echo ' [ req ] default_bits = 2048 default_keyfile = secret.key distinguished_name = req_distinguished_name encrypt_key = no prompt = no [ req_distinguished_name ] C = US ST = Washington L = Seattle O = My Company, Inc OU = Research & Development CN = *.example.com ' > openssl.cnf $ openssl req -new -config openssl.cnf -out request.csr $ openssl req -text -noout -in request.csr

CSR for a UC Certificate

The steps are the same as above for a UC Certificate (aka Unified-Communications Certificate or UCC, which allows you to cover multiple domains with the same cert), except you need to add v3_ext and alt_names sections to the config file. Choose one of your domain names as the "primary" domain, and include it both as the CN value in the req_distinguished_name section, and as the first domain in the alt_names section. (In theory, in a UC cert it doesn't matter at all what you specify in the CN field — it doesn't even have to be a domain name — but in practice it's best if you specify one of the domains from the alternate-names section as the CN; many tools will identify the CSR by what you put in the CN field, and treat it like the cert's primary domain.)

$ mkdir ucc-csr && cd ucc-csr $ echo ' [ req ] default_bits = 2048 default_keyfile = secret.key distinguished_name = req_distinguished_name encrypt_key = no prompt = no req_extensions = v3_ext [ req_distinguished_name ] C = US ST = Washington L = Seattle O = My Company, Inc OU = Research & Development CN = www.example.com [ v3_ext ] subjectAltName = @alt_names [ alt_names ] DNS.1 = www.example.com DNS.2 = www.example.org DNS.3 = static.example.com DNS.4 = www.another-example.com ' > openssl.cnf $ openssl req -new -config openssl.cnf -out request.csr $ openssl req -text -noout -in request.csr

In the output of the resulting request.csr, you should see a X509v3 Subject Alternative Name: section, listing all your configured domains:

Certificate Request: Data: Version: 0 (0x0) Subject: C=US, ST=Washington, L=Seattle, O=My Company, Inc, OU=Research & Development, CN=www.example.com Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): 00:cc:13:7b:91:ee:9f:36:25:4a:d8:ad:ae:20:51: 1e:b1:3d:8e:9e:21:88:2c:78:b9:50:ee:ae:fc:60: 73:6c:b4:42:ad:fc:7c:c8:b2:78:33:84:74:87:9d: 23:07:6e:9b:14:5c:7c:9c:c6:75:05:b7:c7:cf:88: 1f:14:66:30:19:97:fc:f1:1d:6f:ee:16:3c:46:b7: ed:35:ce:a6:49:18:5f:2b:ea:89:69:a3:f1:99:fe: a0:95:9d:a5:d6:e8:a0:f5:38:07:c0:6d:98:2d:0b: 04:f6:a5:32:56:e4:12:ab:ee:34:6b:07:71:06:6f: 58:b6:e7:0d:26:75:6d:06:22:c2:d4:bb:1a:43:9d: a6:09:c1:0c:fe:cb:ad:40:c4:3e:62:2c:49:5d:20: 79:0b:c6:93:27:d2:e1:b9:bd:3b:2e:e4:88:71:c4: 5e:a1:ce:45:14:2c:15:99:a3:ea:fe:77:ea:14:e5: 71:8b:c0:01:57:f5:61:4e:a8:19:92:6d:23:6b:78: 02:fc:54:7f:2a:3c:95:6f:37:b2:63:09:6f:13:9d: 47:47:4f:39:7b:79:f6:60:83:c3:2f:e7:db:1b:58: 6b:1d:3d:d6:c4:be:6a:1a:0c:e1:08:a0:4b:30:aa: 27:a4:e0:4c:eb:ba:2a:64:96:75:fe:c0:01:0d:4c: d5:b3 Exponent: 65537 (0x10001) Attributes: Requested Extensions: X509v3 Subject Alternative Name: DNS:www.example.com, DNS:www.example.org, DNS:static.example.com, DNS:www.another-example.com Signature Algorithm: sha1WithRSAEncryption 92:9f:6e:15:66:12:90:0f:62:6c:f6:ca:79:4b:04:88:35:0c: 10:7b:f5:5c:6d:b7:f5:19:a3:3b:5c:eb:b9:fa:d3:63:95:a0: 1b:7f:69:9a:ad:4d:23:03:7d:fc:83:2c:dd:76:6d:7f:a5:da: 8a:53:34:82:eb:10:12:8c:22:2f:7b:cd:94:3a:8a:7d:fd:33: f5:ca:21:23:37:96:cf:00:64:93:82:ac:41:95:01:74:dd:ed: 83:68:ec:4b:29:87:19:63:fe:72:bf:44:91:ef:ac:a1:50:d9: 63:06:e6:5b:00:42:61:ca:3b:86:01:f9:2e:21:3c:58:4f:a7: d4:97:3d:89:5a:0b:11:c6:0d:49:95:ee:20:80:31:eb:5b:1a: 3c:ef:66:88:5b:12:23:9f:6d:67:ed:eb:18:83:0a:69:e1:82: 2a:46:41:24:48:12:64:42:90:99:7c:8b:bd:6c:65:33:d4:2f: f8:c4:99:b8:95:f7:d6:c1:c0:fc:d7:d4:fd:b7:27:3d:4a:ab: 14:82:4c:17:25:b0:ec:3e:9d:97:ac:8e:f0:1f:e4:92:de:28: a2:36:59:cf:71:fc:81:ed:0a:2a:ba:16:63:35:03:65:17:a2: 7f:13:ac:2a:54:39:ec:f0:1b:9a:7e:c5:3b:d1:74:c5:df:9e: 2f:a9:3e:58

Self-Signed Certificate (for a Single/Wildcard Domain)

To generate a self-signed certificate, you don't need a create a certificate request at all (contrary to other how-tos you might find); you can just create the certificate directly in a single step. You start with the same config as above:

$ mkdir mycert && cd mycert $ echo ' [ req ] default_bits = 2048 default_keyfile = secret.key distinguished_name = req_distinguished_name encrypt_key = no prompt = no [ req_distinguished_name ] C = US ST = Washington L = Seattle O = My Company, Inc OU = Research & Development CN = www.example.com ' > openssl.cnf

While you use the same openssl req command to generate the self-signed cert, you add the -x509 option to generate the certificate file directly (instead of just generating a request). You probably also will want to add the -days option, which specifies for how long the cert is good. The default is just 30 days; let's make it 10 years instead:

$ openssl req -new -x509 -days 3653 \ -config openssl.cnf -out self-signed.crt

This will generate the new cert as self-signed.crt in the current directory, along with a new (2048-bit RSA) secret key for it as secret.key. When you configure a server with this cert, you'll point it at both the (public) cert and the secret key. For example, you'd configure an apache vhost like this:

SSLEngine on SSLCertificateFile /path/to/self-signed.crt SSLCertificateKeyFile /path/to/secret.key

But before you try to use the cert, it's best to double check it, this time with the openssl x509 command:

$ openssl x509 -text -noout -in self-signed.crt

The output should look like this:

Certificate: Data: Version: 1 (0x0) Serial Number: ab:99:22:16:8c:cc:32:cd Signature Algorithm: sha1WithRSAEncryption Issuer: C=US, ST=Washington, L=Seattle, O=My Company, Inc, OU=Research & Development, CN=www.example.com Validity: Not Before: Sep 15 03:39:59 2011 GMT Not After: Sep 15 03:39:59 2021 GMT Subject: C=US, ST=Washington, L=Seattle, O=My Company, Inc, OU=Research & Development, CN=www.example.com Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): 00:cc:13:7b:91:ee:9f:36:25:4a:d8:ad:ae:20:51: 1e:b1:3d:8e:9e:21:88:2c:78:b9:50:ee:ae:fc:60: 73:6c:b4:42:ad:fc:7c:c8:b2:78:33:84:74:87:9d: 23:07:6e:9b:14:5c:7c:9c:c6:75:05:b7:c7:cf:88: 1f:14:66:30:19:97:fc:f1:1d:6f:ee:16:3c:46:b7: ed:35:ce:a6:49:18:5f:2b:ea:89:69:a3:f1:99:fe: a0:95:9d:a5:d6:e8:a0:f5:38:07:c0:6d:98:2d:0b: 04:f6:a5:32:56:e4:12:ab:ee:34:6b:07:71:06:6f: 58:b6:e7:0d:26:75:6d:06:22:c2:d4:bb:1a:43:9d: a6:09:c1:0c:fe:cb:ad:40:c4:3e:62:2c:49:5d:20: 79:0b:c6:93:27:d2:e1:b9:bd:3b:2e:e4:88:71:c4: 5e:a1:ce:45:14:2c:15:99:a3:ea:fe:77:ea:14:e5: 71:8b:c0:01:57:f5:61:4e:a8:19:92:6d:23:6b:78: 02:fc:54:7f:2a:3c:95:6f:37:b2:63:09:6f:13:9d: 47:47:4f:39:7b:79:f6:60:83:c3:2f:e7:db:1b:58: 6b:1d:3d:d6:c4:be:6a:1a:0c:e1:08:a0:4b:30:aa: 27:a4:e0:4c:eb:ba:2a:64:96:75:fe:c0:01:0d:4c: d5:b3 Exponent: 65537 (0x10001) Signature Algorithm: sha1WithRSAEncryption 92:9f:6e:15:66:12:90:0f:62:6c:f6:ca:79:4b:04:88:35:0c: 10:7b:f5:5c:6d:b7:f5:19:a3:3b:5c:eb:b9:fa:d3:63:95:a0: 1b:7f:69:9a:ad:4d:23:03:7d:fc:83:2c:dd:76:6d:7f:a5:da: 8a:53:34:82:eb:10:12:8c:22:2f:7b:cd:94:3a:8a:7d:fd:33: f5:ca:21:23:37:96:cf:00:64:93:82:ac:41:95:01:74:dd:ed: 83:68:ec:4b:29:87:19:63:fe:72:bf:44:91:ef:ac:a1:50:d9: 63:06:e6:5b:00:42:61:ca:3b:86:01:f9:2e:21:3c:58:4f:a7: d4:97:3d:89:5a:0b:11:c6:0d:49:95:ee:20:80:31:eb:5b:1a: 3c:ef:66:88:5b:12:23:9f:6d:67:ed:eb:18:83:0a:69:e1:82: 2a:46:41:24:48:12:64:42:90:99:7c:8b:bd:6c:65:33:d4:2f: f8:c4:99:b8:95:f7:d6:c1:c0:fc:d7:d4:fd:b7:27:3d:4a:ab: 14:82:4c:17:25:b0:ec:3e:9d:97:ac:8e:f0:1f:e4:92:de:28: a2:36:59:cf:71:fc:81:ed:0a:2a:ba:16:63:35:03:65:17:a2: 7f:13:ac:2a:54:39:ec:f0:1b:9a:7e:c5:3b:d1:74:c5:df:9e: 2f:a9:3e:58

On ubuntu/debian, you'll probably want to move self-signed.crt to the /etc/ssl/certs directory (the first place you should check when you forgot where you put it 10 years ago):

$ chmod 444 self-signed.crt $ sudo chown root:root self-signed.crt $ sudo mv self-signed.crt /etc/ssl/certs/www.example.com.crt

And secret.key to the /etc/ssl/private directory:

$ chmod 400 secret.key $ sudo chown root:root secret.key $ sudo mv secret.key /etc/ssl/private/www.example.com.key

Self-Signed UC Certificate

Generating a self-signed UCC is just a matter of using the openssl.cnf file for a UC CSR with the above steps — but with one little tweak to the config file. Rename the req_extensions field to x509_extensions:

$ mkdir uc-cert && cd uc-cert $ echo ' [ req ] default_bits = 2048 default_keyfile = secret.key distinguished_name = req_distinguished_name encrypt_key = no prompt = no req_extensions = v3_ext x509_extensions = v3_ext [ req_distinguished_name ] C = US ST = Washington L = Seattle O = My Company, Inc OU = Research & Development CN = www.example.com [ v3_ext ] subjectAltName = @alt_names [ alt_names ] DNS.1 = www.example.com DNS.2 = www.example.org DNS.3 = static.example.com DNS.4 = www.another-example.com ' > openssl.cnf $ openssl req -new -x509 -days 3653 \ -config openssl.cnf -out self-signed.crt $ openssl x509 -text -noout -in self-signed.crt

In the output of that last command (showing the text of self-signed.crt), you should see a X509v3 Subject Alternative Name: section, listing all your configured domains:

Certificate: Data: Version: 3 (0x2) Serial Number: ab:99:22:16:8c:cc:32:cd Signature Algorithm: sha1WithRSAEncryption Issuer: C=US, ST=Washington, L=Seattle, O=My Company, Inc, OU=Research & Development, CN=www.example.com Validity: Not Before: Sep 15 03:39:59 2011 GMT Not After: Sep 15 03:39:59 2021 GMT Subject: C=US, ST=Washington, L=Seattle, O=My Company, Inc, OU=Research & Development, CN=www.example.com Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): 00:cc:13:7b:91:ee:9f:36:25:4a:d8:ad:ae:20:51: 1e:b1:3d:8e:9e:21:88:2c:78:b9:50:ee:ae:fc:60: 73:6c:b4:42:ad:fc:7c:c8:b2:78:33:84:74:87:9d: 23:07:6e:9b:14:5c:7c:9c:c6:75:05:b7:c7:cf:88: 1f:14:66:30:19:97:fc:f1:1d:6f:ee:16:3c:46:b7: ed:35:ce:a6:49:18:5f:2b:ea:89:69:a3:f1:99:fe: a0:95:9d:a5:d6:e8:a0:f5:38:07:c0:6d:98:2d:0b: 04:f6:a5:32:56:e4:12:ab:ee:34:6b:07:71:06:6f: 58:b6:e7:0d:26:75:6d:06:22:c2:d4:bb:1a:43:9d: a6:09:c1:0c:fe:cb:ad:40:c4:3e:62:2c:49:5d:20: 79:0b:c6:93:27:d2:e1:b9:bd:3b:2e:e4:88:71:c4: 5e:a1:ce:45:14:2c:15:99:a3:ea:fe:77:ea:14:e5: 71:8b:c0:01:57:f5:61:4e:a8:19:92:6d:23:6b:78: 02:fc:54:7f:2a:3c:95:6f:37:b2:63:09:6f:13:9d: 47:47:4f:39:7b:79:f6:60:83:c3:2f:e7:db:1b:58: 6b:1d:3d:d6:c4:be:6a:1a:0c:e1:08:a0:4b:30:aa: 27:a4:e0:4c:eb:ba:2a:64:96:75:fe:c0:01:0d:4c: d5:b3 Exponent: 65537 (0x10001) Requested Extensions: X509v3 Subject Alternative Name: DNS:www.example.com, DNS:www.example.org, DNS:static.example.com, DNS:www.another-example.com Signature Algorithm: sha1WithRSAEncryption 92:9f:6e:15:66:12:90:0f:62:6c:f6:ca:79:4b:04:88:35:0c: 10:7b:f5:5c:6d:b7:f5:19:a3:3b:5c:eb:b9:fa:d3:63:95:a0: 1b:7f:69:9a:ad:4d:23:03:7d:fc:83:2c:dd:76:6d:7f:a5:da: 8a:53:34:82:eb:10:12:8c:22:2f:7b:cd:94:3a:8a:7d:fd:33: f5:ca:21:23:37:96:cf:00:64:93:82:ac:41:95:01:74:dd:ed: 83:68:ec:4b:29:87:19:63:fe:72:bf:44:91:ef:ac:a1:50:d9: 63:06:e6:5b:00:42:61:ca:3b:86:01:f9:2e:21:3c:58:4f:a7: d4:97:3d:89:5a:0b:11:c6:0d:49:95:ee:20:80:31:eb:5b:1a: 3c:ef:66:88:5b:12:23:9f:6d:67:ed:eb:18:83:0a:69:e1:82: 2a:46:41:24:48:12:64:42:90:99:7c:8b:bd:6c:65:33:d4:2f: f8:c4:99:b8:95:f7:d6:c1:c0:fc:d7:d4:fd:b7:27:3d:4a:ab: 14:82:4c:17:25:b0:ec:3e:9d:97:ac:8e:f0:1f:e4:92:de:28: a2:36:59:cf:71:fc:81:ed:0a:2a:ba:16:63:35:03:65:17:a2: 7f:13:ac:2a:54:39:ec:f0:1b:9a:7e:c5:3b:d1:74:c5:df:9e: 2f:a9:3e:58

Grails Foreign ID Generator

I haven't found a complete example on the web of using a foreign id generator in grails, so here's one: Say you have two domains, Primary and Secondary, with a one-to-one relationship (Primary hasOne Secondary and Secondary belongsTo Primary). Primary and Secondary basically represent the same entity, but Secondary has a bunch of data about the entity you rarely use. You map Primary to the DB table named primary, and Secondary to the DB table secondary; and since you've got a one-to-one relationship between Primary and Secondary, you just want the same column in the secondary table to be used as both its primary key and its foreign key to the primary table.

So you define Primary and Secondary like this:

class Primary { int oftUsedInfo int moreOftUsedInfo static hasOne = [ secondary: Secondary ] static mapping = { secondary cascade: 'all-delete-orphan' } } class Secondary { String littleUsedInfo String moreLittleUsedInfo static belongsTo = [ primary: Primary ] static mapping = { id column: 'primary_id', generator: 'foreign', params: [ property: 'primary' ] primary insertable: false, updateable: false } }

With that mapping, hibernate will create tables for you like the following (when using the MySQL InnoDB dialect):

CREATE TABLE `primary` ( `id` BIGINT(20) NOT NULL AUTO_INCREMENT, `version` BIGINT(20) NOT NULL, `oft_used_info` INT(11) NOT NULL, `more_oft_used_info` INT(11) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; CREATE TABLE `secondary` ( `primary_id` BIGINT(20) NOT NULL, `version` BIGINT(20) NOT NULL, `little_used_info` VARCHAR(255) NOT NULL, `more_little_used_info` VARCHAR(255) NOT NULL, PRIMARY KEY (`primary_id`), KEY `FK12344567ABCDEF` (`primary_id`), CONSTRAINT `FK12344567ABCDEF` FOREIGN KEY (`primary_id`) REFERENCES `primary` (`id`) ) ENGINE=InnoDB;

Instead of the secondary table having its own separate AUTO_INCREMENT id column, it just re-uses the primary_id column (referencing the primary table) as its primary key.

When you create a new Primary and Secondary instances programmatically, you'd do it like this:

new Primary( oftUsedInfo: 1, moreOftUsedInfo: 2, secondary: new Secondary( littleUsedInfo: 'foo', moreLittleUsedInfo: 'bar', ), ).save()

Or, if you want to do it property by property:

def primary = new Primary() primary.oftUsedInfo = 1 primary.moreOftUsedInfo = 2 primary.secondary = new Secondary() primary.secondary.littleUsedInfo = 'foo' primary.secondary.moreLittleUsedInfo = 'bar' primary.save()

And when you delete, you only have to delete the Primary domain (because of the all-delete-orphan cascade setting):

Primary.findAllByOftUsedInfo(1).each { it.delete() }

One more thing to note: using the assigned generator like this seems to generate the same database schema:

class Secondary { String littleUsedInfo String moreLittleUsedInfo static belongsTo = [ primary: Primary ] static mapping = { id column: 'primary_id', generator: 'assigned' primary insertable: false, updateable: false } }

Not sure if the behavior is exactly the same, however.

MySQL SSL Implementation Incompatibilities

So I just spent half a day trying to figure out why connecting over SSL to a redhat mysql server from a redhat mysql client works, but not from an ubuntu mysql client. Apparently, redhat is configured to use the OpenSSL implementation by default, whereas most other distributions (and Windows) use yaSSL by default — and OpenSSL and yaSSL aren't completely interoperable.

I initially had tried connecting just by specifying the CA certificate:

mysql --ssl-ca=my-ca-cert.pem -h myhost -u myuser -p

This is the minimum you (should) need to connect when you grant permissions for a mysql user with REQUIRE SSL (and the minimum needed for a worthwhile SSL connection — the CA cert allows the client to verify that you're connecting to the authentic mysql server). This worked fine connecting to a redhat box from a redhat box, but failed with the following inscrutable error-message when connecting to a redhat box from an ubuntu box:

ERROR 2026 (HY000): SSL connection error

This error can mean a whole lot of different things, but none of the common problems (like specifying the wrong certificate or using the same CN for both the CA cert and the client/server certs) turned out to be mine. After digging around a bunch, I found mysql bug 40141, where someone else had discovered connecting over SSL from a non-redhat box to a redhat box wasn't working, and had isolated it to a yaSSL-to-OpenSSL incompatibility. Following that trail, I came across mysql bug 29841, which documents the yaSSL-to-OpenSSL issue pretty clearly, as well as a message on the openssl-users mailing list confirming some of the low-level details:

Apparently, when initiating the SSL connection, the OpenSSL-server implementation expects the client always to send a client certificate; if there's no client certificate to send, it expects a blank cert. The yaSSL client, however, doesn't send any client certificate (even a blank one) when there's none to send.

So I ended up working around the problem by creating a client certificate (which ordinarily you'd use as an alternative to password authentication), and specifying it when connecting:

mysql --ssl-ca=my-ca-cert.pem --ssl-cert=my-client-cert.pem --ssl-key=my-client-key.pem -h myhost -u myuser -p

I didn't change the permissions on the mysql user to REQUIRE X509 (so from OpenSSL clients I can still connect without the client cert) — but specifying the cert does allow the connection to be made, and I haven't noticed any other interoperability issues between OpenSSL and yaSSL (so far).

Handling HTML5 "Multiple" File-Inputs With Grails

HTML5 includes a multiple attribute on file inputs; when set, a user can select multiple files to upload for that input with a single Browse... dialog. Grails makes handling uploads from non-multiple file-inputs really easy. For example, here's an action that echoes the content of a file uploaded via an input named myfile:

class MyController { def echoSimple = { def charset = (params.myfile?.contentType =~ /charset=([^;]+)/). collect { it[1].trim() }.join('') ?: 'ISO-8859-1' def content = new String(params.myfile?.bytes ?: ''.bytes, charset) render contentType: params.myfile?.contentType ?: 'text/plain', text: content } }

Handling multiple file-inputs is still pretty easy, but instead of accessing file objects via the controller's params map, you use the multiFileMap property of the controller's request property. File objects will be instances of spring's MultipartFile interface; for multipart form posts, the request property will be an instance of spring's MultipartRequest interface. The multiFileMap property is a map of file-input names to the list of MultipartFiles uploaded by that input. So if you have a form like the following:

<form action="${g.createLink(controller:'my', action:'echoMultiple')}" method="post" enctype="multipart/form-data"> <input type="file" name="myfile" multiple> <button type="submit">Submit</button> </form>

You can echo the content of all the files selected by the user with this action:

class MyController { def echoMultiple = { def content = request.multiFileMap?.myfile?.collect { file -> def charset = (file.contentType =~ /charset=([^;]+)/). collect { it[1].trim() }.join('') ?: 'ISO-8859-1' new String(file.bytes, charset) }?.join('\n') ?: '' render contentType: 'text/plain', text: content } }

(Note that while other browsers have supported the multiple attribute for some time, no version of IE currently supports it — although it should be supported in IE 10.)

EC2 Sudo: Unable to Resolve Host

Sometimes after changing elastic-IP settings or stopping/starting instances on EC2, I get an irritating error like this when I execute a command with sudo:

sudo: unable to resolve host domU-12-34-ab-cd-56-78

The fix is to lookup the instance's private dns name (via ec2-describe-instances or the AWS console ui) and update the hostname on the instance with the first segment of that DNS name (which is something that looks like ip-12-34-56-78 or domU-12-34-ab-cd-56-78). On ubuntu, this is what you need to do (assuming ip-12-34-56-78 is the new hostname):

$ sudo hostname ip-12-34-56-78 $ echo 'ip-12-34-56-78' | sudo tee /etc/hostname

The first line will set the hostname until you reboot; and the second line will configure the hostname to use once you do reboot.

Transparent Backgrounds in IE6

(And How to Duplicate TweakPNG's Tricks on Linux)

Reading the excellent HTML5 Boilerplate's Notes on Using PNGs for the first time, I found there's this windows-only program, TweakPNG, which lets you set the default background color for pngs. Normally, IE6 displays the alpha (transparency) channel of 32-bit pngs in a lovely blue-gray shade — unless you craft a css hack for each png (using the AlphaImageLoader filter), or do some magic with VML (like the impressive DD_belatedPNG library). The downside of both of these techniques is that they can really affect IE6's page-load time once you start using more than a couple pngs (although in fairness, just about anything other than plain-vanilla HTML hurts IE6's load time).

So if you're using transparent pngs in places where a solid background would be an acceptable alternative (like if the pngs usually are displayed against a plain-white background anyway), you can use TweakPNG to create pngs that IE6 will display with the background color you choose — instead of the default of blue-gray. And after a bunch of googling, I found that there's also a tool for this that linux (and other non-windows) users can use: The already super-useful pngcrush has an underdocumented feature (in the pngcrush man page, but not shown by its -h option) to change this same background-color setting.

For example, the following sets the default background of the new png to white (as well as performing pngcrush's default crushing magicks):

pngcrush -bkgd 255 255 255 original.png new.png

And the following instead sets its default background to dark red:

pngcrush -bkgd 127 0 0 original.png new.png

The new png will still retain the alpha-channel (and other pixel data) of the original — it just now will have a default setting for the background color, to be used by programs which don't support displaying the png with transparency (like IE6).

Getting Started with ELB

We recently started using Amazon's Elastic Load Balancing (ELB) service for our webapp (which is hosted in AWS). We're pretty happy with it so far; although we initially tried using Cherokee as our web server, that didn't seem to work quite right with ELB: About every few hours or so, we'd get a blank page back from the load balancer. I suspect it's a version of a similar issue between ELB and Apache that Amazon fixed back in May 2010. We switched over to using Apache for our web server, and that seemed to fix it, so we didn't dig any further.

One other little disappointing thing about ELB is that it can't serve "naked" domains (ie example.com instead of www.example.com), so we still have a single EC2 instance with a fixed ip address to which we point the DNS record for our naked domain name (and which our web server redirects to our "www" subdomain). Shlomo Swidler has a real nice blog post on why this is, and generally how ELB works (and how to test it).

Quick Start

The AWS docs already have a pretty quick, concise set of steps to setup an ELB for your web servers, so here's a little bit of a higher-level overview of what you have to do:

1 Create the Load Balancer

The only thing that's set in stone when you create the balancer is its name, which is the id you use to reference it, and also becomes the first segment of its dns name; and the region it's in. However, when you create the balancer, you also have to specify at least one port to listen on and at least one availability-zones among which to balance (although you can change both any time you want).

A load balancer can balance among all the availability zones of the region in which you create it. You do have to specifically enable each zone, however. If you don't register with the balancer any of your app's instances from any one particular availability-zone, the balancer will just ignore that zone (ie it won't try to balance to empty zones). You'll want to make sure you have an equivalent number of instances in each zone that you have any instances in, though, since the balancer doesn't adjust its balancing policy to account for the number of instances you have registered in a zone — it will just assume that it can balance to each zone equally.

If you're serving straight HTTP, you'll want to configure the balancer's listener to use the HTTP protocol (the other option is TCP, for everything else). You can configure the balancer to listen on one port and route to another port, if you want (like to listen on port 80, but route to port 8080 on your app instances).

2 Configure the Ping URL

Once you've create the balancer, the next step is to configure it with the URL of your app that the balancer should ping for each instance, to check if the instance is still healthy. You can actually specify a different port for the ping URL, so you could theoretically have the ELB ping a completely different application than the app is balancing (it just has to be some process listening on your instance). But you probably just want it to be some URL in your app that serves a stripped-down (or even empty) HTML page.

Also note that, as long as you specify the HTTP protocol for the ping URL, the ping URL must return an HTTP response status of 200 for the balancer to consider it to be OK. Anything else — including a 30x redirect to some other page — and the balancer will consider it to be unhealthy.

And when you configure the ping URL, you also configure the interval between pings (in seconds), and the max number of seconds in which your server must respond before the balancer times-out the ping. Also, you must set a threshold for the number of good pings before the instance is added to the group balanced by the ELB, and the number of bad pings before it is removed; AWS requires these threshold values to be at least 2. A pretty standard setting for this is to make the interval 30 seconds, the timeout 10 seconds, and the healthy and unhealthy thresholds 2 — so if an instance starts going sideways, the ELB will continue to balance to it for at most a minute and a half before it recognizes that it's gone bad.

3 Configure Sticky Sessions (or Not)

If you need the balancer to do "sticky" sessions (balance to the same instance over the life of your application's sesssion — for example because you have per-session state that's not shared among instances), you need to configure the ELB to use sticky sessions keyed off your application's session cookie. For example, if you're using a standard java servlet engine (like tomcat or jetty), you'd configure the ELB to use the JSESSIONID cookie to route all requests for the same session to the same instance. And note that with the ELB API, this is a two step process — the first step is to create a stickyness "policy", and the second step is to apply it (individually to each listener that needs to use the policy).

If you don't need sticky sessions, then you don't need to do anything special (ELB will do round-robin by default).

4 Register Web-Server Instances

The last step to get going is simply to add your app instances to the load-balancer. The balancer will balance only among instances in the availability zones you specified when you created the balancer (unless/until you separately enable/disable those zones). Note that you should add an equivalent number of instances to each zone to which you've added any instances, as the ELB will try to balance equally among all zones in which you have healthy instances.

5 Update Your DNS Records

The real last step, of course, is to point actual live traffic to the ELB. AWS gives you a unique (and unpredictable) DNS name for the balancer (like my-elb-name-123456789.us-east-1.elb.amazonaws.com; the DescribeLoadBalancers API response gives you it). The DNS name never changes over the life of the instance, though. So to direct your traffic through the ELB, you create/update CNAME records for all the subdomains you want the balancer to serve (ie www.example.com, www.example.org, etc), pointing each record to the balancer's AWS DNS name. As the DNS update propagates across the tubes, your users will start using the ELB.

Custom Grails Constraints

There are already a couple of good tutorials on the web about creating custom grails constraints, but they include some extra cruft that you don't really need in their example classes. All you really need for your custom constraint class is to override the processValidate() method of the AbstractConstraint class to run your custom validation, and call rejectValue() if it fails:

package myapp import org.codehaus.groovy.grails.validation.AbstractConstraint import org.springframework.validation.Errors class LowerCaseConstraint extends AbstractConstraint { static NAME = 'lowerCase' boolean supports(Class type) { true } String getName() { NAME } protected void processValidate(Object target, Object value, Errors errors) { if (constraintParameter && value =~ /[A-Z]/) rejectValue target, errors, "default.invalid.${name}.message", "${name}.invalid", [constraintPropertyName, constraintOwningClass, value] as Object[] } }

The above example checks if the constraintParameter property is true (or at least truthy), and if so, rejects the value if it contains any uppercase ascii chars. The constraintParameter property is the value specified for this constraint in the domain class's constraints DSL. In the following example, it's true (from the lowerCase: true part):

static constraints = { myProperty size: 1..20, lowerCase: true }

The other thing you need to do is register your custom constraint; do this by adding the following line to your conf/Config.groovy (wrapped to fit the content area of this blog):

org.codehaus.groovy.grails.validation. ConstrainedProperty.registerNewConstraint myapp.LowerCaseConstraint.NAME, myapp.LowerCaseConstraint.class

And you're probably better off overriding and implementing a few more of the AbstractConstraint methods just to make sure that the constraint is used correctly (applied to an appropriate property type, and passed an appropriate constraint parameter). Also, it's probably better to segregate your validation logic into a separate method, for clarity and to make it easier to unit test:

package myapp import org.codehaus.groovy.grails.validation.AbstractConstraint import org.springframework.validation.Errors class LowerCaseConstraint extends AbstractConstraint { static NAME = 'lowerCase' // overridden Constraint methods /** Returns true if this constraint can be applied * to a domain property of the specified type. */ boolean supports(Class type) { type != null && String.class.isAssignableFrom(type) } /** Sets the constraintParameter value * (first checking if the constraint parameter is of the correct type). */ void setParameter(Object param) { if (!(param instanceof Boolean)) throw new IllegalArgumentException("Parameter for constraint [$name] of property [$constraintPropertyName] of class [$constraintOwningClass] must be a boolean") super.setParameter(param) } /** Returns the name of this constraint. */ String getName() { NAME } /** Adds an error to the specified errors object * if the specified property-value does not conform to this constraint * for the specified target object. */ protected void processValidate(Object target, Object value, Errors errors) { if (constraintParameter && !validate(target, value)) rejectValue target, errors, "default.invalid.${name}.message", "${name}.invalid", [constraintPropertyName, constraintOwningClass, value] as Object[] } // custom implementation methods /** Returns true if the specified value conforms to this constraint. */ def validate(target, value) { !(value =~ /[A-Z]/) } }

You then can easily write a unit test for your constraint:

package myapp import grails.test.GrailsUnitTestCase class LowerCaseConstraintTests extends GrailsUnitTestCase { def constraint protected void setUp() { super.setUp() constraint = new LowerCaseConstraint() } /** Asserts that the specified string passes this constraint. */ protected validateTrue(s) { assertTrue constraint.validate(null, s) } /** Asserts that the specified string fails this constraint. */ protected validateFalse(s) { assertFalse constraint.validate(null, s) } void testPassesWhenValueIsNull() { validateTrue null } void testPassesWhenValueIsEmpty() { validateTrue '' } void testPassesWhenValueIsLowerCaseString() { validateTrue 'foobar' } void testPassesWhenValueIsStringOfNumbers() { validateTrue '1234' } void testFailsWhenValueIsUpperCaseString() { validateFalse 'FOOBAR' } void testFailsWhenValueIsMixedCaseString() { validateFalse 'fooBar' } }

For a five-star tutorial on how to build your custom constraint as a plugin, check out Geoff Lane's Build a Custom Validator in Grails with a Plugin blog post.

Groovy HTTPBuilder Cookies

Groovy's super-convenient HTTPBuilder makes web-scraping a breeze. Surprisingly, it doesn't seem to have a built-in DSL for adding pre-configured cookies to your client's session. That's okay, because they're pretty easy to add (once you plow through the documentation). Here's a quick example:

#!/usr/bin/groovy import groovyx.net.http.HTTPBuilder import org.apache.http.impl.cookie.BasicClientCookie @Grapes([ @Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.5.1') ]) class MyScraper { static http = new HTTPBuilder('http://www.example.com/') static void main(String[] args) { try { run args } catch (e) { e.printStackTrace() } } static run(args) { // add some cookies addCookie domain: 'www.example.com', path: '/', name: 'mycookie', value: 'myvalue' addCookie domain: 'www.example.com', path: '/', name: 'anothercookie', value: 'anothervalue' // make some requests using the cookies def html = http.get(path: '/search', query: [q:'groovy']) // ... } // adds a cookie to the http client // with the specified name and value, and optional domain and path static addCookie(m) { // create the basic cookie object def cookie = new BasicClientCookie(m.name, m.value) // add optional cookie properties m.findAll { k,v -> !(k in ['name', 'value']) }. each { k,v -> cookie[k] = v } // add the new cookie to the client's cookie-store http.client.cookieStore.addCookie cookie } }

Literal Tabs in Vi

My .vimrc, like any sane developer's, is configured to use spaces instead of tabs (with set expandtab). But sometimes you do actually need to insert a literal tab character (like when you're editing tab-delimited data). The web is pretty unhelpful on this topic — most of its advice is directed toward turning tabs into spaces, not vice-versa. Fortunately, vi's own help content has the answer (:help expandtab): in insert mode, just type [ctrl]-v [tab].

[Ctrl]-v is also useful when you need to insert other control characters, like a carriage return ([ctrl]-v [ctrl]-m), or a null ([ctrl]-v 0), etc. And you can also use it to insert characters via their unicode code-points (like μ [greek mu] with [ctrl]-v u03bc).

The other option for inserting literal tabs (useful if you need a bunch of them) is to temporarily turn off the expandtab setting, so that pressing [tab] in insert mode actually just inserts a literal tab. You can temporarily toggle off the expandtab setting with :set noexpandtab (or just :set noet), and then turn it back on again with :set expandtab (or :set et).

MySQL to Groovy Gotchas

I've been running some raw sql in groovy recently (against a mysql db), and while most of the time the marshalling from sql result-sets to groovy objects is super convenient, there have been a couple things that caught me by surprise.

TINYINT Display Width

I had originally assumed that the "display widths" for integer types (like the (4) in INT(4)) were merely ornamental. But it turns out that, at least for TINYINTs, somewhere along the sql-to-groovy marshalling chain the display width is used to determine whether a TINYINT value should be marshalled as a boolean (for TINYINT(1)) or as a byte (for TINYINT(2)).

That's actually pretty clever, but not what I expected — maybe for BIT(1), where the values could only be 0 or 1— but not for TINYINT(1), where you'd expect the values might at least range from -9 to 9 (and still can actually be -128 to 127).

GROUP_CONCAT

I would have expected the result of this to be marshaled as a String (at least for TEXT fields) — but this seems to be instead marshalled as some sort of primitive array. When I dumped out its class name, it was [B. I think that might be like a primitive byte array; however, using groovy's join() method didn't work on it, so there must be something a little more complicated going on there. I didn't bother to check it out further — I just used the following to convert it to a string:

def sql = new Sql(dataSource) sql.eachRow('SELECT GROUP_CONCAT(my_field) AS concat FROM my_table GROUP BY other_field') { row -> println (row.concat as Character[]).join('') }

Android SDK Upgrade

Just re-built an old android 1.0 application with the latest sdk (r8); was pretty smooth — all I had to do was:

1. Install the official java 6 sdk (apt-get install sun-java6-jdk).
2. Download the latest android sdk.
3. Install the latest android sdk components.
4. Download the latest eclipse ide ("classic" package).
5. Install (and configure) the adt eclipse plugin.
6. Delete the 1.0 project's .classpath and .project files.
7. Create a new "android project" in eclipse in the same directory as the old one (where you deleted the old .classpath and .project.
8. Filter out the .svn directories in eclipse's build path (if I had installed subclipse or something similar I probably could have skipped this).
9. Add a new uses-sdk element to the AndroidManifest.xml to ignore some retro-fitted permission requirements.
10. Increment the android:versionCode attribute in the AndroidManifest.xml.
11. Generate some new android-market graphics (two screenshots, a high-resolution icon, a "promotional" graphic (seems to be used on the market's details page for the app), and a "feature" graphic (seems to be used on the market's top-level page if the app is featured on it).
12. Clean rebuild, export the package, test it, publish!