Pither.com

The Opscode wiki only provides some rather vague and incomplete hints for migrating Chef to a new server. Below is what I actually did and it worked for me. This is all running on Debian Squeeze and using Chef 0.10.8 from the Opscode provided Debian packages.

On the new server:

apt-get install chef-server
/etc/init.d/chef-server-webui stop
/etc/init.d/chef-server stop
/etc/init.d/chef-expander stop
/etc/init.d/chef-solr stop
/etc/init.d/rabbitmq-server stop
/etc/init.d/couchdb stop

On the old server:

curl -H "Content-Type: application/json" -X POST http://localhost:5984/chef/_compact
/etc/init.d/chef-server-webui stop
/etc/init.d/chef-server stop
/etc/init.d/chef-expander stop
/etc/init.d/chef-solr stop
/etc/init.d/rabbitmq-server stop
/etc/init.d/couchdb stop

tar -czf /tmp/chef_server_export.tar.gz /var/lib/chef /etc/chef /etc/couchdb /var/lib/couchdb

Transfer the resulting file from the old machine to the new one (eg via scp, rsync, etc).

Be careful with this next bit - these commands will overwrite existing data on the new server! So with that warning in mind and back on the new machine, with the above tar file now transferred into /tmp/...

cd /
tar -xzf /tmp/chef_server_export.tar.gz
/etc/init.d/couchdb start
/etc/init.d/rabbitmq-server start
/etc/init.d/chef-solr start
/etc/init.d/chef-expander start
/etc/init.d/chef-server start
/etc/init.d/chef-server-webui start

On you workstation edit your .chef/knife.rb file and update the chefserverurl line so it points to your new server. Then run:

knife node list

Just to check you get the right list back from the new server.

Then update the DNS entry for your chef hostname (hopefully you do have a name just used by chef!) and all your clients should catch up as DNS filters through.

Last weekend Sophie drew a picture of me, for me:

For the first time in quite a while, yesterday I caught a commuter train into London. I happened to sit in a section of nine seats. In the random collection of people in that set of seats:

• 2 were sleeping
• 2 were reading kindles
• 1 was reading an iPad
• 1 was using a Blackberry phone
• 3 were using Android phones (all different makes)
• zero were reading newspapers
• zero were reading paper books

I don't remember the split being like that last time I caught a London commuter train.

Every year or so I am reminded that yet another year has gone by without the HMRC and Companies House fixing their awful electronic CT600 corporation tax return. If you're a company you don't really get a lot of choice about using this form once a year and these days it pretty much has to be done electronically.

I'm going to mostly overlook the fact that this form is awfully designed, forces you to type in blocks of standard text and insists on you entering and re-entering identical information several times throughout the form. Even the use of the horrendous Adobe Acrobat forms technology could be forgiveable, if only it actually worked!

However, it sadly does not. There is (and has been every year that I've been involved with submitting this form) a problem with Adobe's Acrobat reader accepting the SSL certificates of the Companies House submission site.

This year, I thought I'd actually write up the process of fixing this. Let's start with the error message:

SSL Error!!! Please install the CA Certificate(s) for SSL
Communication if certificate resides on local disk, try
"acroread -installCertificate [-PEM|-DER] [pathname]" on
the command line. if certificate resides on the server try
"acroread -installCertificate xmlgw.companieshouse.gov.uk 443"
on command line.

It helpfully suggests a solution right there in the error message. Further more if you open a terminal and run the second command that is suggested, it will appear to do the right thing. The only trouble is, that it doesn't! What it will actually do is install just the first certificate in the chain and the full verification of the chain will still fail.

So, whistle stop tour of how to actually fix it (using the command line obviously, none of that nonsense web browser, pointing and clicking stuff that other guides out there already suggest!)...

Check that you can actually verify the certificate (you may need a different CApath, this one is suitable for Ubuntu/Debian):

$ openssl s_client -CApath /etc/ssl/certs/ -connect xmlgw.companieshouse.gov.uk:443
...
    Verify return code: 0 (ok)

Assuming openssl can verify the chain, you have everything you need. First, check how many certificates are involved. Look at the first few lines of output from the command above and you should see a certificate list, something like:

depth=2 C = US, ST = UT, L = Salt Lake City, O = The USERTRUST Network, OU = http://www.usertrust.com, CN = UTN-USERFirst-Hardware
verify return:1
depth=1 C = IE, ST = Dublin, L = Dublin, O = Digi-Sign Limited, OU = Terms and Conditions of use: http://www.digi-sign.com/repository, CN = Digi-Sign CA Digi-SSL Xp
verify return:1
depth=0 C = GB, ST = Wales, L = Cardiff, O = Companies House, OU = IT Infrastructure, OU = Provided by Digi-Sign Limited, OU = Digi-SSL Xp, CN = xmlgw.companieshouse.gov.uk
verify return:1

To collect the certificates we can from s_client, add "-showcerts" to the openssl command (and save a copy of the output):

$ openssl s_client -CApath /etc/ssl/certs/ -showcerts -connect xmlgw.companieshouse.gov.uk:443 | tee certs.txt

Within the output, you should see some certificates, I get two during this example. The important bits are wrapped with begin and end lines like:

-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

For each certificate, copy the output file to a new file and then edit it to remove everything except the certificate. Leaving just the begin/end lines and all the characters between them. Make sure each file only ends up with one certificate (and that they aren't the same one!). I called my two files cert1.pem and cert2.pem:

$ cp certs.txt cert1.pem
$ vi cert1.pem
$ cp certs.txt cert2.pem
$ vi cert2.pem

Then tell acroread about them:

acroread -installCertificate -PEM cert1.pem
acroread -installCertificate -PEM cert2.pem

Each time you should check the issuer displayed and confirm the import.

For me there is one certificate left, which is the top level CA that openssl does not list. To find this file, we'll need the issuer hash of the second certificate from above:

$ openssl x509 -issuer_hash -noout -in /tmp/cert2.pem 
b13cc6df

We can then use that hash directly to import the CA:

acroread -installCertificate -PEM /etc/ssl/certs/b13cc6df.0

Now, finally, we can get can to filling in the CT600!

A few weeks ago I ran the East Grinstead 10k. I managed it slightly slower than I was hoping (I blame the non-road surfaces and hills!), but it still wasn't too bad at 57:44.

My official position was 200/335 with my fellow runners John managing an impressive 139th and Chris just behind me in 202nd place. We all looked a bit worn out afterwards though (I was so bad I even ate the free banana they gave us!):

Quite a long time ago I talked about setting envelope sender addresses so that I could send mailshots and get back any bounces in a useful way. The missing component to it was to process those bounces within my application. Here, finally, is my write up of how I did this.

After much searching I eventually found the excellent SubEtha SMTP project which made including an SMTP server within my Grails application incredibly easy. Just four simple steps are required...

Include the library (grails-app/conf/BuildConfig.groovy):

dependencies {
    compile('org.subethamail:subethasmtp:3.1.4') {
        excludes 'mail'
    }
}

Write a SubEtha message listener (for me: src/groovy/com/supajam/smtp/MessageListener.groovy):

package com.supajam.smtp;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.subethamail.smtp.TooMuchDataException;
import org.subethamail.smtp.helper.SimpleMessageListener
import com.supajam.BounceRecord

public class MessageListener implements SimpleMessageListener {
    private final static Logger log = LoggerFactory.getLogger(MessageListener.class);

    public boolean accept(String from, String recipient) {
        return true;
    }

    public void deliver(String from, String recipient, InputStream data) throws TooMuchDataException, IOException {

        // I use this to process bounce messages, but it's just an
        // email message that could be processed however you like

        BounceRecord.withTransaction {
            if (log.isDebugEnabled())
                log.debug("Processing mail from " + from + " to " + recipient);

            // The regex on the next line is specific to my VERP scheme
            def origAddr = recipient =~ /-(.+=.+)@/

            if(origAddr && origAddr[0].size() > 1) {
                // here I extract the source address, ensure it's
                // valid, record the bounce and remove the address
                // from the mailing list.
            }
        }
    }
}

I've included an outline of the code that I actually use to process incoming email - all of which I know will be bounce messages. There's nothing to force this use though, it's just a generic way to receive emails into an application.

Wire it together with Spring (grails-app/conf/spring/resources.groovy):

beans = {
    // Create an in-process SMTP server for processing bounces
    smtpMessageListener(com.supajam.smtp.MessageListener)

    smtpMessageListenerAdapter(org.subethamail.smtp.helper.SimpleMessageListenerAdapter, ref('smtpMessageListener'))

    smtpServer(org.subethamail.smtp.server.SMTPServer, ref('smtpMessageListenerAdapter')) {
        port = 2500
        hostName = "my.server.name"
        disableTLS = true
        maxConnections = 10
    }
}

To avoid running my grails application as root, I set the SMTP port to a high number (2500 in this example). Getting emails delivered to such a port is beyond this post (but personally I use iptables redirects).

Finally, make it run at application start (grails-app/conf/BootStrap.groovy):

class BootStrap {
    def smtpServer

    def init = { servletContext ->

        log.debug("Starting SMTP server")
        try {
            smtpServer.start()
        }
        catch(Exception e) {
            // Starting the web application is more important
            // than SMTP, so just log the failure and carry on
            log.error("SMTP server could not be started")
        }
    }

    def destroy = {
        log.debug("Stopping SMTP server")
        smtpServer.stop()
    }
}

I guard the SMTP server start just in case there's a problem. No SMTP server is a nuisance but no web application is very bad. Just for cleanliness I also attempt to stop it in the destroy method.

That's it! It really is that simple to have a full SMTP server running within grails and ready to accept and process incoming email.

I administer (mostly through SEOSS) quite a few OpenVZ virtual machines and I recently wanted to create a clone of one onto a local system for some testing. Unfortunately I didn't have an OpenVZ machine locally, so decided I'd have a go at migrating the OpenVZ virtual to a local KVM instance. The process I went through is below.

Create a local disk image for the clone.

qemu-img create -f raw clone.raw 10G

Setup a loop back device so the image can have a partition table created.

sudo losetup /dev/loop0 clone.raw
sudo fdisk /dev/loop0

Create swap (1st primary partition, 2G in size and set type to Linux swap - just the fdisk key presses are listed).

n
p
1
<enter>
+2G
t
82

Create root (2nd primary partition, rest of disk - just the fdisk key presses are listed).

n
p
2
<enter>
<enter>

Finally save and quit fdisk.

w

Remove the simple loopback device.

sudo losetup -d /dev/loop0

Create loopback devices for each of the new partitions.

sudo kpartx -av clone.raw

Initialise the swap partition and create a suitable file system on the root.

sudo mkswap /dev/mapper/loop0p1
sudo mke2fs -j /dev/mapper/loop0p2

Mount the clone's root file system locally and rsync the remote source machine into it. In my case I connect to the actual, running VM as there is nothing running that will break my copy. In other cases it may be necessary to shutdown the source VM and rsync the underlying directory from the host machine.

mkdir mnt
sudo mount /dev/mapper/loop0p2 mnt
sudo rsync -avzx --numeric-ids root@your.source.machine:/ ./mnt/

An OpenVZ VM doesn't have terminals but some will be needed for KVM.

sudo vi mnt/etc/inittab
# Re-enable the tty terminals

Having a DHCP client will also make life easier with KVM, so I install a common one.

sudo chroot mnt
apt-get install isc-dhcp-client
exit

Finished with the root file system and the cloned image loopbacks.

sudo umount mnt
sudo kpartx -d clone.raw

For the next bit, you'll need to have a suitable kernel (to run the VM) installed locally (outside the VM).

sudo kvm -m 512 -kernel /boot/vmlinuz-2.6.38-11-generic -initrd /boot/initrd.img-2.6.38-11-generic -append "root=/dev/vda2" -drive file=clone.raw,if=virtio -net nic,model=virtio -net user

For some reason I had to change VT (ie Alt+F2) within the KVM console in order to get a login prompt. Make sure you login as (or become) root for the next bit.

For me the KVM networking interface appeared as eth3 (you can check with "ip a"), which needs to be used in the next commands.

echo "auto eth3" >> /etc/network/interfaces
echo "iface eth3 inet dhcp" >> /etc/network/interfaces

ifup eth3

Install a kernel (choose one suitable for your VM) and a boot loader (probably grub).

apt-get install linux-image-amd64 grub

The normal grub configuration questions will need answering during this. To tidy up you probably want to enable the swap space and create a simple fstab as well.

vi /etc/fstab

cat /etc/fstab 
/dev/vda2 / ext3 rw,relatime 0 0
/dev/vda1 none swap defaults 0 0

swapon -a

shutdown -h now

The machine can now be run without sudo and with a simpler command line

kvm -m 512 -drive file=clone.raw,if=virtio,boot -net nic,model=virtio -net user

That's it! You may want to reconfigure the networking, perhaps import the VM into libvirt/virt-manager or perform other configuration specific to your setup. For me, I just needed a local clone of the machine for testing so I stuck with the existing config and just ran it from the command line when needed.