last week

Well, that didn't entirely go according to plan!

I haven't published a journal article for four weeks. Simply because other things have had to take priority.

That shift in priorities started on the 12th November when a major server used by my web and email hosting company started to spew errors from both of it's disks! Thankfully they didn't actually die and they were in a RAID which I think helped keep the whole system running, just.

The next 48 hours were spent migrating everything off that server as fast as I could (more on that below), while keeping services running as much as possible. This physical server was running lots of customer websites and all customer email, so some pretty critical services that people quickly notice issues with.

The critical services were all moved within that initial two days but, fully migrating all data and systems away from that computer took another couple of weeks.

In total I've spent roughly 40 hours of, completely unplanned and unscheduled, work recovering from this in the last three weeks.

Hence, writing up my journal slipped a little.

But this week, it's back!

Before the server break

As I post these the week after things actually happen and the server broke on the day I should have made a post, these first few things are from a slightly calmer schedule...

Great support

Not my regular area but I helped with a Magento to WooCommerce migration where the target WordPress is hosted by WordPress. This meant temporarily making a copy of their source database available remotely to WordPress. Initially this wasn't working so I sought support from WordPress to ask about any firewall restrictions. It turns out there shouldn't be any but for some reason remote MySQL database connections have to use the standard 3306 port. My point of posting though is that the support (via online chat) from WordPress was excellent - quick, technically knowledgeable and keen to discuss/suggest options and find a solution.

Old (abandoned) buildings

I managed to discover an abandoned factory in Redhill and an unused old school that's apparently been sold for conversion to housing.

Replacement Fitbit strap

My Fitbit Charge HR strap broke! I couldn't find any sign of official replacements but thankfully there are numerous third party replacements available and cost less than £10. The replacement was really fiddly though and involved several moments where I feared for my Fitbit's life! However the operation was successful and only took about 30 minutes in total.

Redhill Property Meetup

I organised an initial, experimental Redhill Property Meetup. Just three of chatting property in a pub for a few hours. Hopefully there'll be more soon.

After the server break

Bad Disks

Both disks in the server started reporting lots of reallocation errors. The visible symptom of this was prolonged slowness reading, and especially writing, to disk. Occasionally something would apparently get worse and things would pretty much grind to a halt for seconds or even a minute or so!

Definitely time to move everything to a different server.

Of course that means you need another server with enough capacity to take over the workload. I had some spare capacity in other servers but not enough for everything. So while starting to setup virtual machines elsewhere I was also sourcing a new server; then getting that setup, working and ready to install.

Next problem - copying data. The old server had the current data and was still accessible (sort of) but copying data from it was horrendously slow. Thankfully I also have nightly remote backups and the system I use for them has a handy BackupPC_tarCreate tool that can be used to pull out specific directories of content into tar files.

So I ended up restoring some sections of data from backup while simultaneously using direct server copying from the old server for other sections. The direct server copy being restricted by the dying disks and the backup restore being restricted by upload bandwidth.

After about 36 hours these efforts had got all but the largest chunks of data onto a new server and fully working (for email, I made use of the SMTP forwarding and Doveot's IMAP and POP proxying capabilities to keep services running across servers). The remaining data totalled about 110GB. Having extracted it from backup and compressed it before upload, the tar file was about 60GB.

When I stopped the upload, it had an estimated 21 hours left to complete.

Instead I copied that 60GB archive to temporary storage (over a LAN) and drive it to a faster internet connection where completing the upload only took 1 hour and 12 minutes! Using an alternative approach and a bit of physical transportation optimisation I managed to replace a 21 hour upload with 40 minutes of travel and just over an hour of upload.

Having only slept an hour the night before I managed to finish getting all my customer services fully running again before managing a whole six hours sleep.

Along this journey I discovered a handy list of exim commands.

Co-working

Somewhere in the blur of days following, I ran another couple of Redhill Micropreneur co-working days, including one at a brand new (and very impressive) Regus location in Manor Royal.

Theatre

Our two eldest children were also performing in a show at The Harlequin. The show was a version of Thoroughly Modern Millie Junior with the entire cast being local children. In the space of four days (following about three months of weekend rehearsals) the children had two four-hour dress rehearsals at the theatre, then two performances per day on Saturday and Sunday. I think it's fair to say everyone was a little tired and stressed by the end (or perhaps the beginning!) of that! I saw two of the performances and they were absolutely amazing, I was really, really impressed.

More Property Meets

Not only did these weeks see an initial Redhill Property Meetup but they also included another excellent Crawley Property Meet and an extremely informative NLA Crawley meeting with some great legislative update information.

IKEA Issues

Visiting IKEA is, generally speaking, a nightmare that's only just bearable for the gain (eg required furniture). Hence the fact that they now offer reasonably priced delivery is fantastic. Unfortunately our recent experiences of this, hopefully easier interaction method, have not proved ideal. We ordered a small bookcase and an assortment of KALAX drawer and cupboard inserts. Delivery attempt one looked OK on the outside but on the inside the bookcase had a bend/crack across one side panel and the drawer inserts both had damaged corners. Thankfully IKEA arranged collection and replacement delivery quite quickly. The next attempt managed to provide a working bookcase and one drawer unit, however the other drawer unit looked like it had a manufacturing fault - one of the joins/edges was ripped. We're now waiting for delivery attempt three - hopefully it will be the last!

Still a more efficient, and pleasant, use of time than a store visit (or multiple(!) to handle returns) would have been.