BOFH: The ONE-NINE uptime solution

Year 2015 - Episode 2

Episode 2

"So what's the problem again?" the Boss asks, lurching into Mission Control like a Dalek with stepper motor problems.

"We're not sure - it's not internet connectivity because we can get to other sites ok, but we're definitely having problems with the email site."

"We shouldn't be!" he snaps back, presumably thinking that a quick statement of the obvious is a corrective procedure.

"Yes, we shouldn't be - but we are."

"So how long till it's back again?"

"That's largely dependant on our email supplier."

"I thought you said this would never happen?"

And there's the issue. When the mail server hardware you're using is so old that all the plastic in the badges (with a vendor name no-one can remember any more) have started to discolour and peel off you know you have a potential problem. When you suggest to Senior IT Management for the third year running that if they're going to save money on some project, perhaps it should be the intranet portal, which does nothing, links to nothing and produces nothing (apart from seething anger in the users who are forced to use it to submit online holiday and sick leave requests), your problem is a little bit larger.

Instead of raging against the machine as it were, I take the moral high ground and point out to the Boss that this problem has occurred because we were forced into taking our mail into the cloud.

I'm not actually against cloud-based email - so long as people understand that split-second response times they envisage are potentially not going to be realised.

It's my own fault for not nipping this in the bud when the portal idea first raised its head. Miles after everyone else had realised that portals were last year's nightmare the company decides that we need one (thanks to a consultant who appeared out of absolute nowhere, made some outrageous recommendations and disappeared like magic before the project could be implemented. [By 'magic' I mean very much like the cement that was poured into the foundations of the building across the road around three-and-a-half years back when the project kicked off.]).

And the project - unlike the consultant concerned - just would not die. You get a couple of senior managers of some departments keen on an idea with lots of flashing lights and potential glory and they'll be throwing money at it like it's a stripper with a vacuum cleaner. And the more money they tip into it, the less likely they are to axe it, so in the end it's a vast black hole of cash threatening to suck the company into the void.

MY cash - destined for good, not for waste.

So when the time comes to invest in a cloud solution to email problems (because although we don't have a capital budget to buy hardware we do have an operations budget they can fillet) we choose the cheapest solution there possibly is. One with what the PFY has called the ONE NINE uptime solution. That's 9 per cent, not 19 per cent.

Support is non-existent, because the 24-hour support lines are run over a proprietary VoIP audio conference platform which - surprise, surprise - has performance issues at exactly the same time as the mail server has issues (i.e. work hours).

I've tried to track the server location but a traceroute has a stack overflow before it ever gets near a host which looks like it could be legit. Sure, it might be some multihomed dynamic service but I've got 5 quid on it being a coal-powered 386 running in the craphouse of a minicab office in Cairo.

On the plus side though, if I ever kidnap someone I know how I'm going to be anonymising my ransom calls.

"I think you'll find that I didn't say that it'd never happen. In fact I think I said I'd put 50 quid on you begging me to bring our old server back within 2 months. So that's 50 quid you owe me."

"I'm not asking for it back and I'm not paying you 50 quid."

"Fair enough - I've found the best time to call them is sometime after 10:18pm our time. No idea what it is their time - wherever they are, but that's usually when they answer the call."

"I think YOU should call them." he says.

"Oh we are," I say. "I put a call through to them every morning. That's what the music in the background of the office is. And like I say, they generally answer at 10:18pm."

"What did they say?"

"How would I know - it's 10:18pm. I just know when they answer from the phone logs."

"So you've not logged a fault?"

"I would if I could get hold of them."

"Can you email them?"

"There's no email address on their website, just a page which gives you their phone numbers. In fact the ONLY email contact I think we've ever had with them was whoever signed us up for the service."

The Boss looks away guiltily. "So CAN we get the old server up and running?"

"Fifty Quid." I say.

"I'm just asking if it's POSSIBLE, not asking you to do it."

I remain silent and the Boss wanders off. 10 minutes later he returns with freshly dispensed 50 quid note.

"No." I reply.

"Well I'll have my 50 quid back then!"

"I'm sorry; your total credit is now zero."

"The bet was that you'd come wanting the server back. I rest my case."

"Ah, but as you said, I didn't BEG!"

"What's that?" the Director of IT says, entering Mission Control.

"Oh he paid me 50 quid to make him beg."


"Look, as I said to Schrodinger - what happens in the cat box stays in the cat box."

"?" the Boss and Director wonder silently.

"It's a quantum thing - and I can't explain it to you as you might get entangled."


"Playing here all week ladies and gentleman. Was there anything else?"

"We need the mail back online."

"We can't use the old server as it crashed shortly after we changed over?"

"Can't we get new disks?"

"No I mean crashed - as in into the footpath from the 6th floor - because I knew you'd try and resurrect it."

"Look, we're getting a lot of pressure from above!" the Director blurts, attempting sternness.

"You mean like degenerative matter?" I ask.


"Don't forget to pick up my DVD at the door," I chirp. "Look, how do you want to do this: the easy way or the hard way?"

"Uh... the EASY way" the Boss and Director concur.

"Find the budget and replace the servers. Look, here's a purchase order I prepared earlier - the quotes are still valid and I asked them to put the kit on hold. TOTAL, probably 15K, including licenses."

"Uh, what was the hard way?"

"We pick another vendor from undeliverables anonymous and shift our mail to them for another couple of months, and then one of you has the misfortune to visit a building site with me on the night before they pour the foundation concrete while the other one receives cryptically untraceable VoIP calls for the ransom money of $15K."

"I... who..."

"It's a coin toss," I reply, pushing the purchase order over to them.

Clarity. Focus. That's all we need.