Today's lesson: always be smart before you're stupid. This helps you prevent some damage from your stupidity. If you're stupid first, anything can happen.
For the first time in a good long while I missed my evening meditation because I was repairing this site. Sorry if you cam here and got the 502 Bad Gateway message.
I started an upgrade process as I was leaving work, but by the time I got home I realized I had done something bad.
Blog site down. Bad Gateway! Bad! Bad, naughty, awful, shameful gateway:
Then my kids had scouts and I didn't really get to jump in and fix it until about 8:45 PM.
The solution for me was, ultimately, not to fix it. I have backups of everything, and I am not a Linux admin or even much of a user anymore. What I needed to do was:
Start a new VM from my backup disk image.
Delete and reimport content in ghost.
Zip images on old VM and restore them on new VM.
Change the DNS entries to point to new VM.
Shut off old VM and release the IP address.
All of this should have taken about 20 minutes if I did it right the first time.
It took about an hour because I had to figure several of the steps out along the way.
It's kind of amazing that you can do this. You do need to be careful with backups - make sure you have them, make sure you back things up frequently and before you do anything dumb.
Always backup before you do ANYTHING that's dumb.
But once you have your backups, and once you have reached a state of self-awareness that allows you to ensure that you make them before making mistakes, the cloud makes the world a very simple place. Kudos Cloud Makers (Google in particular) you saved me a lot of time. Also, Ghost - their backup and recovery worked pretty well. I shouldn't really have had to do steps 4 and 5, but it was still pretty painless.
I'm a little sad that I couldn't fix whatever problem I created, but sometimes you must cut your losses and provision a new VM.
Those are rockets and they hold up the cloud. Those black hash-mark areas are patches in the cloud where things didn't work right. Because of these flaws someone needed to sew the cloud together. The patches are comprised of 81% code and 22% duct tape.
Why don't the cables burn up in the rocket exhaust? That is because of trained monkeys wearing asbestos suits. When the cable breaks or one monkey burns up, a monkey is added to the chain.
This concludes our brief tutorial on the cloud.
Here's the deal. This was inspired by a discussion with a client about a large cloud service provider. The service provider informed the client that the hardware that their instances ran on was bad and needed to be replaced. This is not the first time this has happened.
To me this revelation of 'hardware problem' feels tone deaf because everything about the cloud (which I mostly love) is predicated on us (the customers) not caring about hardware anymore: pricing structures, product offerings, marketing materials, billing. It seems wrong to then blame the hardware when it is convenient. My client didn't ask for you to tie their instance to some faulty hardware. My client had previously been entirely oblivious (and rightly so) to what hardware the instance ran on. Why don't they just quietly move the VM and fix the problem themselves?
Even if it was totally manual, it would give me a lot more faith in the magic and that they have their stuff together.
Truly, I love the cloud. Even if it is imperfect, it powers a lot of the modern world (including this site). I love it mostly because it is a technical marvel and it empowers business, but this customer service flaw makes me doubt the technical greatness. Why do that?
I really believe that cloud, IoT, and mobile technology are job creators in the long run, because there will just be so much of it.
Just, please, don't blame the hardware when the rest of the time you don't want me to think about it.
I'm in the midst of reading Ubik by Phillip K. Dick.
Here is an interesting theme from the book: people pay for a great many things as a service, instead of owning the use of those things outright. Many of the instances in the book are a bit silly for us in 2016 - a coin operated door to an apartment, for instance. But we have the benefit of an additional 45 years of history and science that PKD did not have in 1969, when Ubik was published.
Today there are a lot of things that we pay for in an ongoing way (albeit not with coins) - phones, books, music, computing power, automobiles, genealogy information, etc.
Phillip K. Dick was able to imagine a version of our electronic, service-based, cloud technology world well before any of it existed.
That he got some of the implementation details wrong is hardly surprising, given when he was writing.
A lot of old science fiction gets those sort of details wrong. It can be hard to look past those limitations sometimes - it's easy to focus on coin-operated doors or old special-effects technologies. But if you do, really good sci-fi gets a lot of things right. Heck, sometimes even bad sci-fi gets a few things right.
It's a healthy exercise in ignoring details when the details aren't the thing that matters - what mattered then (as now) was imagination and storytelling. PKD got those things very right a lot of the time.
Also, if you are ever in Ft. Morgan, CO, USA, you can visit Phillip K. Dick's grave. It is located in Riverside Cemetery. The people there are happy to help show you where his grave is located. He is buried with his twin sister who died when she was an infant.