Fifteen years ago, I worked on a blog that was hosted via Blogger.com (aka blogspot.com aka Google). We had a custom domain name for the blog and everything. It was pretty cool.

Now, many years later, the domain name is finally set to expire. We haven’t touched that blog in eleven years, but it still seems a shame for the content to just vanish. So I thought about making a static copy to host somewhere.

Google makes cloning one of these blogs difficult. They do, however, give you a backup/download capability. I went through re-activating the Google account that was tied to the blog, giving all sorts of identifying information and getting verification emails and texts. That done, I initiated the process to backup the blog, and shortly thereafter received an email that my download was ready. However, now Google is absolutely certain I’m not who I say I am (even with verification emails and texts), and their security locked me out of the account. Also, when I read up on the subject, even if I could download it, their site backup is an XML bundle that only works for reimporting to their blog system anyway.

So I thought I’d use the good old standby wget to build a static copy. I tried:

wget --mirror -w 2 -p --html-extension --convert-links --restrict-file-names=windows http://www.myurl.com

Yes, this site was so old that we didn’t use SSL… Still, Google stores the assets off in a bunch of other subdomains, and I was unable to come up with the correct syntax to allow wget to follow those. I’d get the pages, but everything still linked to the Google servers for the assets. That wasn’t going to work.

So next I used the old, powerful F/OSS friend, httrack. My first attack was as follows:

httrack "http://www.myurl.com/" \
  -O "myurl-offline" \
  -%v \
  --robots=0 \
  "+www.myurl.com/*" \
  "+*.blogspot.com/*" \
  "+*.bp.blogspot.com/*" \
  "+*.googleusercontent.com/*" \
  "+*.jpg +*.jpeg +*.png +*.gif +*.webp" \
  "+*.css +*.js" \
  "+*.mp4 +*.webm" \
  "-*/search?updated-max=*"

This worked — but a little too well. This blog was part of a community of sites, many of which were hosted elsewhere on blogspot. The cloning was slow. Then I noticed it had used up 2G of disk space, whereupon I discovered that I was happily making static copies of twelve other blogs from that community, and possibly more to come! I interrupted the process, and tried again removing the blank check for blogspot sites:

httrack "http://www.myurlcom/" \
    -O "myurl-offline" \
    -%v \
    --robots=0 \
    "+www.myurl.com/*" \
    "+*.bp.blogspot.com/*" \
    "+*.googleusercontent.com/*" \
    "+*.jpg +*.jpeg +*.png +*.gif +*.webp" \
    "+*.css +*.js" \
    "+*.mp4 +*.webm" \
    "-*/search?updated-max=*"

This was successful!

I now have a static version of the site. It’s not perfect; some references like the user profile links still point at blogspot. But if I want to be able to post the static site somewhere, I can do that, and it will be sufficiently usable that people can still experience the postings and articles.

Ugh, so the WordPress built-in gallery content type seems broken again. I’m not sure it’s worth bothering about. If i fix it locally, it’ll just break again on some future update.

We used to have a lot of physical devices on our network^*. Servers, firewalls, file-shares, staging servers, development machines… all sitting on the network with their hard drives endlessly spinning, spinning, spinning!

System administrators are fond of referring to platter-based hard drives as “spinning rust,” partly as a reference to the ferro-magnetic iron crystals that store the actual data, but also to remind us that it’s always decaying and corroding. Over time, drives start generating errors or becoming unreliable. When we had physical devices that exhibited issues, we’d yank the hard drive and replace it. Over the years, we’d accumulated a pile of a dozen or more drives that were unreliable or bad but still contained data.

The data is not especially sensitive, but there could be stuff that could be abused or belongs to other parties. There may well be meeting notes, source code, sample data files, or there could be cached passwords or other credentials. It’s not worth just hoping it’d be OK to release to the world. So it’s a chore to render this data unreadable.

Pulling apart spinning platter hard disks is humbling. These are incredible little devices, with incredibly precise machining and elegant engineering. Going through a pile that spans a decade, you can actually see the improvements in technology: new vibration damping systems, different head-parking strategies, traps for dust, and more. I see these parts, and am inspired by the craftsmanship that goes into them.

So in the spirit of admiration, I offer these (hopefully unreadable) holiday memories.

^* Now, of course, we have few physical devices but all those same services are implemented on “the cloud.” This means that someone else has physical devices somewhere, with their hard drives (or SSDs) endlessly spinning, spinning, spinning (or trimming, trimming, trimming).

Every time this happens, I get confused and lost, and have to rediscover the solution. So here’s a note to future self.

Symptom: PHPStorm stalls on an SVN update, and sits there doing nothing. Network is OK. SSH into server that supplies SVN, watch the WebDAV logs, and there’s nothing even trying to talk to it.

PHPStorm is configured to use an external Subversion client.

Solution: Don’t (necessarily) go and mess around with the settings for PHPStorm. Open a terminal, go to the working set in question, and do an “svn up” from the command line. This is where you’ll discover that SVN has either identified the server certificate as expired or updated, and it’ll ask you to approve the certificate (in that latter case). Log in with your credential again. Now it’ll all be OK again.

I have long wanted to be able to post visual stories where I can show a map of a place with specific pictures. If you go to the “Map View” of iOS Photos, or the “Map” view in Adobe Lightroom Classic, you can see sort of what I was looking for. There are many ways to view geo-tagged photos on a map, but not as many to be able to create a map for sharing.

What I wanted was to be able to tell a visual story, like a hike or travel day, and have it displayed in context. I wanted to be able to place this in a blog (perhaps even here!?). I wanted it to be easy: throw a few geo-tagged photos into a directory, run a script, and get my annotated map and scaled images. I saw there are WordPress plugins that will do this available for purchase, but they all rely on Leaflet or Google Maps and involve a lot of view-time dependencies. Furthermore, I don’t like Google Maps because anyone who views my story would be tracked and analyzed by the evil advertising algorithm.

The WordPress plugins are also more interactive than I need. I don’t necessarily want the ability to zoom, or scroll away from my specified region, or list all of the restaurants within the view.

So, to make a long story short, I’ve started to write the script I want. It’s an ugly PHP script that reads the geographical data from a folder full of JPGs. It uses a free account at MapBox to download map tiles and assembles the background. Then it draws the locations of each photo along with a direction indicator (also extracted from the EXIF data of the photos), ordering them by the time the picture was taken. It scales the images to a web-appropriate size, then it slaps together a primitive web page with the labeled map and just enough Javascript to display the photos in a “lightbox” effect when you click on the location. Once it’s generated, there are no external dependencies and the entire thing fits in about 4 kilobytes (not including the images).

Here’s an example in Ventura, California, and another in Mar Vista, California.

These raw outputs are fine, but if I wanted, the HTML is easy to edit to make more friendly. For example, I could replace the filenames in that right-hand column with descriptions.

There are some interesting challenges. It turns out the coordinates that the iPhone injects into the EXIF data are usually pretty accurate, but when I wandered around a small park and took pictures, in some cases the coordinates were off by a dozen meters or so. The compass direction tends to be more accurate, except in rare cases where it’s completely way the heck off. I don’t know if it’s magnetic interference, the fact that the orientation of the phone changes when one lifts it to take a picture, or what is the cause.

I should probably come up with a better strategy for overlapping photo points. It might be nice to anti-alias the circle that’s drawn. I could also potentially avoid drawing on the map at all, but rather use CSS to draw the locations within the web page. That would allow easier visual customization (at the expense of bigger file sizes and more complexity).

At some point, I should also get over my embarrassment, and open source the code.