I’ve migrated my notes back to Obsidian, and have been using it pretty much exclusively for note-taking since I last posted in September.

There’s a core plugin for Obsidian called “Importer” that will import Apple Notes. However, there is a caveat: the notes have to be unlocked. So if you have a bunch of locked (encrypted) notes, you’ll need to unlock them before doing the import. Another caveat that I discovered today, a month after my switch, is the way the importer uses the first line of an Apple Note as a title. More on this below.

I wanted to be able to encrypt select notes, like I did with note locking in Apple Notes. There’s a community plugin for Obsidian called “MeldEncrypt” that lets me do that. The UI takes a little getting used to, but it seems to work pretty well.

Then there’s syncing. I want to access my notes on my phone, on my notebook, or wherever. I’d like to be able to add a new computer or device to this sharing at any time.

Initially, I set up the Obsidian vault (which is the directory structure on the filesystem where it saves the notes) in an iCloud directory. That way, on my iPhone or notebook computer, I was able to open the vault and let Apple handle the syncing. But this whole process is about moving away from proprietary systems. iCloud is a proprietary dependency and isn’t available on Linux. Obsidian has a subscription sync service that I could use, but again, I’d like to be independent of other providers and keep my stuff on “my” servers.

There’s a plugin called RemotelySave, which allows syncing notes to a wide variety of services. It’s currently free, and will be adding more sophisticated features for paid subscribers, but theoretically will continue to be free for basic sync to standard services. One of those standard services is WEBDAV.

Well, I have a VPS to host web stuff. I know how to set up Apache services. So I went in and configured a WEBDAV service on one of my web servers (not the one hosting this blog). I secured the HTTP connection using a LetsEncrypt certificate and Basic Authentication, and point it at a directory outside the web root (in this case /var/webdav). The Apache configuration looks like:

<IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerName webdav.somedomain.com

    Alias /webdav /var/webdav

    <Directory /var/webdav>
        Options None
        AllowOverride None

        Dav On

        AuthType Basic
        AuthName "WebDAV"
        AuthUserFile /etc/httpd/webdav.passwd
        Require valid-user

        <LimitExcept GET HEAD OPTIONS>
            Require valid-user
        </LimitExcept>
    </Directory>

    ErrorLog /var/log/httpd/webdav_error.log
    CustomLog /var/log/httpd/webdav_access.log combined

SSLCertificateFile /etc/letsencrypt/live/webdav.somedomain.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/webdav.somedomain.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

I had to create the directory /var/webdav, give the web account access to it, and use htpasswd to create a user/password file as /etc/httpd/webdav.passwd.

So then I set up RemotelySave, and watched the data show up on the server! Excellent! You can set RemotelySave to sync on various schedules, on or after Obsidian startup, and (experimentally) upon save. I opted to set it up to sync one second after startup and upon save.

Then I set up Obsidian on a Linux box that has no access to the iCloud directory. Would it sync? Boom! No, it would not. Going into the low-level logs, I could see the problem.

Remember above I said there were issues with the first line of notes and titles? That’s what tripped me up here. I had a big folder of notes that were quotations. Often, I’d paste those into Apple Notes and not do any further editing. The Obsidian Apple Notes importer sees the first line of each note as its title.

Also, the filename for each note created (or imported) by Obsidian is based on the title, and seems to be truncated at 60 characters. Mysteriously, though, some imported Notes had filenames that include the entire title. I can’t find the factor that determined which were truncated and which were not. Many of the un-truncated titles start with quotation marks, but also many notes that have titles starting with quotation marks truncated correctly. In any case, I had a number of files with names over 255 characters. Those crashed the plugin when I tried to sync to Linux.

Another thing I discovered at this point is that the importer truncates the first line of the note after 200 (or so) characters within the newly created note. Up ’til now, I hadn’t noticed that I’d lost significant content on some of these notes that were created when I pasted long quotations into Apple Notes.

Fortunately, even at this late time, I can remediate the problem by going into Apple Notes, changing the title of these notes to something shorter, and re-running the Obsidian Importer plugin for that folder. It cleverly uses the Apple Note opaque ID to decide whether to update or create a note. I then had to delete the long-named files manually on the WEBDAV server filesystem, and re-sync.

It worked! Now the files sync correctly to Linux as well.

Again, I’ll report back here if I have any new conclusions.

I’ve gone through a lot of notes apps in my time. I’ve used SimpleNotes, Joplin, Obsidian, and Apple Notes. My requirements are relatively simple: notes must be fast and easy to enter, I need to be able to search through them expeditiously, I need to be able to organize them by nested folder structure (e.g., work > meetings > clientX), I need to be able to edit them offline, and I need them to sync between my desktop (MacOS) and my phone (iOS). I need them to be encrypted if the notes are sitting on someone else’s server.

There are a bunch of other features I’d really like to have too: ordinary text files, easily exportable to a portable format (pure text ideally, markdown acceptable, Word or PDF as a last resort), local encryption or password protection of notes, versioning. I’d like the program to be free or a one-time payment; I don’t want to be stuck with yet another subscription. Other ideal-but-not-absolutely-necessary features would be scriptability, support for “to-do” style lists, F/OSS.

For various reasons, every year or two, I get frustrated with the note program I’m using, and painfully switch. Most recently, I went from Obsidian to Apple Notes.

Apple, however, has been annoying me. They’re stuffing AI and “Apple Intelligence” into everything, and I’m old, and cranky, and believe in the benefits of human creation, and I just don’t want it. I want a platform that respects my wishes rather than tells me what’s best for me. I know, I know, Apple has always had their own opinion of what I wanted, but for many years it was close enough that I was willing to go with it. But the mandatory AI is crossing the Rubicon.

Despite the fact that Linux ecosystem source code is now widely being written using AI, I still have the ability to turn it off on the user side. But if I switch to a Linux desktop, I have a new requirement for my notes: they need to be cross platform. (Also, dunno what I can do about switching to a non-AI-ridden phone. Maybe it’s time for a dumb phone.)

Anyway, I’m moving back towards Obsidian.

I’ve always used a shared filesystem approach to syncing Obsidian: my “vault” was stored on in an iCloud directory. This isn’t perfect, but works so long as I’m careful not to edit on two devices at the same time or the same note when devices are offline. However, if I go cross-platform, iCloud will no longer be viable. I could probably move to Dropbox — there are clients available for Linux. The Obsidian app on the iPhone can access that through additional plugins, but then I’m beholden to yet another 3rd party service. Maybe it’s time to install my own file-share/sync.

I’m currently working on setting up a VPS with Seafile. If all goes well, I can try syncing Obsidian across platforms using this Seafile community server instance and the Seafile-sync plugin. I’ll report back here if I get anywhere.

The WyzeCam3 is a relatively cheap device that can capture video. It can be set up for motion capture or continuous recording. But even if you install a memory card in the device, it will want to use the company’s app and stream data via their cloud storage. I don’t know much about how much thought went into their codebase with regard to security, but this is the kind of device where the usual answer is “not much.”

Furthermore, the WyzeCam requires an app for installation and configuration, and the WyzeCam3 needs to connect to your 2.4GHz WiFi network. Oops. I have no 2.4GHz WiFi network. It’s all 5Ghz here. Now what?

Enter Thingino, an open-source replacement firmware. One nice feature is that it allows the camera hardware to enable an Wi-Fi access point (“AP mode”) — it creates its own 2.4GHz WiFi network, which my Mac will happily connect to! Once the camera is set up, I can further customize things, and have it write motion capture files to the SD memory card rather than someone’s cloud.

So, there’re a few gotchas that were not obvious to me:

1. When you’ve downloaded the firmware, you need to use something to flash it as an image on your micro-SD (e.g., Balena Etcher, Rufus, or the like). I came across at least one guide that said “simply copy the files onto the SD card” which is incorrect!
2. When the Thingino installation has completed, you connect to the WiFi network it creates. A captive portal form pops up for basic configuration. You’re asked to provide a device name, a root password, and a WiFi SSID and password. There’s also a checkbox to have the camera create its own WiFi hot-spot (the aforementioned AP mode). If you’re going to have it work in AP mode, the SSID and password you provided above are used for the network it creates. If you are not, it will use those to connect to an existing network. Don’t make the mistake of giving the SSID and credentials for your existing network and then checking that checkbox!
3. Once you’ve installed Thingino via the SD card and you’ve connected to the network admin, you might want to set up motion capture to “Storage.” You’re gonna have to reformat that SD card first! The UI won’t complain, or tell you why it’s not working, but it just won’t let you submit the form. Thingino recommends formatting in exFAT.
4. Even if you’ve configured the “Motion” panel, you have to click on the Storage link and tell it that you really mean the storage mounted at /mnt/mmcblk0p1 (or the mountpoint for your SD card ). You can have motion sensing and capture enabled, but it will fail if you haven’t taken that step.
5. You may need to go into storage manager to tell it how to clear off space before you can enable motion capture. The control panel for motion capture does not seem consistent.
6. Even after all that, the motion capture details may not save correctly. At least the firmware version I downloaded, it just wouldn’t save the settings consistently. There is a workaround! You have to ssh in to the camera, and set the values:
   Enable motion detection
   jct /etc/prudynt.json set motion.enabled true
jct /etc/prudynt.json set motion.send2storage true
   Configure the storage destination
   jct /etc/send2.json set storage.mount /mnt/mmcblk0p1
jct /etc/send2.json set storage.send_video true
   Set motion sensitivity, range is 1-8, default is 1. Higher = more sensitive
   jct /etc/prudynt.json set motion.sensitivity 5
   Configure clip length (in seconds)
   jct /etc/prudynt.json set motion.video_length 15
   Restart prudynt
   /etc/init.d/S31prudynt restart
7. If you want to restore the original WyzeCam firmware, the instructions say to copy the original config files that get backed up by the Thingino install into the root directory of the SD card, name the file autoupgrade-full.bin, and then power it up. Oh, did you have that SD card formatted in exFAT? Ha ha, the firmware replacement process requires the SD card to be formatted FAT32 with Master Boot Record (MBR). The failure mode for this is somewhat opaque — it does seem to do something, and then in my case it announced it was ready for setup… in Italian. It was not ready for setup, though, as it hadn’t replaced the firmware.

Other Problems:

1. There’s no way to have the device do its own time management without a network (NTP). You can set the clock (ssh in, use the “date” command in BusyBox) but it doesn’t survive power-cycles.
2. I set up an isolated 2.4GHz guest network for the cameras that only allows DNS and NTP traffic to/from the internet. Then I went to reconfigure the cameras to use that network. From the admin panel, when I changed the network settings and turned off the “create AP,” it never seemed to connect. I had to reflash to the original firmware and then reinstall Thingino.

Other notes:

1. If you set up the Thingino camera with an AP network, it defaults to the IP address 100.64.1.1

We were in the middle of watching a streaming TV show / SAG Award nominee, when the spinny wheel started up and the “TV” computer popped up a “network disconnected” message. I figured our new Frontier Fiber had disconnected, but the lights all looked good. Next to it, though, the Netgate firewall appliance had an angry blinking red LED. I plugged in a serial cable. Not good!

Uncorrectable I/O failure? Hm. This turns out to be an actual physical hardware error.

I’d bought this appliance before the start of the pandemic, so it was about six and a half years old. Shortly after I bought it, there was all sorts of drama with the company behind pfSense (not linking, but you can search) which made me wonder how trustworthy the whole thing was. And just a few months ago, when I ditched the cable modem for a fiber connection, the device had corrupted its entire configuration during a routine software update, and stupid-stupid-stupid … I hadn’t kept a recent backup of the config.xml. So I’d just been through the whole reconfigure process, which requires a hidden trick for VLANs on that specific device that’s not required on other hardware configurations, and which kept me befuddled for many hours more than I’m willing to admit.

The storage RAM on the SG-1100 is soldered on, and it’s beyond my ambition to try to replace it. Instead, I ordered a new piece of hardware.

On this new device, I figured I’d try the OpnSense fork of the firewall software on the assumption that I’d be mostly familiar with it. Well, that’s partly true. The web-based configuration is reorganized. Some things require fewer steps, but they weren’t immediately obvious to me. Dammit Jim, I’m a code monkey, not a network engineer.

Anyway, I spent today restoring my network. It’s not all that complicated. I have a LAN, an isolated guest network, an isolated “internet of things” network, and VPN access from outside, all with special sets of rules to allow or prevent access, depending.

It was pretty smooth. A few gotchas that tripped me up whether reasonable or not:

• ddclient. I use this for pushing my dynamic IP address to the primary DNS provider for a domain, so I can have name-based access to the home network. It kept failing. Eventually, I had to log in to the firewall, and manually edit the configuration file to place quotes around my password. There’s probably some vulnerability in there if it’s passing unescaped strings to the command line — although you already need credentials to get to the interface that would allow it.
• Firewall rules in OpnSense have a “direction” which I don’t remember in pfSense. So when I want to firewall off connections from the Guest network to the LAN, for example, I have to put an “inbound” rule on the Guest network blocking the traffic. “inbound” means that the traffic is blocked when coming to the firewall (I had foolishly thought it was “outbound” to the LAN network). Since rules are specified on interface, network, and can specify both source and destination, I’m not sure why direction is required.
• I created and deleted a VPN instance early in the process. There’s still a tab in the firewall rules area, and automatically generated firewall rules for it, even though the instance has been deleted and has no interfaces. I don’t think it’s a big deal, but it’s confusing.
• So. many. DHCP. options. There’s the default “Dnsmasq DNS & DHCP”, there’s also “ISC DHCPv4” and “ISC DHCPv6”, and “Kea DHCP.” For my little network, I could do everything in the default, just creating separate DHCP ranges for each interface.

Setup wasn’t so bad. I’ll doubtless gripe here if things don’t work the way I want them to.

Fifteen years ago, I worked on a blog that was hosted via Blogger.com (aka blogspot.com aka Google). We had a custom domain name for the blog and everything. It was pretty cool.

Now, many years later, the domain name is finally set to expire. We haven’t touched that blog in eleven years, but it still seems a shame for the content to just vanish. So I thought about making a static copy to host somewhere.

Google makes cloning one of these blogs difficult. They do, however, give you a backup/download capability. I went through re-activating the Google account that was tied to the blog, giving all sorts of identifying information and getting verification emails and texts. That done, I initiated the process to backup the blog, and shortly thereafter received an email that my download was ready. However, now Google is absolutely certain I’m not who I say I am (even with verification emails and texts), and their security locked me out of the account. Also, when I read up on the subject, even if I could download it, their site backup is an XML bundle that only works for reimporting to their blog system anyway.

So I thought I’d use the good old standby wget to build a static copy. I tried:

wget --mirror -w 2 -p --html-extension --convert-links --restrict-file-names=windows http://www.myurl.com

Yes, this site was so old that we didn’t use SSL… Still, Google stores the assets off in a bunch of other subdomains, and I was unable to come up with the correct syntax to allow wget to follow those. I’d get the pages, but everything still linked to the Google servers for the assets. That wasn’t going to work.

So next I used the old, powerful F/OSS friend, httrack. My first attack was as follows:

httrack "http://www.myurl.com/" \
  -O "myurl-offline" \
  -%v \
  --robots=0 \
  "+www.myurl.com/*" \
  "+*.blogspot.com/*" \
  "+*.bp.blogspot.com/*" \
  "+*.googleusercontent.com/*" \
  "+*.jpg +*.jpeg +*.png +*.gif +*.webp" \
  "+*.css +*.js" \
  "+*.mp4 +*.webm" \
  "-*/search?updated-max=*"

This worked — but a little too well. This blog was part of a community of sites, many of which were hosted elsewhere on blogspot. The cloning was slow. Then I noticed it had used up 2G of disk space, whereupon I discovered that I was happily making static copies of twelve other blogs from that community, and possibly more to come! I interrupted the process, and tried again removing the blank check for blogspot sites:

httrack "http://www.myurlcom/" \
    -O "myurl-offline" \
    -%v \
    --robots=0 \
    "+www.myurl.com/*" \
    "+*.bp.blogspot.com/*" \
    "+*.googleusercontent.com/*" \
    "+*.jpg +*.jpeg +*.png +*.gif +*.webp" \
    "+*.css +*.js" \
    "+*.mp4 +*.webm" \
    "-*/search?updated-max=*"

This was successful!

I now have a static version of the site. It’s not perfect; some references like the user profile links still point at blogspot. But if I want to be able to post the static site somewhere, I can do that, and it will be sufficiently usable that people can still experience the postings and articles.