Notes and Replacing Proprietary Services, Part II
I’ve migrated my notes back to Obsidian, and have been using it pretty much exclusively for note-taking since I last posted in September.
There’s a core plugin for Obsidian called “Importer” that will import Apple Notes. However, there is a caveat: the notes have to be unlocked. So if you have a bunch of locked (encrypted) notes, you’ll need to unlock them before doing the import. Another caveat that I discovered today, a month after my switch, is the way the importer uses the first line of an Apple Note as a title. More on this below.
I wanted to be able to encrypt select notes, like I did with note locking in Apple Notes. There’s a community plugin for Obsidian called “MeldEncrypt” that lets me do that. The UI takes a little getting used to, but it seems to work pretty well.
Then there’s syncing. I want to access my notes on my phone, on my notebook, or wherever. I’d like to be able to add a new computer or device to this sharing at any time.
Initially, I set up the Obsidian vault (which is the directory structure on the filesystem where it saves the notes) in an iCloud directory. That way, on my iPhone or notebook computer, I was able to open the vault and let Apple handle the syncing. But this whole process is about moving away from proprietary systems. iCloud is a proprietary dependency and isn’t available on Linux. Obsidian has a subscription sync service that I could use, but again, I’d like to be independent of other providers and keep my stuff on “my” servers.
There’s a plugin called RemotelySave, which allows syncing notes to a wide variety of services. It’s currently free, and will be adding more sophisticated features for paid subscribers, but theoretically will continue to be free for basic sync to standard services. One of those standard services is WEBDAV.
Well, I have a VPS to host web stuff. I know how to set up Apache services. So I went in and configured a WEBDAV service on one of my web servers (not the one hosting this blog). I secured the HTTP connection using a LetsEncrypt certificate and Basic Authentication, and point it at a directory outside the web root (in this case /var/webdav). The Apache configuration looks like:
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName webdav.somedomain.com
Alias /webdav /var/webdav
<Directory /var/webdav>
Options None
AllowOverride None
Dav On
AuthType Basic
AuthName "WebDAV"
AuthUserFile /etc/httpd/webdav.passwd
Require valid-user
<LimitExcept GET HEAD OPTIONS>
Require valid-user
</LimitExcept>
</Directory>
ErrorLog /var/log/httpd/webdav_error.log
CustomLog /var/log/httpd/webdav_access.log combined
SSLCertificateFile /etc/letsencrypt/live/webdav.somedomain.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/webdav.somedomain.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
I had to create the directory /var/webdav, give the web account access to it, and use htpasswd to create a user/password file as /etc/httpd/webdav.passwd.
So then I set up RemotelySave, and watched the data show up on the server! Excellent! You can set RemotelySave to sync on various schedules, on or after Obsidian startup, and (experimentally) upon save. I opted to set it up to sync one second after startup and upon save.
Then I set up Obsidian on a Linux box that has no access to the iCloud directory. Would it sync? Boom! No, it would not. Going into the low-level logs, I could see the problem.
Remember above I said there were issues with the first line of notes and titles? That’s what tripped me up here. I had a big folder of notes that were quotations. Often, I’d paste those into Apple Notes and not do any further editing. The Obsidian Apple Notes importer sees the first line of each note as its title.
Also, the filename for each note created (or imported) by Obsidian is based on the title, and seems to be truncated at 60 characters. Mysteriously, though, some imported Notes had filenames that include the entire title. I can’t find the factor that determined which were truncated and which were not. Many of the un-truncated titles start with quotation marks, but also many notes that have titles starting with quotation marks truncated correctly. In any case, I had a number of files with names over 255 characters. Those crashed the plugin when I tried to sync to Linux.
Another thing I discovered at this point is that the importer truncates the first line of the note after 200 (or so) characters within the newly created note. Up ’til now, I hadn’t noticed that I’d lost significant content on some of these notes that were created when I pasted long quotations into Apple Notes.
Fortunately, even at this late time, I can remediate the problem by going into Apple Notes, changing the title of these notes to something shorter, and re-running the Obsidian Importer plugin for that folder. It cleverly uses the Apple Note opaque ID to decide whether to update or create a note. I then had to delete the long-named files manually on the WEBDAV server filesystem, and re-sync.
It worked! Now the files sync correctly to Linux as well.
Again, I’ll report back here if I have any new conclusions.
Oh! Important note! I moved the Obsidian Vault off of the iCloud drive too! I was worried about the two sync systems interfering.
Also of note. On the iOS side, I needed to create a new vault with the exact same name as the one I want to sync, and specify that it is not to be stored in iCloud. Then, configuring RemotelySave brought all my notes back to the iOS device.