Fri, 21 Mar 2008

Interesting Image Problem

— SjG @ 3:33 pm

So we had a jpeg image from someone, and were distributing it through a web-based system (note that all non-technical details in this whole posting will be presented in annoying vague language). The web-based system is PHP and uses GD-lib. GD-lib successfully thumbnails the images, but when the images are downloaded, both Firefox and IE7 complain that the image has errors:

The image file "" cannot be displayed, because it contains errors.

Windows image browser shows the image successfully, and Photoshop happily opens it. Looking at the file itself, I can see that it *is* a JFIF file (e.g., a valid jpeg). It starts with the FFD8 header, etc. It does have some strange characters in the IPTC data. This turns out to be a red herring, however. The problem turns out to be that it’s a jpeg image, but it’s using an 8-bit CMYK color space, which isn’t supported by Firefox, IE6, or IE7.

Firefox/Mozilla will be supporting CMYK jpegs in the future. Opera already does. I’m not sure if IE8 will.

Later, I found a blog entry on this very topic, that, strangely I didn’t find when I Googled the error message. But there is good information out there. In fact, if you know to include the term “CMYK” you get tons of useful responses.

Thu, 3 Jan 2008

Backups, Updated Again

— SjG @ 5:11 pm

Had some updates to the backup script which I never published.

Here it is.


Backup Scripts

(for background, see Automated Backups.)

Wed, 28 Nov 2007

Aperture Import, Continued.

— SjG @ 1:36 pm

I’ve done some more work on the Aperture Importer (background here), and the latest is attached below. It now does some reformatting of keywords that get split (e.g., “San” and “Diego” can be merged to “San Diego” as a keyword.) It’s hacky and ugly. You’ll have to set up your own keywords for this kind of merge.

I’ve found a couple of apparent Aperture bugs.

If I tell Aperture to import an empty directory from Applescript, it’ll stall and lock up Aperture.

Worse, I find that if I do a large import (more than, say, 5,000 images), Aperture grabs a bunch of memory that never gets released. Well, Aperture itself doesn’t grab the memory, but it causes the kernel_task “process” to allocate a big pile of real memory, which it seems to hold on to until reboot.

It’s a cumulative thing: if I import 5,000 images, the memory gets grabbed. Then, if I do another 5,000 image import, the memory usage doubles. Thinking it would be handled by swapping, I didn’t worry, and continued. This was a bad idea. Aperture locked up, but so did the whole OS. The last thing I could see from top was that 100% of my real memory was allocated, that less than 256M of swap was in use. I had at least 50GB of disk free, so that wasn’t the problem.

Anyway, for safety, if you use this import script, I recommend rebooting between import sessions. Yeah, it’s voodoo, but it’s guaranteed to work.

Aperture Importer Update

Linux Security Camera Server on a Dell Vostro 400

— SjG @ 11:17 am

So I’ve been building a new security camera system. The last time I did this, I bought a Dell dual-core box, and spent about a week installing Debian, and building and rebuilding the kernel to support the dual cores, to support the BT878 video capture chipset, compile and configure motion, etc. It took a week of evenings and a weekend or two, because the Dell hardware wasn’t automatically supported, and it required special boot-time parameters to recognize the SATA controller, for example.

So I’m building a new system on a Dell Vostro 400. Right off the bat, I ran into problems with installing the Debian net-install. I was booting off a CD, but the installer couldn’t find an ATA/IDE controller for which it had a driver. Weird. This article showed me the solution to that — set the Dell controller’s SATA mode to “RAID.”

But then I hit a wall with the Intel Gigabit network controller. I couldn’t find any workarounds for it, but, after extensive Googling, found that some of the Ubuntu people may have a patch. The posting was six months old.

So screw it, thought I, and downloaded a shiny new ISO of Ubuntu 7.10 Server Edition to see whether it would work.

Damn, am I impressed! Not only did it recognize the ATA/IDE controller and the network controller, it happily recognized the BT878-based card and loaded the kernel modules. It even has motion installed as a package for easy installation. I was able to copy over all my support scripts and motion configuration files, and was up an running in less than two hours (and that includes setting up the web server, motd, special sshd tweaks, and all)!

Now, all I have to do is deal with my crisis of faith. Do I leave the Church of Debian for the radical new Ubuntuist movement?

Sat, 17 Nov 2007

Aperture Import Script

— SjG @ 7:14 pm

So, after the demise/µsoftification of iView Media Pro, the time came to switch to Aperture.

However, while Aperture is mighty powerful, its limitation of 10,000 images in a project makes import of my photos difficult. What’s more, my … er … unique system of “organization” doesn’t natively work well with Aperture. My attempt at organization, which predates such things as iView, Aperture, or even a usable Bridge, is predicated on the idea of the filesystem as a hierarchical database of sorts.

For example, I start with a directory called “photos” and within it are directories for “animals,” “events,” “people,” “places,” “things,” “projects,” etc. Within “events” are “political,” “work,” “family,” etc. With each of these are either further taxonomic directories, or what might be equivalent to rolls-of-film directories, e.g., “MothersDay-2007-05-13” or “CocktailsAtBerris-2000-06-14.” Directories are all Unix-friendly (no spaces or crazy punctuation) and are generally CamelCase for multiple words.

So I went through a frustrating attempt to write a good Applescript importer. The problem with languages like Applescript (or Javascript implementations embedded in Adobe products) is that they promise more than they can deliver. They’re designed to interact with Applications, but generally don’t have rich access to application functionality. Why can’t I create a folder in my Aperture library in Applescript? Why can’t I get/set a single pixel in Photoshop with Javascript. Yes, I know both are possible through some crazy GUI-script calls or cryptic Event IDs, but why give me the equivalent of object-oriented access and then leave out all the important methods?

Well, enough ranting. With a lot of help from others who have gone before me and posted comments and, even better, code, I hacked together something that will read in my hierarchy, create a new Aperture Project for each leaf node on my tree, and convert the path to that node into a set of keywords which it will apply.

So “photos/events/family/MothersDay-2007-05-13” will become a project named “MothersDay-2007-05-13” and the images within it will all be tagged with the keywords: Events, Family, Mothers, Day, 2007-05-13. It’l also throw in copyright notices and author name.

There’re provisions for excluding words from becoming tags (e.g., “and”) as well as special case code for directories named “misc,” which I often use as catchalls for a taxonomical branch — these get named for the parent directory plus the “misc” (e.g. “ArthopodsMisc”).

Perfect? No. Better than doing it manually? Yes.

In any case, here it is:
Aperture Importer