I’d upgraded by home machine to Mavericks fairly soon after the OS was released, but hesitated in upgrading my main work machine. I didn’t want to have extensive downtime while tracking down odd dependencies and incompatibilities.

Well, time came to upgrade. It seemed safe. Everything was fine on my home machine. So I went ahead and upgraded my work machine.

Suddenly, I couldn’t get MacPorts to build MySQL.

I had followed the migration guide carefully. I tried all the usual tricks. In the port /opt/local/var/macports/build/…/config.log, the error was:

ld: library not found for -lcrt1.10.6.o

Google seemed to think this indicated that my Xcode command-line tools were not installed correctly. That library should be installed with all of the Unixy support that comes with Xcode’s command-line tools. Within the Xcode application, it told me that I had the command-line tools 5.1.1 (5B1008) properly installed. When trying various command-line options, the command-line tools were, in fact, installed. For example, xcrun gave the exact results one would expect.

Other tests also made it look like everything was good:
# xcode-select -p
/Applications/Xcode.app/Contents/Developer


Numerous Googled sites said to use the “xcode-select” command to install the tools if they were not functioning properly. Eventually, I gave in and tried it. Interestingly, this yielded an unexpected result:
# xcode-select --install
xcode-select: error: no developer tools were found, and no install
could be requested (perhaps no UI is present), please install
manually from 'developer.apple.com'.


Since I had originally installed Xcode under Leopard from a downloaded package rather than recently through the App store, I thought perhaps in the various upgrades something had gotten messed up. I decided to completely uninstalled Xcode:

/Developer/Library/uninstall-devtools --mode=all
I also removed the vestigial /Developer directory, and the /Application/Xcode directory.

I re-installed Xcode from the App store. Everything functioned and/or failed exactly as it had before.

In desperation, I downloaded the Mavericks command line tools package from Apple, and installed it. It should be the same thing as what was installed with Xcode. But it evidently is not, because now I can build the MacPort for MySQL.

edit/update: It may not be clear above, but normally doing the “xcode-select –install” is all that’s needed. It’s also not stated above, but I tried that after re-installing Xcode from scratch, and had the same issue. Evidently, whatever was mis-configured on my machine is quite rare.

Also, don’t trust Xcode when it tells you that it’s installed the command-line tools in the preferences panel like this:
It’s probably lying to you – it’s installed stubs, but not the actual tools.

For a project that communicates over an expensive network connection (i.e., one that charges by the kilobyte), I needed to find out exactly how many bytes a specific process was going to transfer between my source host and a destination machine. For my own nefarious purposes, I need to know how many bytes of payload data I’m sending/receiving, but I also need to know the true total data transfer, including TCP/IP headers, etc.

Over the years, I have accumulated a few tricks for measuring this sort of thing. Usually, though, I’ve had to measure one particular kind of traffic (specifically, HTTP) — in which case, it’s not hard to set up a proxy using nc. In this latest case, however, the process not only uses HTTP/HTTPS, but ssh to issue remote commands, so I need to monitor all TCP/IP traffic between the machines.

There are other tools that are sometimes helpful. For example, to see what’s using up bandwidth at a given moment, a tool like iftop is great. Unfortunately, I need to know the aggregates, and iftop doesn’t log to a file in a way that I can use.

If I were on a pure Linux environment, it looks like IPTraf would do what I want, but I’m using a Mac.

I don’t doubt that there are much better approaches out there¹, but here’s what I used (pretending that the remote host was at IP 192.168.1.100):

sudo tcpdump -e host 192.168.1.100 > net_process_log.txt
perl -p -i.bak -e 's/(.*?)length (\d+):(.*)length (\d+)/$2,$4/g' net_process_log.txt
cut -d , -f 1 net_process_log.txt > actual_size.txt
cut -d , -f 2 net_process_log.txt > data_size.txt
awk '{s+=$1} END {print s}' actual_size.txt
138099
awk '{s+=$1} END {print s}' data_size.txt
102412
So, in my example, I’m using tcpdump to output all traffic between my machine and the host 192.168.1.100. Typical records output from tcpdump looks like:

11:13:23.834080 xx:xx:xx:xx:xx:xx (oui Unknown) > xx:xx:xx:xx:xx:xx (oui Unknown), ethertype IPv4 (0x0800), length 292: dvr.home.http > apotheosis.home.64602: Flags [P.], seq 1:227, ack 468, win 1354, options [nop,nop,TS val 132300159 ecr 1750033024], length 226
11:13:23.834081 xx:xx:xx:xx:xx:xx (oui Unknown) > xx:xx:xx:xx:xx:xx (oui Unknown), ethertype IPv4 (0x0800), length 66: dvr.home.http > apotheosis.home.64602: Flags [.], ack 469, win 1354, options [nop,nop,TS val 132300159 ecr 1750033026], length 0

There are two lengths specified: the first is the actual packet size, and the second is the payload of the packet. As you can see in the second packet, the payload is zero bytes, but the packet length is 66 bytes.

In any case, I use perl to extract the two lengths into a comma-delimited file, cut to split out the columns, and awk to add them up. It’d be trivial to do all these steps together in a short perl program, but I like keeping around tons of obscure text files from forgotten procedures on my hard drive. Well, actually, I did it this way so I could sanity-check the intermediate steps.

So, this taught me that my process transferred 102,412 bytes of payload, and with the TCP/IP packet overhead transmitted a total of 138,099 bytes.

¹ I didn’t discover bwm-ng until after I did my measurement. It looks like it might be a good solution as well. I probably could have used Wireshark too.

Say you’re developing on a Mac, and want to test some PHP code that has calls to the direct IO library. You may not actually have a physical serial port, but your unit tests will fail in the wrong way if the library isn’t present. You want the unit tests to fail in the right way!

If you do the expected thing, you’ll find that dio is currently beta:

root# pecl install dio
Failed to download pecl/dio within preferred state "stable", latest release is version 0.0.5, stability "beta", use "channel://pecl.php.net/dio-0.0.5" to install
install failed


So you have to try to do it the hard way:

samuel:~ root# pecl install channel://pecl.php.net/dio-0.0.5
downloading dio-0.0.5.tgz ...


Easy-peasy, eh? Not so fast!


/private/tmp/pear/temp/dio/dio.c: In function 'zif_dio_fdopen':
/private/tmp/pear/temp/dio/dio.c:138: error: 'EBADFD' undeclared (first use in this function)
/private/tmp/pear/temp/dio/dio.c:138: error: (Each undeclared identifier is reported only once
/private/tmp/pear/temp/dio/dio.c:138: error: for each function it appears in.)
make: *** [dio.lo] Error 1
ERROR: `make' failed


Fortunately, via this bug report, we can see what to do:

# pecl download channel://pecl.php.net/dio-0.0.5
# tar xzvf dio-0.0.5.tgz
# cd dio-0.0.5
# phpize
# ./configure


Then edit dio.c, and change line 138 to:

if ((fcntl(fd, F_GETFL, 0) == -1) && (errno == EBADF)) {


Then, finish up:

# make
# make install


Then, create a file called “dio.ini” in /opt/local/var/db/php5/ containing:

extension=dio.so


Now you can run your tests!

Coder #1: Man, revision control sucks.
Coder #2: What? Are you insane?
Coder #1: Insane? I don’t think so.
Coder #2: How can you say revision control sucks, then?
Coder #1: Well, I’m stuck using SVN.
Coder #2: Subversion isn’t so bad. Especially with the new merge tracking stuff from version 1.5. It’s pretty easy to use.
Coder #1: Yeah, well, I stand by my assertion. It sucks.
Coder #2: Can you be more specific?
Coder #1: Have you tried using SVN in a mixed environment? Say Windows, Linux, and Mac OS?
Coder #2: It’s not going to be line-ending issues. Oh, I see. You’re talking about case-sensitive versus case-insensitive filesystems.
Coder #1: I wish it was that easy. That’s crappy, but you can work around it.
Coder #2: Then?
Coder #1: A riddle for you. When is UTF-8 not UTF-8?
Coder #2: Huh?
Coder #1: When it’s in a filename.
Coder #2: [clicks, reads]
Coder #1: But I guess if you don’t have any Macs or ZFS around, you’re fine.
Coder #2: Holy Linus on a unicycle! Well, I guess it’s back to dated tar files…
Coder #3: Use GIT, you IDIOT!

I’ve played with the Javascript interface to Photoshop for a couple of years. Conceptually, it’s great — a simple, powerful, interpreted language like Javascript, with an API to interface to one of the best image-processing packages available. In practice, it’s not as good as it is in concept, but it’s still pretty good. The API doesn’t include all of Photoshop’s functionality directly, and there are a lot of things you need to execute as fairly obscure event actions. These event actions aren’t documented, but can be determined by activating a plug-in which logs everything that you do using the Photoshop GUI — you can then read through these logs, and copy the actions you need.

Still, there are some real advantages to using this Javascript interface, as opposed to something designed for the purpose like, say, Processing. You can use the Photoshop UI for controlling inputs to your script (set foreground colors, select portions of the image, select specific layers, etc.), and output your manipulations directly into Photoshop layers.

I’ll be posting here shortly a library I’ve created for easily building dialog panels for setting script options. I find that most manipulations I want to do have a set of variables, and I’d rather not tweak the code each time I want to change them.

This library was originally written under Photoshop 10 (aka CS3). Under version 11 (aka CS4), it was less stable. Sometimes it would crash out at odd places complaining that I was referencing properties of undefined objects. Because there have been memory leaks and other issues with the Javascript interpreter, these seemingly random failures were annoying but not too surprising. When it came to version 12 (aka CS5), I was rarely able to run my scripts at all. What made it frustrating was the apparent randomness of the crashes. I could print a variable to the console, and the very next line would crash out with an “undefined object” error when referencing that variable.
To make a long story short, I was able to track down the issue. It turns out that in iterations, declaring variables matters. That is to say:
for (i in someCollection)
{
$.writeln(i['someAttribute']);
}

will cause random crashes, but
for (var i in someCollection)
{
$.writeln(i['someAttribute']);
}

runs beautifully. Now, I “knew” that the var keyword is optional and used for specifying scope, but I never had any idea that there could be an issue within the scope of a simple loop. Obviously, Javascript didn’t know that I intended i to be a variable on each iteration — perhaps it thinks I meant for i to be a 1957 Chevy Belair on some iterations.

In any case, having cracked the code as it were, I have proceeded to enhance and add to my library. After a little more testing, I’ll be posting it here or on GitHub.