Wed, 12 Jun 2013

Smart fixtures for testing

— SjG @ 1:28 pm

Because we’re not completely insane, we run automated testing of the web sites and web-based applications we develop. Because we are busy, we probably don’t do as much testing as we’d like. While there’s always room for improvement, though, having both unit tests and functional tests is a huge, huge win.

For enterprise intranet sites, there are a lot of things which are time-based. There are documents that get published or expired on given dates, users who receive notifications on a schedule, financial tables that depend on the fiscal year, and so on. If you test these functions (and, of course, you do), you may discover your automated tests suddenly fail on the first day of the quarter or some other threshold. This will likely be due to hard-coded dates in your test fixtures (and surely not because of boundary-condition bugs in your code).

There is an easy solution. Where it’s appropriate, you can make your fixtures “smart” by using adaptive logic. Messy, beautiful functions like PHP’s strtotime make this easy.

For example, here’s a snippet from a fixture for a Yii-based project. It’s for a message data table, and I want to be able to guarantee that there’s at least one valid message and one expired message no matter when I’m testing:

"Content"=>"There will be an important announcement on ".date('m/d/Y',strtotime('first day of next month')),
"PublicationDate"=>date('Y-m-d H:i:s',strtotime('midnight first day of last month')),
"ExpirationDate"=>date('Y-m-d H:i:s',strtotime('midnight first day of next month -1 second')),
"Subject"=>"Monthly Results",
"Content"=>"This message is never current",
"PublicationDate"=>date('Y-m-d H:i:s',strtotime('midnight first day of last month -2 month')),
"ExpirationDate"=>date('Y-m-d H:i:s',strtotime('midnight first day of this month -1 day')),

With a fixture like that, I can make safe assumptions about time-related displays no matter what the current date is.

Fri, 24 May 2013

Javascript approximation for Pi

— SjG @ 11:01 am

Based on this tweet, I now have the ultimate JavaScript approximation for Pi (Π), which I think we can all agree is preferable in every way to the outmoded Math.PI:

var pi=((++[+[]][+[]]+[]+ ++[+[]][+[]]+[])* ++[+[]][+[]])*(++[+[]][+[]]+ ++[+[]][+[]])/((+[+[]]+'x'+(![]+[])[[+!+[]+!+[]]*[+!+[]+!+[]]])/(++[+[]][+[]]+ ++[+[]][+[]]));

Sun, 21 Apr 2013

Measuring network traffic between two hosts

— SjG @ 10:28 am

For a project that communicates over an expensive network connection (i.e., one that charges by the kilobyte), I needed to find out exactly how many bytes a specific process was going to transfer between my source host and a destination machine. For my own nefarious purposes, I need to know how many bytes of payload data I’m sending/receiving, but I also need to know the true total data transfer, including TCP/IP headers, etc.

Over the years, I have accumulated a few tricks for measuring this sort of thing. Usually, though, I’ve had to measure one particular kind of traffic (specifically, HTTP) — in which case, it’s not hard to set up a proxy using nc. In this latest case, however, the process not only uses HTTP/HTTPS, but ssh to issue remote commands, so I need to monitor all TCP/IP traffic between the machines.

There are other tools that are sometimes helpful. For example, to see what’s using up bandwidth at a given moment, a tool like iftop is great. Unfortunately, I need to know the aggregates, and iftop doesn’t log to a file in a way that I can use.

If I were on a pure Linux environment, it looks like IPTraf would do what I want, but I’m using a Mac.

I don’t doubt that there are much better approaches out there1, but here’s what I used (pretending that the remote host was at IP

sudo tcpdump -e host > net_process_log.txt
perl -p -i.bak -e 's/(.*?)length (\d+):(.*)length (\d+)/$2,$4/g' net_process_log.txt
cut -d , -f 1 net_process_log.txt > actual_size.txt
cut -d , -f 2 net_process_log.txt > data_size.txt
awk '{s+=$1} END {print s}' actual_size.txt
awk '{s+=$1} END {print s}' data_size.txt

So, in my example, I’m using tcpdump to output all traffic between my machine and the host Typical records output from tcpdump looks like:

11:13:23.834080 xx:xx:xx:xx:xx:xx (oui Unknown) > xx:xx:xx:xx:xx:xx (oui Unknown), ethertype IPv4 (0x0800), length 292: dvr.home.http > apotheosis.home.64602: Flags [P.], seq 1:227, ack 468, win 1354, options [nop,nop,TS val 132300159 ecr 1750033024], length 226
11:13:23.834081 xx:xx:xx:xx:xx:xx (oui Unknown) > xx:xx:xx:xx:xx:xx (oui Unknown), ethertype IPv4 (0x0800), length 66: dvr.home.http > apotheosis.home.64602: Flags [.], ack 469, win 1354, options [nop,nop,TS val 132300159 ecr 1750033026], length 0

There are two lengths specified: the first is the actual packet size, and the second is the payload of the packet. As you can see in the second packet, the payload is zero bytes, but the packet length is 66 bytes.

In any case, I use perl to extract the two lengths into a comma-delimited file, cut to split out the columns, and awk to add them up. It’d be trivial to do all these steps together in a short perl program, but I like keeping around tons of obscure text files from forgotten procedures on my hard drive. Well, actually, I did it this way so I could sanity-check the intermediate steps.

So, this taught me that my process transferred 102,412 bytes of payload, and with the TCP/IP packet overhead transmitted a total of 138,099 bytes.

1 I didn’t discover bwm-ng until after I did my measurement. It looks like it might be a good solution as well. I probably could have used Wireshark too.

Thu, 11 Apr 2013


— SjG @ 9:34 am

That was close.


It almost made me think I’d need to take shelter under a refrigerator.

Filed in:

Mon, 8 Apr 2013

Why I need an extension on my taxes

— SjG @ 9:49 pm

… because I wanted to play with HTML 5 Canvas.

This should run on any reasonably sane browser. Play with it yourself here.