Sat, 1 Sep 2007

How do they know?

— SjG @ 9:29 pm

So, it’s Saturday night, 10pm on a three-day Holiday Weekend. And another important server decides to have a fatal disk error.

How do they f*ing know?

It’s uncanny.

This is just another in a long series of similar failures. The power supply failed in my security-system a few years ago — hours before we were going to leave for a vacation. The previous time the security system crashed (due to unfortunate automatic software update) was — yup — the afternoon Elizabeth and I were leaving for a weekend cruise. And Elizabeth’s Mac had its hard drive fail the day I left for Bulgaria.

Of course, we all know why this is. There have been half a dozen other failures that happened during more mundane times, which evoked the standard rage/repair response. But that was par for the course, and soon forgotten.

So here I am. Saturday night … fixing machines. I should have been a ditch digger or something.

Wed, 15 Aug 2007

Mysterious Ruby on Rails problem — and solution

— SjG @ 1:51 pm

So, I’m working on an old RoR application, and was having a hard time getting it to work on my newer development machine.

The error dump was not at all clear:

C:\project\foo_bar_baz_quux>rake migrate --trace
(in C:/project/foo_bar_baz_quux)
** Invoke migrate (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute migrate
rake aborted!
wrong number of arguments (1 for 0)
l.rb:551:in `initialize'
l.rb:551:in `new'
l.rb:551:in `scramble41'
l.rb:141:in `real_connect'
adapters/mysql_adapter.rb:316:in `connect'
adapters/mysql_adapter.rb:87:in `initialize'
adapters/mysql_adapter.rb:37:in `new'
adapters/mysql_adapter.rb:37:in `mysql_connection'
adapters/abstract/connection_specification.rb:145:in `send'
adapters/abstract/connection_specification.rb:145:in `connection_without_query_c
.rb:54:in `connection='
adapters/abstract/connection_specification.rb:106:in `retrieve_connection'
adapters/abstract/connection_specification.rb:20:in `connection'
b:178:in `migrate'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:399:in `call'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:399:in `execute'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:399:in `each'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:399:in `execute'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:369:in `invoke'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:362:in `synchronize'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:362:in `invoke'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:1935:in `run'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:1935:in `each'
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake.rb:1935:in `run'
c:/ruby/bin/rake.bat:20:in `load'

I went through a lot of work to use older versions of Rails, etc, but none of it worked. I didn’t find much out
there that helped, either, until I finally happened across this page.

Turns out it’s one of those pernicious password / old style password hash issues with MySQL. Simplest solution for now: use no database password on the dev box.

Mon, 9 Jul 2007

Preventing “Overlapping” cron Processes

— SjG @ 9:43 am

I have a number of very time consuming processes that get run by cron on various machines. Some of these processes would cause problems if they “overlapped” — e.g., a new one gets started before the old one is done.

Now, there are a lot of ways to make sure you’re unique if you’re a process, but often I don’t want to modify the source of the process to add that (for many packages, I’d rather not patch and merge and recompile every time a new version comes out). So I write a simple shell script to run the process; cron calls my shell script, and prevents the overlap.

This uses the magic of “pgrep” — unfortunately, different versions of pgrep have different flags, so the code I originally wrote (which used the “-c” flag, which counts the matching processes) didn’t port to most systems. It’s easy enough to pipe the output through a “wc -l”.

I did have to move the pgrep exec out of my if statement, though, since the comparison was going against the return code, not the output. Doh!


RUNNING_PROCS = `pgrep -f longRunningProcess | wc -l`
if [ "$RUNNING_PROCS" -gt "0" ]
        echo `date` longRunningProcess still running. I\'ll let it finish. 
        echo `date` Starting longRunningProcess.
        /path/to/longRunningProcess -flags 
echo "----------------------------"

Filed in:

Wed, 27 Jun 2007

Unix: How to find files lacking certain strings

— SjG @ 4:10 pm

So, I’m working on a convoluted web site, and a problem comes up. It seems that some vitally important code was not included in some pages (for the sake of argument, let’s say it’s a copyright string). This particular site has an ungodly mix of files, including .htm, .html, and .jsp files. Some of the .jsp files are actual pages, and others are stubs to be included in other .jsp pages. The majority of the full .jsp pages include a “footer.jsp” that has the desired string, so they’re good. But I need to generate a list of the full pages, of whatever sort, that lack this string.

The inverse of this problem is easy, and is the kind of thing I use all the time:
find . -name \*.htm -o -name \*.html -o -name \*.jsp -exec grep -il "myString" {} \;

Initially, I thought using the -v flag to grep would work for me, but grep -vl returns all files it sees, because -v returns the lines that match the invert expression, not the files that match the invert expression. Then there’s the problem that I need to match “full” pages rather than included .jsp stubs.

So here’s how the Mighty Power of Unix came to my rescue:

find . -name \*.htm -o -name \*.html -o -name \*.jsp | xargs grep -il "</html>" | sort -u > full_pages.txt

provides me with a list of pages that are not mere inclusions, if you accept my assumption that an inclusion won’t match the closing HTML tag.

Then I generate a list of full pages that contain the magic string and or include the footer.jsp that would contain the magic string:
find . -name \*.htm -o -name \*.html -o -name \*.jsp | xargs grep -il "</html>" | xargs grep -le "uniqueCopyrightTag\|footer\.jsp" | sort -u > pages_no_string.txt

Then I compare the files to find out which full pages lack both the magic string and the include:
comm -3 pages_no_string.txt full_pages.txt

Wow. There it is!

I bet there’s an easier way. Post an example in the comments if you know of one!

NOTE: All commands are on a single line, regardless of whether they wrap in this particular display.

Wed, 9 May 2007

Extracting Scripts from Javascript pages using Javascript

— SjG @ 1:56 pm

Here’s a weird one. There was the need to extract the contents of all Javascript <script> … </script> tags from an html page, using Javascript in an Ajax-y environment*. I tried using a similar regular expression to the one published by Matt Mecham, but found that IE threw an error. IE didn’t like the [^] construct.

So, since I knew that the pages that this would need to process would be standard strings with nothing odd in them, I substituted [^\0]. Works in Firefox and IE. I don’t know if it breaks under different encodings, though.

The other problems was conceptual — I didn’t remember that regex.exec() only gives you the first match in the resultant array (but gives you your submatches); I confused it with the behavior of string.match() which doesn’t give you your submatches. *sigh*

So the code looks like this:

var reg = new RegExp("<script[^>]*>([^\\0]*?)<\\/script>","ig");
while( (m2 = reg.exec(http.responseText))  != null )
        for( i = 1; i < m2.length; i++ )
           alert(i + '('+m2[i].length+')' + m2[i]);
           // do other stuff

(Please note that WordPress seems insistent on munging that code. Spacing, in particular, might be corrupted.)

(* note that use of the passive voice. To protect the innocent, we won’t say who/why it was needed.)