Wed, 17 Sep 2014

PHP 5.5, PHPUnit/Selenium, Yii, and Mac Ports on Mavericks.

— SjG @ 4:07 pm

There is a special place in hell where every programming project creates its own package manager and you have to use it if you want to use the project. Oh, wait. That’s not a special place in hell. That’s here. Crap.

Some days, I shouldn’t bother getting out of bed. On those days, I have that weird broken cognition that I call “user brain” — I don’t read error messages, I don’t stop to think, and instead of looking at problems and trying to solve them, I randomly bounce around and break things.

So I had User Brain. I was setting up a new name-based SSL virtual host in my development environment, and dropped a line from the configuration. Then I didn’t read the Apache error message carefully, and foolishly thought “Oh, I must need to update my certificate trust chain.” This resulted in my updating my environment via mac ports — and when it asked me if I wanted to migrate from php5 to php5.4, I said OK.

When I finished that process, my development environment was completely b0rken, and I hadn’t solved the original problem. Fast forward an hour of questionable language, and I had a new strategy. Install PHP 5.5!

There are several guides out there on how to make all this stuff work, but they didn’t really work for me. What did, was the following. First, I logged in as root. If I’m going to make stupid mistakes, it’s best to do it with unlimited power. Anything else is unsporting.

# port uninstall --follow-dependents php54

That got rid of all the PHP 5.4 stuff left over from my previous debacle. So now it’s time to install PHP 5.5, and the various modules I need:

# port selfupdate
# port upgrade outdated
# port install php55 php55-gd php55-mbstring php55-iconv php55-curl php55-zip php55-soap php55-pcntl php55-xmlrpc php55-openssl php55-sockets php55-imagick php55-mcrypt php55-solr php55-mysql
# port select --set php php55
# port install php55-apache2handler
# cd /opt/local/apache2/modules
# /opt/local/apache2/bin/apxs -a -e -n php5

That’s all pretty straightforward. But my httpd.conf had a few stray lines in it that needed cleaning up. I had to replace LoadModule php5_module modules/
LoadModule php5_module modules/
and change
Include conf/extra/mod_php54.conf


Include conf/extra/mod_php55.conf

Now I could start up Apache and see what kinds of errors I had.

# /opt/local/apache2/bin/apachectl restart

Well, this is where I finally found and fixed the dropped line that got me started on this whole sordid adventure. I also discovered I didn’t have a good php.ini file, so I copied the distribution one and customized it.

# cp /opt/local/etc/php55/php.ini-development /opt/local/etc/php55/php.ini
# vi !$
# /opt/local/apache2/bin/apachectl restart

Now my dev sites were back up, but I couldn’t run tests. The way that phpunit is structured has changed since the days when it was a Pear package. It now uses a phar package and composer. Now, at some point, I’ll restructure my Yii projects the New Right Way and have a phpunit distribution per project, but I need to get back productive, so I set it up globally for the time being.

# cd /opt/local/bin
# curl -sS | php
# mv composer.phar composer
# wget
# chmod +x phpunit.phar
# mv phpunit.phar phpunit
# composer global require 'phpunit/phpunit-selenium=*'
# composer global require 'phpunit/phpunit-story=*'
# composer global require 'phpunit/dbunit=*'
# composer global require 'phpunit/php-invoker=*'

This is where I discovered the php.ini had set the default include path to a not-quite-right location. I ended up changing



and then once again restarted the environment.
# /opt/local/apache2/bin/apachectl restart

Now there were only two things wrong. The current project is based on Yii 1.1.x, and it still tries to include individual PHPUnit files.

Based on numerous forum posts and false starts, I edited framework/test/CtestCase.php, and changed those “requires” up top to:

if (!method_exists('PHPUnit_Runner_Version', 'id') ||
version_compare(PHPUnit_Runner_Version::id(), '4', '< =') ) { require_once('PHPUnit/Runner/Version.php'); require_once('PHPUnit/Autoload.php'); }

PHPUnit unit tests now run, as do Selenium tests, but they do throw an ugly warning on test failures to tell me:

Warning: include(): Failed opening 'PHPUnit_Extensions_Story_TestCase.php' for inclusion (include_path='.:/Users/samuelg/project/X/Y/protected/commands:/Users/samuelg/project/X/Y/protected/behaviors:/Users/samuelg/project/X/Y/protected/components:/Users/samuelg/project/X/Y/protected/nuclear_secrets:/Users/samuelg/project/X/Y/protected/models:/opt/local/lib/php') in /Users/samuelg/project/X/Y/framework/YiiBase.php on line 418

Nor does it give me line numbers of failures in Selenium tests if I enable screenshots. It’s also seemingly taking screen shots of the wrong pages of the site when there’s an error. I don’t yet have a solution to any of these problems.

Lastly, PHP 5.5 is more sensitive about some things than I expected. First, I had to migrate some crappy developer support code from using direct mysql commands (e.g., mysql_connect, mysql_query, etc.) as they’re deprecated. I now use PDO, and that’s all fine.

But beware PHP being stupid about what’s a variable and what’s a reference. In PHP 5.5 strict mode, it whimpers about things like

$foo = end(my_func());

even if my_func() returns an array. Now, in a sane language, I’d be able to do stuff like that, because hey! an array’s an array, even if it’s being returned by a function! I mean, not to devolve into pointless discussion of language implementations and flaws, but flaming owls of imminent death, is PHP a mess!

Anyway, this post is a mostly cleaned-up version of my little adventure. If it spares anyone else some of the same pain, it will be worth while.

No, the project doesn’t really have a nuclear secrets directory. I just felt stupid nerfing the project name in the first place.

Tue, 22 Apr 2014

Annoying Xcode issue and resolution

— SjG @ 3:08 pm

I’d upgraded by home machine to Mavericks fairly soon after the OS was released, but hesitated in upgrading my main work machine. I didn’t want to have extensive downtime while tracking down odd dependencies and incompatibilities.

Well, time came to upgrade. It seemed safe. Everything was fine on my home machine. So I went ahead and upgraded my work machine.

Suddenly, I couldn’t get MacPorts to build MySQL.

I had followed the migration guide carefully. I tried all the usual tricks. In the port /opt/local/var/macports/build/…/config.log, the error was:

ld: library not found for -lcrt1.10.6.o

Google seemed to think this indicated that my Xcode command-line tools were not installed correctly. That library should be installed with all of the Unixy support that comes with Xcode’s command-line tools. Within the Xcode application, it told me that I had the command-line tools 5.1.1 (5B1008) properly installed. When trying various command-line options, the command-line tools were, in fact, installed. For example, xcrun gave the exact results one would expect.

Other tests also made it look like everything was good:
# xcode-select -p

Numerous Googled sites said to use the “xcode-select” command to install the tools if they were not functioning properly. Eventually, I gave in and tried it. Interestingly, this yielded an unexpected result:
# xcode-select --install
xcode-select: error: no developer tools were found, and no install
could be requested (perhaps no UI is present), please install
manually from ''.

Since I had originally installed Xcode under Leopard from a downloaded package rather than recently through the App store, I thought perhaps in the various upgrades something had gotten messed up. I decided to completely uninstalled Xcode:

/Developer/Library/uninstall-devtools --mode=all
I also removed the vestigial /Developer directory, and the /Application/Xcode directory.

I re-installed Xcode from the App store. Everything functioned and/or failed exactly as it had before.

In desperation, I downloaded the Mavericks command line tools package from Apple, and installed it. It should be the same thing as what was installed with Xcode. But it evidently is not, because now I can build the MacPort for MySQL.

edit/update: It may not be clear above, but normally doing the “xcode-select –install” is all that’s needed. It’s also not stated above, but I tried that after re-installing Xcode from scratch, and had the same issue. Evidently, whatever was mis-configured on my machine is quite rare.

Also, don’t trust Xcode when it tells you that it’s installed the command-line tools in the preferences panel like this: Locations_and_Macintosh_HD
It’s probably lying to you – it’s installed stubs, but not the actual tools.

Fri, 10 Jan 2014

Page loading twice … but why?

— SjG @ 11:15 am

Oh, what a journey of discovery, and what a diet of red herring.

I’m developing some pages in Yii, and noticed that — in Firefox — pages containing CGridViews were actually loading twice. The pages seemed to flicker on load, and in my server logs, I’d see two requests. I went down a lot of dark and twisty alleys trying to figure out what was wrong with my Ajax calls, to no avail. Turning off Javascript did solve the problem, so, of course, I was focused on finding a Javascript bug.

When searching for others experiencing the same problem, I found this posting. “Curious,” thought I.

I tried re-arranging the HTML layout, and damn me twice if putting the character encoding declaration earlier in the file doesn’t fix the problem.

The CGridView red herring comes from more Javascript includes being injected in the header on pages that use CGridViews. So that makes sense.

I’m still puzzled why disabling Javascript in the browser makes any difference. It does not change the location of the character encoding declaration in the file. Maybe it’s a Firefox- or Firebug- specific “feature.”

I have to admit being surprised that the spec dictates the encoding be declared in the first 1024 bytes (or 512 bytes before the 2011 version of the HTML 5 spec). I’m even more surprised that a browser would actually re-submit the request to load the page in the case where the page was out of spec. Redraw? Sure! Reload? That’s just crazy.

Wed, 18 Dec 2013

Angular.js and IE8 Problem

— SjG @ 10:00 pm

I was just finishing up a Angular.js application. It was then time to test it under Internet Explorer 8 — not for purely masochistic reasons, mind you, but because it’s still the standard browser for some people, including the people for whom this particular app was written.

I’d already tested under Chrome, Firefox, Safari, and IE10, so I wasn’t expecting any problems. There you go. I’ve put it in writing. I’m an idiot.

So IE8 didn’t render the app at all. No error messages. I switched to IE9 to test, and it worked there, unless I put it into “IE8 Standards Mode” when it would silently fail again.

Now, this may not be news to some people, but IE8 really is pretty crappy. I spent several hours tracing through things, which is not easy with Angular.js due to the asynchronous way that most apps are built. This app has services that create Ajax promises and controllers that rely on services. Figuring out what’s supposed to be happening when can be challenging.

To make a long story short, this turned out to be a case of IE8’s non-crappiness. One of the views that was being included had a spurious close div in it. All of the other browsers happily rendered things the way I expected and didn’t even bother to warn me that the HTML was malformed. Only IE8 stuck to the standard and refused to participate in the charade. Yes, you read that right. IE8 was being a stickler for standards. I just don’t even know how to process that fact.

For what it’s worth, it can be hard to find this kind of error. When you have partial views that get included dynamically, and/or they’re ng-templates, it’s hard to see your completed HTML document, much less validate it. I even tried things like the Web Developer plugin’s “View Generated Source” and then submitting it to the W3C Validator to no avail. The way I ended up diagnosing it was the “fire the canon and see what dies” approach of commenting out different chunks of the code until I could isolate the section. From there, it was manual analysis. Yuck.

Fri, 6 Dec 2013

Another SELinux Lesson

— SjG @ 6:17 pm

So there’s this project that requires a ridiculously complicated communication protocol involving lots of byte-wrangling and formatting and weird transports. For the sake of brevity, I’ll only mention one of the endpoints, which requires decoding an email attachment.

Of course, this means a procmail script sending the email to PHP for processing and much ceremonious mucking about. On first go, it was failing. In debug mode, procmail was telling me that permission was denied. But it wasn’t user permissions: the file was owned by the same user as receiving the email and running the procmail script.

Naturally, when faced with cryptic permission failures, the first thing I did was look at /var/log/audit/audit.log and /var/log/messages for SELinux denials. There I found nothing at all. No errors, no warnings, no ugly “avc: denied” splatters.

Finally, this page here explained it to me. Rebuilding the policies with semodule -DB quickly revealed that my problem was, in fact, SELinux (as it all so often is). Once I could see the policies that were marked “dontaudit,” it was just an hour of building more and more complicated policies for procmail before stuff started working.

Once everything was good, happy, and shiny, a simple semodule -B returned the SELinux logging to the previous state, and I could once again spend my time fighting the convoluted bit-twiddling of the communication protocol.