atomicbird

Last year I posted a script that could be used in place of the Unix "man" command to generate PDFs of the man pages in question. If Mint is to be believed, this continues to be a fairly popular page, so here's an update on the old version.

When I posted it, Tiger (Mac OS X 10.4) was still current. After Leopard (10.5) came out I found it necessary to update the script slightly. Not because the old one didn't work, but because Leopard brought many updated man pages with it. The caching mechanism used by the script had no way to detect that the cached PDF was out of date relative to the man page it was based on.

Anyway, here's an updated version of the script. PDFs are cached, as before, but are re-generated if the cached PDF is stale.

As with the previous version, you can use it just like "man": "manpdf time" and "manpdf 3 time", for example, differ from the stock "man" command only in that a PDF is generated.

Source is below, you can also download it directly.

#!/bin/bash

# Convert man pages to PDF and open them in the default PDF viewer.
# PDFs are cached when created.
# By Tom Harrington, tph at atomicbird dot com, 16 March 2005

# Directory to save cached PDFs in 
# (if you don't want long-term caching, you could use /tmp/).
CACHEDIR=~/Library/Caches

# Command to read PS from stdin and write PDF to an output file.
PS2PDF_CMD="/usr/bin/pstopdf -i -o"
# Command to open the resulting PDF once it's created.
OPEN_CMD="/usr/bin/open"
# Path to "man", which is expected to take the -w, -S, and -t args.
MAN_CMD="/usr/bin/man"

CACHEDIR="$CACHEDIR/manpdf"
if [ ! -e $CACHEDIR ]; then
	mkdir -p $CACHEDIR
fi

if [ $# -eq 1 ]
then
	MANSECT=""
	MANARG=$1
elif [ $# -eq 2 ]
then
	MANSECT=$1
	MANARG=$2
else
	echo "Usage: " `basename $0` "[section] name"
	exit
fi

if [ "$MANSECT" != "" ]; then
	MANSECTARG="-S $MANSECT"
fi

# Look for the command's man page.
MANFILE=`$MAN_CMD $MANSECTARG -w $MANARG 2>/dev/null`
if [ -z $MANFILE ]; then
	echo No manual entry for $MANARG
	exit
fi
# OK, got the man page.  Now get the name for the corresponding
# PDF.  The goal here is to organize cached PDFs in CACHEDIR
# according to man-page section, similar to the way /usr/share/man
# is organized into "man1", "man3", etc.
echo file: $MANFILE
MANSECT=`echo "$MANFILE" | sed 's/.*\.\([0-9]\).*/\1/'`
MANPDF=$CACHEDIR/pdf$MANSECT/$MANARG.$MANSECT.pdf
FINALDIR=`dirname $MANPDF`
echo PDF: $MANPDF
# Create the PDF if it's not in the cache dir or if the cached version is obsolete.
CREATEPDF=0
if [ ! -e $MANPDF ]; then
	echo Generating man page...
	CREATEPDF=1
else
	if [ $MANFILE -nt $MANPDF ]; then
		echo Man page newer than PDF, regenerating...
		CREATEPDF=1
	fi
fi

if [ $CREATEPDF -eq 1 ]; then
	mkdir -p $FINALDIR
	$MAN_CMD $MANSECTARG -t $MANARG | $PS2PDF_CMD $MANPDF
else
	echo Reading man page from cache...
fi
# Now open the PDF.
$OPEN_CMD $MANPDF

The fun started yesterday when I was testing an application, and I noticed this little gem in Xcode's debug console:

CoreEndianFlipData: error -4940 returned for rsrc type FREF
     (id 128, length 7, native = no)

At about the time this message appeared, my application seemed to stop responding. I didn't know what was causing that, but this mysterious message had to be a clue. I was also, sometimes, getting a "Choose Application" window appearing on the screen, for no readily apparent reason.

I scratched my head for a minute and asked Google what it thought about it. Google wasn't sure. The FREF suggested some old-school pre-X Mac OS programming, but I wasn't doing anything of the sort. The part about endian flipping suggested an Intel vs. PowerPC issue, but I've been doing everything universal for a while now, so that shouldn't be it.

So I asked a team of experts. Thanks to IRC I was able to get some friends on the case, and Daniel Jalkut, Jon Wight, Manton Reece and I spent an interesting and bizarre afternoon trying to figure out what the hell was going on.

Early theories held that, while I wasn't personally using FREFs, perhaps I had included some code that did. That briefly made NDResourceFork a suspect. NDResourceFork is a Cocoa-style interface for dealing with HFS+ resources, and one of its associated classes mentioned FREFs. That ended up being a red herring; I have a graphical front-end app and a back-end worker app, and NDResourceFork is used in the front end while the error was coming from the back end. It's in my project but not actually in the application printing the error message.

So we looked at what the error message appeared to be trying to tell us. Clearly, something must going on with FREFs and/or switching from one endian style to another. Fortunately the experts know a thing or two about old-style Mac stuff, and before long I was setting breakpoints on functions like GetResource() and GetIndResource(), and using PrintResourceChain() in the debugger (functions which, I believe, are older than Mac OS X and which I had never heard of before-- heck, some of them aren't even documented anymore). I also found I could reproduce the problem in the debugger, and it's always nice when the application behaves the same way in a debugger as when it's running normally.

That led to a surprising discovery, that the mystery message was occurring when my application was making AppleScript calls to another application via NSAppleScript. In a way that just confused things for a while, though. There didn't seem to be any reason for this script to be using FREFs. And I verified that my application, the target application, and the AppeScript (compiled with osacompile) were all universal binaries, so there shouldn't have been any question of endianness. It was about at this time I noticed that, for some reason, the bug only manifested itself when the AppleScript was targeting OmniGraffle Pro. Cue ominous music here.

The debugging also led to some seriously weird results, in cases. Trying PrintResourceChain() at the gdb prompt showed resources loaded from completely unrelated applications-- not mine, not the target app, not the source of a scripting addition, not anything that should have been remotely involved.

Trying to pin down exactly when the mystery message appeared, Daniel suggested I set a breakpoint on write(), which is pretty near the lowest-level print statement that should get used by nearly everything printing anything. That was when I regretted all of those NSLog() statements I had added for what Jon Wight called "nuke 'em from orbit style debugging". Some careful disabling and reenabling of breakpoints got me through that without too much trouble, though. And, aha! CoreEndianFlipData appeared in the stack trace! And that was a direct result of my own NSAppleScript call, albeit about 45 levels down in the stack (not an exaggeration, BTW) from there. As for what was going on in between, well, that took some interpreting.

Daniel and Jon both noticed the presence of AEVTsysoppcb in the stack which is, they tell me, AppleScript deciding it needs to ask you where the target application is. The "choose application" window loads icons for all applications. And THAT, apparently, is where FREFs come into all of this. Remember the FREFs? This started with an error message about FREFs. I was now able to start reproducing the error message from Script Editor, which simplified things a bit and, frankly, made me feel a little better by knowing this might not be my fault.

Reducing the AppleScript to the bare essentials needed to demonstrate the behavior, I get this:

if application "OmniGraffle Pro" is running then
	beep
end if

Do this in Script Editor, and a "Choose Application" window appears, asking "Where is OmniGraffle Pro?", even if you have it installed. Click "Cancel", and Script Editor crashes. Watch the console, and you'll see the mystery message that started all of this. Of course OmniGraffle's full name is "OmniGraffle Professional", but fixing that doesn't change the behavior.

Out of curiosity I tried a bunch of other applications. The only other one I've found that causes this is RealPlayer. There might be others.

Apparently then, the chain of events is:

1. My app calls its AppleScript, targeting OmniGraffle
2. The AppleScript tries to find out if OmniGraffle is running
3. For some reason it can't figure this out, because it can't find OmniGraffle. I can only guess this is because of something weird in OmniGraffle's Info.plist, though I'm not sure what.
4. AppleScript helpfully asks the user where to find it.
5. The "Choose Application" window, in trying to find application icons, runs smack into that unrelated application that PrintResourceChain showed. It prints out a weird message about FREFs and endianness
6. If the user cancels, AppleScript freaks out and explodes, killing innocent bystanders like Script Editor or my application.

Now, all I want to do is run this damn AppleScript. If I can find a workaround, I don't actually care why OmniGraffle is making the existing script choke. Daniel pointed out that if an AppleScript says "application appname" somewhere, it causes a full name resolution for the application. If I could make the script check for a running application without that, no name resolution would happen, and tragedy would be avoided. The Pre-Leopard way to do this does just that:

tell application "System Events"
	return first application process of application
		"System Events" whose name is "OmniGraffle"
end tell

Unfortunately it's not workable for me. My script's not actually about OmniGraffle, it targets a bunch of different applications, and it gets the application name from NSWorkspace. NSWorkspace tells me that OmniGraffle is named "OmniGraffle Pro". But AppleScript sees it as "OmniGraffle Professional", and if I use anything but that then it can't find the process. AppleScript then reports that it's not running, even if it is. There's no name resolution, which is nice, but there's also no reliable result.

I'm still beating on this a bit, but it looks like another Leopard-ism may save the day. Beginning with 10.5 it's possible to target an AppleScript based on an application's bundle ID instead of its name. Bundle IDs are, fortunately, consistent where application names may not be. And since they're not application names, they don't get resolved in the same way. So I can do something like this:

if application id "com.omnigroup.OmniGrafflePro" is running then
	beep
end if

And... it works! I think. I need to do some testing to see if it's as reliable as I need it to be. But it looks like what I need.

I'll be reporting this to Apple, because whatever OmniGraffle is doing, AppleScript shouldn't crash and burn like that. I'll probably also report it to Omni, who may well be interested to know.

I'd also like to thank Daniel, Jon, and Manton for taking so much time to help track this down. I don't think I would have got this far without their help.

Update: Daniel Jalkut pointed out that it's possible to address applications by bundle ID in AppleScript on Mac OS X 10.4, although it involves what Apple's AppleScript release notes describe as "...a multi-line incantation using Finder." That incantation turns out to be something like this:

tell application "Finder"
	set appname to displayed name of application
		file id "com.omnigroup.OmniGrafflePro"
end tell
tell application "System Events"
	if exists process appname then
		beep
	end if
end tell

I recently wrote about problems using Leopard code signing with Mac OS X frameworks. I've since gotten feedback on my bug report. It looks like the problem isn't so much that frameworks can't be signed but that the correct signing procedure isn't documented.

The code signing documentation indicates that bundles should be signed. Frameworks are bundles, so if you're looking to sign your code you'll likely be tempted to sign a framework like this:

$ codesign -s "authority info" Sparkle.framework

But if you do that you run into the confusing situation I encountered, with your framework structure modified and difficulty knowing if the bundle is valid or not.

The feedback I got on my bug report explains that code signing should be done differently for versioned bundles like frameworks and... whatever other versioned bundles there might be. Although frameworks commonly contain only one version, they're designed so that multiple versions can be present. When signing a framework then, you sign the specific version, not the entire framework bundle. So instead of the above, you instead do something like this:

$ codesign -s "authority info" Sparkle.framework/Versions/A

If there are other versions, sign them separately.

This leaves the framework structure unmodified:

$ find Sparkle.framework/ -name Sparkle -exec ls -l {} \;
lrwxr-xr-x  1 tph  wheel  24 Nov 29 11:44 Sparkle.framework/
    /Sparkle -> Versions/Current/Sparkle
-rwxr-xr-x  1 tph  wheel  242928 Nov 29 11:44 Sparkle.framework/
    /Versions/A/Sparkle

Of course when verifying the signature, you also need to verify the versions independently of the framework bundle. The framework itself won't have a valid signature, but that's not what you should be looking at anyway:

$ codesign -vvv Sparkle.framework/Versions/A
Sparkle.framework/Versions/A: valid on disk
$ codesign -vvv Sparkle.framework/
Sparkle.framework/: code or signature modified

This seems to make sense. A framework is designed to allow multiple independent bundles, with symbolic links to indicate which is current. So, sign each bundle on its own.

Yesterday I was working on a forthcoming update to Chimey and I noticed something odd. Chimey of course makes use of SparklePlus for automatic updates, and after a test run of my build-for-release script, Sparkle was looking a little odd.

Normally a framework has one or more binaries, with a symbolic link pointing to the current version. From the command line you'd expect something like this:

$ find Sparkle.framework/ -name Sparkle -exec ls -l {} \;
lrwxr-xr-x  1 tph  wheel  24 Nov 21 11:20 Sparkle.framework/
    /Sparkle -> Versions/Current/Sparkle
-rwxr-xr-x  1 tph  wheel  233088 Nov 21 11:20 Sparkle.framework/
    /Versions/A/Sparkle

Instead I was seeing this:

find Sparkle.framework/ -name Sparkle -exec ls -l {} \;
-rwxr-xr-x  1 tph  wheel  242928 Nov 21 11:20 Sparkle.framework/
    /Sparkle
-rwxr-xr-x  1 tph  wheel  233088 Nov 21 11:20 Sparkle.framework/
    /Versions/A/Sparkle

Yow, how the hell did that happen? I thought it might have something to do with copying the framework, either when compiling or when building the disk image, but that wouldn't account for the different file sizes. A quick check on the current version of Chimey showed that this was something new, not something I'd been doing all along without realizing it.

So what was different? My build-for-release script now signs my code using Leopard code signing.

Apple's documentation on code signing indicates that "You should sign every program in your product, including applications, tools, hidden helper tools, utilities and so forth." Chimey's main bundle includes a preference pane and two helper tools, so I was making sure to sign all of them. The main documentation doesn't mention frameworks specifically, but the Code Signing Release notes indicate that "You may also sign any libraries, frameworks, plugins, and scripts you ship, whether they are delivered with an application or separately." Framework signatures don't get checked yet but might be in the future. I had made my script as future-proof as possible by signing the Sparkle framework now.

But if you sign a framework, the codesign tool modifies the structure of the framework, as I found with Sparkle. Just for sanity's sake I made sure that this affects any Mac OS X framework and is not some kind of Sparkle-specific behavior.

And the file sizes? I can guess what's going on, but I made sure:

$ codesign -vvv Sparkle.framework/Sparkle 
Sparkle.framework/Sparkle: valid on disk
$ codesign -vvv Sparkle.framework/Versions/A/Sparkle
Sparkle.framework/Versions/A/Sparkle: code object is not signed

Not only is codesign changing the framework structure, it's also leaving the framework in an inconsistent state with regard to whether it's signed. Instead of one binary I've got two, and even though I signed the framework, one of those two is still unsigned.

At the same time, checking on the framework bundle still returns a valid signature, despite the presence of unsigned code in there:

$ codesign -vvv Sparkle.framework
Sparkle.framework: valid on disk

It's a good thing that Leopard doesn't currently check on framework signatures. For now it seems it's probably best not to bother signing a framework. Although codesign leaves you with something that should work, it's not clear that it's actually doing anything useful, and it's bloating the framework size in the process.

This has been filed as bug #5609522 with Apple, in case anyone from Apple reads this.

The next Iron Coder has been announced. This time it'll be a nine-day event instead of a weekend. The prize is an iPod Touch. And while the API hasn't been announced yet, it's going to be something new in Leopard.

If you read my previous post about expanding my product lineup, you know that MondoMouse was the ultimate result of the first Iron Coder. Since then I've participated whenever I've been able.

If you're a Mac developer, or want to be, this is a great event to take part in. The constraints of the event focus your attention on getting something done and ready to at least show to other people if not actually to release to the general public (and who knows, maybe it'll evolve into something you can release to the world). It's a good way to dive in and learn a new and unfamiliar API, a sort of friendly competition combined with a self-taught immersion course. Win or lose, I expect this Iron Coder will be most valuable to those with the least amount of time working on Leopard so far. If you're just now wanting to come up to speed with Leopard, participating would be a really good way to jump-start things.

It's also a good way to get to know other developers-- and not just any developers, but those who are actually motivated to try something like Iron Coder because it sounds like fun to them. The #macsb IRC channel (irc.freenode.net) serves as an unofficial discussion center for Iron Coder. It's normally a friendly competition, so you can feel free to ask for help along the way and usually get it.

So mark your calendars, and make sure you know your way around Xcode 3.