Unix Libraries and Mac OS X

One of the great strengths of Apple's Mac OS X operating system is its unix core. A veritable world of existing applications are instantly available to it, providing you have persistence and the patience to install the prerequisite software libraries.

As regulars to this column know, we have attempted to build various unix applications on OS X with mixed degrees of success. You can't imagine how many I've played with prior to writing a word about it. Or perhaps you can, if you apparently have as much time on your hands as I do. One of the persistent roadblocks to a successful build has been the following message:

header file 'dlfcn.h' not found

That little file called dlfcn.h is part of the DLopen interface. Heretofore the DLopen interface was available from osxgnu, but does not contain (or at least apparently does not install) that critical little file called dlfcn.h. Today I discovered the missing link. Thanks to a serendipitous web reference, I found said libraries on the SourceForge site. At press time, the latest version is called dlcompat-20021117.tar.gz. StuffIt should unpack this to your desktop automatically. We will install it presently.

Well, DLopen is one of many components that unix applications may assume to be present on your machine. So, if you get the itch to try your hand at building and installing a freely-available unix application from source code, then you'll quickly learn about a concept called dependencies. This isn't a welfare thing. It's a sensible approach to programming that all desktop operating systems incorporate nowadays. Instead of including the same parts over and over again with each application you install, the concept is to install them once, and make them available to all applications that need them.

Linux aficionados have long been acquainted with the dependencies process, and dependencies are handled in a fairly sophisticated way there, not in small part due to the collective hard experience of some very clever programmers. Mac OS X, being late to the unix party, can tap into that existing body of knowledge, but also has to develop its own system of handling dependencies. For standard applications such as Secure Shell, or ssh for short, Software Update handles everything transparently. Having run OS X from the public beta days, I can tell you that it's been interesting to watch the development of how OS X has handled this stuff. In those early days, you had to build OpenSSH yourself if you wanted the Latest and Greatest. Now these applications are kept up-to-date automatically, making it somewhat counterproductive to do it yourself. How far we've come.

For applications you choose to build from source code, dependencies have to be handled manually. That means installing funnily-named libraries such as dlcompat above. Fortunately, the build process will fail at the point where some library is missing, and you actually have a reasonable chance of locating the thing. Unfortunately, dependencies often have dependencies themselves, leading to a process that can try the patience of Job himself, never mind me. Knowing at least a couple of good sources can save the day.

If you have been building unix applications on OS X, then you will have already installed the Developer Tools, which came as a separate CD with your OS X kit, and is also available for the download at the Apple Developer site. Beyond that, specific applications will assume specific libraries and other dependencies, but without the developer tools you won't be able to build anything.

Let's get back to dlcompat for a moment and build the thing. Open a Terminal window and type cd and a space. Next, drag the dlcompat-20021117 folder icon onto your Terminal window. Lo and behold, it writes the complete name and path into your Terminal window automatically. Now press Return. We're ready. Cut and paste the following:

./configure
make
sudo make install

This is the standard build sequence used for many unix applications. Under OS X, it should fly without a hitch. The sudo command requires your administrator-level password, which is the default if it's your Mac.

A related error is this one: can't open library: /usr/lib/libdl.0.dylib (No such file or directory, errno = 2). Whether or not it is orthodox to do so, I worked around this problem by making a link to the libdl.1.dylib library file installed by DLcompat, like so:

ls /usr/lib/libdl.0.dylib
ls /usr/local/lib/libdl.0.dylib
Assuming that the file lib/libdl.0.dylib doesn't exist, then continue:
cd /usr/local/lib
sudo ln -s libdl.dylib libdl.0.dylib

Your mileage may vary. What this does is make a logical link from the file name libdl.0.dylib to the generic file name libdl.dylib. One reference suggests deleting the files libdl.x.dylib altogether, leaving only libdl.dylib installed, but I found this dicey. I prefer a set-and-forget solution myself, such as the link above. As much as I can glean from it all, the onus is on the programmer to only reference libdl.dylib, and allow DLcompat to set a link from that to the actual latest version, e.g. libdl.0.dylib, libdl.1.dylib, libdl.2.dylib, etc. Not being a programmer myself, I stand to be corrected, of course. However, if this hifalutin theory is correct, then my workaround above is for the sake of source code that mistakenly uses a "hard" reference to a particular version of libdl.dylib instead of the generic libdl.dylib. (As future versions of libdl.dylib get developed, in the same way as above you'll have to cook links from the next-to-latest versions, e.g. libdl.1.dylib, to libdl.dylib if you have source code looking for them.) Whatever.

I also came across an interesting short list of related fix-its called Compiling Applications on Mac OS X. This site definitely assumes a certain degree of unix savviness. You'll find it interesting, to say the least. On that site I found a reference to something I've experienced many times: "Error: can not guess host type; you must specify one ." The proposed solution is to copy a pair of files, config.guess and config.sub, into your application's source folder, replacing the ones supplied. But none exist where they say. Instead, I found a set in /usr/local/share/automake-1.6 and copied those. As always, your mileage may vary.

The GNU's Not Unix! site is a great way to get oriented with this stuff. And it's chock full of unix applications that for the most part will build on practically any unix platform. If you never download a thing from this site (highly unlikely), it makes for a good browse, even the licensing agreements.

One way to head off the dependencies conundrum is to Fink out. Fink is a tool made for OS X that installs many well-known unix applications for you, dependencies and all. While Fink itself is not for the faint-of-heart in the first place, nevertheless it works well, and slices and dices those dependencies famously. Basically, Fink checks the application for required libraries, your system for their presence, and installs them if necessary. Fink installs into a somewhat non-standard reserved location, making it easier to keep track of what's where. You might call that cheating. A mere detail at this point. Oh, need I mention to read the documentation carefully?

Eventually you'll encounter build instructions that include the gmake command. As delivered, OS X doesn't support gmake. But it's the same thing as make, which is supported in the Developer Tools. Here are the steps to make the link for gmake:

which make
cd /usr/bin
sudo ln -s make gmake
exit

As always, your mileage may vary, but not by much. Once done, any build scripts that include the gmake command will run the command just fine. I include this here because it fits in perfectly with the living mystery novel one lives when attempting to build stuff on OS X, or any other unix operating system for that matter. Applications assume that certain things are installed, including gmake. It's a reasonable expectation, but someone has to hunt down those things, and that person will be you.

I propose this particular article as being somewhat open-ended, to be continued, as it were. Without a doubt, there are many worthy sites out there. Feel free to contact me with your suggestions for standard libraries and others that might have particular value in an OS X context, and I'll update this article with the contributions. Ciao.