Mac OS X hosts File for Panther

A new year is upon us. Times change. Situations change. Operating systems change. And so does the procedure for short-circuiting unwanted internet advertising while browsing on Mac OS X, Apple's world-class computer operating system. Once again, here is the procedure, freshened up for Panther.

It's a funny thing about procedures. Time has a way of distilling them down to essentials. As my notes begin to morph into this article, it's looking a lot like a fairly short one, at least when compared to the Jaguar and previous versions. One of the reasons is a technical change in Panther. Another is that the probability of losing my audience goes up exponentially with every extra step. Believe me, this is worth the effort - and expenditure of patience.

From the first incarnation of this procedure as given here in this column, I give all the credit for the various steps to those who deserve it. I merely see the Mac OS X hosts File series as the organizing of these steps into one place. So I refer all kudos to those clever people.

Before going further, allow me to repeat my disclaimers. First, I reserve the right to copy and paste from my previous articles with abandon. Why walk when you can Cmd-C, Cmd-V? Second, I offer this procedure freely. You can choose to use it or not. Any side effects on your systems, whether real or perceived, are totally your responsibility. I'm always happy to help, but I'm not your punching bag! For more disclaimers, see my previous article. By the way, macwrite.com, which hosts this column, has some legalese of its own. Good, we're done with that stuff. Let's get to work.


Purpose

The goal of this procedure is to redirect unsolicited requests for internet advertisements to a non-existent internet address. Practically every news site and many others have embedded links in their web pages. A certain level of advertising is okay, but these days it's scatter-bombing. Enough is enough. The basic technique will be to install a pre-configured database file which explicitly lists known ad servers. When OS X goes to look up the numeric internet address for any host name such as "ad.server.com", it will check this list first. If that host is listed, it will accept the corresponding address and attempt a connection, which will fail due to a phony address. If it isn't listed, OS X will go out onto the internet and look up the name on a remote database server called a Domain Name Server, or DNS for short.


Checking Host Name Lookup Order

In earlier versions of OS X, there was a problem with the order in which host names got looked up. Very early versions went out to a DNS first before consulting any local resources. Jaguar partially corrected this by making the operating system consult a certain text file (called a flat file) first. To this day, OS X does not consult its internal database early in the process. To check your system, open a Terminal window and copy and paste the following command:

lookupd -configuration

A page of information should be returned that looks something like this:

[Mac:~] damien % lookupd -configuration

ConfigSource: file://etc/lookupd
LookupOrder: Cache NI DS
MaxIdleServers: 4
MaxIdleThreads: 2
MaxThreads: 64
TimeToLive: 43200
Timeout: 30
ValidateCache: YES
ValidationLatency: 15
_config_name: Global Configuration

LookupOrder: Cache FF DNS NI DS
_config_name: Host Configuration

<-snip->
[Mac:~] damien %

The correct lookup order for the Host configuration should be Cache FF NI DNS DS, i.e. both FF (Flat File) and NI (NetInfo) ahead of DNS. Anyway, so long as FF is up-front, we're good to go today. If you wish to play with this or if indeed FF is out of order or missing altogether, the details for fixing it are laid out below (red text) and in the earlier article. (1)


Setting Up For The First Time

A couple of simple utilities will simplify life. Download and install the unix2dos package from the GNU Mac OS X Public Archive. Also download this simple script, with thanks to the Linux guy in my office who cooked it for me. Unpack it to your desktop, and drag the single file called akamai into your Home folder. Optionally drag it onto TextEdit to view its contents before proceeding. Now open a Terminal window and move the file to a good place like so:

sudo mv akamai /usr/bin
(Enter your administrator password) (2)

We're ready. (3)


Preparing the hosts Database File

Download and unpack this hosts file onto your desktop, with thanks and credit to the maintainers. Drag the HOSTS file out of its folder and into your Home folder. Open a new Terminal window. Rename the file to lowercase hosts. Now run dos2unix on it, to fix the line endings. (4)

The syntax for these two operations is:

mv HOSTS hosts
dos2unix hosts

It won't make any difference to run dos2unix multiple times on the same file, as the endings are fixed the first time and there is nothing to do in successive times. Be sure to do this each time you download a fresh HOSTS file.

Next, a little edit. You see, Apple uses third-party servers at akamai.net to serve up graphics for its web site. However, akamai.net is also listed in the standard HOSTS ad server database file. You need to edit these out; otherwise you will find an uncanny number of apparently broken graphics on the Apple site. The script which you installed a moment ago is made for the purpose. You've already dragged the HOSTS file into your Home folder and renamed it. Your new Terminal window will open in the same folder by default. Simply type

akamai

to remove every instance of an Akamai server in your hosts file. Done.

Now your hosts file is ready for loading.


Installing the hosts File

By default, OS X does not install a hosts file. But just in case one exists, back it up first. Then install the new one. Finally, restart lookupd for good measure. With thanks and credit to MB for the lookupd syntax, here are the steps:

sudo cp /etc/hosts /etc/hosts.previous
sudo mv ~/hosts /etc
sudo killall -HUP lookupd

Done!


Recapping the hosts Procedure

To recap, here are the steps required each time you update a fresh ad server HOSTS file, most of which you can copy and paste into a Terminal window in one (or two) fell swoop(s):

Download and unpack hosts
Drag HOSTS to Home folder
mv HOSTS hosts
dos2unix hosts
akamai
sudo mv /etc/hosts hosts.previous
sudo mv hosts /etc
sudo killall -HUP lookupd
exit

Apart from checking lookup order and installing dos2unix and akamai the one time in the beginning (5), the procedure is fairly manageable, wouldn't you say? You should be able to cook it in under two minutes. You may have to cut and paste the last couple of commands a second time if sudo prompts you for a password.


Adding Custom Hosts

Advanced users may like to add a few oft-used hosts listed on their Macs for quick lookups. While it's possible to edit these into the large ad server hosts file, doing so will compromise the ease of updating the ad server list from time to time. So don't do that. (6) Instead, as discussed earlier, first ensure that your machine's lookup order is:

LookupOrder: Cache NI FF DNS DS
_config_name: Host Configuration

The procedure for this is to create another flat file with a single line of contents that lists the correct lookup order. I'll just warn you ahead, though, that by unhappy coincidence the name of said flat file is the same as the one we've been dealing with above. There is nothing I can do about it except show you the commands, which you can blindly copy and paste into Terminal. Here we go.

cd /etc
sudo mkdir lookupd
cd lookupd
sudo cp hosts hosts.original
sudo echo LookupOrder Cache NI FF DNS DS > hosts
sudo killall -HUP lookupd
exit

Now open NetInfo Manager from /Applications/Utilities, unlock it, then select the /machines NetInfo database directory. At a minimum, you'll see two hosts listed: localhost and broadcasthost. Duplicate one of them, then edit the name and address of the duplicate to match the host name you wish to add. You can delete the "serves" property for it. Click away to save your changes. You can also use this procedure to add the odd ad server or two you discover yourself, so long as you don't overdo it. For the grizzly explanation of this stuff, see my previous article.

That's it. Enjoy the quiet. Happy New Year to you and yours.

1. Prior articles in the Mac OS X hosts File series showed how the hosts file got installed into OS X's NetInfo database, its main repository of operating system information. In this article, hosts gets installed as a flat file into the standard unix location. The reason for this change, which could be construed as a step backwards, is because lookupd, the engine that actually performs the host name lookups, has been changed in Panther to such an extent that overhead during a NetInfo database lookup is substantial enough to cripple a G4's performance. The overhead that naturally attends delving through a long text file is far less, and should be fairly negligible on a G4 machine or better. Lookup performance on a G3 running Jaguar or earlier was substantially better doing NetInfo lookups than flat file lookups, so a G3 upgraded to Panther might suffer a small, yet noticeable performance hit if an ad server hosts file is installed in the standard unix way (but will almost certainly grind to a halt if hosts was installed into the NetInfo database). That said, the file is easily uninstalled like so:

sudo mv /etc/hosts /etc/hosts.broken

To uninstall a previous hosts NetInfo installation, I refer you to the earlier article. The key to it is renaming NetInfo's machines database directory to machines.broken, say, and restoring the original two entries. I did the latter by having a Terminal window ready with the following pre-tested command:

sudo niload -r /machines / < machines.original

If the backup file machines.original doesn't exist, then you know who to blame. Download and unpack this one, cooked from a clean Panther installation, into your Home folder and go from there. Be absolutely sure you pre-test the niload command before touching NetInfo Manager, so that you can quickly up-arrow to recall and execute it as soon as you rename machines. Timing is everything. It won't hurt anything to run the command a couple of times for testing.

2. If you have any trouble with sudo, try su by itself first. If that doesn't work, then you need to enable root user. Open NetInfo Manager from Applications/Utilities, unlock it, and pull down Security, Enable Root User. When you're done, it's probably a good idea to disable it again.

3. An alternate method is to use a nice paste feature of Terminal. Type sudo mv   (leave a space), then drag the akamai file from your desktop onto the Terminal window. Its full path and name will be pasted in for you. Finish the command with  /etc (space in front), and press Return. Type exit to finish any Terminal session nicely.

4. If you'd prefer not to install dos2unix, then run this alternate but equivalent series of commands (with credit and thanks to SE), which I will just quote:

"I found the following sed script to work... NOTE: It didn't work in the default Mac OS X shell, tcsh, so I changed to the Bourne shell first. Also, the ^M is a single character, generated by typing Ctrl-V Ctrl-M."

sh
sh$ sed s/^M//g hosts > foo
sh$ mv foo hosts
sh$ exit
exit

5. In all likelihood dos2unix and akamai, along with any other unix applications and tweaks you may have installed, will require reinstallation each time you perform anything that remotely resembles a clean install of OS X. For most people, this probably includes major OS upgrades, such as from Jaguar to Panther.

6. When you see just how large this file is, I'm hoping you'll leave it alone. But if you absolutely insist on editing hosts after the fact, then use the following command:

sudo /Applications/TextEdit.app/Contents/MacOS/TextEdit /etc/hosts
sudo killall -HUP lookupd

The reason is that everything in /etc is system stuff, and requires appropriate privileges to edit. In case you want to submit your own pet method of starting TextEdit from the command line, allow me to save you the trouble. While it certainly starts, it doesn't have system privileges unless you start it the way I just did. The beauty of my method is that all you need to do is copy and paste onto a Terminal window and go! Remember to close this privileged TextEdit session with Cmd-Q.

Ciao.