Posted By Rogue Amoeba Alumni on February 21st, 2008

I’m a real sucker for cute and slick little tricks, those small things that seem pretty cool when you see them and leave you thinking, “How’d they do that?”. For instance, if you’ve used an iPhone, you’ve no doubt seen that it bounces when it hits the bottom of a list. It doesn’t have to do this, but it’s certainly a cool effect. It’s also more natural, which is always a positive thing. This week, I was happy to be able to help add a similarly-cool trick to the newest version of Airfoil.

Perhaps the most interesting new feature of the the “Apple TV Take 2” update, as far as Rogue Amoeba is concerned, is its ability to receive streamed audio via iTunes. Because the Apple TV is an AirTunes client, you can now use Airfoil to send any audio to it. It’s in sending to the Apple TV that I was able to inject a bit more spice into Airfoil.

What we did
When you use iTunes to send audio to the Apple TV over AirTunes, iTunes sends metadata as well, causing the Apple TV to display the track name, the artist, the album, and album artwork. Once we had Airfoil properly talking to the Apple TV, we turned to working out what exactly we’d send for metadata.

In Leopard, Apple has shipped beautiful icons that represent all the recent models of Mac capable of running Mac OS X 10.5. From Mac Pros and Minis to the new MacBook Air, there’s an icon for each sitting on your system. You can see a slightly scaled-down version of their iMac icon to the right. We actually make use of these icons Airfoil already – when an Airfoil Speakers client shows up in Airfoil’s list, we show the associated artwork to help identify the client machine. However, these images are tiny, just 16×16, so you might not even notice that they’re customized for your machine. With Apple TV, however, we thought it’d be a nice touch to display a full-sized 512×512 image of the Mac that’s sending the audio.

Mike set this up easily enough and it certainly looked nice. On Tiger, however, we could only use a generic icon of an Apple Display. That set Paul to thinking about showing something on that display to help identify the machine. He hit on the idea of showing the desktop of the machine, superimposed on the icon. To quote him, “It’d only be mildly useful, but it would be sexy, and it might be easy?”.

I agreed that this was a sexy idea, but would it be easy? There was only one way to find out. Ultimately, it took some fancy footwork and a little bit of craziness on my part, but I was able to make this happen in just a few hours. Along the way, this actually morphed into a Leopard-only feature. So when you’re streaming to an Apple TV via Airfoil running on Leopard, we send along the image of your Mac and overlay your desktop on it. Put simply, it’s damn cool to see. You can see a sample below, – we’re generating this image live, as we’re sending iTunes audio to the Apple TV via Airfoil running on a MacBook Air¹. It’s really best experienced in person though, so if you’ve got a Leopard mac and an Apple TV, stop reading and grab Airfoil right now to check it out.

“How’d they do that?”
So, how did we make this happen? To start, we ask the system for the image of the Mac model we’re running on (using [NSImage imageNamed: NSImageNameComputer]). That gives us one of those giant 512×512 pixel images representing the local machine. Then we need to figure out where on that image the “screen area” is – the blue desktop that we can overwrite. We debated hard-coding that information for all the models that were there, but that’s not terribly elegant, as it would fail any time Apple brings out a new Mac or if they changed the included images even slightly.

Instead, I decided we could do this with some image processing magic, finding the part of the image that shows the blue Mac OS X default desktop image. By finding colors between certain hue, brightness and saturation ranges, it’s possible to work out which part of the image represents the Mac’s screen. If a screen isn’t found, as in the cases of a G5 tower or Mac mini, then an image of a nice Apple Cinema Display is substituted in. The Windows Blue Screen Of Death beige monitor seen to the left was also under consideration, but the Cinema Display is just a tiny bit nicer.

At this point, we’ve got an image of a Mac or an Apple display and we know where the “desktop” area is on that image. That means it’s time to get the new desktop image to be superimposed over this. Originally we thought of doing a simple screen shot, but this raised security concerns. Mike envisioned, quote, “some upset user coming to us because he forgot to close his porno browser before he started playing music on his 52″ plasma screen”. A simple “Sorry Mom!” probably doesn’t fix the trauma there.

With a bit more discussion, we settled on the “Apple Product Shot” style – just the Desktop, Dock and Menubar. This provides a nice, clean look but it’s still obviously recognizable as your machine. Using the great new CGWindow API in Leopard, I was able to get just that without much difficulty. We also decided to add in the source application from which Airfoil is pulling audio. Here are a couple more examples, with Pandora from a Mac mini (so we substituted the generic Apple display) and Front Row from a MacBook Pro.

So that’s “how we did that”, and the final results are really pretty exciting. When you use Airfoil to send audio to your AppleTV, you’ll be treated to a great-looking rendition of your Mac, replete with its desktop and source application visible. It’s like magic! Don’t take my word for it though – you can get Airfoil 3.1 and see for yourself!

1. Why would we send iTunes via Airfoil? For use with Airfoil Speakers, of course. iTunes doesn’t acknowledge Airfoil Speakers as it can’t authenticate with it. Instead, we get Airfoil to pull audio from iTunes. Then we can send audio to the Apple TV, along with Airfoil Speakers on Mac or PC and of course, AirPort Express units. ↩