Advanced programming tips, tricks and hacks for Mac development in C/Objective-C and Cocoa.

Optimizing the loading of a very large table on the iPhone

In this post, I look at a UITableView on the iPhone which loads its data from a server and look at how its performance scales from single rows to tens of thousands of rows. I'll examine which aspects of the iPhone scale well and which become a burden as a displayed dataset moves from trivially sized to large sizes.

Introduction

Last week, I received an email asking if StreamToMe would be able to handle 20,000 files in a media collection. This person probably meant "20,000 files categorized into subfolders" but the performance geek in me immediately thought that 20,000 files in a single directory was a far more interesting question.

It's easy to find programs on the iPhone that become slower and less responsive when dealing with tables that only contain a few hundred rows. In my own experience, I've rarely tested with more than a couple hundred items in a UITableView. I had no idea what a UITableView would do with 20,000 items.

Purely looking at data sizes, it seems like it should work: the 128MB versions of the iPhone (all devices prior to the 3Gs) allow applications to use between 24MB and 64MB of RAM before they are killed for excess memory use. This should allow for between 1-3kB per row within available memory — far more than I need.

Of course, this won't be a synthetic test with efficient but trivial rows: this is a program with real data, transferred from a server, parsed, constructed and displayed, which must remain capable of media playback after everything else is loaded into memory.

Generating test data

I wanted real test data so I decided to replicate a small MP3 file on the server. I used a 1 kilobyte file containing a single tone that plays for 4 seconds. I used UUID strings appended with ".mp3" for filenames so that file sorting algorithms would still have some real work to do.

for (NSInteger i = 0; i < MAX_FILES; i++)
{
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    
    CFUUIDRef uuid = CFUUIDCreate(NULL);
    NSString *uuidString = (NSString *)CFUUIDCreateString(NULL, uuid);
    CFRelease(uuid);
    [uuidString autorelease];

    NSString *filePath =
        [directoryPath stringByAppendingPathComponent:
            [NSString stringWithFormat:@"%@.mp3", uuidString]];
    [[NSFileManager defaultManager]
        createFileAtPath:filePath
        contents:mp3Data
        attributes:nil];
    
    if ((i % 1000) == 0)
    {
        NSLog(@"%ld files remaining.", MAX_FILES - i);
    }
    
    [pool drain];
}

Using this code, I created directories with 1, 10, 100, 1000, 10,000, 20,000, 100,000 and 1,000,000 files to see how far I would get with the application.

A few interesting points came out of this work, unrelated to the iPhone itself:

  • Yes, you can now create more than 65536 files in a directory. I remember when Macs couldn't create more than 65536 files on the entire disk.
  • Curiously, when you select files and use the "Copy" menu item in the Finder, the maximum number of items is still 65536.
  • Don't try to drag 10,000 (or more) items from one window to another — I gave up watching the beachball on this action and hard-booted.
  • -[NSFileManager createFileAtPath:contents:attributes:] is a little slower than other approaches because it creates the file in a temporary location and moves it to the target location (a single create at the target location would have been faster).
  • if you try to create a million MP3s on your computer, be prepared to wait 3 hours while it churns away and then have the Spotlight metadata indexer slow your computer down for a further few hours.

Initial results

Which data sets will load and how long will they take? With no preparation of StreamToMe in anticipation of this test, I immediately pointed it at the test directories.

Number of filesTotal time from touch event to display of directory
1131ms
10128ms
100424ms
1,0003,108ms
10,00030,587ms
20,00064,191ms*
100,000N/A
1,000,000N/A

The asterisk here indicates that the 20,000 row run was not able to reliably play the audio files after loading (it would sometimes quit due to low memory). The one hundred thousand and million row tests both failed to load at all.

All tests performed on an iPhone 3G connected via 802.11g to a Mac Pro Quad 2.66Ghz (the iPhone was also connected to the Mac Pro via USB for logging purposes). Times are taken from a single cold run.

Initial analysis

Scaling

Looking first at the way the results scale, this table shows the expected behavior with the iPhone's memory arrangement:

  1. The iPhone has a 16k data cache so tests that operate within this limit (fewer than a couple hundred rows) are more bound by the network latency and fixed-duration setup costs than any row-specific work performed. This leads to better than linear (less than O(n)) scaling for the 1, 10 and 100 tests.
  2. Tests that exceed the 16k data limit (one thousand through twenty thousand) scale almost perfectly linearly as they push a consistent amount of data through main memory.
  3. There is no virtual memory on the iPhone, so you don't see a greater than linear increase in time as memory runs out (thrashing) — instead, there's an abrupt point at which things simply fail. More memory will not make a iPhone faster in the same way that it will make a memory constrained Mac faster.
Speed

Looking at performance, it's a little disappointing — no one would want to wait over a minute for their file list to load. Where is that time taken? Looking at timing results for the 20,000 row test:

  • 6,563ms on the server, loading the directory listing and formatting the response.
  • 3,111ms transferring data.
  • 12,771ms on the client parsing the response from the server.
  • 32,098ms on the client converting the parsed response into row-ready classes
  • 9,453ms in autorelease pool cleanup
Memory use

The memory footprint with 20,000 files loaded is around 6.8MB with peak memory use of 38.2MB. This leads to two questions:

  1. With final memory so low, why is the program behaving as though its memory is constrained?
  2. What is causing the peak memory to be so high? If it were much lower, 100,000 rows might be possible.

Code changes and improvements

stat is the biggest constraint on filesystem lookups

The first change I made was on the server. 6.5 seconds to load 20,000 files is too slow. The key constraining factor here is reading the basic metadata for tens of thousands of small files. The low level file function stat (or in this case, lstat) is the limiting factor.

Technically, I wasn't using lstat but -[NSFileManager contentsOfDirectoryAtPath:error:] was invoking it for every file and then -[NSFileManager fileExistsAtPath:isDirectory:] was invoking it again to see if each file was a directory.

In 10.6, you can replace -[NSFileManager contentsOfDirectoryAtPath:error:] with -[NSFileManager contentsOfDirectoryAtURL:includingPropertiesForKeys:options:error:] so that these two commands can be rolled into one (halving the number of calls to lstat). I want to keep 10.5 compatibility, so instead, I wrote my own traversal using readdir and lstat directly.

This change doubled the directory reading speed of the server.

Lowering memory footprint

In Cocoa, peak memory is often caused by autoreleased memory that accumulates during loops. You can address this by inserting NSAutoreleasePool allocations and drains into your loops but this is slow. The best approach is to eliminate autoreleased memory in memory constrained areas. I did this throughout the parsing and conversion code on the iPhone.

There was also some unnecessary copying of memory (from the network data buffer to an NSString and back to a UTF8 string) that I removed (by passing the network data buffer directly as the UTF8 string).

More than simply lowering the memory footprint, these changed almost doubled the speed of parsing on the iPhone.

Memory fragmentation

Even after lowering peak memory usage, I still encountered out of memory errors when trying to allocate larger objects, even though my memory usage was only 16MB.

After some investigation, I realized that my parser was fragmenting memory by allocating smaller and larger string fragments in a leapfrogging effect so that consecutive strings in the final data structure were not actually adjacent in memory. Even though memory usage was only around 50%, there was not a single, contiguous 2MB space within this for media loading and playback due to the scattered pattern of string, array and dictionary allocations following parsing.

The simplest solution (one that didn't involve rewriting the parser) was to copy the parsed data every few rows into properly contiguous locations, releasing the old non-contiguous locations. After this, memory allocation issues went away.

Of course, this did result in a minor increase in parsing time but the improved memory performance was worth the minor performance cost.

Moving work out of critical loops

Finally, I looked at the conversion of parsed data into classes representing each row. This work was primarily assigning strings to properties of an object and couldn't be easily avoided.

Of course, in an ideal case, parsing and object construction would be an integrated process but since the parser is generic and wasn't written for this program, it doesn't produce data in the best format, requiring this extra "converting" pass through the data. For development time constraints, I didn't consider integrating these two components although this is certainly a point where further speed improvements could be gained.

During this process, I also created an NSInvocation for each object to handle its user-interface action when tapped in the table (media rows play the file, folder rows open the folder) and assigned a named UIImage (either the file or the folder icon) to the object.

Since there are only two images and two possible actions, these objects could be created outside the loop and either minimally modified for each row (with different parameters in the case of the NSInvocation) or assigned as-is (in the case of the UIImage).

These seemingly tiny changes (in addition to the memory changes mentioned above) resulted in a better than tenfold performance increase for the converting stage (which had been the most time consuming stage).

Results revisited

With these changes, the results became.

Number of filesNew time takenOld time taken
1135ms131ms
10137ms128ms
100250ms424ms
1,000845ms3,108ms
10,0007,998ms30,587ms
20,00014,606ms64,191ms*
100,00084,594msN/A
1,000,000N/AN/A

The 20,000 row test case now runs more than 4 times faster and the 100,000 test case completes successfully and can play the audio files once loaded.

Running one million rows in the simulator

The million row test set involved 121MB of data sent over the network — it was never going to work on a 128MB iPhone.

Without the memory constraints of the iPhone, I ran a million rows in the simulator. It took 7.5 minutes to load (almost entirely bound by lstat).

After around 800,000 rows (40 pixels high each), UITableView can no longer address each pixel accurately with single precision CGFloats used on the iPhone, so every second row was misplaced by 16 pixels or so making the result almost unreadable beyond that point. In short: UITableView isn't really meant to handle a million rows.

Conclusion

The iPhone can handle tables with 100,000 rows — and it continues to scroll as smoothly as though it were only 100 rows.

Optimization doesn't mean writing assembly code and bleeding from your fingernails. A couple hours work resulted in a 4 times performance increase and dramatically better memory usage from very simple code rearrangement.

The biggest improvements to the performance came from three points:

  • replacing autoreleased objects with tight alloc and release pairs in just two loops (in some cases removing the allocation entirely)
  • removing the NSInvocation generation and UIImage lookup work from the key construction loop
  • reducing the calls to lstat on the server (although this wasn't part of iPhone UITableView optimization per se)

And increasing available productive memory actually involved performing more allocations — reallocating discontiguous collections of objects to contiguous memory locations. There was also the typical elimination of unnecessary copying.

StreamToMe Version 1.1 available

The latest version of StreamToMe — for streaming audio and video from your Mac to your iPhone/iPod Touch — is now available on the App Store. It has only been one month since I released version 1.0 but I have lots of new changes to share.

New Features

You will need to download the latest version of ServeToMe to take advantage of these new features.
The "Seek to anywhere" update

StreamToMe version 1.1 adds a number of requested features, most prominent of which is "seek to anywhere". You no longer need to wait for the end of the file to be encoded before you jump ahead — you can seek to anywhere at anytime and it will "just work".

Remote WiFi and 3G access

StreamToMe now supports connections via 3G and from non-local WiFi locations. Bitrates between 96k and 1600k are chosen by the iPhone based on the available data rate. These lower bitrates are also available on local WiFi connections for situations where interference means a lower bitrate is required.

A remote connection requires that you have configured your Mac's network to make ServeToMe's port accessible remotely. This configuration is left to you since it is dependent on how you are connected to the internet, your modem/router and firewalls.

Password protection

To protect access to your files (especially when made available over the internet) you can now password protect the server.

Minor changes and fixes

Of course, there are numerous little fixes and changes too:

  • Fixed playback of many common MOV codecs, so many more Quicktime MOV files will be supported.
  • The server now only encodes video as needed, resulting in much lower average CPU usage.
  • Fixes for occasional broken socket (network dropout) problems on Snow Leopard.
  • Fixes for session ID problems that caused "The requested file couldn't be converted for streaming" to be incorrectly sent for supported files.
  • Scroll indexes now used for large directories.
  • Shinier file and folder icons.

Still coming...

I can't deliver everything all at once but the following frequently requested features are still planned for a future version:

  • Windows support for ServeToMe
  • Alternate audio tracks
  • Subtitles
  • Thumbnail previews

Screenshots

Here are some screenshots of the updated application in action:

screenshot3.png screenshot1.png screenshot5.png

WhereIsMyMac, a Snow Leopard CoreLocation project

In Snow Leopard, you can ask for the computer's location. Without a GPS, how accurate could that be? The answer in my case is: very accurate. In this post, I'll show you how to write a CoreLocation app for the Mac that shows the current location in Google Maps, so you can see exactly where your computer thinks it is.

Where Is My Mac?

In this post, I present the following sample project:

whereismymac.png
You can download the Xcode 3.2 project here: WhereIsMyMac.zip (33kB). Mac OS X 10.6 is required.

The program shows your current location, centered in the map. The zoom level is set so that the accuracy radius reported by CoreLocation is exactly half the width of the window.

So how accurate is it? For me, my actual location was within about 50 meters of the detected location.

First, I'll talk about the extremely simple code involved. Afterwards, I'll discuss how CoreLocation gets this information.

Implementation

If you haven't already used CoreLocation on the iPhone, it's really simple: just turn it on and let it update you when it has location information.

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
    // Turn on CoreLocation
    locationManager = [[CLLocationManager alloc] init];
    locationManager.delegate = self;
    [locationManager startUpdatingLocation];
}

Then all you need to do is receive the location updates and load the map at the new location when received. We do this in the CLLocationManagerDelegate methods.

- (void)locationManager:(CLLocationManager *)manager
    didUpdateToLocation:(CLLocation *)newLocation
    fromLocation:(CLLocation *)oldLocation
{
    // Ignore updates where nothing we care about changed
    if (newLocation.coordinate.longitude == oldLocation.coordinate.longitude &&
        newLocation.coordinate.latitude == oldLocation.coordinate.latitude &&
        newLocation.horizontalAccuracy == oldLocation.horizontalAccuracy)
    {
        return;
    }

    // Load the HTML for displaying the Google map from a file and replace the
    // format placeholders with our location data
    NSString *htmlString = [NSString stringWithFormat:
        [NSString 
            stringWithContentsOfFile:
                [[NSBundle mainBundle]
                    pathForResource:@"HTMLFormatString" ofType:@"html"]
            encoding:NSUTF8StringEncoding
            error:NULL],
        newLocation.coordinate.latitude,
        newLocation.coordinate.longitude,
        [WhereIsMyMacAppDelegate latitudeRangeForLocation:newLocation],
        [WhereIsMyMacAppDelegate longitudeRangeForLocation:newLocation]];
    
    // Load the HTML in the WebView and set the labels
    [[webView mainFrame] loadHTMLString:htmlString baseURL:nil];
    [locationLabel setStringValue:[NSString stringWithFormat:@"%f, %f",
        newLocation.coordinate.latitude, newLocation.coordinate.longitude]];
    [accuracyLabel setStringValue:[NSString stringWithFormat:@"%f",
        newLocation.horizontalAccuracy]];
}

Notice here that I load the HTML from a file, then use it as a format string, replacing the % sequences. This means that I need to escape the two percent characters in the file that need to remain percents (this is done by turning them into double percents).

The only other relevant code is the code to convert from meters (the unit for accuracy in CoreLocation) to Longitude and Latitude (used to specify the zoom factor to Google Maps).

This too is pretty simple: it's just a scale factor for the latitude and a scale plus a trigonometric function for the longitude:

+ (double)latitudeRangeForLocation:(CLLocation *)aLocation
{
    const double M = 6367000.0; // mean meridional radius of curvature of Earth
    const double metersToLatitude = 1.0 / ((M_PI / 180.0) * M);
    const double accuracyToWindowScale = 2.0;
    
    return aLocation.horizontalAccuracy * metersToLatitude * accuracyToWindowScale;
}

+ (double)longitudeRangeForLocation:(CLLocation *)aLocation
{
    double latitudeRange =
        [WhereIsMyMacAppDelegate latitudeRangeForLocation:aLocation];
    
    return latitudeRange * cos(aLocation.coordinate.latitude * M_PI / 180.0);
}

People who are extremely fussy about distance to latitude conversions don't use a constant meridional radius of curvature for Earth (since Earth is an oblate ellipsoid, this value varies from around 6,330,000 to almost 6,400,000) but a constant value is sufficient for most purposes.

The accuracyToWindowScale value is used so that the accuracy range reported by CoreLocation ends up as half the window width (instead of 100% of the window width).

Where does CoreLocation get this information?

The documentation is somewhat vague on where CoreLocation gets this information. Apple imply that there are a few different sources of information that may be used, depending on what is available.

My first thought was: maybe it is using my IP address to determine my location.

Looking up my IP address using two different online IP geocoding services revealed the exact center of Australia as my location from my IP address — almost 2000 kilometers from where I actually am.

My second thought was maybe it is geocoding my address in Address Book. That's not the answer either, since that address is out-of-date and is at least 5 kilometers away.

The answer is that I have WiFi on my Mac. Apple uses the WiFi networks you can see from your current location and correlates that against a database of known locations of WiFi networks (yes, companies drive down streets and record the approximate street addresses of WiFi networks).

So my neighbouring networks: "SweetCheeks", "TheSherriff", "MrBojangles" and "Netgear5" are revealing my location (although I doubt the names are important — its probably the WiFi MAC addresses that are tracked).

Conclusion

You can download the Xcode 3.2 project here: WhereIsMyMac.zip (33kB). Mac OS X 10.6 is required

The only use I've seen for CoreLocation in Snow Leopard so far is setting the Time Zone automatically in the Date & Time System Preferences panel. This doesn't require a great deal of accuracy but it turns out that CoreLocation is capable of much more.

Of course, I'm only reporting on the accuracy that I see, which is anecdotal evidence only. I'd be interested to know how accurate (or inaccurate) this is for other people.

As you can see in this sample application, the code involved is very simple. Even if it isn't accurate for everyone, it would make a good option in many applications.

Building for earlier OS versions from Snow Leopard

It is very easy, when developing on a new operating system, to create projects that won't run on any previous OS version. To ensure backwards compatibility, there are Xcode and gcc options that allow you to build while maintaining support for earlier OS versions. In this post, I'll look at the ways in which this compatibility is controlled and some of the new ways it can go wrong on Snow Leopard.

The cost of backwards compatibility

Customers are regularly unhappy at programmers for dropping support for older operating systems as soon as possible. As with most decisions like this, the reason tends to be a little bit feature-driven and a little bit economic: development support costs increase for every supported version of the OS older than the most recent supported version.

Supporting older operating systems requires:

  1. When creating a build on a newer OS, you must be careful to write the entire program relying only on behaviors that existing on your earliest targetted OS.
  2. A test system running an earlier version of the OS to verify that point 1 succeeded.

Point 1 tends to be where most programmers rule out support for earlier systems. I know that as much as 20% of the Mac user-base are still running Mac OS X 10.3.9 and 10.4.11 but I'm unlikely to ever support these operating systems again in one of my releases because these markets aren't big enough to warrant giving up features like Obj-C 2.0's fast iteration, NSOperationQueue or CoreAnimation which have changed how I write programs.

Point 2 can also be a serious impediment for small developers who simply don't maintain a series of test machines for their software. Larger developers can simply buy extra machines or buy time in Apple's Compatibility Labs but even large companies need to weigh this cost against the potential return.

Easy part: building against earlier SDKs in Xcode

With Snow Leopard now installed on my main development machine though, I'm forced to go through the backwards compatibility rigamarole just to ensure Leopard compatibility. As fast as the uptake of Snow Leopard was, it still isn't the dominant Mac OS X version — it is far too soon for me to demand users upgrade (although Gus Mueller has already ripped that band-aid off with the 10.6-only release of Acorn 2).

While I'm talking about Mac OS X here, these settings are the same when building for different iPhone OS versions.

Building against earlier SDKs in Xcode is the easy part. There are two settings involved:

  1. Base SDK (the OS version whose headers you'll use and the newest OS version from which you'll use optional features)
  2. Mac OS X Deployment Target (the oldest OS version supported)

The Base SDK controls what SDK you actually link against and the Mac OS X Deployment Target controls the minimum OS version allowed. In simple cases, just set both of these to the same value.

You set the Base SDK in the Project settings:

xcodesdksetting.png

and further down the settings list, the Mac OS X Deployment Target:

xcodedeploymentsetting.png

Weak linking (using newer features if they are available)

Weak linking allows you to link against a newer SDK but deploy on an older operating system.

You should use weak linking in cases where you:

  • want the program to run on a earlier version of the operating system
  • want to use a feature from a newer OS version if it is available

Set the Base SDK to the newer OS version that contains the newer features you will use if available and set the Mac OS X Deployment Target to the older version.

This causes everything newer than the Mac OS X Deployment Target to be weak linked — meaning that you will try to link against the newer features if available but can live without them if they are not available.

If a program is run on an older OS than the Base SDK:

  • unavailable weak linked functions will have NULL function pointers
  • unavailable weak linked class names will return nil from NSClassFromString
  • unavailable weak linked methods will return NO from the containing objects' respondsToSelector: method.

The important point to remember is that anything weak-linked should be checked to ensure it is non-zero before use.

For example, suppose to wanted to set the Dock to autohide on Mac OS X 10.6 (using the new Snow Leopard -[NSApplication setPresentationOptions:] method) but you want your application to also run on Mac OS X 10.5 and leave the Dock as-is, you could set the Base SDK to Mac OS X 10.6, the Deployment Target to Mac OS X 10.5 and run the following code:

if ([[NSApplication sharedApplication]
    respondsToSelector:@selector(setPresentationOptions:)])
{
    [[NSApplication sharedApplication]
        setPresentationOptions:NSApplicationPresentationAutoHideDock];
}

Command-line building for earlier OS versions

You might think POSIX/Open Source libraries that are written to be cross-platform and don't link directly against an Mac OS X specific libraries shouldn't need to change from OS version to OS version.

However, most C/C++ libraries do link against libc (which is a part of libSystem on Mac OS X). Since libSystem is a dynamic library and changes between OS versions, this means that you must be mindful of exactly what version of libSystem you build against.

A common example of getting this wrong is the following linker error:

Undefined symbols:
  "_fopen$UNIX2003", referenced from:
      _some_function in somefile.o

You will get this and similar errors when trying to link two different components which are themselves linked against different versions of libSystem. In this case, the application was linked against Base SDK Mac OS X 10.5 but the static library was linked against the current libSystem.B.dylib in Mac OS X 10.6.

Obviously, fopen is a Standard C function and it is in every version of libSystem but different versions of libSystem have subtly different versions of the function. To fix the bug, you must ensure that all components you link together themselves link against the same versions of the standard libraries.

In the example above, the solution is to rebuild the static library from the source using the following gcc link-line options:

-isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5

That's right, it's the same settings as in Xcode but on the command line — the Base SDK (isysroot) and the Mac OS X Deployment Target (mmacosx-version-min).

Once you've built an executable or a dynamic library, you can use otool to check that they're not linking against any unexpected dynamic libraries (which may not be present on older OS versions) with the following command:

otool -L myAppExecutable

This reports the required dynamic libraries and their version numbers.

Run the same command on the dynamic libraries in your targetted SDK directory to check their version numbers against the library numbers required by your executable.

Be wary of changed gcc

In Mac OS X 10.5, if you didn't specify an architecture, gcc would build a 32-bit binary. In Snow Leopard, this has changed: if no architecture is specified, a 64-bit binary will be built.

You'll need to pay attention to this, particularly for auto-configured builds which may need to coaxing now to build 32-bit binaries.

Of course, this is an easy thing to do: just specify "-arch i386" or "-arch x86_64" or both on the gcc compile command line — but you will need to remember.

This problem can manifest in an annoying way: a file not found error when linking, even when there is a file of the desired name in the search path. If this occurs, be sure to check that the file in the search path contains the right architecture for your build.

Check an architecture with the following:

file libSomeLibrary.dylib

If you're trying to build a 32-bit binary and you only see:

libSomeLibrary.dylib (for architecture x86_64):	Mach-O 64-bit executable x86_64

Then you need to rebuild libSomeLibrary.dylib or find a 32-bit version.

Conclusion

Programmers don't like supporting earlier OS versions because it means more work and fewer cool features to play with. Despite this, there are often economic reasons to put in the effort and forgo the more modern features to support earlier OS versions.

You will want to choose an OS version when you start a project, since removing OS-specific features can be hugely time consuming — it is always easier to drop support later rather than gain support.

Set the Base SDK, set the Deployment Target and make certain all your components remain in-sync on these points. Remember to double check the build target and architecture for all components you build for yourself and always be wary of dynamic libraries.

Creating alpha masks from text on the iPhone and Mac

Alpha masks are a powerful way to create graphical effects in your program. In this post, I'll show you how to create an alpha mask from a text string and use that mask to create text-based effects with an image. I'll also show you how to do this on the iPhone and the Mac, so you can see the differences between these platforms in this area.

Introduction

In this post I will present the following sample application:

textmasking.png

The program shows the current time, updating every second, on the Mac or iPhone. Within the bounds of the text, the image of the space shuttle is displayed at 100% opacity. Everywhere else, it displays at 30% opacity.

This is done by drawing the image of the shuttle over a white background using an alpha mask — where the mask is white (100% opaque) for the text and dark gray (30% opaque) for the background.

Download the TextMasking-iPhone.zip (37kB) and the TextMasking-Mac.zip (42kB) projects.

Clipping regions in Core Graphics

Clipping and masking are the two key ways of limiting the effect that drawing has in CoreGraphics.

With a clipping region, we create a hard boundary. When a clipping region is used, every pixel drawn inside the region is fully shown (completely opaque), and no pixels outside the region are affected (completely transparent). For example, the following code:

CGContextBeginPath(context);
CGContextAddRect(context, CGRectMake(10, 10, 80, 80));
CGContextAddRect(context, CGRectMake(20, 20, 40, 40));
CGContextEOClip(context);

will ensure that subsequent drawing only has an effect if it is inside the rectangle (x=10, y=10, width=80, height=80) but outside the rectangle (x=20, y=20, width=10, height=10).

The "EO" in CGContextEOClip stands for "Even-Odd". When you use this function instead of CGContextClip, subsequent nested regions (an even or an odd number of nestings) continue to toggle clipping on and off. That is why the second CGContextAddRect is excluded from the clipping region. If I had used CGContextClip, the nested rectangle would have no effect (it is already inside the clipping region).

Clipping with image masks in Core Graphics

A mask affects the opacity/transparency of drawn pixels like a clipping region but the affected areas are specified by the color values in an image, not from a region. This is the approach used in the sample applications. A mask is used instead of a clipping region for two reasons:

  • A mask allows varying levels of transparency, not just on or off (for the partially transparent regions in the image).
  • On the iPhone, it's very difficult to get the region outline of text characters (on the Mac you can use appendBezierPathWithGlyph:inFont:), making a clipping region from a text boundary impractical.

Creating a mask

The Mac and the iPhone differ significantly on how the mask image must be created.

On the iPhone, it is possible to copy the current graphics context to an image before it is drawn to screen and use that as the mask.

On the Mac this doesn't work. The Mac requires that masking images be grayscale without an alpha channel — you could copy the current graphics context in the same way but it can't be used as a mask image. We'll need to create a context that meets these specific requirements.

On the iPhone:

Drawing the text to the current graphics context and copying that to an image on the iPhone:

CGContextRef context = UIGraphicsGetCurrentContext();

// Draw a dark gray background
[[UIColor darkGrayColor] setFill];
CGContextFillRect(context, rect);

// Draw the text upside-down
CGContextSaveGState(context);
CGContextTranslateCTM(context, 0, rect.size.height);
CGContextScaleCTM(context, 1.0, -1.0);
[[UIColor whiteColor] setFill];
[text drawInRect:rect withFont:[UIFont fontWithName:@"HelveticaNeue-Bold" size:124]];
CGContextRestoreGState(context);

// Create an image mask from what we've drawn so far
CGImageRef alphaMask = CGBitmapContextCreateImage(context);

The only tricky part here is that the image will be flipped when we draw it back again, so we need to draw the text upside-down — other than that, the iPhone has it easy here.

On the Mac

Since the Mac must have the bitmap image as a CGColorSpaceCreateDeviceGray(); image without an alpha channel, we can't just copy a screen context to use as our mask. Instead, we need to create a new context with the settings we require and perform all drawing there.

This has one advantage: we can set the context to flipped:NO so that we don't need to draw upside-down to have it render correctly.

// Create a grayscale context for the mask
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceGray();
CGContextRef maskContext =
CGBitmapContextCreate(
    NULL,
    self.bounds.size.width,
    self.bounds.size.height,
    8,
    self.bounds.size.width,
    colorspace,
    0);
CGColorSpaceRelease(colorspace);

// Switch to the context for drawing
NSGraphicsContext *maskGraphicsContext =
    [NSGraphicsContext
        graphicsContextWithGraphicsPort:maskContext
        flipped:NO];
[NSGraphicsContext saveGraphicsState];
[NSGraphicsContext setCurrentContext:maskGraphicsContext];

// Draw a black background
[[NSColor darkGrayColor] setFill];
CGContextFillRect(maskContext, rect);

// Draw the text right-way-up (non-flipped context)
[text
    drawInRect:rect
    withAttributes:
        [NSDictionary dictionaryWithObjectsAndKeys:
            [NSFont fontWithName:@"HelveticaNeue-Bold" size:124], NSFontAttributeName,
            [NSColor whiteColor], NSForegroundColorAttributeName,
        nil]];

// Switch back to the window's context
[NSGraphicsContext restoreGraphicsState];

// Create an image mask from what we've drawn so far
CGImageRef alphaMask = CGBitmapContextCreateImage(maskContext);

Using the mask

Using the mask is much easier than creating it:

  1. Save the state of the current context (so that we can go back to a non-masked state when we're done).
  2. Apply the mask using CGContextClipToMask.
  3. Perform whatever drawing we want masked.
  4. Restore the saved context state to remove the mask again.

The iPhone version is:

// Draw a white background (clear the window)
[[UIColor whiteColor] setFill];
CGContextFillRect(context, rect);

// Draw the image, clipped by the mask
CGContextSaveGState(context);
CGContextClipToMask(context, rect, alphaMask);
[[UIImage imageNamed:@"shuttle.jpg"] drawInRect:rect];
CGContextRestoreGState(context);
CGImageRelease(alphaMask);

The Mac version substitutes NSColor and NSImage for UIColor and UIImage but is otherwise the same.

Conclusion

Download the TextMasking-iPhone.zip (37kB) and the TextMasking-Mac.zip (42kB) projects.

Clipping and masking are two of the most powerful operations available when drawing in code but as highly abstract concepts they can be difficult to use for the first time since a mistake normally results in nothing happening (a difficult scenario from which to learn).

I have only briefly shown region-based clipping. If you don't need the partial transparency offered by masks and can draw your shapes as a region then it is faster than a mask (since it doesn't need to allocate or load the mask image). Obviously, the option you choose should be based on the output you require.

The examples shown here don't cache anything — they allocate and dispose of all data structures every time the drawRect: method is called. In a proper program, you should not allocate the mask image every time you draw. You can allocate the mask once and simply set the context back to it to update it.

An NSSplitView delegate for priority based resizing

The default resizing mechanism in NSSplitView is proportional resizing — if the NSSplitView changes size, each column resizes by an equal percent. This works badly in the common case where the columns in a split view are used to separate a side panels from a main view area (for example the "source list" in iTunes or the file tree in Xcode). In this post, I'll show you a delegate class that configures a split view for this side panel and main view behavior — resizing the views in a split view based on a priority list.

Snow Leopard?

Every Mac programming blog that I read is overflowing this week with Snow Leopard information. To celebrate this exciting event, I proudly ignore the trend and present code that (aside from a little Objective-C 2.0 syntax) would run on Mac OS 10.0.

Proportional versus priority-based resizing

For a three section NSSplitView, the default proportional resizing behaves like this:

proportionalsmall.pngproportionallarge.png

In proportional resizing, as the window grows, each column grows by the same percentage.

By comparison, priority-based resizing works like this:

prioritysmall.pngprioritylarge.png

Priority-based resizing nominates 1 view as the most important. This is normally the window's "main" view. This highest priority view is the only view that grows in size as the window grows.

You can download the sample project ColumnSplitView.zip (60kb) to see the priority resizing at work.

Proportional resizing in reverse

The flip side to priority resizing is that the highest priority view is also the first to compress to zero. For this reason, the priority-based resizing should also implement minimum sizes so that the main view never actually reaches zero size.

Once the highest priority view reaches minimum, remaining views are collapsed by priority until all views are at their minimum size. The window or enclosing scroll view's minimum size should be constrained so that the split view is never forced past the point where columns are all at minimum size.

Controlling an NSSplitView

NSSplitView takes a delegate. The delegate methods are where we control the minimum sizes of sections and which views expand or collapse by what amount.

The class I'll present will be a dedicated delegate class named PrioritySplitViewDelegate that allows you to configure priorities and minimum sizes for the NSSplitView's subviews. This generic delegate can then be constructed, configured and attached to the NSSplitView in your controller code.

@interface PrioritySplitViewDelegate : NSObject
{
    NSMutableDictionary *lengthsByViewIndex;
    NSMutableDictionary *viewIndicesByPriority;
}

- (void)setMinimumLength:(CGFloat)minLength
    forViewAtIndex:(NSInteger)viewIndex;
- (void)setPriority:(NSInteger)priorityIndex
    forViewAtIndex:(NSInteger)viewIndex;

@end

Some usage cautions about this design: the delegate does not know in advance how many sections the split view will have, so it will let you specify priorities or minimum sizes for views that don't exist.

Specifying a priority for every view is mandatory so if you forget to specify a priority for a view, an exception will be thrown when the NSSplitView is resized. Minimum sizes are optional but will be zero if not specified. Keep these points in mind when using this class.

Implementation

Constraining the coordinates in the splitView:constrainMinCoordinate:ofSubviewAt: and splitView:constrainMaxCoordinate:ofSubviewAt: happens when the user drags the boundary between two split view sections. In these methods, we need to ensure that the view which is getting smaller does not exceed its minimum size.

When dragging the boundary between two columns to the left, this means that the minimum coordinate is that which would collapse the left view to its minimum size.

- (CGFloat)splitView:(NSSplitView *)sender
    constrainMinCoordinate:(CGFloat)proposedMin ofSubviewAt:(NSInteger)offset
{
    NSView *subview = [[sender subviews] objectAtIndex:offset];
    NSRect subviewFrame = subview.frame;
    CGFloat frameOrigin;
    if ([sender isVertical])
    {
        frameOrigin = subviewFrame.origin.x;
    }
    else
    {
        frameOrigin = subviewFrame.origin.y;
    }
    
    CGFloat minimumSize =
        [[lengthsByViewIndex objectForKey:[NSNumber numberWithInteger:offset]]
            doubleValue];
    
    return frameOrigin + minimumSize;
}

The splitView:constrainMaxCoordinate:ofSubviewAt: is similar. You can download the sample project to see the implementation of this method.

Finally, we need to handle the priority resizing itself. This happens in the implementation of the delegate method splitView:resizeSubviewsWithOldSize:. This method is invoked when the split view is resized (normally because the enclosing window has resized).

A brief description of the work involved is:

  1. Iterate over the list of views, sorted by priority.
  2. As each view is reached, attempt to apply the entire size change to this view.
  3. If applying the size to the view would cause it to become smaller than its minimum size, apply as much as possible and proceed to the next view by priority.

The size change for the split view is named delta in the following code taken from the splitView:resizeSubviewsWithOldSize: method.

for (NSNumber *priorityIndex in
    [[viewIndicesByPriority allKeys] sortedArrayUsingSelector:@selector(compare:)])
{
    NSNumber *viewIndex = [viewIndicesByPriority objectForKey:priorityIndex];
    NSInteger viewIndexValue = [viewIndex integerValue];
    if (viewIndexValue >= subviewsCount)
    {
        continue;
    }
    
    NSView *view = [subviews objectAtIndex:viewIndexValue];
    
    NSSize frameSize = [view frame].size;
    NSNumber *minLength = [lengthsByViewIndex objectForKey:viewIndex];
    CGFloat minLengthValue = [minLength doubleValue];
    
    if (isVertical)
    {
        frameSize.height = sender.bounds.size.height;
        if (delta > 0 ||
            frameSize.width + delta >= minLengthValue)
        {
            frameSize.width += delta;
            delta = 0;
        }
        else if (delta < 0)
        {
            delta += frameSize.width - minLengthValue;
            frameSize.width = minLengthValue;
        }
    }
    else
    {
        frameSize.width = sender.bounds.size.width;
        if (delta > 0 ||
            frameSize.height + delta >= minLengthValue)
        {
            frameSize.height += delta;
            delta = 0;
        }
        else if (delta < 0)
        {
            delta += frameSize.height - minLengthValue;
            frameSize.height = minLengthValue;
        }
    }
    
    [view setFrameSize:frameSize];
    viewCountCheck++;
}

The "continue" skips invalid priorities. You might replace this in your own code with an NSAssert instead, depending on how you like to handle minor errors.

The other point to notice is that we don't break out of the loop once the delta is fully applied — we still need to run setFrameSize: on each view to apply any size change in the perpendicular direction (vertically for columns or horizontally for rows).

This fragment doesn't show it but the code includes a second iteration over all the views, in order, which sets the origins of each view following the resize so they are all positioned correctly for their new sizes.

Hitting the minimum size

If all views are at their minimum and the split view cannot contract any further, the current implementation throws an exception giving the minimum size. This is so that you can configure the containing view (often a window) to respect this minimum and never try to make the NSSplitView smaller than this. If you don't like this behavior, you can remove the NSAssert3 statement in the setMinimumLength:forViewAtIndex: method.

Conclusion

You can download the sample project ColumnSplitView.zip (60kb) to see the full PrioritySplitViewDelegate class.

The advantage to the PrioritySplitViewDelegate class is that it is generic: you don't need to write this code each time and it handles the common case of using an NSSplitView to contain columns and a main view. It offers an easy plug in solution for managing a split view in this arrangement.

It could probably be improved by changing the setMinimumLength:forViewAtIndex: and setPriority:forViewAtIndex: methods to something that prevents you from providing the wrong values for indices or number of lengths or priorities but the NSAsserts in the splitView:resizeSubviewsWithOldSize: method will pick up the most critical errors you might make.