Advanced programming tips, tricks and hacks for Mac development in C/Objective-C and Cocoa.

Streaming and playing an MP3 stream

This week, I present a sample application that streams and plays an audio file from a URL on the iPhone or Mac. I'll show how the application was written by expanding upon Apple's AudioFileStreamExample, including a work-around for an Audio File Stream Services' crash bug when handling streaming MP3s.

Update (2009-06-17): I have written a new post detailing and fixing the problems in this implementation titled: Revisiting an old post: Streaming and playing an MP3 stream.

Introduction

Playing an MP3 stream seems like it should be a straightforward task that the Cocoa APIs would handle easily. Unfortunately, the normal approach to handling media (i.e. "Let Quicktime Handle It") fails here — a quick attempt to play streaming MP3s in Apple's QTKitPlayer example results in a few seconds of no response, followed by a "-2048" error.

Of course, there's a way to play an MP3 stream without using QTKit. I'll show you how and the final result will be a sample application that looks like this:

audiostreamer.png
Since I link to their stream by default in the application, I should probably point out that Triple J is an Australian radio station.

You can download:

Update (2009-06-17): The location of the code has changed. The new, updated version of the code is now located at AudioStreamer (you can also browse the source code repository). The same repository includes both iPhone and Mac versions.

AudioToolbox

There is probably a way to make QTKit play streaming MP3s. I decided to go a different way instead.

In Mac OS X 10.5 (Snowless Leopard), Apple introduced the AudioToolbox framework which contains the Audio File Stream Services and Audio Queue Services that we'll use to solve the problem. These are pure C APIs: not as clean and simple to use as the Cocoa Objective-C APIs, but once written, should get the job done.

Audio File Stream reads the raw bytes and finds audio packets within them. The Audio Queue takes packets and plays them on the sound hardware. Between the two, they should handle streaming playback.

AudioFileStreamExample

At the moment, the AudioToolbox doesn't have any beginner-friendly "Guide" documentation. The only detailed introduction is the afsclient.cpp file in the AudioFileStreamExample that Apple provide (you'll find it in /Developer/Examples/CoreAudio/Services/AudioFileStreamExample).

Sadly, this example is missing a few things to make it work as a proper player:

  • Doesn't wait for audio to finish (the program quits when the data finishes loading, not when it finishes playing)
  • Never plays the final audio buffer
  • Only plays variable bit-rate data
  • Doesn't provide "hints" about the data format (many file types won't be recognized)

Addressing the issues

Waiting until the playback is finished

Immediately after the Audio Queue is created in the MyPropertyListenerProc, you can add a listener to the kAudioQueueProperty_IsRunning property of the Audio Queue. This will allow us to determine when playback has started and (more importantly) when playback has properly finished.

// listen to the "isRunning" property
err = AudioQueueAddPropertyListener(myData->audioQueue, kAudioQueueProperty_IsRunning, MyAudioQueueIsRunningCallback, myData);
if (err) { PRINTERROR("AudioQueueAddPropertyListener"); myData->failed = true; break; }

With this in place, we can implement the MyAudioQueueIsRunningCallback function and use it to wait until the audio has finished playing before we exit the program.

The documentation doesn't point it out but the MyAudioQueueIsRunningCallback function will not be called when the audio stops unless the thread from which the stop was issued has a run loop (e.g. call CFRunLoopRunInMode in a loop while waiting for completion).
Play the final audio buffer

This is a simple problem. All that is needed is to call the MyEnqueueBuffer function once more, after the data has finished loading, to ensure that the buffer in progress is sent to the Audio Queue. It may help to flush the queue as well.

MyEnqueueBuffer(myData);

err = AudioQueueFlush(myData->audioQueue);
if (err) { PRINTERROR("AudioQueueFlush"); return 1; }
Handle CBR data too

The "for CBR data, you'd need another code branch here" comment in the AudioFileStreamExample is a bit of a giveaway on this point. Basically, the code which follows that comment should be wrapped in an "if (inPacketDescriptions)" conditional, followed by an else that looks like this:

else
{
    // if the space remaining in the buffer is not enough for this packet, then enqueue the buffer.
    size_t bufSpaceRemaining = kAQBufSize - myData->bytesFilled;
    if (bufSpaceRemaining < inNumberBytes) {
        MyEnqueueBuffer(myData);
    }
    
    // copy data to the audio queue buffer
    AudioQueueBufferRef fillBuf = myData->audioQueueBuffer[myData->fillBufferIndex];
    memcpy((char*)fillBuf->mAudioData + myData->bytesFilled, (const char*)inInputData, inNumberBytes);

    // keep track of bytes filled and packets filled
    myData->bytesFilled += inNumberBytes;
    myData->packetsFilled = 0;
}

Straightforward stuff: you just copy all the data into the buffer, without needing to worry about packet sizes.

Hinting about data types

This is a bit more of an open ended problem. A few different approaches can work here:

  • Use file extensions to guess the file type
  • Use mime types provided in HTTP headers to determine the file type
  • Continuously invoke AudioFileStreamParseBytes on the first chunk of the file until it returns without an error
  • Hardcode the type, if you can presume it in all cases

I only implemented the first of these options. If you know the URL of the source file, it goes a little something like this:

AudioFileTypeID fileTypeHint = 0;
NSString *fileExtension = [[url path] pathExtension];
if ([fileExtension isEqual:@"mp3"])
{
    fileTypeHint = kAudioFileMP3Type;
}
// ... and so on for a range of other file types

Then you pass the fileTypeHint into the call to AudioFileStreamOpen.

Final nasty bug

After making all these changes, the Audio File Stream Services hit me with a nasty bug: AudioFileStreamParseBytes will crash when trying to parse a streaming MP3.

Of course, if you let bugs in other people's code discourage you, you won't get too far as a programmer. Even if the bug is truly in someone else's code (99% of the time the real cause is in your own code), there's often a way around the problem.

In this case, if we pass the kAudioFileStreamParseFlag_Discontinuity flag to AudioFileStreamParseBytes on every invocation between receiving kAudioFileStreamProperty_ReadyToProducePackets and the first successful call to MyPacketsProc, then AudioFileStreamParseBytes will be extra cautious in its approach and won't crash.

So, set a boolean named discontinuous in the myData struct to true after:

case kAudioFileStreamProperty_ReadyToProducePackets:

and set it to false again at the start of MyPacketsProc, then replace the call to AudioFileStreamParseBytes with:

if (myData->discontinuous)
{
    err = AudioFileStreamParseBytes(myData->audioFileStream, bytesRecvd, buf, kAudioFileStreamParseFlag_Discontinuity);
    if (err) { PRINTERROR("AudioFileStreamParseBytes"); myData->failed = true; break;}
}
else
{
    err = AudioFileStreamParseBytes(myData->audioFileStream, bytesRecvd, buf, 0);
    if (err) { PRINTERROR("AudioFileStreamParseBytes"); myData->failed = true; break; }
}

and all should be well.

Making a proper Cocoa application out of it

The final step was to take the reworked example and set it up as part of a proper Cocoa application. For this, I decided to further add the following:

  • Load the data over an NSURLConnection instead of a socket connection.
  • Handle the connection in a separate thread, so any potential blocking won't affect the user-interface.
  • Wrap the construction and invocation in an Objective-C class.
  • Make the isPlaying state an NSKeyValueObserving compliant variable so the user-interface can update to reflect the state.
  • Since the program always fills one buffer completely before audio starts, I halved the kAQBufSize to reduce waiting for audio to start.

I invite you to look at the AudioStreamer code in the sample application to see how this was done. It is fairly straightforward. Where possible, AudioStreamer keeps the code, style and approach of the AudioFileStreamExample. I don't advocate using so many boolean flags or public instance variables in normal situations.

Conclusion

Download: You can download the complete source code for this post AudioStreamer from Github (you can also browse the source code repository). The same repository includes both iPhone and Mac versions. This code includes the improvements from my later post Revisiting an old post: Streaming and playing an MP3 stream

The application works. Given the learning curve of a new API and the MP3 parsing bug, I'm fairly pleased I succeeded.

The program will handle other types of stream (not just MP3s) as well as non-streaming files downloaded over HTTP (although the player will block the download so that it only downloads at playback speed — this implementation doesn't cache ahead).

I think the biggest limitation in the implementation's current form is that it doesn't read mime types or aggressively try different file types, so URLs without a file extension may not work.

With regards to the bug in AudioFileStreamParseBytes, if this discussion thread is accurate, it appears that Apple have already fixed the MP3 parsing bug in the iPhone version of the function, so the fix should make it into an upcoming version of Mac OS X.

A Cocoa application driven by HTTP data

Here's a tiny application that queries a webpage via HTTP, parses and searches the response and presents the results in a neatly formatted window. In essence, it's what many Dashboard widgets and iPhone apps do but I'll show you how to do it in a regular Cocoa application.

Brought to you by FuelView, an iPhone application I wrote for fetching FuelWatch information in Western Australia.

Introduction

Most Dashboard widgets work by fetching data via HTTP and formatting the response for easy viewing. This type of widget does nothing your web browser can't do but because it's focussed on a specific goal, it can be faster and present a better experience within its more narrow bounds.

Similar but Cocoa

This fetch/parse/search/present behavior need not be limited to widgets. I'll show you how you can replicate this in a Cocoa application. The example I'll present will show the "New to the store" items from the Apple Store.

Here's a screenshot of this application in action:

newtothestore.png

The data presented in the application's window comes from the list of "New to the store" items on the http://store.apple.com webpage, as circled in the following screenshot:

storewebpage.png  

The steps involved

As I've already stated, this type of web retrieval application requires the following steps:

  • Fetch over HTTP
  • Parse the response into a structured format
  • Search the parsed response and extract the desired information
  • Format and present to the user

Fetch over HTTP

NSURLConnection handles data transfer over HTTP in Cocoa. All you need to do is give it a URL to fetch and it make it happen.

You can drive an NSURLConnection synchronously using sendSynchronousRequest:returningResponse:error: (which admittedly takes less code than what I'm about to show you) but that will block the entire thread until the response is received (almost always a bad idea).

For a class with an NSMutableData member named responseData and an NSURL member named baseURL, here's how to fetch the page at http://store.apple.com into that data member asynchronously:

    responseData = [[NSMutableData data] retain];
    baseURL = [[NSURL URLWithString:@"http://store.apple.com"] retain];

    NSURLRequest *request =
        [NSURLRequest requestWithURL:[NSURL URLWithString:@"http://store.apple.com"]];
    [[[NSURLConnection alloc] initWithRequest:request delegate:self] autorelease];

For this asynchronous approach to work, the delegate object (self in this case) must also implement the following methods:

- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
    [responseData setLength:0];
}

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
    [responseData appendData:data];
}

- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error
{
    [[NSAlert alertWithError:error] runModal];
}

- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
    // Once this method is invoked, "responseData" contains the complete result
}

In addition to this required NSURLConnection functionality, the sample app implements another method here to track what the final URL will be (since redirections may occur).

- (NSURLRequest *)connection:(NSURLConnection *)connection
    willSendRequest:(NSURLRequest *)request
    redirectResponse:(NSURLResponse *)redirectResponse
{
    [baseURL autorelease];
    baseURL = [[request URL] retain];
    return request;
}

Having the correct base URL for the webpage will allow us to perform relative to absolute URL conversion below if needed.

Parsing and Searching an HTML document

The response from an HTTP request is normally HTML — a long chunk of text. In order to make useful sense of this, you'll want to use the right tools to extract useful information.

There are lots of examples of people using text searching and regular expressions to find data in webpages. These examples are doing it wrong.

NSXMLDocument and an XPath query are your friends. They really make finding elements within a webpage, RSS feed or XML documents very easy.

The following code is inserted in the body of the connectionDidFinishLoading: method shown above:

    NSError *error;
    NSXMLDocument *document =
        [[NSXMLDocument alloc] initWithData:responseData options:NSXMLDocumentTidyHTML error:&error];
    
    // Deliberately ignore the error: with most HTML it will be filled with
    // numerous "tidy" warnings.
    
    NSXMLElement *rootNode = [document rootElement];
    
    NSString *xpathQueryString =
        @"//div[@id='newtothestore']/div[@class='modulecontent']/div[@id='new-to-store']/div[@class='list_content']/ul/li/a";
    NSArray *newItemsNodes = [rootNode nodesForXPath:xpathQueryString error:&error];
    if (error)
    {
        [[NSAlert alertWithError:error] runModal];
        return;
    }

The XPath query is the core of the search operation. Simple, yet incredibly adept at extracting data from a structured document.

This query here looks for all the "a" tags (HTML links) in the list under the "newtothestore" div.

Format and present to the user

In the sample application, the table view at the left of the window is populated from the NSArray named newItems on the NewItemsClient object (to which all methods discussed so far belong). Since the user interface has been configured in Interface Builder to handle this populating automatically, all that needs to be done is to update newItems in a Key-Value-Observing compliant fashion.

This code fragment should be inserted immediately below the previous code fragment:

    [self willChangeValueForKey:@"newItems"];
    [newItems release];
    newItems = [[NSMutableArray array] retain];
    for (NSXMLElement *node in newItemsNodes)
    {
        NSString *relativeString = [[node attributeForName:@"href"] stringValue];
        NSURL *url = [NSURL URLWithString:relativeString relativeToURL:baseURL];
        
        NSString *linkText = [[node childAtIndex:0] stringValue];
        
        [newItems addObject:
            [NSDictionary dictionaryWithObjectsAndKeys:
                [url absoluteString], @"linkURL",
                linkText, @"linkText",
                nil]];
    }
    [self didChangeValueForKey:@"newItems"];

The final result has the newItems array filled with absolute URLs and link text from the items in the "New to the store" list.

The user interface, as assembled in Interface Builder, handles the details of displaying the "linkText" from each item. A connector object tracks selection changes in the table and displays the "linkURL" of the selected item in the WebView.

You can download the complete XCode project for the example presented. It's XCode 3.1 but should also load in 3.0 (with a warning you can ignore).

Conclusion

I hope I've shown that the Dashboard widget niche of "small applications driven by web-retrieved data" is quickly and simply achievable in Cocoa.

The example fetched from HTML but the techniques would be similar or identical for RSS or general XML data fetched over HTTP.

Drawing gloss gradients in CoreGraphics

This post presents a function &mdash DrawGlossGradient(CGContextRef context, NSColor *color, NSRect inRect) &mdash that will draw a "gloss" gradient in a single statement. All colors in the gradient are calculated from the single color parameter.

Introduction

The following samples are gloss gradients. They are all made by the DrawGlossGradient function described below.

gradientSample.png

This type of gradient is common on buttons or other graphical adornments on webpages. The "aqua" aesthetic of Mac OS X also uses this type of gradient in numerous places.

The gradient is actually composed of a number of components, all intended to simulate a translucent glass or plastic lens-shaped object that is lit from above.

A diagram of the physical structure being modelled would look like this:

glossDiagram.png

The light gray "lens" shape is the glass or plastic translucent object being modelled.

The top half, as seen by the viewer, is dominated by the arc labelled "B" which is light from the light-source reflected directly to the viewer.

The bottom half, as seen by the viewer, contains the effects of two arcs: C and A. Arc C is a "caustic" highlight, where the light from the light source is focussed to a higher intensity by the lens shape of the translucent material. Arc A is darker because the recessed nature of the translucent material casts a shadow over this area.

The final point to note is that the lens shape is not flat at the back, so these light and dark components attenuate in a non-linear fashion.

Creating the effect in code

We need four different color values:

  • The top of the gloss highlight (whitest due to incident angle of reflection)
  • The bottom of the gloss highlight (white-ish but not as white as top)
  • The background color - darkest visible part of shadow (will be provided as input to the function)
  • The caustic color (brighter than background and incorporating a subtle hue change)

Once we have these values, we can simply create a gradient out of them.

I'm going to use a CoreGraphics CGShadingRef. It would be possible to produce a fairly similar effect using an NSGradient but that class only handles constant-slope gradients and I want to incorporate a subtle exponential in the gradients.

The gloss highlight color

The two gloss highlight colors will just be a blend of white and the background color. Picking relative intensities of these two colors is not very hard. I chose a 0.6 fraction of white for my top gloss color and a 0.2 for the bottom of the gloss (although these fractions will be reduced by the scaling below).

When using a range of background colors, I found that dark colors needed a smaller fraction of white than light colors to appear similarly glossy, so I had to scale the effect based on background brightness.

I chose the following function to produce a scaling coefficient for my gloss brightness, based on brightness of the background color:

float perceptualGlossFractionForColor(float *inputComponents)
{
    const float REFLECTION_SCALE_NUMBER = 0.2;
    const float NTSC_RED_FRACTION = 0.299;
    const float NTSC_GREEN_FRACTION = 0.587;
    const float NTSC_BLUE_FRACTION = 0.114;

    float glossScale =
        NTSC_RED_FRACTION * inputComponents[0] +
        NTSC_GREEN_FRACTION * inputComponents[1] +
        NTSC_BLUE_FRACTION * inputComponents[2];
    glossScale = pow(glossScale, REFLECTION_SCALE_NUMBER);
    return glossScale;
}

The input components are 3 floats (RGB). The coefficients with which I multiply them are the NTSC color-to-luminance conversion coefficients. It's an acceptable "perceptual brightness" conversion for color and far easier than RGB to LUV. I then raise this value to a fractional power — value chosen experimentally as it seemed to give about the right final value across the range of brightnesses.

The caustic highlight color

The caustic color is a harder problem. We need to achieve a hue and brightness shift of the background color towards yellow, while retaining the background's saturation.

Again, as with gloss, there was a non-linearity to account for: colors further in hue from yellow required proportionally less hue shift to maintain the appearance of the same hue-shift effect. I chose to scale the hue shift by a cosine such that the hue shift seemed perceptually appropriate.

In addition, grays (having no real hue) need special handling. Reds needed special handling to account for the fact that hue wraps around at red. Purples didn't really look good hued towards yellow, so I decided to make them hue towards magenta.

void perceptualCausticColorForColor(float *inputComponents, float *outputComponents)
{
    const float CAUSTIC_FRACTION = 0.60;
    const float COSINE_ANGLE_SCALE = 1.4;
    const float MIN_RED_THRESHOLD = 0.95;
    const float MAX_BLUE_THRESHOLD = 0.7;
    const float GRAYSCALE_CAUSTIC_SATURATION = 0.2;
    
    NSColor *source =
        [NSColor
            colorWithCalibratedRed:inputComponents[0]
            green:inputComponents[1]
            blue:inputComponents[2]
            alpha:inputComponents[3]];

    float hue, saturation, brightness, alpha;
    [source getHue:&hue saturation:&saturation brightness:&brightness alpha:&alpha];

    float targetHue, targetSaturation, targetBrightness;
    [[NSColor yellowColor] getHue:&targetHue saturation:&targetSaturation brightness:&targetBrightness alpha:&alpha];
    
    if (saturation < 1e-3)
    {
        hue = targetHue;
        saturation = GRAYSCALE_CAUSTIC_SATURATION;
    }

    if (hue > MIN_RED_THRESHOLD)
    {
        hue -= 1.0;
    }
    else if (hue > MAX_BLUE_THRESHOLD)
    {
        [[NSColor magentaColor] getHue:&targetHue saturation:&targetSaturation brightness:&targetBrightness alpha:&alpha];
    }

    float scaledCaustic = CAUSTIC_FRACTION * 0.5 * (1.0 + cos(COSINE_ANGLE_SCALE * M_PI * (hue - targetHue)));

    NSColor *targetColor =
        [NSColor
            colorWithCalibratedHue:hue * (1.0 - scaledCaustic) + targetHue * scaledCaustic
            saturation:saturation
            brightness:brightness * (1.0 - scaledCaustic) + targetBrightness * scaledCaustic
            alpha:inputComponents[3]];
    [targetColor getComponents:outputComponents];
}

So this function is really just an HSV conversion of the inputComponents and the yellowColor, and the blending of the two.

Composing into a single gradient

Now to assemble the colors into a gradient. We'll need to implement an interpolation function that will return the correct color for a given progress point in the gradient.

With the aforementioned "background color", "caustic color", "top gloss white fraction" and "bottom gloss white fraction" passed into this function as the "color", "caustic", "initialWhite" and "finalWhite" parameters of the GlossParameters struct, the function looks like this:

typedef struct
{
    float color[4];
    float caustic[4];
    float expCoefficient;
    float expScale;
    float expOffset;
    float initialWhite;
    float finalWhite;
} GlossParameters;

static void glossInterpolation(void *info, const float *input,
    float *output)
{
    GlossParameters *params = (GlossParameters *)info;

    float progress = *input;
    if (progress < 0.5)
    {
        progress = progress * 2.0;

        progress =
            1.0 - params->expScale * (expf(progress * -params->expCoefficient) - params->expOffset);

        float currentWhite = progress * (params->finalWhite - params->initialWhite) + params->initialWhite;
        
        output[0] = params->color[0] * (1.0 - currentWhite) + currentWhite;
        output[1] = params->color[1] * (1.0 - currentWhite) + currentWhite;
        output[2] = params->color[2] * (1.0 - currentWhite) + currentWhite;
        output[3] = params->color[3] * (1.0 - currentWhite) + currentWhite;
    }
    else
    {
        progress = (progress - 0.5) * 2.0;

        progress = params->expScale *
            (expf((1.0 - progress) * -params->expCoefficient) - params->expOffset);

        output[0] = params->color[0] * (1.0 - progress) + params->caustic[0] * progress;
        output[1] = params->color[1] * (1.0 - progress) + params->caustic[1] * progress;
        output[2] = params->color[2] * (1.0 - progress) + params->caustic[2] * progress;
        output[3] = params->color[3] * (1.0 - progress) + params->caustic[3] * progress;
    }
}

As you can see, the function is split into two halves: the first half handles the gloss and the second half handles the caustic. An exponential is used to create an attenuating effect on the gradient.

Draw the gradient

The draw function is pretty straightforward. Most of it is configuring the GlossParameters struct with the coefficient and offsets of the exponential, invoking the functions to generate the required colors and performing the mechanics of drawing a gradient using CGShadingCreateAxial and CGContextDrawShading.

void DrawGlossGradient(CGContextRef context, NSColor *color, NSRect inRect)
{
    const float EXP_COEFFICIENT = 1.2;
    const float REFLECTION_MAX = 0.60;
    const float REFLECTION_MIN = 0.20;
    
    GlossParameters params;
    
    params.expCoefficient = EXP_COEFFICIENT;
    params.expOffset = expf(-params.expCoefficient);
    params.expScale = 1.0 / (1.0 - params.expOffset);

    NSColor *source =
        [color colorUsingColorSpaceName:NSCalibratedRGBColorSpace];
    [source getComponents:params.color];
    if ([source numberOfComponents] == 3)
    {
        params.color[3] = 1.0;
    }
    
    perceptualCausticColorForColor(params.color, params.caustic);
    
    float glossScale = perceptualGlossFractionForColor(params.color);

    params.initialWhite = glossScale * REFLECTION_MAX;
    params.finalWhite = glossScale * REFLECTION_MIN;

    static const float input_value_range[2] = {0, 1};
    static const float output_value_ranges[8] = {0, 1, 0, 1, 0, 1, 0, 1};
    CGFunctionCallbacks callbacks = {0, glossInterpolation, NULL};
    
    CGFunctionRef gradientFunction = CGFunctionCreate(
        (void *)&params,
        1, // number of input values to the callback
        input_value_range,
        4, // number of components (r, g, b, a)
        output_value_ranges,
        &callbacks);
    
    CGPoint startPoint = CGPointMake(NSMinX(inRect), NSMaxY(inRect));
    CGPoint endPoint = CGPointMake(NSMinX(inRect), NSMinY(inRect));

    CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
    CGShadingRef shading = CGShadingCreateAxial(colorspace, startPoint,
        endPoint, gradientFunction, FALSE, FALSE);
    
    CGContextSaveGState(context);
    CGContextClipToRect(context, NSRectToCGRect(inRect));
    CGContextDrawShading(context, shading);
    CGContextRestoreGState(context);
    
    CGShadingRelease(shading);
    CGColorSpaceRelease(colorspace);
    CGFunctionRelease(gradientFunction);
}

Conclusion

There are lots of parameters in these functions that can be tweaked to personal preference. You can enhance the hue change, the gradient slopes and the gloss intensity very simply.

I'm fairly happy with the gloss and its brightness. I think that's worked well.

The hue shift for the caustic works well but the brightness of the caustic seems a little inconsistent. This could be tweaked a little.

Purples on the boundary with blue or red can look a little strange. Maybe there's a way to smooth this, I don't know. I haven't really thought about it.

This approach doesn't work well with bright colors provided as the input color, since the input color is used as the darkest color in the gradient. This is not a problem as much as it is a consideration when choosing the input color.

Parametric acceleration curves in Core Animation

CAMediaTimerFunction is used to control basic acceleration along a path in Core Animation but is very limited in what in can do. In this post, I look at the mathematics behind CAMediaTimerFunction and present a sample application that simulates some functions that CAMediaTimerFunction can't achieve by using parametric CAKeyframeAnimation values.

Introduction

If you animate an object along a straight line in Core Animation, you can have the object move at a constant speed along the whole path or you can specify that the object accelerate up to speed at the beginning and decelerate to a halt at the end (ease-in/ease-out). The class CAMediaTimerFunction specifies this behaviour.

If you want something quite different, like exponential slow-down along the path, then sadly, CAMediaTimerFunction can't do that. Furthermore, you can't subclass CAMediaTimerFunction to modify its behavior. We can still perform exponential slow-down in Core Animation but we'll need to do it a different way.

mediatimercapabilities.png

These graphs are screenshots from the "AnimationAcceleration" sample app (described below). It can handle all of these acceleration types.

The mathematics of ease-in/ease-out animation

In Core Animation, the animation classes derived from CAAnimation allow you to set the timingFunction (a CAMediaTimerFunction object). The primary purpose of this property is to allow you to control the "smoothness" of the animation at the endpoints (linear speed or ease-in/ease-out).

CAMediaTimerFunction uses a cubic Bézier to convert input time values into output time values. The documentation doesn't make it explicitly clear how the Bézier is used to map time values. So let's look at that first.

A cubic Bézier is described by the parametric equation:

F(t) = (1 - t)3P0 + 3t(1 - t)2P1 + 3t2(1 - t)P2 + t3P3

where the Bézier is specified by the four control points P0, P1, P2 and P3 (each of which is a pair of coordinates — an X and a Y coordinate) and the parameter "t" moves from zero at P0 to one at P3 (i.e. t ∈ [0, 1]). The whole function produces an output point F for every t value (where F contains an X and a Y coordinate like each of the P values).

CAMediaTimerFunction does not use the input time as the "t" value in this equation. Instead, input time is used as the X value of F and the output time is obtained by solving for the Y value of F at the same "t" point. This is more flexible than using "t" as the input value but is more computationally complex since you must solve the cubic.

Things CAMediaTimerFunction can't do

Since it uses a cubic Bézier, CAMediaTimerFunction can only provide time mappings that can be described by a 3rd-order polynomial.

This means the following are achievable:

  • constant slope mappings
  • quadratic mappings (parabolic)
  • cubic mapping (including simple "s"-shaped curves)

but the following can't be achieved:

  • exponentials
  • sine waves
  • polynomials higher than 3rd order

Solution: CAKeyframeAnimation

The CAKeyframeAnimation class lets us specify every single point along an animation path. If we generate enough keyframes, we can model any equation we want. In this way, we can overcome the limitations of CAMediaTimerFunction — we use flat linear time but distribute the keyframes to achieve the same effect.

To solve the problem, we'll need to recreate the parametric mapping nature of CAMediaTimerFunction but using any arbitrary mapping function that we choose.

Sample app: AnimationAcceleration

Download the "AnimationAcceleration" sample app. It is an XCode 3.1 project but should load correctly XCode 3.0 or later.

The sample application shows uses parametric functions to generate keyframes so that the red dot moves along its linear path (the vertical axis at the left of screen) at different rates of acceleration. The gray dot at the bottom moves at a constant speed, marking out time.

animationaccelerationscreenshot.png

The acceleration curves used include

  • Linear
      Achievable using CAMediaTimerFunction.
  • Ease-in/Ease-out
      I've implemented this using a cubic Bézier mapping from "t" to X (to contrast with CAMediaTimerFunction's X of F to Y of F mapping). This acceleration uses the maximum amount of "t" to X ease-in/ease-out acceleration possible but is still weak compared with X to Y mapping — demonstrating why Apple went through the extra effort to provide X to Y mapping instead.
  • Quadratic
      Achievable using CAMediaTimerFunction.
  • Exponential Decay
      Cannot be achieved using CAMediaTimerFunction.
  • Second-order Response Curve
      Technically, this curve models the voltage response of a second order circuit to a step change in voltage. I like it because it also produces a spring-like oscillation around the destination similar to damped harmonic motion. Since this curve actually overshoots the destination, it demonstrates a result that would be impossible in a bounded mapping function like CAMediaTimerFunction, even if subclassing CAMediaTimerFunction was possible.

In the program, each of the curves is described parametrically by an NSObject<Evaluate> object which implements the evaluateAt: method. The CAKeyframeAnimation subclass AccelerationAnimation then implements the following method to generate all the keyframe values using the results from this method.

- (void)calculateKeyFramesWithEvaluationObject:(NSObject *)evaluationObject
    startValue:(double)startValue
    endValue:(double)endValue
    interstitialSteps:(NSUInteger)steps
{
    NSUInteger count = steps + 2;
    
    NSMutableArray *valueArray = [NSMutableArray arrayWithCapacity:count];

    double progress = 0.0;
    double increment = 1.0 / (double)(count - 1);
    NSUInteger i;
    for (i = 0; i < count; i++)
    {
        double value =
            startValue +
            [evaluationObject evaluateAt:progress] * (endValue - startValue);
        [valueArray addObject:[NSNumber numberWithDouble:value]];
        
        progress += increment;
    }
    
    [self setValues:valueArray];
}

The keyframe values generated by this method will work for single property animation (in this case, the "position.y" coordinate of the red dot). If you wanted to animate in two dimension (e.g. "position" instead of "position.y"), then you could implement a similar method that accepted startValue and endValue as NSPoints and performed similar parametric interpolation for the X and Y coordinates.

This animation is applied to the acceleratedDot (the red dot) layer as follows:

[CATransaction begin];
[CATransaction
    setValue:[NSNumber numberWithFloat:2.5]
    forKey:kCATransactionAnimationDuration];
AccelerationAnimation *animation =
    [AccelerationAnimation
        animationWithKeyPath:@"position.y"
        startValue:[self originPoint].y
        endValue:[self maxYPoint].y
        evaluationObject:[currentConfiguration objectForKey:@"evaluator"]
        interstitialSteps:INTERSTITIAL_STEPS];
[animation setDelegate:self];
[[acceleratedDot layer]
    setValue:[NSNumber numberWithDouble:[self maxYPoint].y]
    forKeyPath:@"position.y"];
[[acceleratedDot layer] addAnimation:animation forKey:@"position"];
[CATransaction commit];

where the originPoint and maxYPoint methods return the two endpoints of the red dot's path. The destination point is applied using setValue:forKeyPath: so that the object will remain at the destination after the AccelerationAnimation completes.

Conclusion

Core Animation is heavily geared towards implicit animations that are simple point-to-point transitions — not a bad thing since this is the overwhelmingly common case.

Explicit animations along elaborate paths (in this case, elaborate along the time dimension) take significantly more code. You also must choose how many keyframes are required for a smooth path. However, parametric animation using CAKeyframeAnimation opens up any acceleration curve you want and you still gain the benefits of the rest of Core Animation (CALayer, separate animation thread, NSView integration, etc).