CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation 95 Figure 4-10. The two bumpers were created Figure 4-11. The bumpers’ class was then by simply dragging the images from the changed from UIImageView to GadgetView Interface Builder Media library into the using the Identity Inspector. document window. We don’t process any gestures for three or more fingers, so we filter them out early. There’s no reason you couldn’t add your own processing here, at least, no technical reason. Whether it’s advisable or not from a user-experience perspective is another question. So, this is how we handle a two-finger gesture: else if ([allTouches count]==2) { NSArray* twoTouches = [allTouches allObjects]; NSSet is unordered; we get the touches into an array, which doesn’t sort it for us, but at least we can address each touch individually even if we don’t know which is which yet. lastPinch = [self calculatePinch:twoTouches]; lastRotation = [self calculateAngle:twoTouches]; lastCenter = [self averageTouchPoint:twoTouches]; We’ll look at the implementations of these methods later; the important thing is that we’re initializing three tracking variables: a floating-point number to track the distance between the fingers, another to track their relative angle, and a CGPoint to track the midpoint between them. CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation96 We'll also need to track whether we’ve locked the interface into a particular gesture. When performing two-finger drags we’ll set the modeLock to either lockToXAxis or lockToYAxis, and when interacting with the mode-locked gadget, we may set it to lockToRotation or lockToScale. Until then, it defaults to lockNotYetChosen: if (modeLock==lockNotYetChosen) { axisLockedDrag = YES; originalCenter = lastCenter; } Until we get some sign that the user’s performing a more specialized gesture, we can only assume that two fingers on the screen could mean a two-finger drag. We’ll initialize for this, but note the test we do first, as it’s a subtle but important one: modeLock is set once a ges- ture has been detected and locked into place—and only cleared when all fingers involved in the gesture have left the screen. So, this test means you can temporarily lift and replace one finger and not reset the gesture. It’s particularly important, because it’s very easy for a finger to brush over the edge of the display when dragging or flicking, and the iPhone will register this as the touch ending. We need to account for this situation; otherwise, we’ll get really unpleasant behavior near the edge of the screen. That’s all for two-finger gestures, we just need to handle the simple one-finger case now: } else { UITouch* touch = [allTouches anyObject]; We just grab the only touch there is, and it turns out we don’t actually need to initialize any state for single-finger drag operations. We just check for double-taps, which reset the gad- get to its default size (stored earlier, in awakeFromNib, not shown here): if (touch.tapCount==2) { CGPoint ctr = self.center; CGRect bounds = self.bounds; bounds.size.width = defaultSize; bounds.size.height = defaultSize; self.bounds = bounds; self.center = ctr; } } } CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation 97 Deciding What Movement Means What about when a finger moves? Just like before, we get all the view-specific touches and see how many there are to determine our response. The handling for each case is more com- plex, though: - (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event { NSSet* allTouches = [event touchesForView:self]; if ([allTouches count]==1) { if (modeLock>lockNotYetChosen) return; This test is another part of dealing with finger-ups at the edge of the screen: if one of the fingers in a two-finger gesture temporarily loses contact with the screen, we don’t want to revert to single-finger drag behavior, so we check and lock the behavior accordingly. But, if we’re just doing a regular drag, the code is pretty simple. Touches remember their last position as well as their current one, so we just get both these positions and subtract one from the other to get the finger’s relative movement: UITouch* anyTouch = [touches anyObject]; lastMove = anyTouch.timestamp; CGPoint now = [anyTouch locationInView: self.superview]; CGPoint then = [anyTouch previousLocationInView: self.superview]; dragDelta = CGPointDelta(now, then); self.center = CGPointApplyDelta(self.center, dragDelta); [self stopTimer]; } Notice that we don’t get the touch’s location in the current view! This is because we’re mov- ing the view itself, which means the coordinate system would be continuously changing and drive us nuts. I mean, since we’re dragging the view along with the finger, if we’re doing our job right, the finger shouldn’t move relative to the view at all! So, we find the finger’s posi- tion in the superview instead, which gives us a fixed frame of reference. This technique applies to any animated view, although bear in mind that for many animated views, you’ll want to take the simpler approach of just disabling interaction with a view while it’s animating. We also record when the touch occurred, as we’ll need this later. Once we’ve done all this, all we need to do is update the graphics to match. We also stop any background animation that’s occurring on this view (of which more later in the section “Applying Weight and Iner- tia”) since the user’s finger movement takes priority. CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation98 Now for the fun, two-finger gestures. We start the usual way, by collecting together all the data we may need: else if ([allTouches count]==2) { NSArray* twoTouches = [allTouches allObjects]; float pinch = [self calculatePinch:twoTouches]; float rotation = [self calculateAngle:twoTouches]; CGPoint touchCenter = [self averageTouchPoint:twoTouches]; CGSize delta = CGPointDelta(touchCenter, lastCenter); CGPoint gadgetCenter = self.center; This block is for the first few frames of the gesture, before we know what kind it is. It’s going to keep an eye out for clues as to the user’s intent. if (axisLockedDrag && (modeLock==lockNotYetChosen)) { if (fabsf(pinch-lastPinch)>PINCH_THRESHOLD) { axisLockedDrag = NO; if (modal) modeLock = lockToScale; } else if (fabsf(rotation-lastRotation)>ROTATION_THRESHOLD) { axisLockedDrag = NO; if (modal) modeLock = lockToRotation; } If the fingers get closer together or further apart, or rotate relative to each other, we don’t have a two-fingered drag (which assumes constant relative position between the fingers). Of course, a little wobble is to be expected, so the relative motion has to exceed a threshold. If this happens, we disable dragging. One of the gadgets, if you remember, allows free-form editing (you can scale and rotate at the same time), but the other locks itself to whichever happens first. If the fingers are moving together, we’re still doing a drag operation, but again, we check against thresholds. Once we’re sure the user has a particular axis in mind, we lock to it: else { CGSize dragDistance = CGPointDelta(touchCenter, £ originalCenter); if (fabsf(dragDistance.width)>AXIS_LOCK_THRESHOLD) { modeLock = lockToXAxis; CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation 99 } else if (fabsf(dragDistance.height)>AXIS_LOCK_THRESHOLD) { modeLock = lockToYAxis; } } } Applying the Movement Now we know what’s going on! It’s just a matter of putting the appropriate gesture into effect, depending on which we’ve detected: if (axisLockedDrag) { switch(modeLock) { case lockToXAxis: delta.height = 0; gadgetCenter.y = Interpolate(gadgetCenter.y, originalCenter.y, 0.1f); break; case lockToYAxis: delta.width = 0; gadgetCenter.x = Interpolate(gadgetCenter.x, originalCenter.x, 0.1f); break; } self.center = CGPointApplyDelta(gadgetCenter, delta); } The axis-locked drag is the same as the single-finger drag from before, except we cancel out the motion on whichever axis we haven’t locked to. Also, because we don’t know which axis to lock to until the user starts moving that way, we allowed some free-form dragging at first. We need to cancel this out, but simply snapping the gadget into place is jarring. We linearly interpolate it back instead. Another approach is to simply refuse to move the gadget at all until an axis has been locked in. This method is less responsive, so the user will perceive a need to pull harder on the gadget before it will start moving, but the interface can feel more stable or even slightly magnetic, like a MagSafe power adapter. Here, we start to handle the various transformations that can be applied to the gadget: else { if (modeLock!=lockToScale) CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation100 { CGAffineTransform transform = self.transform; transform = CGAffineTransformRotate(transform, rotation-lastRotation); self.transform = transform; } Note that rather than testing for the current lock, we apply any transformations that aren’t locked out, since in free-form mode several transformations may be applied at once. if (modeLock!=lockToRotation) { float scale = pinch/lastPinch; CGRect bounds = self.bounds; bounds.size.width *= scale; bounds.size.height *= scale; self.bounds = bounds; } There’s one more thing to take care of in free-form mode: when you’re rotating and scaling a gadget, you may not move both fingers evenly. In fact, you can slip-and-slide all over the place, and because we’re only tracking relative motion, the control will still work. But it’ll look weird. If we update the position of the gadget to the midpoint of the fingers, it’ll maintain the illusion that we’re manipulating a physical object: if (modeLock==lockNotYetChosen) self.center = CGPointApplyDelta(self.center, delta); Finally, update some of the tracking state, so that we make the comparisons against the right values next time round. And we’re done! lastPinch = pinch; lastRotation = rotation; } lastCenter = touchCenter; } } Applying Weight and Inertia We’ve now implemented all the gestures, but there’s something missing. I mentioned earlier the importance of weight and inertia, and it’s time to put it into practice. If we flick an object, it should keep going. All the work for this is done at the moment the finger is lifted, starting off as usual by acquiring the current touches. Remember, at this stage, the touches being removed from the screen are still included. We put into practice the standard check for all fingers being CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation 101 removed: if any fingers still touch the view, it’s considered as being gripped, and we leave it alone. - (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event { NSSet* allTouches = [event touchesForView:self]; if ([touches count]==[allTouches count]) { modeLock = lockNotYetChosen; The gadget’s been released. The obvious thing is to cancel the lock at this point, but before we can implement inertia, we must check when was the gadget last moved: if ((event.timestamp - lastMove) > MOVEMENT_PAUSE_THRESHOLD) return; If the gadget has been stationary for a while, it shouldn’t start moving just because we let go of it. It’s also common for your finger to roll or shift position slightly as you lift it off the screen, so we also ignore small movements: if ((fabsf(dragDelta.width)>INERTIA_THRESHOLD) || (fabsf(dragDelta.height)>INERTIA_THRESHOLD)) { [self startTimer]; } } } The gadget needs to move without the user’s interaction, so we run a timer at 60 frames per second (the iPhone’s screen refresh rate) to update it. I assume you know how to use NSTimer by now, so we’ll just look at what happens when the timer fires: - (void) timerTick: (NSTimer*)timer { dragDelta = CGSizeScale(dragDelta, INERTIAL_DAMPING); When we calculated the user’s finger movement earlier, we stashed it in a member, rather than local, variable so that we’d have access to it later. This way, the gadget keeps moving at the speed the user last moved it. Each time the timer fires, we multiply this vector by a frac- tion a little under one to simulate the friction of the surface below the gadget. This simple simulation would never quite reach zero, so we only process it if it’s above a cer- tain level: if ((fabsf(dragDelta.width)>DELTA_ZERO_THRESHOLD) || (fabsf(dragDelta.height)>DELTA_ZERO_THRESHOLD)) { CGPoint ctr = CGPointApplyDelta(self.center, dragDelta); CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation102 We calculate the new position of the gadget, but we don’t apply it right away. We need to test it first; otherwise, it’s really easy to just flick a gadget into outer space. Just like a scroll- ing list, if our gadget does hit the edge, it’ll bounce off. Collision detection’s a little outside the scope of this chapter, but you can look it up in the sample code if you need to. CGSize halfSize = CGSizeMake(self.bounds.size.width/2, self.bounds.size.height/2); [self check:&ctr delta:&dragDelta halfSize:halfSize forBouncingAgainst:self.superview.bounds.size]; self.center = ctr; } else { If the inertial movement drops below a noticeable level, we just kill it off outright—saving the user’s batteries, apart from anything else. You’ll remember from earlier that we also stop it if you place a finger on the gadget, allowing you to trap it in flight. [self stopTimer]; } } Tying Up Loose Ends We’ve now looked at the processing for every part of the gesture, and by now, you should be ready to implement your own. We’ll just close this section by noting that we also clear the gesture lock if a touch is cancelled by the operating system and, as promised earlier, by hav- ing a quick look at how we calculate the values used to track the fingers. The calculateAngle method is the most complex of the set, so we’ll pick it apart first: - (float) calculateAngle:(NSArray*)twoTouches { NSParameterAssert([twoTouches count]==2); UITouch* firstTouch = [twoTouches objectAtIndex:0]; UITouch* secondTouch = [twoTouches objectAtIndex:1]; After a quick safety check, we simply pull out the two touches and arbitrarily decide that one of them is the first. We poured them into an array so we could use objectAtIndex, but the array itself can be in arbitrary order. This is the only method that actually cares what order the touches come in, so we simply compare the pointers and swap them if necessary: if (firstTouch>secondTouch) { UITouch* temp = firstTouch; firstTouch = secondTouch; CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation 103 secondTouch = temp; } The same UITouch object representing each touch remains the same throughout the event, so if we sort them like this, we know we’re preserving an order—any order. The order doesn’t matter as long as it doesn’t change midgesture. Having done this, we can extract the coor- dinates, again in the parent view’s reference frame so that the rotation of the gadget itself doesn’t throw our calculations off: CGPoint first = [firstTouch locationInView:self.superview]; CGPoint second = [secondTouch locationInView:self.superview]; CGSize delta = CGPointDelta(first, second); return atan2f(delta.height, delta.width); } Finally, an atan2f from the C standard library gets us the angle described by the line between the points. The other calculations are equally straightforward and follow the same patterns, so there’s no need to rehash them. The calculatePinch method just gets the length of the line, instead of the angle. averageTouchPoint is even simpler, calculating the mean of the array of coordinates passed to it. Summary It’s important to remember that the sample code in this chapter is an example of the tech- niques, not of good application design! It’s rarely a good idea to overload a single gadget with this many behaviors. Read Apple’s iPhone Human Interface Guidelines (log in to http:// developer.apple.com/iphone/ to find this document). Don’t think of them just as some rules to stick to, but consider the decisions behind them. It’s the spirit, not the letter, that is important. iPhone applications are not only different from desktop applications but also have a wider range of variation in their own right, because they’re not all used seated at a flat, controlled surface. Some, like Stage Hand, may be used in a relatively controlled envi- ronment but by a user under stress. Others may be purely for relaxation but could be used while being jolted around on public transit with a full bag of groceries in the other hand. It’s far better to design something cleanly in the first place than to rely on fancy tricks to get you out of trouble. Sometimes, though, you are painted into a corner or have no choice, and it’s better to have these tools at your disposal, especially if they’re for power or optional, extra features rather than the application’s core task. If you have to use advanced gestures, try to keep the processing as simple as possible. It’s easy to overthink it! You don’t need to write a handwriting-recognition system. Instead, focus as much as possible on the user’s intent and do the minimum to detect that intent and CHAPTER 4: All Fingers and Thumbs: Multitouch Interface Design and Implementation104 differentiate it from others. Definitely don’t try to be too restrictive and require too much accuracy from the user. Users sometimes do the strangest things, but if you can put yourself in their places (or if you have the opportunity to ask them) and find out why, the results can be very illuminating. What’s best is when you can put some of that understanding into the application. When you consider why your users want to do something, you can be a step ahead of them and present sensible choices, or enable only the appropriate gestures, which typically increases the reliability of detection. You don’t have to be psychic, but by putting yourself in the user’s head, you can make the iPhone seem to be! [...]... Animation with the cocos2diPhone Framework s o you want to write an iPhone game with realistic physics This chapter will get you started Here’s our story: Brainjuice was founded in 2007 by myself and Ivan Neto as the product division of our agency, INCOMUM Design & Concept That December, we launched Blogo, a weblog editor and our first Mac desktop application After the first version of the iPhone SDK was released,... variables 115 1 16 CHAPTER 5: Physics, Sprites, and Animation with the cocos2d -iPhone Framework The first step to detecting collisions is initializing the types of collisions in an enum We’ll use these later to associate collision areas with C functions @implementation GameLayer - (id) init { // snip sets up the board, players etc // add the puck puck = [self addSpriteNamed:@"puck.png" x: 160 y:240 type:kColl_Puck];... look at the unique challenges presented by developing a 2D game on the iPhone To do this, we’re going to use OpenGL ES with help from the cocos2d and Chipmunk libraries to create a simple miniature golf game (as shown in Figure 5-1) Figure 5-1 The miniature golf game layer CHAPTER 5: Physics, Sprites, and Animation with the cocos2d -iPhone Framework Introducing OpenGL ES OpenGL ES is the most widely used... Flash/ActionScript, and Ruby/Rails Development environments: Eclipse, Visual Studio, and Flash Programs used: TextMate, Creative Suite, and Terminal.app Life as an iPhone developer: Created the Arcade Hockey game and Town Hall reference application iPhone development toolset: TextMate for editing, Xcode for building, and instruments and clang static analyzer for detecting leaks What’s in this chapter: This... (void)pushPuckFromCorner{ cpBodyApplyImpulse(puck, cpvsub(cpv( 160 ,240), puck->p), cpvzero); } Here’s the key to the fix: we define a function to push the puck from the corner that will be called later from our C function callback This just applies an impulse on the puck toward the center of the table CHAPTER 5: Physics, Sprites, and Animation with the cocos2d -iPhone Framework - (cpBody *) addSpriteNamed: (NSString... accelerometer, the possibilities for different interfaces and controls are endless Gestural controls mixed with realistic physics can create a scarily immersive game experience We made the choice to hold off an iPhone version of Blogo, which we knew would be a complex problem both in terms of the interface design and the number of different services we would have to integrate with Instead, we wanted to cut our... what it takes to make a game not only playable but fun, and we ran into more than a few stops that required some creative workarounds 109 110 CHAPTER 5: Physics, Sprites, and Animation with the cocos2d -iPhone Framework Figure 5-2 The main menu of Arcade Hockey We began by studying the other hockey games in the App Store, and a couple of paid competitors gave us an early scare We downloaded and tested... 5-3) We mocked up a prototype with a bare table and placeholder graphics to begin working on the physics while the sprites were being designed CHAPTER 5: Physics, Sprites, and Animation with the cocos2d -iPhone Framework Figure 5-3 The Options screen Once the mallets felt right, we began sketching out the basic game play with the puck We quickly learned that collision detection was far from plug and play,... the program, as well as some final touches like the simulated game noises in the first screen, and the flaming puck that appears in a 111 112 CHAPTER 5: Physics, Sprites, and Animation with the cocos2d -iPhone Framework sudden-death match point We also did some fine-tuning on the game play, adjusting the maximum speed of the puck to keep the game from getting out of hand Next, I’ll go into some more detail... move forward as the mallet approached the center line Figure 5-4 Finger tracking while guarding the goal (shaded circles show finger positions) CHAPTER 5: Physics, Sprites, and Animation with the cocos2d -iPhone Framework Listing 5-1 shows a code snippet for compensating for the user’s finger Listing 5-1 Compensating for the User’s Finger int finger_padding = 30; int fat_fingers_offset = 40; // we add a . } } The gadget needs to move without the user’s interaction, so we run a timer at 60 frames per second (the iPhone s screen refresh rate) to update it. I assume you know how to use NSTimer. overload a single gadget with this many behaviors. Read Apple’s iPhone Human Interface Guidelines (log in to http:// developer.apple.com /iphone/ to find this document). Don’t think of them just as. TextMate, Creative Suite, and Terminal.app. Life as an iPhone developer: Created the Arcade Hockey game and Town Hall reference application iPhone development toolset: TextMate for editing, Xcode