When you tap on your iPhone’s screen, capacitive sensors recognize that a circuit has been completed at a certain point on the screen. That gets translated at a low level into, “hey, someone touched the screen at these (x,y) coordinates.” That’s easy enough to understand.
But if you tap an on-screen button, how does iOS translate from recognizing that it’s received a touch event to actually pressing that button? And how can you trick the system into thinking it’s received a button tap that didn’t actually happen?
A brief primer on UIEvents and UITouches
If you read Apple’s public API documentation, there are a handful of high-level event objects that are handled by your application layer. When a touch event is received, the active UIApplication receives a sendEvent: message with a UIEvent object (technically, a private subclass called UITouchesEvent) containing a set of UITouch object.
From there, it delegates down to the sendEvent: method on the active UIWindow, which in turn figures out the appropriate UIResponder object to forward the event on to, which for a touch event is probably a UIView. That UIView will then receive a message like touchesBegan:withEvent:, touchesMoved:withEvent:, touchesEnded:withEvent, or touchesCancelled:withEvent, depending on the event. If you’ve been doing iOS development for a while, it’s likely you’ve had to subclass one of those methods.
I’m working on a project right now where I need to be able to programmatically create and dispatch touch events to a UIWebView. Since a UIWebView is a black box with a fairly limited public interface, there’s no fudging it; I need a proper UIEvent object that the UIWebView will accept and respond to.
UITouch Me
It would be seemingly sensible to figure out how to instantiate your own UITouch and UIEvent objects. If you Google around for faking touch events, that’s the route a lot of people take.
Conceptually, though, it’s not a great approach. Dealing with private frameworks is a necessary evil; minimizing the surface area of private code you’re reaching into is essential since Apple could introduce breaking changes at any time. Faking events at the UITouch and UIEvent level requires touching three different private APIs, which creates three distinct possible points of failure.
A practical example of this: the most widely-read article on faking iOS touch events is a 2008 Cocoa with Love article. Even if you fix the basic syntax errors in the included code (it’s not the author’s fault; Objective-C has changed a lot in five years), it flat-out doesn’t work any more.
GraphicsServices
Enter GraphicsServices.framework, the private framework responsible for handling events at a system level. When you touch the screen, that gets translated at a low level into a GSEvent object, a C struct containing various information about the event. GSEvents represent all kinds of system events, from hardware button presses to low memory warnings. It turns out that UIEvent objects are little more than light wrappers around GSEvent objects; you can even access a UIEvent object’s underlying GSEvent by calling the private _gsEvent method on it.
After a GSEvent has been created, it gets dispatched using a function called GSSendEvent() that takes in a reference to a GSEvent and the internal port of an application to receive the event (some events get handled by system processes like the Springboard rather than the frontmost application).
If that target is the frontmost application, the GSEvent object somehow gets wrapped in a UIEvent and passed to the application via the sendEvent: method described above. (I don’t know whether that’s the same for events bound for system processes, but that’s not particularly relevant for our immediate purposes.)
This suggests an easier route: if we can create a fake GSEvent and use GSSendEvent() to route it directly to our application, we can bypass needing to manually create UITouch and UIEvent objects by letting the system do it for us.
It turns out that this is a fairly regular pattern for developers of jailbroken apps, who often need to do things like simulate pressing the home button or tapping the status bar. Despite the regularity with which jailbroken app developers dive into this framework, though, there is very little documentation of how to actually do it.
Putting it all together
First, you need to import both GraphicsServices.framework and its header files into your application.
In my personal development environment (OS X 10.8.5, Xcode 5.0 build 5A1412), the framework was located at
~/Library/Developer/Xcode/iOS\ DeviceSupport/7.0\ \(11A4449d\)/Symbols/System/Library/PrivateFrameworks/GraphicsServices.framework`
I used a copy of the headers I downloaded from a GitHub repo containing an iOS SDK header dump. I used nst/iOS-Runtime-Headers, but others exist. Throw the GraphicsServices folder into your project, and #import "GraphicsServices.h" where relevant.
After that’s all set up, there are three things to do: construct a GSEvent object, get the port of your application, and tell the system to send your GSEvent to your application’s port.
Constructing a GSEvent
If you look at GSEvent.h, you can see all of the properties that make up the GSEvent, GSEventRecord, and GSHandInfo structs.
For me, finding the proper values to set was a process of trial and error. The GSEvent.h header is useful, as is manually inspecting real GSEvent objects (with the help of a custom UIWindow subclass with an overwritten sendEvent: method). It’s possible some of the properties I’m setting below aren’t necessary, or might need to be changed for your purposes.
One thing to note: the CGPoint you pass into the event’s location field and the pathInfo’s pathLocation field are coordinates relative to the entire window, not the UIView that will ultimately receive the event. If you have coordinates relative to a view and a reference to the view itself, you can convert the coordinates using [view convertPoint:point toView:view.window].
Getting your application’s port
This function was cribbed pretty much wholesale from Stack Overflow, although I ripped out a bunch of unrelated code related handling the lock screen.
Sending the event
We have a GSEvent object, event, and a function that will return the frontmost app port. Let’s put ‘em together.
And there you have it!
If you want to simulate a full tap event, you want to construct two events, one where the phase is UITouchPhaseBegan and one where it’s UITouchPhaseEnded. Similarly, a gesture would have you sending UITouchPhaseMoved as often as necessary between those two.
If you want to see the final result in production code, check out the UIFakeTouch class I ended up writing for use in my project.
A word of warning
It goes without saying that this isn’t a technique that should be used in applications destined for release in the App Store. By all means use it for your Cydia apps or for applications not designed for production release, but if you submit an app to Apple that links against GraphicsServices.framework it will almost certainly be rejected.
A second word of warning: iOS 7
This approach doesn’t work with iOS 7. While it doesn’t appear that the header files of GraphicsServices.framework have been modified, Apple has changed the way that it works under the hood such that it silently fails.
You can still use this technique using the iOS 7 SDK and Xcode 5, just make sure you’re targetting iOS 6.1 or earlier. When I have the time to dive into it, I hope to write a follow-up post on iOS 7 compatibility. Let me know if you have any success!