Phone Harness is a tool that enables AI agents to control and interact with real phones through programmatic interfaces.
The tool solves the problem of giving AI agents the ability to perform real-world mobile tasks by connecting them to actual devices. For iPhones, it captures the screen through macOS iPhone Mirroring, performs optical character recognition to identify text and tap coordinates, and sends touch and keyboard input as HID-level events. For Android devices, it uses adb as the transport layer, capturing screenshots and reading the accessibility tree to identify interactive elements, then sending input commands. Both platforms expose the same set of helper functions to the agent, allowing identical code to work across device types. The agent sees the screen, can tap specific locations, type text, and read the results of its actions.
Developers should choose this tool if they need to automate mobile app interactions or test mobile workflows with AI agents. It works well for tasks that require real device behavior without requiring installation of custom software on the phone itself. The tool supports both local phones connected to a Mac or Linux machine and cloud-hosted Android devices through a paid service. For iPhone users without an Android device available, the cloud option provides a way to test on Android without local setup. The approach differs from alternatives by operating at the screen and accessibility level rather than requiring app-level integration or device jailbreaking.
The project maintains active development with regular updates to both the core helpers and documentation. The tool includes diagnostic capabilities through a doctor command to verify the connection chain is working correctly. Setup is designed to be agent-friendly, with the ability to send a setup prompt directly to Claude Code or Codex that walks through necessary pairing steps. The project is maintained as a personal effort and accepts sponsorship to sustain ongoing work.
