# Image Recognition Macro Concepts: How MAS Works

> How an image recognition macro works in MAS: template images and OCR read the Android screen, humanized input goes over adb, and where macros run.

Source: https://automationmacro.com/docs/concepts (Start, updated 2026-09-04)

Macro Automation Studio (MAS) automates Android apps the way a person does: it looks at the screen, decides, and taps. This page explains how an image recognition macro works, the words you meet in the app, and where each part runs. Read it once and the rest of the docs will make sense.

## How an image recognition macro sees the screen

MAS does not modify APKs and does not read app memory. It works from the outside, through adb, with three building blocks.

### Template images

A template is a small crop of the screen: a button, an icon, a badge. You crop it in Asset Lab from the live device, it lands in your Image Library with a numeric ID, and your script refers to it by that ID. At run time `find_object` takes a fresh screenshot and searches it for the template with OpenCV template matching. A match needs a similarity of 0.8 by default (`threshold=0.8`) and returns the centre point, so the next line is a tap.

Image files can be jpg, png, gif or webp, up to 10 MB each. Keep templates small and distinctive: a whole screen matches only that exact screen, while a button matches wherever the button appears.

### OCR automation on Android

Text is read with Tesseract OCR through `read_text`. You pass a `Region(x1, y1, x2, y2)` so the engine looks at one counter or one label rather than the whole screen, and a page segmentation mode (`psm=7` for a single line, `psm=8` for a single word). `ColorConversion.BLACK_WHITE` helps with coloured text on noisy backgrounds. Asset Lab has a live OCR test so you can tune a region before it goes into code.

### Humanized input

Input is sent as real touch and key events: `click`, `swipe`, `input_text`, `key_press`, `zoom_in` and `zoom_out`. Timing is yours to set. A `click` waits `delay_ms=1000` after the tap, a `swipe` takes `duration_ms=1000`, and a key held for 500 ms or more counts as a long press. Slower, spaced-out macros look less mechanical.

### Coordinates and screen size

Every coordinate is a pixel offset from the top-left corner of the screen. `get_screen_size()` returns the width and height; `get_device_info()` returns the same plus the device name. Templates and regions are tied to the resolution and DPI you captured them on, so keep every device that runs a macro on the same resolution as the one you built it on. The emulator guides recommend a size for each emulator, and cloud devices are created with a fixed display preset.

## Vocabulary

| Term | Meaning in MAS |
|---|---|
| Macro | A Python program that automates one task on a device. The unit you run, schedule, share and sell. |
| Project | The folder behind a macro: `src/app.py`, an optional argument form, assets and a git history. |
| Device | One Android target MAS can drive, identified by its adb port or, for a cloud device, by its ID. |
| Device group | A named set of devices on the **Device Groups** page. Each device in a group has its own macro, arguments, proxy and webhooks; **Start All** runs them one after another. |
| Settings profile | A named bundle of argument values for one macro. Devices follow a profile, so changing it updates every device that uses it. "Default" always exists. |
| Cloud device | An Android device MAS runs for you on its own servers. Macros on it run server-side and its screen streams over WebRTC. |
| MAS Agent | The AI author on the **Agent** page. Describe a task, it explores the device, asks when it must decide, and compiles a macro that passes 3 validation runs. |
| Marketplace | The catalog of ready-made bots inside the app. Download one and it becomes a macro in **Macros**. |
| Asset Lab | The helper app for cropping templates, picking points and testing OCR regions on a live screen. |
| UI Builder | The designer for argument forms (`.uibproj`) and runtime dashboards (`.uibrt`). |
| Code Editor | The Python IDE inside MAS, opened per project from **Macros**, with a debugger and a run console. |

## Where things run

Three places do the work, and it helps to know which one you are looking at.

### Your computer drives emulators

MAS on your PC or Mac finds emulators by their process name and the ports they listen on, then talks to each one with `adb connect 127.0.0.1:<port>`. Screenshots, taps and OCR for a local device all happen on your machine, so the app must stay open while a macro runs. The same is true of the scheduler.

### Cloud devices run in MAS's cloud

A cloud device lives on MAS's servers. When you run a macro on it, MAS uploads a snapshot of your project and runs it next to the device. You can watch the screen live in the app or close the app entirely; the run continues either way. Idle devices stop after about an hour of inactivity and keep their data.

### Scripts talk to the app over JSON-RPC

A macro is a normal Python process. MAS starts it as `python -u -m src.app` inside its own Python 3.13 environment and hands it the connection details in environment variables (`MAS_RPC_PORT`, `MAS_RPC_HOST`, `MAS_DEVICE_ID`, `MAS_SESSION_TOKEN`). Every `mas.*` call is a JSON-RPC 2.0 request over a local socket; the app owns adb, OpenCV and Tesseract and sends the result back. On a cloud device the same calls travel over a unix socket inside the sandbox. This is why `import mas` needs no install and why a script started outside MAS reports "Could not discover RPC port".

### AI at authoring time, none at run time

MAS Agent spends AI credits while it explores and compiles. The macro it produces is plain Python and runs like any other, on your emulator or on a cloud device, without the agent in the loop.

## The subscription

Every page in the app apart from sign-in and **Subscription** needs an active entitlement: the free trial or a paid plan. A plan is sized by devices, meaning how many of your own emulators or phones can run at the same time. Cloud devices are an add-on with their own monthly credits, and extra credits can be topped up for agent sessions. Cancelling keeps access until the end of the paid period. The [Billing](/docs/billing) page explains the mechanics; the figures are on the [pricing page](/pricing).

## Where to go next

Set up a device on the [Devices](/docs/devices) page, crop your first template in [Asset Lab](/docs/asset-lab), or read the [SDK overview](/docs/sdk) if you would rather start from code.