# Python ADB Screenshot, Template Matching and OCR

> Python ADB screenshot, OpenCV template matching and Tesseract OCR on an Android emulator with the MAS SDK, with the retry helpers and real defaults.

Source: https://automationmacro.com/docs/sdk/vision (Python SDK, updated 2026-09-04)

Vision is how a Macro Automation Studio (MAS) macro sees the screen from Python. The app takes a screenshot of the device, over adb on an emulator, searches it for your template images with OpenCV template matching, and reads text with Tesseract OCR. The SDK only sends the request over JSON-RPC, so there is nothing to install. This page explains the search functions, the wait helpers, OCR, and the types they return. Full signatures are on the [API reference](/docs/api-reference). When you describe the task to [MAS Agent](/agent), it writes this code for you.

## Templates for OpenCV template matching on Android

A template is a small crop of the screen: a button, an icon, a badge. You crop it in [Asset Lab](/docs/asset-lab) from the live device, it lands in your Image Library with a numeric ID, and your script refers to it by that ID. Small, distinctive crops match reliably; a whole screen matches only that exact screen.

Declare the IDs once at the top of the script with [`mas.images`](/docs/api-reference#images). It returns an `ImageMap` with one `ImageRef` per name, so the rest of the script reads like prose.

```python
import mas

images = mas.images({
    "claim_btn": 784,
    "energy_icon": 785,
    "close_x": 786,
})

match = mas.find_object(images.claim_btn)
```

Three rules come from the source. Names must be valid Python identifiers (`claim_btn`, not `claim-btn`), IDs must be integers, and the map is read-only. Every search function also accepts a raw integer ID. When a script mixes `mas.images()` with raw integers, the SDK prints a hint on exit listing the unregistered IDs, because the packer reads image IDs from the `mas.images()` call when it bundles a project for a cloud run or the Marketplace. An ID that is not in your library raises `ImageNotFoundError`.

## take_screenshot

[`take_screenshot()`](/docs/api-reference#take_screenshot) captures the device screen and returns a `Screenshot` with `base64` (PNG data), `width`, `height` and an ISO 8601 `timestamp`. Every search function takes a fresh screenshot unless you pass one in, so capture once when you need to look for several things on the same frame.

```python
import base64
import mas

images = mas.images({"claim_btn": 784, "close_x": 786})

shot = mas.take_screenshot()
claim = mas.find_object(images.claim_btn, screenshot=shot)
close = mas.find_object(images.close_x, screenshot=shot)

with open("frame.png", "wb") as f:
    f.write(base64.b64decode(shot.base64))
```

## find_object

[`find_object(image_id, *, threshold=0.8, screenshot=None, continuous_mode=False, timeout_ms=1000, capture_interval_ms=500, max_matches=1, search_region=None)`](/docs/api-reference#find_object) searches the screen for one template. It returns an `ObjectMatch` when the best match scores at least `threshold`, and `None` otherwise. A miss never raises. Every parameter after `image_id` is keyword-only.

- `threshold`: similarity from 0.0 to 1.0, higher is stricter. 0.8 is the default. Go up to 0.9 when a template has near-identical siblings, and down to 0.7 only for compressed or animated art.
- `search_region`: a `Region(x1, y1, x2, y2)` that limits the search. A tight region is faster and rules out look-alikes elsewhere on the screen.
- `continuous_mode`: when `True`, the app keeps capturing every `capture_interval_ms` (500 ms) until the template appears or `timeout_ms` (1000 ms) elapses. `timeout_ms` has no effect outside continuous mode.
- `max_matches`: leave it at 1 here and use `find_objects` when you want several.
- `screenshot`: a frame from `take_screenshot()` to reuse.

```python
import mas
from mas import Region

images = mas.images({"claim_btn": 784})

match = mas.find_object(images.claim_btn)
match = mas.find_object(images.claim_btn, threshold=0.9,
                        search_region=Region(x1=0, y1=1400, x2=1080, y2=1920))
```

## find_objects

[`find_objects(image_id, *, ..., max_matches=10, ...)`](/docs/api-reference#find_objects) takes the same parameters, with `max_matches` defaulting to 10, and returns a list of `ObjectMatch` sorted best first. The list is empty on a miss. Use it for grids of identical items: reward chests, checkboxes, list rows that share an icon.

## find_any_object

[`find_any_object(image_ids, *, ..., search_strategy="first_match", ...)`](/docs/api-reference#find_any_object) searches for several templates in one call and returns one `ObjectMatch`, or `None`. It is for variants of the same control: two languages, two themes, a pressed and an unpressed state. `matched_template_id` on the result tells you which template hit.

`search_strategy` takes one of three values. `"first_match"` and `"priority_order"` try the templates in list order and stop at the first hit. `"best_match"` tries every template and returns the strongest score.

```python
import mas

images = mas.images({"ok_en": 301, "ok_de": 302, "ok_fr": 303})

match = mas.find_any_object([images.ok_en, images.ok_de, images.ok_fr])
if match:
    mas.log(f"matched template {match.matched_template_id}")
    mas.click(match.x, match.y)
```

## Waiting: find_object_retry and find_any_object_retry

[`find_object_retry(image_id, *, total_tries=3, time_sleep=2.0, **kwargs)`](/docs/api-reference#find_object_retry) is the recommended way to wait for something to appear. It calls `find_object` up to `total_tries` times, sleeps `time_sleep` seconds between failed attempts, never sleeps after the last one, and returns the first match or `None`. The cost is bounded and easy to read: three tries two seconds apart is at most four seconds of waiting plus three searches.

`total_tries` and `time_sleep` are keyword-only. Every other keyword (`threshold`, `search_region`, `max_matches`, `screenshot`) is forwarded to `find_object` by name, so order never matters. If you pass a `screenshot`, the call collapses to a single attempt, because retrying a frozen frame would search the same pixels again.

[`find_any_object_retry(image_ids, *, total_tries=3, time_sleep=2.0, **kwargs)`](/docs/api-reference#find_any_object_retry) does the same over a list of templates and forwards to `find_any_object`.

```python
import mas
from mas import Region

images = mas.images({"claim_btn": 784})

btn = mas.find_object_retry(
    images.claim_btn,
    total_tries=5,
    time_sleep=1.5,
    threshold=0.85,
    search_region=Region(x1=0, y1=1200, x2=1080, y2=1920),
)
if btn is None:
    mas.log("claim button never appeared", level="warning")
```

## wait_for_object

[`wait_for_object(image_id, timeout_ms=10000, threshold=0.8)`](/docs/api-reference#wait_for_object) waits on the app side until the template appears or the timeout passes, and returns the match or `None`. It is kept for existing macros. Prefer `find_object_retry` in new code: discrete tries with a flat pause read plainly, take a predictable amount of time, and behave the same on every host. `wait_for_object` also has no `search_region`.

## read_text

[`read_text(region=None, screenshot=None, model="eng_best", psm=7, color_conversion=ColorConversion.NONE, timeout_ms=30000)`](/docs/api-reference#read_text) runs Tesseract OCR and returns a `TextRecognitionResult` with `text`, `confidence` (0.0 to 1.0) and the `region` it read.

- `region`: a `Region` around the text. Without it the whole screen is read, which is slow and noisy. Draw the region in Asset Lab, where you can test it live.
- `model`: `"eng_best"` (default), `"eng_fast"` or `"eng"`.
- `psm`: the Tesseract page segmentation mode. `7` treats the region as one line (default), `8` as one word, `6` as a uniform block of text, `11` as sparse text in no particular order.
- `color_conversion`: a `ColorConversion` member applied before OCR. `NONE` is the default. `BGR_TO_GRAY` or `RGB_TO_GRAY` turns the region grayscale. `BLACK_WHITE` is a MAS extension that thresholds to pure black and white, the best choice for bright text on a busy background. There is no `GRAYSCALE` or `BINARY` member.
- `timeout_ms`: 30000 by default.

OCR returns text, not numbers. Strip everything that is not a digit before you compare.

## ObjectMatch, Region and the other types

[`ObjectMatch`](/docs/api-reference#objectmatch) has exactly three fields: `x` and `y`, the center of the matched area in screen pixels, and `matched_template_id`, the image ID that hit. `match.center` returns `(x, y)` as a tuple and `match.template_id` is an alias. There is no confidence, width or height on a match; if you need a stricter answer, raise `threshold`.

[`Region`](/docs/api-reference#region) is a dataclass with `x1`, `y1` (top-left) and `x2`, `y2` (bottom-right), in pixels. `Screenshot`, `TextRecognitionResult`, `ColorConversion` and `SearchStrategy` are all importable from `mas`.

## Example: click a button when it appears

```python
import mas

images = mas.images({"claim_btn": 784})

btn = mas.find_object_retry(images.claim_btn, total_tries=4, time_sleep=2.0)
if btn:
    mas.click(btn.x, btn.y, delay_ms=1000)
else:
    mas.log("nothing to claim this round")
```

## Example: read an energy counter

```python
import re
import mas
from mas import Region, ColorConversion

ENERGY = Region(x1=880, y1=40, x2=1060, y2=90)

result = mas.read_text(region=ENERGY, psm=7, color_conversion=ColorConversion.BLACK_WHITE)
digits = re.sub(r"[^0-9]", "", result.text)
energy = int(digits) if digits else 0
mas.log(f"energy={energy} confidence={result.confidence:.2f}")

if energy < 20:
    mas.log("low energy, stopping", level="warning")
    raise SystemExit(0)
```

## Example: find all matches on one Python ADB screenshot

Capture one frame, find every match on it, then tap them in order. Take a new frame afterwards, because the screen has changed.

```python
import mas

images = mas.images({"chest": 412})

shot = mas.take_screenshot()
chests = mas.find_objects(images.chest, screenshot=shot, max_matches=10, threshold=0.85)
mas.log(f"found {len(chests)} chests")

for chest in chests:
    mas.click(chest.x, chest.y, delay_ms=800)
```

## Errors

A miss is `None` or an empty list, never an exception. The exceptions you will meet are `ImageNotFoundError` (the ID is not in your Image Library), `DeviceNotConnectedError` (no device bound to the run) and `TimeoutError` from `wait_for_object`. All three extend `RPCError`; see [Errors and exceptions](/docs/sdk/errors).