UI-TARS Desktop: ByteDance's Open-Source GUI Agent Stack

By Prahlad Menon 3 min read

Most of the “computer use” conversation in 2026 centers on closed models — Claude’s computer use, OpenAI’s Operator, Google’s Project Mariner. They work, but they’re black boxes: you pay per action, you can’t self-host, and you can’t inspect how the agent decides where to click.

ByteDance’s UI-TARS-desktop takes the opposite bet. It’s a fully open-source multimodal agent stack — model weights, desktop app, and CLI all in the open — that controls your computer and browser by seeing the screen and acting on it like a person would.

Two Projects, One Stack

The repo ships two things under the “TARS” umbrella:

UI-TARS Desktop — a native desktop application that turns the UI-TARS vision-language model into a GUI agent. It can drive a local computer, a remote computer, or a browser. The June 2025 v0.2.0 release added free Remote Computer and Remote Browser operators — click once and remotely control a machine, no configuration.

Agent TARS — the broader multimodal agent, shipped as a CLI and Web UI. It integrates GUI-agent and vision capabilities into your terminal and browser, and connects to real-world tools via MCP. The v0.3.0 CLI added streaming for shell commands and multi-file output, timing stats for tool calls, and an Event Stream Viewer for debugging the agent’s data flow.

The Model Underneath

What makes this more than a wrapper is the underlying model. UI-TARS is a vision-language model that reasons about screenshots before acting — no accessibility tree, no DOM scraping, just pixels in and actions out.

  • UI-TARS-1.5 (April 2025) added reinforcement-learning-based reasoning, letting the model “think before it acts” and posting state-of-the-art results on GUI and game benchmarks. The 7B version is open on Hugging Face.
  • UI-TARS-2 (September 2025) is an “All-In-One” upgrade spanning GUI, game-playing, code, and tool use in a single model. Details are in the technical report.

Because the model uses absolute coordinates to ground objects on screen (a Qwen 2.5-VL lineage), the repo includes a specific coordinate-processing guide — a small but important detail if you’re wiring it into your own environment.

Why It Matters

The strategic value here is ownership. If you’re building agentic automation on top of closed computer-use APIs, you’re exposed on three fronts: cost (every click is metered), privacy (screenshots leave your machine), and lock-in (your workflows are tied to one vendor’s model and pricing).

An open GUI-agent stack changes the calculus:

  • Self-host the whole loop — run UI-TARS-1.5-7B locally on your own GPU for sensitive workflows where screenshots can’t leave the building.
  • No per-action tax — automate high-volume repetitive GUI tasks without a metered API bill.
  • Inspect and fine-tune — open weights mean you can adapt the model to your specific applications, not just prompt around a frozen one.

Where It Fits

For quick web automation you can pair UI-TARS with Midscene.js. For full desktop control, the desktop app is the fastest way to try it — and the free remote operators let you evaluate without touching model deployment at all. When you’re ready to own the stack, the open weights and Agent TARS CLI are there.

The closed players still lead on raw capability today. But UI-TARS is the clearest signal yet that open, self-hostable computer-use agents are a real alternative — and for privacy-sensitive or cost-sensitive automation, that alternative is worth building on.

Links: UI-TARS Desktop · UI-TARS model · UI-TARS-1.5-7B weights · UI-TARS-2 report