Google SIMA AI

What is Google SIMA AI?

SIMA (Scalable Instructable Multiworld Agent) is an AI agent developed by Google DeepMind. It is designed to understand a variety of virtual environments and complete tasks by following plain English instructions.

SIMA does not need access to a game’s source code or special developer APIs. It works from two inputs only: what it sees on screen and natural language commands. It then translates those instructions into keyboard and mouse actions to control the main character in the game.

SIMA has been tested on over 600 basic skills, such as navigating environments, picking up objects, using menus, turning, climbing ladders, and opening maps. These simple tasks typically take around 10 seconds to complete.

Who created Google SIMA AI?

SIMA was created by Google DeepMind, the AI research company founded by Demis Hassabis. DeepMind focuses on building general-purpose AI systems, from game-playing agents like those for StarCraft II to instruction-following agents like SIMA.

How does Google SIMA AI work?

• Input: SIMA requires only on-screen images and simple natural language commands.
• Processing: It matches what it sees on screen with the language instructions and predicts what to do next, without needing the game’s underlying code or special developer connections.
• Output: It converts instructions into keyboard and mouse actions to move around, interact with objects, and use menus.

Because it relies only on visual input and standard controls, SIMA can potentially work with virtually any virtual environment without complicated setup or special interfaces.