Artificial intelligence itself can control computers

Earlier, Anthropic announced the development of an AI that can perform office tasks, automating a significant part of the economy.

Already today, they presented an updated Claude 3.5 Sonnet model, which can interact with any application via API.

The Claude 3.5 Sonnet mimics human actions: keystrokes, mouse clicks and other gestures, writes Hitech.

It also recognizes what’s happening on the screen, analyzes the screenshot, and calculates how to act to complete tasks.

Anthropic calls this an “action-execution layer” that allows Claude to execute commands at the desktop level, using any program or website.

To do this, users grant access and AI uses it to perform actions, such as filling out forms or interacting with software.

AI agents like Claude promise to simplify automation, but it’s not a new idea. Similar tools are being developed by Microsoft, OpenAI, and other companies.

However, Claude 3.5 features improved performance: solving complex tasks that require hundreds of steps and adjusts its actions if it encounters problems.

However, the model faces difficulties such as scrolling and scaling the screen, which reduces its effectiveness in some tasks.

Security issues are also relevant. Models that use desktop programs can be vulnerable to attacks and data breaches.

Anthropic recognizes the risks, but believes it is better to control processes with less powerful models and explore potential threats by gradually improving protection.