Browser

Agents open web pages, read them and check them

In the work panel’s browser, the agent opens its own tab, reads the page, then follows links and fills in forms. Clicking and typing ask before they run, and in tabs you opened those actions ask every time.

The browser in the main area has moved to the Ridge Loop detail page; the side chat beside it shows the browser snapshot and browser navigate steps and reports that the page’s elevation has no unit
The browser in the main area shows the trail list; the side chat has read it with browser snapshot and asks for approval to run browser_navigate

The agent’s tools

Pages read as text, elements picked by reference

The agent reads the page’s accessibility structure, not screen coordinates. Buttons and fields carry references, so it names exactly what it will press.

  • browser_snapshot reads the page as an accessibility outline of up to 50,000 characters, with references such as [ref=e12] on each element.
  • browser_click, browser_type, browser_press_key press, type into, or send keys such as Enter, Tab and the arrows to an element by its reference.
  • browser_navigate opens http and https addresses only and does not follow redirects to another site.
  • browser_wait waits for text to appear, or for up to 30 seconds.
  • browser_tabs, browser_tab_new, browser_tab_close list, open and close tabs in the window where the conversation is open, your own tabs included. Routines and headless runs get no browser.

Safeguards

It asks differently depending on whose tab it is

Approval cards name the page and who opened the tab. What the agent reads from a page reaches the model marked as untrusted web data.

An approval card for browser_click: the page is in a tab the agent opened, and the approval holds on this site until Ave restarts

Tabs the agent opened

Allow for this session holds per site until Ave restarts. There is no always-allow for browser actions.

An approval card for browser_tab_new in the network category, with the address it will open and Reject and Allow buttons

Opening a tab asks too

browser_tab_new counts as a network action, so it shows the address it will open and waits for approval.

The main area with the trail-notes preview in a web panel beside a terminal

A web panel in the main area

Move a page from the side panel to the main area and it becomes a web panel; the page keeps its state.

The work panel’s Browser tab showing the trail-notes preview beside the conversation

The work panel’s browser

One address bar and the list of visited sites, each marked as requested or found in a result.

It chooses how to read, too

Only what can be used

A snapshot can list just the elements a person can act on, such as links, buttons and fields, as a flat list.

Only what changed

Reading the same tab again, the agent can ask for just what changed since its previous snapshot instead of the whole page.

Data, not instructions

Text read from a page is marked as written by whoever runs the site, so instructions written inside a page are treated as data.

Questions

Can the agent use a tab I’m signed in to?

In a tab you opened, clicking, typing, pressing keys, navigating and closing ask every time, in every permission mode, and nothing is remembered; with no one to ask, they are denied. Reading or capturing that tab is a network action: it asks once per site for the session, and passes under Allow all.

Does it type passwords?

No. The agent does not type into password, payment-card or one-time-code fields.

What if the page changes while the approval card is open?

The action does nothing. If the agent tries again, it asks about the page as it is now.

What if I touch the page while the agent works?

The agent holds the page’s focus only while it acts, then gives it back. Press Stop on the tab, or click or type in the page, and its pending actions stop.

What about addresses it can’t open?

Anything that isn’t http or https, or that carries credentials in the address, is refused without showing an approval card.

Can it take a screenshot?

Yes. browser_screenshot captures what a tab shows, for when the text outline isn’t enough.

Can routines use the browser?

No. Routines and headless runs are not given the browser tools.

Read on