使用 Gemini + Playwright MCP 构建浏览器 Agent

在许多实际场景中，网站和应用程序不提供用于程序化访问的 API。

这时浏览器代理就特别有用：它们可以像人类一样与网页交互，点击按钮、填写表单、滚动和提取内容。

在本 Notebook 中，我们将探索如何构建浏览器代理以执行各种任务，主要侧重于信息收集，甚至扩展到图像生成。

🧰 我们的栈

• Haystack Agentic 框架
• Google Gemini 2.5 Flash，一个功能强大的模型，有慷慨的免费套餐
• Playwright MCP 服务器，提供浏览器自动化工具

设置

首先，我们将安装所有必需的依赖项

• Haystack 集成：MCP 和 Google GenAI
• Playwright 依赖项：这些是在 Colab 上成功运行它所必需的

! uv pip install mcp-haystack google-genai-haystack gdown

! npm i -D @playwright/test && npx playwright install --with-deps chrome

要获取免费的 Gemini API 密钥，请访问 Google AI Studio。

from getpass import getpass
import os

os.environ["GEMINI_API_KEY"] = getpass("Enter your Gemini API Key: ")

启动 Playwright MCP 服务器并创建工具集

我们首先启动 Playwright MCP 服务器，它将为我们提供 Web 导航和交互所需的工具。

请注意，我们在前面加上了 nohup 以在 Colab 中以非阻塞方式运行服务器，并且还需要一些其他选项才能使其在 Colab 环境中正常工作。

! nohup npx @playwright/mcp@latest  --headless --isolated --no-sandbox --port 8931 > playwright.log &

nohup: redirecting stderr to stdout

片刻之后，如果服务器成功启动，我们应该会看到一条成功的日志消息。

! cat playwright.log

Listening on https://:8931
Put this in your client config:
{
  "mcpServers": {
    "playwright": {
      "url": "https://:8931/mcp"
    }
  }
}
For legacy SSE transport support, you can use the /sse endpoint instead.

然后，我们创建一个连接到我们启动的服务器的 MCPToolset。

📌 在使用 LLM 的工具时，最好只选择你实际需要的工具。这有助于避免用过多的选项让 LLM 感到困惑。

因此，我们将从一个工具开始：browser_navigate，它导航到一个给定的 URL。

您可以在 Playwright MCP 服务器文档中找到可用工具的完整列表。

from haystack_integrations.tools.mcp import MCPToolset, StreamableHttpServerInfo

server_info = StreamableHttpServerInfo(url="https://:8931/mcp")

toolset = MCPToolset(server_info=server_info, tool_names=["browser_navigate"])

print(toolset.tools)

[Tool(name='browser_navigate', description='Navigate to a URL', parameters={'type': 'object', 'properties': {'url': {'type': 'string', 'description': 'The URL to navigate to'}}, 'required': ['url'], 'additionalProperties': False, '$schema': 'https://json-schema.fullstack.org.cn/draft-07/schema#'}, function=<function MCPToolset.__init__.<locals>.create_invoke_tool.<locals>.invoke_tool at 0x79b2e079a840>, outputs_to_string=None, inputs_from_state=None, outputs_to_state=None)]

我们第一个具有 URL 导航功能的浏览器代理

我们现在可以使用 Haystack Agent 组件来构建浏览器代理。

我们需要指定

• 一个聊天生成器，即 LLM（gemini-2.5-flash）。
• 一个好的系统消息。这个模型很强大，但有时可能会过于谨慎，拒绝执行它能够执行的任务，因此我们包含了一个系统消息来鼓励它。
• 工具：我们传递了之前定义的工具集。

我们还指定了一些可选参数

• exit_conditions=["text"]。一旦 LLM 只回复文本响应而不进行工具调用，Agent 就会退出。
• streaming_callback=print_streaming_chunk。使用实用函数，将从 LLM 发送过来的文本和工具调用，以及来自工具的工具调用结果一起流式传输。

from haystack.components.agents import Agent
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator

chat_generator = GoogleGenAIChatGenerator(
    model="gemini-2.5-flash",
)

system_message = """
You are an intelligent assistant equipped with tools for navigating the web.

You can use tools when appropriate, but not every task requires them — you also have strong reasoning and language capabilities.
If a request seems challenging, don't default to refusal due to perceived tool limitations. Instead, think creatively and attempt a solution using the skills you do have.
You are more capable than you might assume. Trust your abilities.
"""

agent = Agent(
        chat_generator=chat_generator,
        tools=toolset,
        system_prompt=system_message,
        exit_conditions=["text"],
        streaming_callback=print_streaming_chunk
)

📰 查找美国相关新闻

让我们从一个简单的任务开始：导航到一个网站并过滤美国相关新闻。

messages = [
    ChatMessage.from_user("navigate to https://www.theguardian.com/world and list all US-related news")
]
result = agent.run(messages=messages)

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.theguardian.com/world"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.theguardian.com/world\nawait page.goto('https://www.theguardian.com/world');\n```\n\n### New c...

[ASSISTANT]
Here are some US-related news articles from The Guardian:

**Africa Section:**
*   US destruction of contraceptives denies 1.4m African women and girls lifesaving care, NGO says
*   US hunter reportedly killed by buffalo during expedition in South Africa
*   Rwanda agrees to take up to 250 migrants from the US

**Americas Section:**
*   Dozens more countries face higher levies on exports to US as new Trump tariffs come into effect
*   US Coast Guard releases report calling Titan disaster a ‘preventable tragedy’
*   Canada wildfires: 81 million Americans under air quality alerts as blazes rage

**Asia Pacific Section:**
*   US research station staff evacuated from Antarctica in high-risk operation

**Europe Section:**
*   ‘No clear public response from Russia’ on plans to end war, Zelenskyy says after Macron call – Europe live
*   Kremlin denies Putin agreed to trilateral meeting with Trump and Zelenskyy
*   Donald Trump says ‘everyone agrees this war must come to a close’ after Witkoff meets Putin – as it happened

**Middle East Section:**
*   Médecins Sans Frontières calls for immediate closure of US and Israeli backed aid group in Gaza – Middle East crisis live
*   Democrats sound alarm over Trump’s reported $1bn plan to retrofit gifted Qatari jet
*   US demands for Hezbollah disarmament may force Lebanon into dangerous choice

**South and Central Asia Section:**
*   Trump’s demand that India stop buying Russian oil puts Modi in tight spot
*   India to still buy oil from Russia despite Trump threats, say officials

**US Section:**
*   US treasury secretary praises ‘manufacturing renaissance’ as Trump tariffs snap into effect – live updates
*   Chris Cuomo mocked for response after falling for deepfake AOC video
*   ‘It’s not a coincidence’: after Ghislaine Maxwell’s prison move … what next?
*   Alleged killer of Minnesota politician and her husband expected to plead not guilty in court
*   James Cameron warns of ‘Terminator-style apocalypse’ if AI weaponised

好的。

📅 今天谁出生？

另一个简单的任务。看看我们的 Agent 的表现。

messages = [
    ChatMessage.from_user("I was born on October 17th. Find five notable people born on the same day using Wikipedia.")
]
result = agent.run(messages=messages)

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://en.wikipedia.org/wiki/October_17"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://en.wikipedia.org/wiki/October_17\nawait page.goto('https://en.wikipedia.org/wiki/October_17');\n``...

[ASSISTANT]
Here are five notable people born on October 17th:

1.  **Pope John Paul I** (1912) - Head of the Catholic Church and sovereign of the Vatican City State from August 26, 1978, until his death in September of the same year.
2.  **Arthur Miller** (1915) - American playwright and screenwriter, known for plays like *Death of a Salesman* and *The Crucible*.
3.  **Rita Hayworth** (1918) - American actress, dancer, and producer, a major glamour icon in the 1940s.
4.  **Evel Knievel** (1938) - American daredevil and entertainer, famous for his motorcycle stunts.
5.  **Eminem** (1972) - American rapper, songwriter, and record producer, widely considered one of the greatest and most influential hip-hop artists.

具有 URL 导航 + 可访问性快照的浏览器代理

有时网页的原始版本不易解释或缺少相关信息。在这种情况下，文本格式的可访问性快照可以帮助 LLM。

根据 Playwright MCP 服务器维护者的说法，这种方法通常比提供屏幕截图效果更好。

让我们重建我们的 Agent。

toolset = MCPToolset(server_info=server_info, tool_names=["browser_navigate", "browser_snapshot"])

agent = Agent(
        chat_generator=chat_generator,
        tools=toolset,
        system_prompt=system_message,
        exit_conditions=["text"],
        streaming_callback=print_streaming_chunk
)

🗞️🗞️ 比较在线新闻网站

现在让我们使用我们的 Agent 在不同网站上查找相似的新闻报道。

messages = [ChatMessage.from_user("""
1. Visit www.nbcnews.com
2. Visit www.bbc.com
3. Identify news stories that appear on both sites.
4. Your final response must contain only a Markdown table listing the shared news stories and their respective links from each site.
""")
]
result = agent.run(messages=messages)

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "www.nbcnews.com"}



ERROR:haystack.components.tools.tool_invoker:Failed to invoke Tool `browser_navigate` with parameters {'url': 'www.nbcnews.com'}. Error: Failed to invoke tool 'browser_navigate' with args: {'url': 'www.nbcnews.com'}, got error: Tool 'browser_navigate' returned an error: [TextContent(type='text', text='Error: page.goto: Protocol error (Page.navigate): Cannot navigate to invalid URL\nCall log:\n\x1b[2m  - navigating to "www.nbcnews.com", waiting until "domcontentloaded"\x1b[22m\n', annotations=None, meta=None)]




[ASSISTANT]
I apologize, but I encountered an error when trying to navigate to www.nbcnews.com. The error indicates an invalid URL. I will try again using the full URL including "https://".



[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.nbcnews.com"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.nbcnews.com\nawait page.goto('https://www.nbcnews.com');\n```\n\n### New console messages\n- [...

[ASSISTANT]
I have successfully visited NBCNews.com. Here's a summary of the main news stories and their links:

**NBC News Stories:**

*   **"New tariffs snap into effect, putting Trump’s global trade reset into high gear"**: https://www.nbcnews.com/business/business-news/trump-tariffs-latest-round-takes-effect-thursday-august-7-2025-rcna223461
*   **"Trump demands resignation of Intel CEO over alleged China ties"**: https://www.nbcnews.com/business/business-news/trump-demands-intel-ceo-resign-rcna223594
*   **"Trump hikes India tariffs to 50% for buying Russian oil"**: https://www.nbcnews.com/business/business-news/trump-hikes-india-tariffs-50-percent-buying-russian-oil-rcna223374
*   **"Kremlin says Putin-Trump meeting will happen in 'coming days'"**: https://www.nbcnews.com/world/russia/russia-putin-trump-meeting-confirmed-ukraine-rcna223572
*   **"Fort Stewart suspect shot five coworkers with personal handgun"**: https://www.nbcnews.com/news/us-news/quornelius-radford-know-fort-stewart-shooting-suspect-rcna223529
*   **"Decade of Supreme Court rulings have given states more unfettered power in redistricting"**: https://www.nbcnews.com/politics/supreme-court/supreme-court-rulings-states-unfettered-power-redistricting-rcna223146
*   **"Scientists want to prevent AI from going rogue by teaching it to be bad first"**: https://www.nbcnews.com/tech/tech-news/ai-anthropic-researchers-predicting-dangerous-behavior-rcna223236
*   **"Surgeon says UnitedHealthcare dispute may force her into bankruptcy"**: https://www.nbcnews.com/health/health-news/texas-surgeon-says-unitedhealthcare-dispute-may-force-bankruptcy-rcna223519
*   **"New research reveals clues about memory and longevity from the brains of 'SuperAgers'"**: https://www.nbcnews.com/health/aging/superaging-memory-social-aging-northwestern-university-study-over-80-rcna222908
*   **"Ultra-processed foods make up the majority of kids' diet, CDC finds"**: https://www.nbcnews.com/health/health-news/ultra-processed-foods-make-majority-kids-diet-cdc-report-finds-rcna223481
*   **"A salty twist: Diabetes risk study says french fries are a culprit"**: https://www.nbcnews.com/health/health-news/french-fries-diabetes-risk-type-2-study-fast-food-potatoes-rcna222961
*   **"Florida woman arrested after allegedly posing as nurse and treating patients with stolen license"**: https://www.nbcnews.com/news/us-news/florida-woman-arrested-allegedly-posing-nurse-treating-4000-patients-s-rcna223580
*   **"New photos show scale of destruction in Gaza as Israel weighs expanded military operation"**: https://www.nbcnews.com/world/gaza/gaza-israel-photos-destruction-expand-war-military-operation-rcna223575
*   **"Red Sox rookie who has just 159 at-bats signs $130 million contract"**: https://www.nbcnews.com/sports/mlb/red-sox-roman-anthony-130-million-contract-rcna223591
*   **"Man accused of killing Minnesota lawmaker to appear in court on murder charges"**: https://www.nbcnews.com/news/us-news/man-accused-killing-minnesota-lawmaker-appear-court-murder-charges-rcna223576
*   **"Man accused in D.C. Jewish museum killings indicted on federal hate crime charges"**: https://www.nbcnews.com/news/us-news/man-accused-killing-israeli-embassy-staffers-indicted-federal-hate-cri-rcna223534
*   **"Assemblies of God leaders address sex abuse of boys and young men in Texas sauna"**: https://www.nbcnews.com/news/us-news/assemblies-god-leaders-sex-abuse-scandal-chi-alpha-rcna223510
*   **"United Airlines outage that caused ground stops across the U.S. is resolved"**: https://www.nbcnews..com/news/us-news/united-airlines-outage-prompts-ground-stop-major-us-hubs-rcna223550
*   **"Trump threatens to take over D.C. police and says he might deploy the National Guard"**: https://www.nbcnews.com/politics/trump-administration/trump-threatens-dc-police-takeover-might-deploy-national-guard-rcna223535
*   **"UCLA says Trump administration has suspended $584 million in grants"**: https://www.nbcnews.com/news/us-news/ucla-says-trump-administration-suspended-584-million-grants-rcna223531
*   **"Congress doesn't want to talk to Alex Acosta, Jeffrey Epstein's 'sweetheart deal' maker"**: https://www.nbcnews.com/politics/congress/congress-doesnt-want-talk-alex-acosta-epsteins-sweetheart-deal-maker-rcna223543
*   **"Tulsi Gabbard declassified documents over CIA objections, sources say"**: https://www.nbcnews.com/politics/politics-news/tulsi-gabbard-declassified-documents-objections-cia-sources-say-rcna223548
*   **"Sen. Marsha Blackburn announces she's running for governor in Tennessee"**: https://www.nbcnews.com/politics/politics-news/sen-marsha-blackburn-announces-running-governor-tennessee-rcna223338
*   **"MLB is calling up its first female umpire"**: https://www.nbcnews.com/sports/mlb/mlb-first-female-umpire-jen-pawol-rcna223446
*   **"U.S. Open singles winners will get a record $5 million each"**: https://www.nbcnews.com/sports/tennis/us-open-singles-winners-5-million-rcna223442
*   **"Seattle Storm acquire All-Star guard Brittney Sykes in trade with Washington Mystics"**: https://www.nbcsports.com/wnba/news/seattle-storm-acquire-all-star-guard-brittney-sykes-in-trade-with-washington-mystics?cid=nbcnews
*   **"Canadian teenager Mboko reaches semifinals in Montreal and will face Rybakina"**: https://www.nbcsports.com/tennis/news/canadian-teenager-mboko-reaches-semifinals-in-montreal-and-will-face-rybakina?cid=nbcnews
*   **"Drake Maye on Mike Vrabel breaking up fight: We're trying to build that intensity"**: https://www.nbcsports.com/nfl/profootballtalk/rumor-mill/news/drake-maye-on-mike-vrabel-breaking-up-fight-were-trying-to-build-that-intensity?cid=nbcnews
*   **"It takes a village"**: https://www.nbcnews.com/news/nbcblk/trump-supporting-ohio-community-beloved-haitian-immigrant-faces-deport-rcna221579
*   **"First class (lounge)"**: https://www.nbcnews.com/business/travel/airport-vip-perks-expand-rcna221997
*   **"80 years later"**: https://www.nbcnews.com/world/asia/80-years-later-hiroshima-bombing-survivors-warn-new-nuclear-warfare-rcna220177
*   **"'Walking Dead' actor Kelley Mack dies at 33"**: https://www.nbcnews.com/pop-culture/pop-culture-news/walking-dead-actress-kelley-mack-dies-33-rcna223329
*   **"Some immigrants seeking green cards may be placed in removal proceedings, USCIS says"**: https://www.nbcnews.com/news/us-news/green-card-applicants-immigrants-removal-proceedings-uscis-rcna223099
*   **"France's biggest wildfire of the summer has eclipsed the size of Paris and is still spreading"**: https://www.nbcnews.com/world/europe/frances-biggest-wildfire-summer-eclipsed-size-paris-still-spreading-rcna223559
*   **"Myanmar’s U.S.-sanctioned acting President Myint Swe dies after long illness"**: https://www.nbcnews.com/world/myanmar/myanmars-us-sanctioned-acting-president-myint-swe-dies-long-illness-rcna223568
*   **"Eddie Palmieri, pioneering Latin jazz musician, dies at 88"**: https://www.nbcnews.com/news/us-news/eddie-palmieri-pioneering-latin-jazz-musician-dies-88-rcna223561
*   **"Kelly Clarkson says her ex-husband has been ill, postpones August concerts in Las Vegas"**: https://www.nbcnews.com/pop-culture/pop-culture-news/kelly-clarkson-says-ex-husband-ill-postpones-august-concerts-las-vegas-rcna223563
*   **"Wednesday Addams is back at Nevermore after an ominous season 1 cliffhanger"**: https://www.nbcnews.com/pop-culture/pop-culture-news/wednesday-season-1-recap-everything-remember-starting-season-2-rcna223367
*   **"Apple, under pressure from Trump, says it's boosting U.S. investment"**: https://www.nbcnews.com/business/business-news/apple-trump-pressure-us-investment-rcna223444
*   **"Claire's, known for piercing millions of teens' ears, files for Chapter 11"**: https://www.nbcnews.com/business/business-news/claires-known-piercing-millions-teens-ears-files-chapter-11-2nd-time-2-rcna223447
*   **"Student loan borrowers face new challenges trying to get Public Service Loan Forgiveness"**: https://www.nbcnews.com/business/personal-finance/student-loan-borrowers-face-new-challenges-trying-get-public-service-l-rcna223448
*   **"Wall Street gains as oil ends lower in volatile trade"**: https://www.nbcnews.com/business/markets/wall-street-gains-oil-ends-lower-volatile-trade-rcna223482

Now I will navigate to www.bbc.com.

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.bbc.com"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.bbc.com\nawait page.goto('https://www.bbc.com');\n```\n\n### New console messages\n- [ERROR] F...

[ASSISTANT]
| News Story | NBC News Link | BBC News Link |
|---|---|---|
| Trump tariffs / trade with India/China | https://www.nbcnews.com/business/business-news/trump-tariffs-latest-round-takes-effect-thursday-august-7-2025-rcna223461 <br> https://www.nbcnews.com/business/business-news/trump-demands-intel-ceo-resign-rcna223594 <br> https://www.nbcnews.com/business/business-news/trump-hikes-india-tariffs-50-percent-buying-russian-oil-rcna223374 | https://www.bbc.com/news/articles/c1w83j35jjjo <br> https://www.bbc.com/news/articles/c1dxr1g4y7yo <br> https://www.bbc.com/news/articles/cx23jmvn5yzo <br> https://www.bbc.com/news/articles/cly3r664q1eo <br> https://www.bbc.com/news/articles/cdx0n7y29kdo |
| Putin-Trump meeting | https://www.nbcnews.com/world/russia/russia-putin-trump-meeting-confirmed-ukraine-rcna223572 | https://www.bbc.com/news/articles/c14gkkzvpx8o <br> https://www.bbc.com/news/articles/cr5rdl1y8ndo |
| Gaza/Israel destruction/military operation | https://www.nbcnews.com/world/gaza/gaza-israel-photos-destruction-expand-war-military-operation-rcna223575 | https://www.bbc.com/news/articles/c8ryekj1m1do |
| Florida nurse arrest/impersonation | https://www.nbcnews.com/news/us-news/florida-woman-arrested-allegedly-posing-nurse-treating-4000-patients-s-rcna223580 | https://www.bbc.com/news/articles/clyrg2z350vo |
| United Airlines outage/delays | https://www.nbcnews.com/news/us-news/united-airlines-outage-prompts-ground-stop-major-us-hubs-rcna223550 | https://www.bbc.com/news/articles/cx29q585g3go |
| Apple investment/pressure from Trump | https://www.nbcnews.com/business/business-news/apple-trump-pressure-us-investment-rcna223444 | https://www.bbc.com/news/articles/cp8zyyygxv7o <br> https://www.bbc.com/news/articles/cdx0n7y29kdo |
| France wildfires | https://www.nbcnews.com/world/europe/frances-biggest-wildfire-summer-eclipsed-size-paris-still-spreading-rcna223559 | https://www.bbc.com/news/articles/cm2vk4xl204o <br> https://www.bbc.com/news/videos/cqxg47ggj29o <br> https://www.bbc.com/news/articles/cj6y803pjkwo |
| Myanmar's acting President Myint Swe dies | https://www.nbcnews.com/world/myanmar/myanmars-us-sanctioned-acting-president-myint-swe-dies-long-illness-rcna223568 | https://www.bbc.com/news/articles/c78m0d29rvno |
| 80th anniversary of Hiroshima atomic bombing | https://www.nbcnews.com/world/asia/80-years-later-hiroshima-bombing-survivors-warn-new-nuclear-warfare-rcna220177 | https://www.bbc.com/reel/video/p0lvdqmt/what-happened-at-hiroshima- <br> https://www.bbc.com/news/videos/cwye6k0xn7zo |

from IPython.display import display, Markdown

display(Markdown(result["last_message"].text))

绝对不差！

如果您查看流式传输的工具调用，您还会注意到 Agent 有时能够纠正自己的错误。

👨🏻‍💻 查找 GitHub 贡献者信息

这个例子不是原创的：我把它摘自一个 Hugging Face 教程，但它很棒。

messages = [ChatMessage.from_user("""
I'm trying to find how hard I have to work to get a repo in github.com/trending.
Can you navigate to the profile for the top author of the top trending repo,
and give me their total number of contributions over the last year?
""")
]

result = agent.run(messages=messages)

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://github.com/trending"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://github.com/trending\nawait page.goto('https://github.com/trending');\n```\n\n### Page state\n- Pag...

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://github.com/cjdsellers"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://github.com/cjdsellers\nawait page.goto('https://github.com/cjdsellers');\n```\n\n### Page state\n-...

[ASSISTANT]
The top trending repository is `nautechsystems/nautilus_trader`, and the top author is `cjdsellers`. They have made 4,203 contributions in the last year.

更多工具，用于更高级的浏览器代理！

要解锁高级用例，我们需要更多工具：点击、输入内容、返回和等待的能力。

toolset = MCPToolset(server_info=server_info,
                     tool_names=["browser_navigate", "browser_snapshot",
                                "browser_click", "browser_type",
                                 "browser_navigate_back","browser_wait_for",
                                ])

agent = Agent(
        chat_generator=chat_generator,
        tools=toolset,
        system_prompt=system_message,
        exit_conditions=["text"],
        streaming_callback=print_streaming_chunk
)

🖱️ 在亚马逊上查找产品的价格范围

我想买一个新鼠标。让我们用我们的 Agent 来发现亚马逊上这款产品的价格范围。

prompt="""
1. Go to Amazon Italy and find all available prices for the Logitech MX Master 3S mouse.
2. Exclude any items that:
- are not computer mice;
- are not the exact Logitech MX Master 3S model;
- do not display a price;
- are bundled with other products or accessories.
3. Your final message must only contain a Markdown table with two columns: Name and Price.
"""

result = agent.run(messages=[ChatMessage.from_user(prompt)])

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.amazon.it"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.amazon.it\nawait page.goto('https://www.amazon.it');\n```\n\n### Page state\n- Page URL: https...

[TOOL CALL]
Tool: browser_type 
Arguments: {"submit": true, "element": "Search Amazon.it", "ref": "e141", "text": "Logitech MX Master 3S"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Fill \"Logitech MX Master 3S\" into \"Search Amazon.it\"\nawait page.getByRole('searchbox', { name: 'Search Amazon.it'...

[ASSISTANT]
Here are the Logitech MX Master 3S mice available on Amazon Italy with their prices:

| Name | Price |
|---|---|
| MX Master 3S - Mouse Wireless ad Alte Prestazioni, Scorrimento Ultraveloce, Ergo, 8K DPI, Tracciamento su Vetro, Clic Silenziosi, USB C, Bluetooth, Windows, Linux, Chrome - Grigio scuro | €68.53 |
| MX Master 3S - High Performance Wireless Mouse, Ultra Fast Scrolling, Ergo, 8K DPI, Glass Tracking, Silent Clicks, USB C, Bluetooth, Windows, Linux, Chrome - Light Grey | €71.16 |
| MX Master 3S for Mac - Wireless Bluetooth Mouse with Ultra Fast Scrolling, Ergonomic, 8000 DPI, Glass Tracking, Silent Clicks, USB-C, Apple, iPad - Space Grey | €65.08 |
| MX Master 3S for Business, Wireless Mouse with Silent Click, 8K DPI, USB Receiver with Logi Bolt Protected, Bluetooth, USB-C, MagSpeed, Windows/Mac/Chrome/Linux - Grey | €79.82 |
| MX Master 3S for Mac - Ultra Fast Scrolling Bluetooth Mouse, Ergonomic 8K DPI, Glass Tracking, Silent Clicks, USB-C, Apple, iPad - Light Grey | €74.02 |

from IPython.display import display, Markdown

display(Markdown(result["last_message"].text))

以下是亚马逊意大利站上 Logitech MX Master 3S 鼠标及其价格

🖥️ GitHub 探索

prompt = "List some recent PRs merged by anakin87 on the deepset-ai/haystack repo on GitHub. Max 5."
result = agent.run(messages=[ChatMessage.from_user(prompt)])

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://github.com/deepset-ai/haystack/pulls"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://github.com/deepset-ai/haystack/pulls\nawait page.goto('https://github.com/deepset-ai/haystack/pull...

[TOOL CALL]
Tool: browser_type 
Arguments: {"element": "textbox \"Search all issues\"", "submit": true, "ref": "e115", "text": "is:pr author:anakin87 is:merged"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Fill \"is:pr author:anakin87 is:merged\" into \"textbox \"Search all issues\"\"\nawait page.getByRole('textbox', { nam...

[ASSISTANT]
Here are 5 recent PRs merged by anakin87 on the deepset-ai/haystack repo:

1.  **#9690**: chore: remove unused `type:ignore` and `cast` (merged Aug 7, 2025)
2.  **#9682**: fix: `ChatMessage` bugfixes (merged Aug 6, 2025)
3.  **#9679**: feat: add serde methods to `ImageContent` and `TextContent` (merged Aug 5, 2025)
4.  **#9662**: chore: re-export missing symbols in `__init__.py` files (merged Jul 30, 2025)
5.  **#9659**: test: fix Datadog tests and unpin `ddtrace` test dependency (merged Jul 30, 2025)

▶️ 从 YouTube 查找内容创建者的社交资料

从 YouTube 视频开始，查找创作者的社交资料。由于搜索引擎有时会阻止自动化工具，我们要求模型找到智能的替代方法来查找信息。

prompt="""
1. Open this YouTube video: https://www.youtube.com/watch?v=axmaslLO4-4 and extract the channel author’s username.
2. Then, try to find all their social media profiles by searching the web using the username.
If you cannot perform a web search, try other available methods to find the profiles.
3. Return only the links to their social media accounts along with the platform names.
"""

result = agent.run(messages=[ChatMessage.from_user(prompt)])

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.youtube.com/watch?v=axmaslLO4-4"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.youtube.com/watch?v=axmaslLO4-4\nawait page.goto('https://www.youtube.com/watch?v=axmaslLO4-4'...

[TOOL CALL]
Tool: browser_snapshot 
Arguments: 

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### New console messages\n- [INFO] LegacyDataMixin will be applied to all legacy elements.\nSet `_legacyUndefinedCheck: true` on ...\n- [WARNING] [GroupM...

[TOOL CALL]
Tool: browser_snapshot 
Arguments: 

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### New console messages\n- [INFO] LegacyDataMixin will be applied to all legacy elements.\nSet `_legacyUndefinedCheck: true` on ...\n- [WARNING] [GroupM...

[TOOL CALL]
Tool: browser_click 
Arguments: {"element": "Mr. RIP channel link", "ref": "e220"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Click Mr. RIP channel link\nawait page.getByRole('link', { name: 'Mr. RIP' }).click();\n```\n\n### New console message...

[ASSISTANT]
The channel author's username is "Mr. RIP".


[TOOL CALL]
Tool: browser_snapshot 
Arguments: 

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### New console messages\n- [INFO] LegacyDataMixin will be applied to all legacy elements.\nSet `_legacyUndefinedCheck: true` on ...\n- [WARNING] [GroupM...

[TOOL CALL]
Tool: browser_click 
Arguments: {"element": "and 4 more links", "ref": "e1199"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Click and 4 more links\nawait page.getByRole('button', { name: 'and 4 more links' }).click();\n```\n\n### New console ...

[ASSISTANT]
Here are the social media profiles for Mr. RIP:

*   **Twitch:** https://www.twitch.tv/retireinprogress
*   **Blog:** https://retireinprogress.com/
*   **Telegram:** https://t.me/newsrip
*   **Discord:** mr.rip/discord

🗺️ 使用 Google 地图查找位置

result = agent.run(messages=[ChatMessage.from_user("Use Google Maps to find the deepset HQ in Berlin")])

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.google.com/maps"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.google.com/maps\nawait page.goto('https://www.google.com/maps');\n```\n\n### New console messa...

[TOOL CALL]
Tool: browser_type 
Arguments: {"ref": "e35", "element": "Search Google Maps", "text": "deepset HQ Berlin", "submit": true}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Fill \"deepset HQ Berlin\" into \"Search Google Maps\"\nawait page.getByRole('combobox', { name: 'Search Google Maps' ...

[TOOL CALL]
Tool: browser_click 
Arguments: {"element": "deepset HQ Berlin", "ref": "e182"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Click deepset HQ Berlin\nawait page.locator('#searchboxinput').click();\n```\n\n### New console messages\n- [WARNING] ...

[ASSISTANT]
The deepset HQ in Berlin is located at Zinnowitzer Str. 1, 10115 Berlin, Germany.

🚂 🚌 查找公共交通出行选项

使用 Google 地图，让我们问一个更难的问题，这需要更多的导航。

prompt = """
1. Using Google Maps, find the next 3 available public transportation travel options from Paris to Berlin departing today.
2. For each option, provide a detailed description of the route (e.g., transfers, stations, duration).
3. Include a direct link to the corresponding Google Maps route for each travel option.
"""

result = agent.run(messages=[ChatMessage.from_user(prompt)])

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.google.com/maps"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.google.com/maps\nawait page.goto('https://www.google.com/maps');\n```\n\n### New console messa...

[TOOL CALL]
Tool: browser_type 
Arguments: {"ref": "e37", "element": "Search Google Maps combobox", "text": "Paris to Berlin"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Fill \"Paris to Berlin\" into \"Search Google Maps combobox\"\nawait page.getByRole('combobox', { name: 'Search Google...

[TOOL CALL]
Tool: browser_click 
Arguments: {"ref": "e39", "element": "Search button"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Click Search button\nawait page.getByRole('button', { name: 'Search' }).click();\n```\n\n### New console messages\n- [...

[TOOL CALL]
Tool: browser_click 
Arguments: {"element": "Directions button", "ref": "e42"}



ERROR:haystack.components.tools.tool_invoker:Failed to invoke Tool `browser_click` with parameters {'element': 'Directions button', 'ref': 'e42'}. Error: Failed to invoke tool 'browser_click' with args: {'element': 'Directions button', 'ref': 'e42'}, got error: Tool 'browser_click' returned an error: [TextContent(type='text', text='Error: Ref e42 not found in the current page snapshot. Try capturing new snapshot.', annotations=None, meta=None)]




[TOOL CALL]
Tool: browser_snapshot 
Arguments: 

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### New console messages\n- [WARNING] [GroupMarkerNotSet(crbug.com/242999)!:A0702B00AC0E0000]Automatic fallback to software WebG...\n- [ERROR] Failed to ...

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://www.google.com/maps/dir/Paris/Berlin/data=!4m2!4m1!3e3"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://www.google.com/maps/dir/Paris/Berlin/data=!4m2!4m1!3e3\nawait page.goto('https://www.google.com/ma...

[ASSISTANT]
Here are the next 3 available public transportation travel options from Paris to Berlin departing today:

---

**Option 1:**

*   **Duration:** 10 hours 55 minutes
*   **Departure/Arrival:** 5:10 PM from Gare de l'Est (Paris) – 4:05 AM (Friday) in Berlin
*   **Route Description:**
    *   Train ICE9555 from Gare de l'Est (Paris).
    *   Walk for 4 minutes.
    *   Bus FlixBus.
*   **Google Maps Link:** https://www.google.com/maps/dir/Paris/Berlin/data=!4m2!4m1!3e3

---

**Option 2:**

*   **Duration:** 11 hours 11 minutes
*   **Departure/Arrival:** 6:25 PM from Paris – 5:36 AM (Friday) in Berlin
*   **Route Description:**
    *   High-speed train Eurostar.
    *   Transfer to High-speed train ICE319.
    *   Transfer to High-speed train ICE100.
*   **Google Maps Link:** https://www.google.com/maps/dir/Paris/Berlin/data=!4m2!4m1!3e3

---

**Option 3:**

*   **Duration:** 10 hours 41 minutes
*   **Departure/Arrival:** 6:55 PM from Paris – 5:36 AM (Friday) in Berlin
*   **Route Description:**
    *   High-speed train Eurostar.
    *   Transfer to High-speed train ICE100.
*   **Google Maps Link:** https://www.google.com/maps/dir/Paris/Berlin/data=!4m2!4m1!3e3

---

Please note that Google Maps sometimes shows general options first, and clicking into them provides more granular details. The provided links are for the general Paris to Berlin transit route, on which these specific options are displayed.

准备就绪！

🖼️ 🤗 通过 Hugging Face Spaces 生成图像

现在让我们尝试更高级的操作：访问 Hugging Face Spaces 来生成图像；请注意，我们还要求我们的 Agent 为图像生成模型创建好的提示。

prompt="""
1. Visit Hugging Face Spaces, search the Space named exactly "FLUX.1 [schnell]" and enter it.
2. Craft a detailed, descriptive prompt using your language skills to depict: "my holiday on Lake Como".
3. Use this prompt to generate an image within the Space.
4. After prompting, wait 5 seconds to allow the image to fully generate. Repeat until the image is generated.
5. Your final response must contain only the direct link to the generated image — no additional text.
"""

result = agent.run(messages=[ChatMessage.from_user(prompt)])

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://hugging-face.cn/spaces/flux.1/schnell"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://hugging-face.cn/spaces/flux.1/schnell\nawait page.goto('https://hugging-face.cn/spaces/flux.1/schnel...

[TOOL CALL]
Tool: browser_navigate 
Arguments: {"url": "https://hugging-face.cn/spaces"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Navigate to https://hugging-face.cn/spaces\nawait page.goto('https://hugging-face.cn/spaces');\n```\n\n### Page state\n-...

[TOOL CALL]
Tool: browser_type 
Arguments: {"element": "Search models, datasets, users...", "text": "flux.1/schnell", "ref": "e11"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Fill \"flux.1/schnell\" into \"Search models, datasets, users...\"\nawait page.getByRole('textbox', { name: 'Search mo...

[TOOL CALL]
Tool: browser_click 
Arguments: {"element": "img", "ref": "e12"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Click img\nawait page.locator('.absolute').first().click();\n```\n\n### Page state\n- Page URL: https://hugging-face.cn...

[TOOL CALL]
Tool: browser_click 
Arguments: {"ref": "e1115", "element": "\ud83c\udfce\ufe0f\ud83d\udca8 black-forest-labs/FLUX.1-schnell"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Click 🏎️💨 black-forest-labs/FLUX.1-schnell\nawait page.getByRole('link', { name: '🏎️💨 black-forest-labs/FLUX.' }).clic...

[TOOL CALL]
Tool: browser_type 
Arguments: {"text": "A serene, sun-drenched afternoon on Lake Como. A classic Riva speedboat gently glides across the azure water, leaving a soft wake. Elegant villas with terracotta roofs and lush gardens dot the shoreline, backed by rolling green hills. In the distance, the majestic Italian Alps stand guard under a clear, cerulean sky. The air is warm and filled with the scent of blooming flowers and fresh lake water. A sense of tranquil luxury pervades the scene.", "ref": "f1e23", "element": "Prompt"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Fill \"A serene, sun-drenched afternoon on Lake Como. A classic Riva speedboat gently glides across the azure water, l...

[TOOL CALL]
Tool: browser_click 
Arguments: {"ref": "f1e24", "element": "Run"}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Ran Playwright code\n```js\n// Click Run\nawait page.locator('#iFrameResizer0').contentFrame().getByRole('button', { name: 'Run' }).click();\n```\n\n...

[TOOL CALL]
Tool: browser_wait_for 
Arguments: {"time": 5}

[TOOL RESULT]
{"meta":null,"content":[{"type":"text","text":"### Result\nWaited for 5\n\n### New console messages\n- [WARNING] Unrecognized feature: 'ambient-light-sensor'. @ https://hugging-face.cn/spaces/black-fo...

[ASSISTANT]
https://black-forest-labs-flux-1-schnell.hf.space/file=/tmp/gradio/3a1d55dec7f02816a136f855fafe9c754364d14f/image.webp

from PIL import Image
import requests
from io import BytesIO
from IPython.display import display

response = requests.get(result["last_message"].text)
image = Image.open(BytesIO(response.content))
display(image)

🏞️ 令人印象深刻！

下面的视频演示了一个类似的本地设置。看着 Agent 像人类一样在网络上导航，真是令人着迷。

import gdown
from ipywidgets import Video

url = "https://drive.google.com/drive/folders/1HyPdNxpzi3svmPVYGXak7mqAjVWztwsH"

gdown.download_folder(url, quiet=True, output=".")

Video.from_file('/content/browser_agent.mp4', autoplay=False, loop=False)

Video(value=b'\x00\x00\x00 ftypisom\x00\x00\x02\x00isomiso2avc1mp41\x00\x00\x00\x08free\x01SU<mdat\x00\x00\x00…

下一步是什么？

当我开始探索浏览器代理时，我曾预期会很快遇到障碍。但有了合适的框架、一个强大的 LLM 和 Playwright MCP 服务器（即使没有截图功能），您实际上可以走得很远。本 Notebook 展示了这个过程中的一些有趣且有时令人印象深刻的演示。

我们还希望探索如何将这些 Agent 从 Notebook 迁移到生产环境：在 Docker 环境中部署，通过 API 公开，并集成到简单的 UI 中。未来将有更多关于这方面的内容。

当然，有些用例需要视觉能力或身份验证处理。如果您对这些功能的兴趣，请随时打开一个 GitHub 讨论。

笔记本作者：Stefano Fiorucci