2025-10-14

實(shí)戰(zhàn)教程:構(gòu)建能交互網(wǎng)頁的 AI 助手——基于 Playwright MCP 的完整項(xiàng)目

項(xiàng)目概述:打造智能網(wǎng)頁操作助手

在本教程中,我們將構(gòu)建一個(gè)完整的、能夠?qū)嶋H交互網(wǎng)頁的AI助手。這個(gè)助手不僅能理解自然語言指令,還能通過 Playwright MCP 執(zhí)行復(fù)雜的網(wǎng)頁操作。我們將從零開始,搭建一個(gè)功能完備的系統(tǒng),涵蓋從環(huán)境配置到實(shí)際部署的全流程。

項(xiàng)目目標(biāo)

構(gòu)建一個(gè)能夠執(zhí)行以下任務(wù)的AI助手:

  • 自動(dòng)登錄網(wǎng)站并處理認(rèn)證
  • 填寫復(fù)雜表單和交互元素
  • 提取、分析和結(jié)構(gòu)化網(wǎng)頁數(shù)據(jù)
  • 處理多步驟工作流程
  • 應(yīng)對網(wǎng)頁異常和動(dòng)態(tài)內(nèi)容

一、項(xiàng)目架構(gòu)設(shè)計(jì)

技術(shù)棧選擇

  • 后端框架: Node.js + Express
  • 瀏覽器自動(dòng)化: Playwright
  • AI 模型集成: Anthropic Claude API
  • 協(xié)議層: 自定義 MCP (Model Context Protocol) Server
  • 前端界面: React + Tailwind CSS
  • 數(shù)據(jù)庫: SQLite (用于會(huì)話存儲(chǔ))
  • 任務(wù)隊(duì)列: Bull (用于異步任務(wù)處理)

系統(tǒng)架構(gòu)

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">用戶界面 (React) ↓ (HTTP/REST API) 后端服務(wù)器 (Express + AI 路由) ↓ (MCP 協(xié)議) Playwright MCP Server ↓ (瀏覽器控制) Chromium/Firefox 實(shí)例 </pre>

二、環(huán)境準(zhǔn)備與項(xiàng)目初始化

步驟1:創(chuàng)建項(xiàng)目結(jié)構(gòu)

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">mkdir ai-web-assistant cd ai-web-assistant mkdir -p src/{mcp,ai,routes,models,utils} public/{css,js} tests touch package.json server.js .env.example README.md </pre>

步驟2:定義項(xiàng)目依賴

創(chuàng)建 package.json

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">{ "name": "ai-web-assistant", "version": "1.0.0", "type": "module", "scripts": { "start": "node server.js", "dev": "nodemon server.js", "test": "jest", "mcp:dev": "node src/mcp/server.js" }, "dependencies": { "express": "^4.18.2", "cors": "^2.8.5", "dotenv": "^16.3.0", "playwright": "^1.40.0", "@anthropic-ai/sdk": "^0.7.0", "sqlite3": "^5.1.6", "bull": "^4.11.0", "express-rate-limit": "^7.1.0", "helmet": "^7.0.0" }, "devDependencies": { "nodemon": "^3.0.0", "jest": "^29.6.0" } } </pre>

運(yùn)行 npm install 安裝依賴。

步驟3:環(huán)境配置

創(chuàng)建 .env 文件:

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"># API 配置 ANTHROPIC_API_KEY=your_anthropic_api_key_here PORT=3000 NODE_ENV=development # 瀏覽器配置 BROWSER_TYPE=chromium HEADLESS_MODE=false BROWSER_TIMEOUT=30000 # 數(shù)據(jù)庫配置 DB_PATH=./data/sessions.db # 安全配置 SESSION_SECRET=your_session_secret_here RATE_LIMIT_WINDOW=900000 RATE_LIMIT_MAX=100 </pre>

三、核心模塊實(shí)現(xiàn)

1. Playwright MCP Server 實(shí)現(xiàn)

創(chuàng)建 src/mcp/server.js

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">import { chromium, firefox, webkit } from'playwright'; import { EventEmitter } from'events'; class PlaywrightMCPServer extends EventEmitter { constructor(config = {}) { super(); this.config = { browserType: config.browserType || 'chromium', headless: config.headless !== false, timeout: config.timeout || 30000, ...config }; this.browser = null; this.context = null; this.page = null; this.isInitialized = false; this.sessionId = null; } // 初始化瀏覽器實(shí)例 async initialize(sessionId = null) { try { this.sessionId = sessionId ||session_{Date.now()}`; const browserMap = { chromium, firefox, webkit }; const BrowserClass = browserMap[this.config.browserType] || chromium; this.browser = await BrowserClass.launch({ headless: this.config.headless, timeout: this.config.timeout, args: ['--no-sandbox', '--disable-dev-shm-usage'] }); this.context = awaitthis.browser.newContext({ viewport: { width: 1280, height: 720 }, userAgent: 'AI-Web-Assistant/1.0', acceptDownloads: true, ignoreHTTPSErrors: true }); // 添加頁面錯(cuò)誤處理 this.context.on('page', page => { page.on('pageerror', error => { this.emit('pageError', { sessionId: this.sessionId, error }); }); }); this.page = awaitthis.context.newPage(); // 設(shè)置默認(rèn)超時(shí) this.page.setDefaultTimeout(this.config.timeout); this.page.setDefaultNavigationTimeout(this.config.timeout * 2); this.isInitialized = true; this.emit('initialized', { sessionId: this.sessionId }); return { success: true, message: 'Playwright MCP Server initialized successfully', sessionId: this.sessionId }; } catch (error) { console.error('Failed to initialize Playwright:', error); this.emit('error', error); return { success: false, error: error.message }; } } // 工具定義 - MCP 協(xié)議核心 getTools() { return { navigate: { name: 'navigate', description: 'Navigate to a specific URL', parameters: { url: { type: 'string', description: 'The URL to navigate to' }, waitUntil: { type: 'string', description: 'When to consider navigation successful', enum: ['load', 'domcontentloaded', 'networkidle'], default: 'networkidle' } } }, click: { name: 'click', description: 'Click on an element using CSS selector, XPath, or text', parameters: { selector: { type: 'string', description: 'CSS selector, XPath, or text to identify the element' }, selectorType: { type: 'string', description: 'Type of selector: css, xpath, or text', enum: ['css', 'xpath', 'text'], default: 'css' }, waitForNavigation: { type: 'boolean', description: 'Whether to wait for navigation after click', default: false } } }, fill_form: { name: 'fill_form', description: 'Fill a form with multiple fields', parameters: { fields: { type: 'object', description: 'Object mapping selectors to values' } } }, extract_data: { name: 'extract_data', description: 'Extract structured data from the page', parameters: { schema: { type: 'object', description: 'Schema defining what data to extract' } } }, wait_for_element: { name: 'wait_for_element', description: 'Wait for an element to appear', parameters: { selector: { type: 'string', description: 'CSS selector for the element' }, state: { type: 'string', description: 'Element state to wait for', enum: ['attached', 'detached', 'visible', 'hidden'], default: 'visible' }, timeout: { type: 'number', description: 'Timeout in milliseconds', default: 10000 } } }, screenshot: { name: 'screenshot', description: 'Take a screenshot for debugging', parameters: { fullPage: { type: 'boolean', description: 'Whether to capture full page', default: false } } }, get_page_info: { name: 'get_page_info', description: 'Get comprehensive information about the current page' } }; } // 工具執(zhí)行引擎 async executeTool(toolName, parameters = {}) { if (!this.isInitialized) { thrownewError('Playwright not initialized. Call initialize() first.'); } try { let result; switch (toolName) { case'navigate': result = awaitthis.navigateToUrl(parameters.url, parameters.waitUntil); break; case'click': result = awaitthis.clickElement(parameters.selector, parameters.selectorType, parameters.waitForNavigation); break; case'fill_form': result = awaitthis.fillForm(parameters.fields); break; case'extract_data': result = awaitthis.extractData(parameters.schema); break; case'wait_for_element': result = awaitthis.waitForElement(parameters.selector, parameters.state, parameters.timeout); break; case'screenshot': result = awaitthis.takeScreenshot(parameters.fullPage); break; case'get_page_info': result = awaitthis.getPageInfo(); break; default: thrownewError(`Unknown tool:{toolName}); } this.emit('toolExecuted', { sessionId: this.sessionId, toolName, parameters, result }); return { success: true, data: result }; } catch (error) { console.error(Tool execution failed: {toolName}`, error); this.emit('toolError', { sessionId: this.sessionId, toolName, parameters, error: error.message }); return { success: false, error: error.message, suggestion: this.getErrorSuggestion(error.message) }; } } // 具體的工具實(shí)現(xiàn)方法 async navigateToUrl(url, waitUntil = 'networkidle') { if (!url.startsWith('http')) { url = 'https://' + url; } const response = awaitthis.page.goto(url, { waitUntil, timeout: this.config.timeout }); return { url: this.page.url(), status: response?.status(), title: awaitthis.page.title(), finalUrl: this.page.url() }; } async clickElement(selector, selectorType = 'css', waitForNavigation = false) { let element; switch (selectorType) { case'css': element = this.page.locator(selector); break; case'xpath': element = this.page.locator(`xpath={selector}); break; case'text': element = this.page.getByText(selector, { exact: false }); break; default: thrownewError(Unsupported selector type: ${selectorType}); } await element.waitFor({ state: 'visible' }); if (waitForNavigation) { awaitPromise.all([ this.page.waitForNavigation({ waitUntil: 'networkidle' }), element.click() ]); } else { await element.click(); } return { success: true, element: awaitthis.getElementInfo(element) }; } async fillForm(fields) { const results = {}; for (const [selector, value] ofObject.entries(fields)) { try { const element = this.page.locator(selector); await element.waitFor({ state: 'visible' }); await element.fill(value); results[selector] = { success: true, value }; } catch (error) { results[selector] = { success: false, error: error.message }; } } return results; } async extractData(schema) { const data = {}; for (const [key, config] ofObject.entries(schema)) { try { const { selector, type = 'text', attribute } = config; const element = this.page.locator(selector); switch (type) { case'text': data[key] = await element.textContent(); break; case'attribute': data[key] = await element.getAttribute(attribute); break; case'multiple': data[key] = await element.allTextContents(); break; default: data[key] = await element.textContent(); } } catch (error) { data[key] = null; } } return data; } async getElementInfo(element) { try { const boundingBox = await element.boundingBox(); const isVisible = await element.isVisible(); return { visible: isVisible, boundingBox, tagName: await element.evaluate(el => el.tagName.toLowerCase()) }; } catch (error) { return { error: error.message }; } } async takeScreenshot(fullPage = false) { const screenshot = awaitthis.page.screenshot({ fullPage, type: 'png' }); return { screenshot: screenshot.toString('base64'), type: 'png', fullPage }; } async getPageInfo() { return { url: this.page.url(), title: awaitthis.page.title(), content: awaitthis.page.content(), viewport: this.page.viewportSize() }; } // 錯(cuò)誤處理和建議 getErrorSuggestion(errorMessage) { const suggestions = { 'timeout': '嘗試增加等待時(shí)間或檢查網(wǎng)絡(luò)連接', 'element not found': '檢查選擇器是否正確,或等待元素加載', 'navigation failed': '檢查URL是否正確,或網(wǎng)站是否可訪問', 'target closed': '瀏覽器頁面已關(guān)閉,需要重新初始化' }; for (const [key, suggestion] ofObject.entries(suggestions)) { if (errorMessage.toLowerCase().includes(key)) { return suggestion; } } return'請檢查網(wǎng)絡(luò)連接和頁面狀態(tài)后重試'; } // 清理資源 async cleanup() { try { if (this.page) { awaitthis.page.close(); } if (this.context) { awaitthis.context.close(); } if (this.browser) { awaitthis.browser.close(); } this.isInitialized = false; this.emit('cleanedUp', { sessionId: this.sessionId }); return { success: true, message: 'Resources cleaned up successfully' }; } catch (error) { console.error('Cleanup failed:', error); return { success: false, error: error.message }; } } } exportdefault PlaywrightMCPServer; </pre>

2. AI 處理模塊

創(chuàng)建 src/ai/handler.js

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">import Anthropic from'@anthropic-ai/sdk'; import PlaywrightMCPServer from'../mcp/server.js'; class AIHandler { constructor(apiKey) { this.anthropic = new Anthropic({ apiKey }); this.mcpServer = new PlaywrightMCPServer(); this.conversationHistory = newMap(); } // 初始化會(huì)話 async initializeSession(sessionId) { const result = awaitthis.mcpServer.initialize(sessionId); if (!this.conversationHistory.has(sessionId)) { this.conversationHistory.set(sessionId, []); } return result; } // 處理用戶指令 async processInstruction(sessionId, instruction, context = {}) { try { const history = this.conversationHistory.get(sessionId) || []; // 構(gòu)建系統(tǒng)提示詞 const systemPrompt = this.buildSystemPrompt(context); // 獲取可用工具 const availableTools = this.mcpServer.getTools(); // 調(diào)用 Claude 模型 const message = awaitthis.anthropic.messages.create({ model: "claude-3-sonnet-20240229", max_tokens: 4096, system: systemPrompt, messages: [ ...history, { role: "user", content: instruction } ], tools: Object.values(availableTools) }); let finalResponse = ''; let currentMessage = message; // 處理工具調(diào)用 while (currentMessage.content.some(item => item.type === 'tool_use')) { const toolResults = []; for (const contentItem of currentMessage.content) { if (contentItem.type === 'tool_use') { const toolName = contentItem.name; const parameters = contentItem.input; // 執(zhí)行工具 const toolResult = awaitthis.mcpServer.executeTool(toolName, parameters); toolResults.push({ type: 'tool_result', tool_use_id: contentItem.id, content: JSON.stringify(toolResult) }); } } // 繼續(xù)對話 currentMessage = awaitthis.anthropic.messages.create({ model: "claude-3-sonnet-20240229", max_tokens: 4096, messages: [ ...history, { role: "user", content: instruction }, { role: "assistant", content: currentMessage.content }, { role: "user", content: toolResults } ], tools: Object.values(availableTools) }); } // 提取最終響應(yīng) const textContent = currentMessage.content.find(item => item.type === 'text'); finalResponse = textContent ? textContent.text : '操作完成'; // 更新對話歷史 history.push( { role: "user", content: instruction }, { role: "assistant", content: currentMessage.content } ); // 保持最近10輪對話 if (history.length > 20) { history.splice(0, 4); } return { success: true, response: finalResponse, sessionId }; } catch (error) { console.error('AI processing failed:', error); return { success: false, error: error.message, sessionId }; } } // 構(gòu)建系統(tǒng)提示詞 buildSystemPrompt(context) { return你是一個(gè)專業(yè)的網(wǎng)頁操作助手,可以通過瀏覽器自動(dòng)化工具執(zhí)行各種網(wǎng)頁任務(wù)。 你的能力包括: - 導(dǎo)航到指定網(wǎng)址 - 點(diǎn)擊按鈕和鏈接 - 填寫表單和輸入框 - 提取網(wǎng)頁數(shù)據(jù) - 等待頁面加載 - 處理復(fù)雜交互 重要指導(dǎo)原則: 1. 在執(zhí)行操作前先分析頁面結(jié)構(gòu) 2. 使用合適的選擇器定位元素 3. 處理可能出現(xiàn)的錯(cuò)誤和異常 4. 提供清晰的操作反饋 5. 對于復(fù)雜任務(wù),分解為多個(gè)步驟執(zhí)行 當(dāng)前上下文:${JSON.stringify(context)} 請謹(jǐn)慎操作,確保每一步都正確執(zhí)行。如果遇到錯(cuò)誤,請分析原因并提供解決方案。; } // 獲取會(huì)話歷史 getSessionHistory(sessionId) { returnthis.conversationHistory.get(sessionId) || []; } // 清理會(huì)話 async cleanupSession(sessionId) { this.conversationHistory.delete(sessionId); returnawaitthis.mcpServer.cleanup(); } } exportdefault AIHandler; </pre>

3. Express 服務(wù)器和路由

創(chuàng)建 server.js

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">import express from'express'; import cors from'cors'; import helmet from'helmet'; import rateLimit from'express-rate-limit'; import dotenv from'dotenv'; import AIHandler from'./src/ai/handler.js'; // 加載環(huán)境變量 dotenv.config(); const app = express(); const PORT = process.env.PORT || 3000; // 初始化 AI 處理器 const aiHandler = new AIHandler(process.env.ANTHROPIC_API_KEY); // 中間件配置 app.use(helmet()); app.use(cors()); app.use(express.json({ limit: '10mb' })); // 速率限制 const limiter = rateLimit({ windowMs: parseInt(process.env.RATE_LIMIT_WINDOW) || 15 * 60 * 1000, max: parseInt(process.env.RATE_LIMIT_MAX) || 100, message: '請求過于頻繁,請稍后再試' }); app.use(limiter); // 會(huì)話存儲(chǔ) const sessions = newMap(); // API 路由 // 健康檢查 app.get('/health', (req, res) => { res.json({ status: 'ok', timestamp: newDate().toISOString() }); }); // 初始化會(huì)話 app.post('/api/session/init', async (req, res) => { try { const sessionId = req.body.sessionId ||session_{Date.now()}_{Math.random().toString(36).substr(2, 9)}; const result = await aiHandler.initializeSession(sessionId); if (result.success) { sessions.set(sessionId, { createdAt: newDate(), lastActivity: newDate() }); res.json({ success: true, sessionId, message: '會(huì)話初始化成功' }); } else { res.status(500).json({ success: false, error: result.error }); } } catch (error) { console.error('Session init error:', error); res.status(500).json({ success: false, error: error.message }); } }); // 處理用戶指令 app.post('/api/instruction', async (req, res) => { try { const { sessionId, instruction, context = {} } = req.body; if (!sessionId || !instruction) { return res.status(400).json({ success: false, error: '缺少必要參數(shù):sessionId 和 instruction' }); } // 更新會(huì)話活動(dòng)時(shí)間 const session = sessions.get(sessionId); if (session) { session.lastActivity = newDate(); } const result = await aiHandler.processInstruction(sessionId, instruction, context); res.json(result); } catch (error) { console.error('Instruction processing error:', error); res.status(500).json({ success: false, error: error.message }); } }); // 獲取會(huì)話歷史 app.get('/api/session/:sessionId/history', (req, res) => { const { sessionId } = req.params; const history = aiHandler.getSessionHistory(sessionId); res.json({ success: true, sessionId, history }); }); // 清理會(huì)話 app.delete('/api/session/:sessionId', async (req, res) => { try { const { sessionId } = req.params; const result = await aiHandler.cleanupSession(sessionId); sessions.delete(sessionId); res.json({ success: true, sessionId, message: '會(huì)話清理成功' }); } catch (error) { console.error('Session cleanup error:', error); res.status(500).json({ success: false, error: error.message }); } }); // 會(huì)話清理任務(wù)(定期清理過期會(huì)話) setInterval(() => { const now = newDate(); const SESSION_TIMEOUT = 30 * 60 * 1000; // 30分鐘 for (const [sessionId, session] of sessions.entries()) { if (now - session.lastActivity > SESSION_TIMEOUT) { console.log(清理過期會(huì)話: {sessionId}`); aiHandler.cleanupSession(sessionId); sessions.delete(sessionId); } } }, 5 * 60 * 1000); // 每5分鐘檢查一次 // 錯(cuò)誤處理中間件 app.use((error, req, res, next) => { console.error('Unhandled error:', error); res.status(500).json({ success: false, error: '服務(wù)器內(nèi)部錯(cuò)誤' }); }); // 404 處理 app.use('*', (req, res) => { res.status(404).json({ success: false, error: '接口不存在' }); }); // 啟動(dòng)服務(wù)器 app.listen(PORT, () => { console.log(`AI Web Assistant 服務(wù)器運(yùn)行在端口{PORT}); console.log(環(huán)境: ${process.env.NODE_ENV}); }); exportdefault app; </pre>

四、前端界面實(shí)現(xiàn)

創(chuàng)建 public/index.html

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>AI 網(wǎng)頁操作助手</title> <script src="https://cdn.tailwindcss.com"></script> <style> .message-user { background-color: [#3b82f6](javascript:;); color: white; } .message-assistant { background-color: [#e5e7eb](javascript:;); color: [#374151](javascript:;); } .typing-indicator { display: inline-block; } .typing-dot { display: inline-block; width: 8px; height: 8px; background-color: [#9ca3af](javascript:;); border-radius: 50%; margin: 02px; animation: typing 1.4s infinite ease-in-out; } .typing-dot:nth-child(1) { animation-delay: -0.32s; } .typing-dot:nth-child(2) { animation-delay: -0.16s; } @keyframes typing { 0%, 80%, 100% { transform: scale(0); } 40% { transform: scale(1); } }</style> </head> <body class="bg-gray-100 min-h-screen"> <div class="container mx-auto px-4 py-8 max-w-4xl"> <header class="text-center mb-8"> <h1 class="text-3xl font-bold text-gray-800 mb-2">AI 網(wǎng)頁操作助手</h1> <p class="text-gray-600">使用自然語言指令自動(dòng)化網(wǎng)頁操作</p> </header> <div class="bg-white rounded-lg shadow-lg overflow-hidden"> <div class="bg-gray-800 text-white p-4 flex justify-between items-center"> <div> <span id="sessionStatus" class="text-sm">未連接</span> </div> <div class="space-x-2"> <button id="initSession" class="bg-green-600 hover:bg-green-700 px-4 py-2 rounded text-sm"> 開始新會(huì)話 </button> <button id="clearSession" class="bg-red-600 hover:bg-red-700 px-4 py-2 rounded text-sm" disabled> 結(jié)束會(huì)話 </button> </div> </div> <div class="h-96 overflow-y-auto p-4 space-y-4" id="chatMessages"> <div class="text-center text-gray-500 py-8"> 發(fā)送指令開始與AI助手對話 </div> </div> <div class="border-t p-4"> <div class="flex space-x-2"> <input type="text" id="instructionInput" placeholder="輸入你的指令,例如:打開百度并搜索AI最新進(jìn)展..." class="flex-1 border rounded-lg px-4 py-2 focus:outline-none focus:ring-2 focus:ring-blue-500" disabled > <button id="sendButton" class="bg-blue-600 hover:bg-blue-700 text-white px-6 py-2 rounded-lg disabled:bg-gray-400 disabled:cursor-not-allowed" disabled > 發(fā)送 </button> </div> <div class="mt-2 text-sm text-gray-500"> <p>示例指令:</p> <div class="flex flex-wrap gap-2 mt-1"> <button class="example-instruction text-xs bg-gray-200 hover:bg-gray-300 px-2 py-1 rounded" data-instruction="打開百度首頁">打開百度</button> <button class="example-instruction text-xs bg-gray-200 hover:bg-gray-300 px-2 py-1 rounded" data-instruction="搜索今天的熱門新聞">搜索新聞</button> <button class="example-instruction text-xs bg-gray-200 hover:bg-gray-300 px-2 py-1 rounded" data-instruction="提取當(dāng)前頁面的所有標(biāo)題">提取標(biāo)題</button> </div> </div> </div> </div> <div class="mt-4 bg-white rounded-lg shadow p-4"> <h3 class="font-semibold mb-2">會(huì)話信息</h3> <div class="text-sm space-y-1"> <div>會(huì)話ID: <span id="sessionIdDisplay" class="font-mono">-</span></div> <div>狀態(tài): <span id="connectionStatus">未連接</span></div> <div>消息數(shù): <span id="messageCount">0</span></div> </div> </div> </div> <script> class AIAssistant { constructor() { this.sessionId = null; this.isConnected = false; this.messageCount = 0; this.initializeElements(); this.attachEventListeners(); } initializeElements() { this.sessionStatus = document.getElementById('sessionStatus'); this.sessionIdDisplay = document.getElementById('sessionIdDisplay'); this.connectionStatus = document.getElementById('connectionStatus'); this.messageCountDisplay = document.getElementById('messageCount'); this.chatMessages = document.getElementById('chatMessages'); this.instructionInput = document.getElementById('instructionInput'); this.sendButton = document.getElementById('sendButton'); this.initSessionBtn = document.getElementById('initSession'); this.clearSessionBtn = document.getElementById('clearSession'); } attachEventListeners() { this.initSessionBtn.addEventListener('click', () => this.initializeSession()); this.clearSessionBtn.addEventListener('click', () => this.clearSession()); this.sendButton.addEventListener('click', () => this.sendInstruction()); this.instructionInput.addEventListener('keypress', (e) => { if (e.key === 'Enter') this.sendInstruction(); }); // 示例指令點(diǎn)擊事件 document.querySelectorAll('.example-instruction').forEach(btn => { btn.addEventListener('click', (e) => { this.instructionInput.value = e.target.dataset.instruction; this.sendInstruction(); }); }); } async initializeSession() { try { this.showLoading('正在初始化會(huì)話...'); const response = await fetch('/api/session/init', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({}) }); const data = await response.json(); if (data.success) { this.sessionId = data.sessionId; this.isConnected = true; this.messageCount = 0; this.updateUI(); this.addMessage('system', '會(huì)話已初始化,你可以開始發(fā)送指令了。'); } else { thrownewError(data.error); } } catch (error) { this.addMessage('error',初始化失敗: {error.message}`); } finally { this.hideLoading(); } } async sendInstruction() { const instruction = this.instructionInput.value.trim(); if (!instruction || !this.isConnected) return; // 添加用戶消息 this.addMessage('user', instruction); this.instructionInput.value = ''; // 顯示輸入狀態(tài) const thinkingMessage = this.addMessage('assistant', ''); this.showTypingIndicator(thinkingMessage); try { const response = await fetch('/api/instruction', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ sessionId: this.sessionId, instruction: instruction }) }); const data = await response.json(); // 移除輸入狀態(tài) this.removeTypingIndicator(thinkingMessage); if (data.success) { this.addMessage('assistant', data.response); } else { this.addMessage('error', `操作失敗:{data.error}); } } catch (error) { this.removeTypingIndicator(thinkingMessage); this.addMessage('error',網(wǎng)絡(luò)錯(cuò)誤: {error.message}`); } } async clearSession() { if (!this.sessionId) return; try { await fetch(`/api/session/{this.sessionId}, { method: 'DELETE' }); } catch (error) { console.error('清理會(huì)話失敗:', error); } this.sessionId = null; this.isConnected = false; this.messageCount = 0; this.updateUI(); this.clearMessages(); this.addMessage('system', '會(huì)話已結(jié)束。點(diǎn)擊"開始新會(huì)話"重新開始。'); } addMessage(role, content) { this.messageCount++; this.messageCountDisplay.textContent = this.messageCount; const messageDiv = document.createElement('div'); messageDiv.className =p-3 rounded-lg max-w-3/4 ${ role === 'user' ? 'message-user ml-auto' : role === 'error' ? 'bg-red-100 text-red-800 border border-red-200' : 'message-assistant' }; if (role === 'thinking') { messageDiv.innerHTML = '<div class="typing-indicator"><span class="typing-dot"></span><span class="typing-dot"></span><span class="typing-dot"></span></div>'; } else { messageDiv.textContent = content; } this.chatMessages.appendChild(messageDiv); this.chatMessages.scrollTop = this.chatMessages.scrollHeight; return messageDiv; } showTypingIndicator(messageElement) { messageElement.innerHTML = '<div class="typing-indicator"><span class="typing-dot"></span><span class="typing-dot"></span><span class="typing-dot"></span></div>'; } removeTypingIndicator(messageElement) { messageElement.innerHTML = ''; } clearMessages() { this.chatMessages.innerHTML = '<div class="text-center text-gray-500 py-8">發(fā)送指令開始與AI助手對話</div>'; } showLoading(message) { this.initSessionBtn.disabled = true; this.initSessionBtn.textContent = message; } hideLoading() { this.initSessionBtn.disabled = false; this.initSessionBtn.textContent = '開始新會(huì)話'; } updateUI() { this.sessionStatus.textContent = this.isConnected ? '已連接' : '未連接'; this.sessionIdDisplay.textContent = this.sessionId || '-'; this.connectionStatus.textContent = this.isConnected ? '活躍' : '未連接'; this.connectionStatus.className = this.isConnected ? 'text-green-600' : 'text-red-600'; this.instructionInput.disabled = !this.isConnected; this.sendButton.disabled = !this.isConnected; this.clearSessionBtn.disabled = !this.isConnected; } } // 初始化應(yīng)用 document.addEventListener('DOMContentLoaded', () => { new AIAssistant(); });</script> </body> </html> </pre>

五、測試與驗(yàn)證

1. 創(chuàng)建測試腳本

創(chuàng)建 tests/integration.test.js

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">import { test, expect } from'@playwright/test'; import AIHandler from'../src/ai/handler.js'; import dotenv from'dotenv'; dotenv.config(); test.describe('AI Web Assistant Integration Tests', () => { let aiHandler; let sessionId; test.beforeEach(async () => { aiHandler = new AIHandler(process.env.ANTHROPIC_API_KEY); const initResult = await aiHandler.initializeSession(); sessionId = initResult.sessionId; }); test.afterEach(async () => { await aiHandler.cleanupSession(sessionId); }); test('should initialize session successfully', async () => { expect(sessionId).toBeDefined(); expect(typeof sessionId).toBe('string'); }); test('should process simple navigation instruction', async () => { const result = await aiHandler.processInstruction( sessionId, '請打開百度首頁 https://www.baidu.com' ); expect(result.success).toBe(true); expect(result.response).toBeDefined(); }); test('should handle invalid instruction gracefully', async () => { const result = await aiHandler.processInstruction( sessionId, '執(zhí)行一個(gè)不存在的操作' ); // 即使指令有問題,也應(yīng)該有合理的響應(yīng) expect(result.response).toBeDefined(); }); }); </pre>

2. 運(yùn)行測試

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">npm test </pre>

六、部署與運(yùn)行

1. 生產(chǎn)環(huán)境配置

創(chuàng)建 ecosystem.config.js

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">module.exports = { apps: [{ name: 'ai-web-assistant', script: 'server.js', instances: 'max', exec_mode: 'cluster', env: { NODE_ENV: 'production', PORT: 3000 }, env_production: { NODE_ENV: 'production' } }] }; </pre>

2. Docker 配置

創(chuàng)建 Dockerfile

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">FROM node:18-alpine WORKDIR /app # 安裝 Playwright 依賴 RUN apk add --no-cache \ chromium \ nss \ freetype \ freetype-dev \ harfbuzz \ ca-certificates \ ttf-freefont # 設(shè)置環(huán)境變量 ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser # 復(fù)制 package.json 并安裝依賴 COPY package*.json ./ RUN npm ci --only=production # 復(fù)制源代碼 COPY . . # 創(chuàng)建非root用戶 RUN addgroup -g 1001 -S nodejs RUN adduser -S nextjs -u 1001 USER nextjs EXPOSE3000 CMD ["npm", "start"] </pre>

3. 啟動(dòng)應(yīng)用

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"># 開發(fā)模式 npm run dev # 生產(chǎn)模式 npm start </pre>

七、實(shí)際應(yīng)用場景

場景1:自動(dòng)化數(shù)據(jù)收集

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">// 指令:收集 GitHub 趨勢項(xiàng)目 const instruction = 請?jiān)L問 GitHub Trending 頁面 (https://github.com/trending), 收集今天最流行的 JavaScript 項(xiàng)目的前5名, 包括項(xiàng)目名稱、星標(biāo)數(shù)和描述, 并整理成 JSON 格式返回。 ; </pre>

場景2:自動(dòng)化表單填寫

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">// 指令:注冊測試用戶 const instruction = 請打開我們的測試注冊頁面 http://localhost:3000/register, 填寫以下信息: - 用戶名: testuser_{Date.now()} - 郵箱: test{Date.now()}@example.com - 密碼: TestPassword123 然后點(diǎn)擊注冊按鈕,并確認(rèn)注冊成功。 ; </pre>

場景3:復(fù)雜工作流程

<pre data-tool="mdnice編輯器" style="-webkit-tap-highlight-color: transparent; margin: 10px 0px; padding: 0px; outline: 0px; max-width: 100%; box-sizing: border-box !important; overflow-wrap: break-word !important; color: rgb(0, 0, 0); font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;">// 指令:完整的電商流程測試 const instruction = 請執(zhí)行以下電商購物流程: 1. 登錄到測試電商網(wǎng)站 2. 搜索"筆記本電腦" 3. 選擇第一個(gè)商品 4. 添加到購物車 5. 進(jìn)入結(jié)算流程 6. 填寫測試配送信息 7. 確認(rèn)訂單 請?jiān)诿總€(gè)步驟完成后報(bào)告狀態(tài)。 ; </pre>

總結(jié)

通過本教程,我們成功構(gòu)建了一個(gè)功能完整的 AI 網(wǎng)頁操作助手,具備以下特點(diǎn):

  1. 完整的架構(gòu):從前端界面到后端服務(wù),再到瀏覽器自動(dòng)化層
  2. 靈活的 MCP 協(xié)議:支持多種網(wǎng)頁操作工具
  3. 智能的 AI 集成:利用 Claude 模型理解自然語言指令
  4. 健壯的錯(cuò)誤處理:能夠應(yīng)對各種網(wǎng)頁異常情況
  5. 可擴(kuò)展的設(shè)計(jì):易于添加新的工具和功能

這個(gè)項(xiàng)目展示了如何將現(xiàn)代 AI 技術(shù)與瀏覽器自動(dòng)化相結(jié)合,創(chuàng)造出能夠理解并執(zhí)行復(fù)雜網(wǎng)頁操作的智能助手。你可以在此基礎(chǔ)上繼續(xù)擴(kuò)展,比如添加視覺識(shí)別、多瀏覽器支持、分布式任務(wù)處理等功能,打造更強(qiáng)大的自動(dòng)化解決方案。

立即開始構(gòu)建你自己的 AI 網(wǎng)頁助手,釋放自動(dòng)化的無限可能!

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容