refactor: 代码和文档优化

Author: NanmiCoder

docs/爬虫进价/01_工程化爬虫开发规范.md

@@ -16,27 +16,22 @@
 
 ### 1.2 验证码的工作原理
 
-```
-用户                    前端                    服务器
- |                       |                        |
- |   1.请求验证码        |                        |
- | --------------------> |                        |
- |                       |   2.请求验证码         |
- |                       | ---------------------> |
- |                       |                        |
- |                       |   3.返回验证码图片+ID  |
- |                       | <--------------------- |
- |                       |                        |
- |   4.显示验证码        |                        |
- | <-------------------- |                        |
- |                       |                        |
- |   5.输入验证码        |                        |
- | --------------------> |                        |
- |                       |   6.提交验证           |
- |                       | ---------------------> |
- |                       |                        |
- |                       |   7.返回验证结果       |
- |                       | <--------------------- |
+```mermaid
+sequenceDiagram
+    participant User as 用户
+    participant Frontend as 前端
+    participant Server as 服务器
+
+    User->>Frontend: 1. 请求验证码
+    Frontend->>Server: 2. 请求验证码
+    Server-->>Frontend: 3. 返回验证码图片+ID
+    Frontend-->>User: 4. 显示验证码
+
+    User->>Frontend: 5. 输入验证码
+    Frontend->>Server: 6. 提交验证
+    Server-->>Frontend: 7. 返回验证结果
+
+    Note over User,Server: 验证通过后继续操作
 ```
 
 ## 二、图片验证码识别
@@ -872,6 +867,333 @@ async def demo_captcha_solving():
         await browser.close()
 ```
 
+## 七、B站验证码处理实战
+
+B站在特定场景下会触发验证码，本节介绍B站验证码的特点和处理方法。
+
+### 7.1 B站验证码触发场景
+
+```mermaid
+flowchart TD
+    request[发起请求] --> check{B站风控检测}
+    check -->|正常| success[返回数据]
+    check -->|异常| trigger{触发验证码}
+
+    trigger -->|高频请求| slider[滑块验证码]
+    trigger -->|IP异常| geetest[极验验证码]
+    trigger -->|敏感操作| click[点选验证码]
+
+    slider --> verify{验证}
+    geetest --> verify
+    click --> verify
+
+    verify -->|成功| success
+    verify -->|失败| block[临时封禁]
+```
+
+### 7.2 B站常见验证码类型
+
+| 场景 | 验证码类型 | 触发条件 | 处理难度 |
+|-----|-----------|---------|---------|
+| 登录保护 | 滑块验证码 | 异地登录、频繁登录 | ⭐⭐⭐ |
+| 接口防护 | 极验验证码 | 请求频率过高 | ⭐⭐⭐⭐ |
+| 评论/弹幕 | 点选验证码 | 短时间大量发送 | ⭐⭐⭐⭐⭐ |
+| 关注/收藏 | 简单确认 | 批量操作 | ⭐ |
+
+### 7.3 B站滑块验证码处理
+
+```python
+import asyncio
+import httpx
+from playwright.async_api import async_playwright, Page
+from loguru import logger
+
+
+class BilibiliSliderCaptcha:
+    """B站滑块验证码处理器"""
+
+    def __init__(self, page: Page):
+        self.page = page
+
+    async def detect_and_solve(self) -> bool:
+        """
+        检测并解决滑块验证码
+
+        Returns:
+            是否成功解决
+        """
+        try:
+            # 检测是否出现滑块验证码
+            slider_frame = self.page.frame_locator("iframe[src*='captcha']")
+
+            # 等待滑块出现（最多5秒）
+            try:
+                await slider_frame.locator(".geetest_slider_button").wait_for(
+                    timeout=5000
+                )
+            except Exception:
+                # 没有验证码，正常情况
+                return True
+
+            logger.info("检测到B站滑块验证码")
+
+            # 获取滑块和背景图
+            bg_element = slider_frame.locator(".geetest_canvas_bg")
+            slider_element = slider_frame.locator(".geetest_canvas_slice")
+
+            bg_bytes = await bg_element.screenshot()
+            slider_bytes = await slider_element.screenshot()
+
+            # 检测缺口位置
+            gap_x = self._detect_gap(bg_bytes, slider_bytes)
+
+            if not gap_x:
+                logger.error("无法检测缺口位置")
+                return False
+
+            # 执行拖拽
+            await self._drag_slider(slider_frame, gap_x)
+
+            # 等待验证结果
+            await asyncio.sleep(2)
+
+            # 检查是否成功
+            try:
+                await slider_frame.locator(".geetest_success").wait_for(
+                    timeout=3000
+                )
+                logger.info("B站滑块验证码通过")
+                return True
+            except Exception:
+                logger.warning("B站滑块验证码验证失败")
+                return False
+
+        except Exception as e:
+            logger.error(f"B站滑块验证码处理异常: {e}")
+            return False
+
+    def _detect_gap(self, bg_bytes: bytes, slider_bytes: bytes) -> int:
+        """检测缺口位置"""
+        import cv2
+        import numpy as np
+
+        bg = cv2.imdecode(np.frombuffer(bg_bytes, np.uint8), cv2.IMREAD_COLOR)
+        slider = cv2.imdecode(np.frombuffer(slider_bytes, np.uint8), cv2.IMREAD_COLOR)
+
+        # 边缘检测
+        bg_edges = cv2.Canny(cv2.cvtColor(bg, cv2.COLOR_BGR2GRAY), 100, 200)
+        slider_edges = cv2.Canny(cv2.cvtColor(slider, cv2.COLOR_BGR2GRAY), 100, 200)
+
+        # 模板匹配
+        result = cv2.matchTemplate(bg_edges, slider_edges, cv2.TM_CCOEFF_NORMED)
+        _, _, _, max_loc = cv2.minMaxLoc(result)
+
+        return max_loc[0]
+
+    async def _drag_slider(self, frame, distance: int):
+        """拖拽滑块"""
+        import random
+
+        slider_btn = frame.locator(".geetest_slider_button")
+        box = await slider_btn.bounding_box()
+
+        if not box:
+            raise Exception("无法获取滑块位置")
+
+        start_x = box['x'] + box['width'] / 2
+        start_y = box['y'] + box['height'] / 2
+
+        # 生成人类轨迹
+        trajectory = self._generate_human_trajectory(distance)
+
+        await self.page.mouse.move(start_x, start_y)
+        await asyncio.sleep(random.uniform(0.1, 0.2))
+
+        await self.page.mouse.down()
+
+        for x, y, delay in trajectory:
+            await asyncio.sleep(delay)
+            await self.page.mouse.move(start_x + x, start_y + y)
+
+        await asyncio.sleep(random.uniform(0.05, 0.1))
+        await self.page.mouse.up()
+
+    def _generate_human_trajectory(self, distance: int):
+        """生成人类轨迹"""
+        import random
+
+        trajectory = []
+        current_x = 0
+        steps = random.randint(20, 30)
+
+        for i in range(steps):
+            progress = i / steps
+            # 缓动函数
+            eased = progress * (2 - progress)
+            target_x = int(distance * eased)
+
+            x = target_x
+            y = random.randint(-3, 3)
+            delay = random.uniform(0.01, 0.03)
+
+            trajectory.append((x, y, delay))
+
+        trajectory.append((distance, 0, 0.05))
+        return trajectory
+
+
+async def bilibili_with_captcha_handling():
+    """带验证码处理的B站访问示例"""
+
+    async with async_playwright() as p:
+        browser = await p.chromium.launch(headless=False)
+        context = await browser.new_context()
+        page = await context.new_page()
+
+        # 访问B站
+        await page.goto("https://www.bilibili.com")
+
+        # 模拟一些操作...
+        # 如果触发验证码，自动处理
+        captcha_handler = BilibiliSliderCaptcha(page)
+
+        # 在关键操作后检查验证码
+        success = await captcha_handler.detect_and_solve()
+
+        if success:
+            print("操作成功，无验证码或验证码已处理")
+        else:
+            print("验证码处理失败")
+
+        await browser.close()
+
+
+if __name__ == "__main__":
+    asyncio.run(bilibili_with_captcha_handling())
+```
+
+### 7.4 避免触发验证码的策略
+
+在B站爬虫中，预防优于处理：
+
+```mermaid
+flowchart LR
+    subgraph 预防策略
+        A[控制请求频率] --> B[模拟真实行为]
+        B --> C[使用登录态]
+        C --> D[IP轮换]
+    end
+
+    subgraph B站建议配置
+        E[每分钟<30请求]
+        F[随机延迟2-5秒]
+        G[保持Cookie有效]
+        H[高匿代理池]
+    end
+
+    A --> E
+    B --> F
+    C --> G
+    D --> H
+```
+
+```python
+import asyncio
+import random
+from typing import Optional
+
+
+class BilibiliRateLimiter:
+    """B站请求频率控制器"""
+
+    def __init__(
+        self,
+        requests_per_minute: int = 20,
+        min_delay: float = 2.0,
+        max_delay: float = 5.0
+    ):
+        self.requests_per_minute = requests_per_minute
+        self.min_delay = min_delay
+        self.max_delay = max_delay
+        self._last_request_time: Optional[float] = None
+        self._request_count = 0
+        self._minute_start: Optional[float] = None
+
+    async def wait(self):
+        """等待直到可以发送下一个请求"""
+        now = asyncio.get_event_loop().time()
+
+        # 重置分钟计数
+        if self._minute_start is None or now - self._minute_start > 60:
+            self._minute_start = now
+            self._request_count = 0
+
+        # 检查是否超过频率限制
+        if self._request_count >= self.requests_per_minute:
+            wait_time = 60 - (now - self._minute_start)
+            if wait_time > 0:
+                await asyncio.sleep(wait_time)
+            self._minute_start = asyncio.get_event_loop().time()
+            self._request_count = 0
+
+        # 随机延迟
+        if self._last_request_time:
+            elapsed = now - self._last_request_time
+            if elapsed < self.min_delay:
+                delay = random.uniform(self.min_delay, self.max_delay)
+                await asyncio.sleep(delay - elapsed)
+
+        self._last_request_time = asyncio.get_event_loop().time()
+        self._request_count += 1
+
+
+# 使用示例
+rate_limiter = BilibiliRateLimiter(
+    requests_per_minute=20,
+    min_delay=2.0,
+    max_delay=5.0
+)
+
+async def safe_bilibili_request(client, url):
+    """安全的B站请求（带频率控制）"""
+    await rate_limiter.wait()
+    return await client.get(url)
+```
+
+---
+
+## 八、与第11章的关联
+
+本章介绍的验证码处理技术在第11章综合实战项目中有实际应用场景：
+
+### 代码位置
+- **验证码检测**：`源代码/爬虫进阶/11_进阶综合实战项目/tools/captcha.py`
+- **频率控制**：`源代码/爬虫进阶/11_进阶综合实战项目/client/bilibili_client.py`
+
+### 技术要点对应
+| 本章内容 | 第11章实现 |
+|---------|-----------|
+| `OCRCaptchaSolver` | `tools/captcha.py` 中的识别模块 |
+| `SliderCaptchaSolver` | 滑块验证码处理逻辑 |
+| `BilibiliRateLimiter` | `client/` 中的请求频率控制 |
+| 人类轨迹生成 | 防检测行为模拟 |
+
+### 学习路径
+
+```mermaid
+graph LR
+    A[第07章<br/>扫码登录] --> B[第08章<br/>验证码处理]
+    B --> C[第09章<br/>数据清洗]
+    C --> D[第11章<br/>综合实战]
+
+    style B fill:#e1f5fe
+    style D fill:#fff9c4
+```
+
+掌握本章的验证码处理技术后，你能够处理B站爬虫中可能遇到的各类验证码场景，为第11章的综合项目打下基础。
+
+---
+
 ## 本章小结
 
 本章介绍了验证码识别与处理的核心技术：
@@ -881,6 +1203,7 @@ async def demo_captcha_solving():
 3. **滑块处理**：缺口检测和人类轨迹模拟
 4. **打码平台**：第三方服务的接入和成本控制
 5. **合规考虑**：法律边界和替代方案
+6. **B站实战**：B站验证码触发场景和处理策略
 
 验证码处理是爬虫进阶的重要技能，但务必在合法合规的前提下使用。