Skip to content

delete useless code#7959

Open
zhoutianzi666 wants to merge 2 commits into
PaddlePaddle:developfrom
zhoutianzi666:remove_decoder_cache
Open

delete useless code#7959
zhoutianzi666 wants to merge 2 commits into
PaddlePaddle:developfrom
zhoutianzi666:remove_decoder_cache

Conversation

@zhoutianzi666
Copy link
Copy Markdown
Collaborator

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick,PR标题需遵循格式,在最开始加上[Cherry-Pick]标签,以及最后面加上原PR ID,例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

Modifications

Usage or Command

Accuracy Tests

Checklist

  • Add at least a tag in the PR title.
    • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
    • You can add new tags based on the PR content, but the semantics must be clear.
  • Format your code, run pre-commit before commit.
  • Add unit tests. Please write the reason in this PR if no unit tests.
  • Provide accuracy results.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

@paddle-bot
Copy link
Copy Markdown

paddle-bot Bot commented May 29, 2026

Thanks for your contribution!

@zhoutianzi666 zhoutianzi666 changed the title commit DO NOT May 29, 2026
@zhoutianzi666 zhoutianzi666 changed the title DO NOT DO NOT MAKE CODE May 29, 2026
@zhoutianzi666 zhoutianzi666 changed the title DO NOT MAKE CODE delete useless code May 29, 2026
@PaddlePaddle-bot
Copy link
Copy Markdown

PaddlePaddle-bot commented May 29, 2026

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-05-29 23:43:43

CI报告基于以下代码生成(30分钟更新一次):


1 任务总览

存在 1 个 required 失败任务,需优先处理后方可合并。

总执行(rerun次数) 总任务 ✅ 通过 ❌ 失败 ⏳ 运行中 ⏸️ 等待中 跳过
42(0) 42 37 5 0 0 0

2 任务状态汇总

2.1 Required任务 : 9/10 通过

必选任务阻塞合并,失败需优先处理。

状态 任务 耗时 根因 修复建议 日志 重跑
Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage 1h31m PR问题:decode 写KV cache逻辑被替换 恢复 decoder/speculate 写 cache 分支 Job -
其余 9 个必选任务通过 - - - - -

2.2 可选任务 — 28/32 通过

可选任务不阻塞合并,失败仅供参考。

状态 任务 耗时 日志 重跑
Run iluvatar Tests / run_iluvatar_cases 15m11s Job -
Check PR Template 20s Job -
CI_HPU 1h7m Job -
Trigger Jenkins for PR 15s Job -
其余 28 个可选任务通过 - - -

3 失败详情(仅 required)

Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage — 测试失败(置信度: 中)

Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage

  • 状态: ❌ 失败
  • 错误类型: 测试失败
  • 置信度: 中
  • 根因摘要: decode 写KV cache逻辑被替换
  • 分析器: ci_analyze_unittest_fastdeploy

失败用例:

测试 错误 根因
tests/e2e/test_ernie_03b_pd_decode_unified_attention.py::test_non_chat_usage_non_stream AttributeError: 'NoneType' object has no attribute 'json' HTTP 请求超时返回 None,服务端 decode worker 已异常断连
tests/e2e/test_EB_Lite_serving_R3.py::test_r3_accuracy EOFError: Ran out of input routing replay 输出 layer_0.pdtensor 为空,生成流程未正常完成

根因详情:
本 PR 修改 custom_ops/gpu_ops/append_attention.cu,将 EncoderWriteCacheWithRopeKernelmax_enc_len_this_time > 0 分支中提出并无条件执行,同时删除了原 decode/speculate 阶段的 DecoderWriteCacheWithRoPEKernel / SpeculateWriteCacheWithRoPEKernel 写 cache 逻辑。新的调用还把 encoder kernel 的 seq_lens_encoder 参数传成 seq_lens_this_time,会改变 decode 阶段 KV cache 写入/跳过 batch 的语义。

失败用例覆盖 pd_decode_unified_attention 与 routing replay,运行时日志出现 ConnectionResetError: [Errno 104] Connection reset by peerFailed to connect to engine worker queue,随后测试侧请求超时或读取到空 routing 文件,和本次 append attention decode 写 cache 变更高度相关。

关键日志:

Failed test cases:
tests/e2e/test_ernie_03b_pd_decode_unified_attention.py
E AttributeError: 'NoneType' object has no attribute 'json'
E EOFError: Ran out of input
ConnectionResetError: [Errno 104] Connection reset by peer
ERROR engine_worker_queue.py[line:857] Failed to connect to engine worker queue

修复建议:

  1. custom_ops/gpu_ops/append_attention.cu L183-L227/L282 附近:不要用 encoder 写 cache kernel 无条件替代 decode/speculate 写 cache;恢复或等价迁移原 DecoderWriteCacheWithRoPEKernel / SpeculateWriteCacheWithRoPEKernel 分支,确保 decode-only 请求仍按 seq_lens_decoder/seq_lens_encoder 语义写入 KV cache。
  2. custom_ops/gpu_ops/append_attn/get_block_shape_and_split_kv_block.cu L386-L414:确认 kv_batch_ids/kv_tile_ids_per_batch/kv_num_blocks_x_cpu 对 encoder、decode、speculate 三种路径的含义一致;如只服务 write-cache-kv,需要补充 decode unified attention e2e 覆盖后再合入。

修复建议摘要: 恢复 decoder/speculate 写 cache 分支

关联变更: custom_ops/gpu_ops/append_attention.cucustom_ops/gpu_ops/append_attn/get_block_shape_and_split_kv_block.cu

链接: 查看日志

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (develop@791d77f). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #7959   +/-   ##
==========================================
  Coverage           ?   67.52%           
==========================================
  Files              ?      467           
  Lines              ?    65179           
  Branches           ?    10007           
==========================================
  Hits               ?    44013           
  Misses             ?    18349           
  Partials           ?     2817           
Flag Coverage Δ
GPU 77.76% <ø> (?)
XPU 7.08% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Paddle-CI-Agent | pr_review | 2026-05-29 13:51:55

📋 Review 摘要

PR 概述:删除冗余的 Decoder/Speculate KV 写缓存代码路径,将 encoder/decoder 写缓存统一到 EncoderWriteCacheWithRopeKernel 并无条件执行
变更范围custom_ops/gpu_ops/append_attention.cucustom_ops/gpu_ops/append_attn/get_block_shape_and_split_kv_block.cu
影响面 Tag[OP] [KVCache]

问题

级别 文件 概述
❓ 疑问 append_attention.cu EncoderWriteCacheWithRopeKernel 无条件调用且第 4 参数改为 seq_lens_this_time,纯 Decode 批次下行为需确认
❓ 疑问 get_block_shape_and_split_kv_block.cu split_kv_block 提升为无条件执行且切换为 seq_lens_this_time,纯 Decode 批次下 kv_batch_ids 填充正确性需确认

疑问 1 — append_attention.cu

EncoderWriteCacheWithRopeKernel 现在在 if (max_enc_len_this_time > 0) 之外被无条件调用,同时第 4 个参数(原 seq_lens_encoder)改为 seq_lens_this_time

// 新代码(两个位置均传 seq_lens_this_time)
EncoderWriteCacheWithRopeKernel<...>(
    meta_data, qkv,
    seq_lens_this_time,   // param3: seq_lens_this_time ✓
    seq_lens_this_time,   // param4: 原 seq_lens_encoder ← 已变更
    seq_lens_decoder, ...);

函数内部(encoder_write_cache_with_rope_kernel.h)将此参数传给 rotary_qk_variable 以及 CascadeAppendWriteCacheKVQKV,后者的底层 cache_kernel 用它做跳过判断:

if (seq_lens[ori_bi] == 0) continue;  // 原语义: 无 encoder token 则跳过

纯 Decode 批次seq_lens_encoder[i] == 0,但 seq_lens_this_time[i] = 1),旧代码因 if (max_enc_len_this_time > 0) 不进入此 kernel;新代码将进入且 seq_lens[ori_bi] = 1 ≠ 0,不会跳过。同时 DecoderWriteCacheWithRoPEKernel 已被删除,请确认 EncoderWriteCacheWithRopeKernel 能完整替代其功能(尤其 ori_seq_id 计算是否与 decoder 场景一致)。

疑问 2 — get_block_shape_and_split_kv_block.cu

split_kv_block 第 2 参数改为 seq_lens_this_time,且整个 KV block 切分逻辑由 if (max_enc_len_this_time > 0) 内提升为无条件执行。请确认:

  • 对纯 Decode 批次(全部请求均为 1 token decoder),split_kv_block 填充后的 kv_batch_ids / kv_tile_ids_per_batch 是否会被 EncoderWriteCacheWithRopeKernel 内的量化写缓存路径(CascadeAppendWriteCacheKVC8QKV / CascadeAppendWriteCacheKVC4QKV)正确消费。

📝 PR 规范检查

标题缺少官方 Tag,且 Motivation / Modifications / Usage or Command / Accuracy Tests 各段均为空,Checklist 全部未勾选。

标题建议(可直接复制):

  • [OP] Delete useless decoder and speculate write cache kernel calls
PR 描述建议(点击展开,可直接复制)
## Motivation
将 encoder / decoder / speculate 三条 KV 写缓存路径统一,删除冗余的 `SpeculateWriteCacheWithRoPEKernel``DecoderWriteCacheWithRoPEKernel` 调用,简化 `AppendAttentionKernel` 逻辑。同时在 `GetBlockShapeAndSplitKVBlock` 中将 KV block 切分无条件执行并改用 `seq_lens_this_time`,与上层统一路径对齐。

## Modifications
- `custom_ops/gpu_ops/append_attention.cu`:将 `EncoderWriteCacheWithRopeKernel` 调用提升到 `if (max_enc_len_this_time > 0)` 之外,无条件执行;第 4 参数改为 `seq_lens_this_time`;删除 `SpeculateWriteCacheWithRoPEKernel``DecoderWriteCacheWithRoPEKernel` 共 4 个调用点
- `custom_ops/gpu_ops/append_attn/get_block_shape_and_split_kv_block.cu`:将 KV block 切分(`split_kv_block`)从 encoder-only 条件块改为无条件执行;参数改为 `seq_lens_this_time`;为 `kv_num_blocks_x_cpu.copy_` 添加 CUDAGraph capture 保护宏

## Usage or Command
N/A

## Accuracy Tests
N/A

## Checklist

- [x] Add at least a tag in the PR title.
  - Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
  - You can add new tags based on the PR content, but the semantics must be clear.
- [ ] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.

总体评价

PR 变更思路清晰——将三条 KV 写缓存路径合一是合理重构,代码量大幅减少。但统一后 EncoderWriteCacheWithRopeKernel 在纯 Decode 批次下的语义正确性(第 4 参数语义替换 + 无条件调用)建议在 PR 描述中给出说明或补充测试覆盖,以便 Reviewer 确认功能等价。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants