[ISSUE #10442] Introduce FlatPropertiesMap and optimize message properties encode/decode

[wang-jiahua]

Replace per-message HashMap allocation in the properties encode/decode hot path with a compact FlatPropertiesMap backed by a flat Object[] array. Add ThreadLocal reuse for the map, StringBuilder, and char[] buffers. Intern high-frequency property keys and values to eliminate redundant String allocation.

Which Issue(s) This PR Fixes

• Fixes [Enhancement] Introduce FlatPropertiesMap and optimize message properties encode/decode to reduce allocation #10442

Brief Description

Heap dump shows HashMap$Node[] occupies 21% of live heap and String has only 21,873 distinct values out of 100,000 sampled (e.g., "true" duplicated 7,142
times). The string2messageProperties / messageProperties2String path is the largest allocation hotspot on the broker send path.

Changes:

• FlatPropertiesMap (new): compact Map<String, String> backed by flat Object[], supports reset() for ThreadLocal reuse, computeEncodedLength(), and
  encodeTo(ByteBuffer) for zero-copy serialization
• MessageDecoder: byte-level bytes2messageProperties(ByteBuffer) parsing without new String() + split(); ThreadLocal REUSABLE_PROPS_MAP and
  REUSABLE_SB
• MessageConst: STRING_INTERN_BY_LEN length-bucketed key intern array
• MessageDecoder: VALUE_INTERN_BY_LEN for common value interning ("true", "false", "DefaultRegion", etc.)
• MessageClientIDSetter: ThreadLocal char[] for createUniqID
• MessageVersion: direct if-else lookup replacing values() iteration

JFR-measured bytes/msg reduction: -42.7% from this change alone.

How Did You Test This Change?

• JFR profiling (ObjectAllocationSample) on 4×ECS (8c/30G), 128 producer threads, 1KB body, Temurin JDK 21
• Verified message encode/decode round-trip correctness
• Send-path string2messageProperties returns HashMap (not FlatPropertiesMap) to maintain downstream compatibility

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR focuses on reducing allocations and improving performance on the broker hot path by optimizing message property serialization/parsing and reusing buffers.

Changes:

• Added UTF-8 byte-level encode/decode paths for message properties to avoid intermediate String allocations.
• Introduced reusable ThreadLocal buffers (StringBuilder, char[], and a properties map) to reduce per-message allocations.
• Added key/value interning helpers to reduce duplicated String allocations when parsing properties.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
common/src/main/java/org/apache/rocketmq/common/message/MessageVersion.java	Simplifies magic-code to version mapping with direct comparisons.
common/src/main/java/org/apache/rocketmq/common/message/MessageExtBrokerInner.java	Adds cached UTF-8 propertiesData and lazy encoding path.
common/src/main/java/org/apache/rocketmq/common/message/MessageDecoder.java	Adds byte-based properties encode/decode, interning, and ThreadLocal reuse.
common/src/main/java/org/apache/rocketmq/common/message/MessageConst.java	Adds intern tables for frequent property keys to avoid substring allocations.
common/src/main/java/org/apache/rocketmq/common/message/MessageClientIDSetter.java	Reuses a ThreadLocal char[] buffer for uniq ID creation.
common/src/main/java/org/apache/rocketmq/common/message/Message.java	Removes side-effect allocation in getProperty and optimizes priority parsing.
common/src/main/java/org/apache/rocketmq/common/message/FlatPropertiesMap.java	Introduces a compact map for decoded properties with fast encoding support.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[qianye1001]

有个问题要注意下， 这个map的put 也需要有后覆盖前的语义（包括entry的正确性），以及多次decode会不会有问题

[oss-taishan-ai]

Review by github-manager-bot

Summary

This PR introduces performance optimizations for message properties encoding/decoding by replacing HashMap allocation with a compact FlatPropertiesMap backed by a flat Object[] array, adding ThreadLocal reuse for StringBuilder and char[] buffers, and interning high-frequency property keys.

Findings

• [Good] MessageDecoder.java:605-635 — ThreadLocal StringBuilder reuse with proper capacity cap (REUSABLE_SB_CAP_LIMIT = 64KB) prevents unbounded memory growth.
• [Good] MessageDecoder.java:644-673 — New messageProperties2Bytes() method directly encodes to UTF-8 bytes, avoiding String allocation on the hot path.
• [Good] MessageDecoder.java:675-720 — Custom UTF-8 encoder correctly handles edge cases including unpaired surrogates (matches JDK behavior).
• [Good] MessageDecoder.java:722-770 — bytes2messageProperties() returns independent HashMap (not ThreadLocal-shared), preventing cross-decode corruption.
• [Good] MessageConst.java:1-35 — Interning of high-frequency property keys reduces String allocation.
• [Good] MessageDecoderTest.java:431-556 — Comprehensive test coverage including ASCII, multi-byte UTF-8, surrogate pairs, and null handling.

Suggestions

1. Consider adding a benchmark (JMH) to quantify the performance improvement on the encode/decode hot path.
2. The REUSABLE_SB_CAP_LIMIT of 64KB is reasonable; document the rationale for this specific value.

Verdict

✅ Approved — Well-designed performance optimization with proper memory safety guards and comprehensive test coverage.

---

Automated review by github-manager-bot