-
Notifications
You must be signed in to change notification settings - Fork 5.5k
feat: Make SSD cache maxEntries limit configurable #26795
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
zacw7
wants to merge
1
commit into
prestodb:master
Choose a base branch
from
zacw7:config-ssd-cache-max-entries
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+25
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Reviewer's guide (collapsed on small PRs)Reviewer's GuideAdds a configurable upper bound on the number of SSD cache entries and wires it through system configuration to the SSD cache setup, with a default of 10M entries (~500MB metadata). Sequence diagram for propagating SSD cache maxEntries into SsdCachesequenceDiagram
participant PrestoServer
participant SystemConfig
participant SsdCacheConfig
participant SsdCache
PrestoServer->>SystemConfig: ssdCacheReadVerificationEnabled()
SystemConfig-->>PrestoServer: bool
PrestoServer->>SystemConfig: ssdCacheMaxEntries()
SystemConfig-->>PrestoServer: uint64_t
PrestoServer->>SsdCacheConfig: create cacheConfig(..., ssdCacheReadVerificationEnabled, ssdCacheMaxEntries)
SsdCacheConfig-->>PrestoServer: cacheConfig
PrestoServer->>SsdCache: create SsdCache(cacheConfig)
Class diagram for new SSD cache maxEntries configuration flowclassDiagram
class SystemConfig {
+bool ssdCacheReadVerificationEnabled()
+uint64_t ssdCacheMaxEntries()
}
class PrestoServer {
+std::unique_ptr~velox::cache::SsdCache~ setupSsdCache()
}
class SsdCacheConfig {
+SsdCacheConfig(uint64_t ssdCacheSize, uint64_t ssdCacheCheckpointSize, bool asyncCacheSsdDisableFileCow, bool ssdCacheChecksumEnabled, bool ssdCacheReadVerificationEnabled, uint64_t ssdCacheMaxEntries)
}
class SsdCache {
+SsdCache(SsdCacheConfig cacheConfig)
}
PrestoServer --> SystemConfig : uses
PrestoServer --> SsdCacheConfig : creates
PrestoServer --> SsdCache : creates
SsdCacheConfig --> SsdCache : configures
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there - I've reviewed your changes - here's some feedback:
- The new
ssdCacheMaxEntriesconfig usesuint64_t, but most config numerics are signed; consider using a signed type and explicitly validating non-negative values to avoid surprising wraparound if a negative value is configured. - The comment states that
0means "no limit" forssdCacheMaxEntries, but this behavior isn't visible in the changes here; ensure that the underlyingSsdCache/cacheConfigactually implements this semantic so the config description matches behavior.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The new `ssdCacheMaxEntries` config uses `uint64_t`, but most config numerics are signed; consider using a signed type and explicitly validating non-negative values to avoid surprising wraparound if a negative value is configured.
- The comment states that `0` means "no limit" for `ssdCacheMaxEntries`, but this behavior isn't visible in the changes here; ensure that the underlying `SsdCache` / `cacheConfig` actually implements this semantic so the config description matches behavior.
## Individual Comments
### Comment 1
<location> `presto-docs/src/main/sphinx/presto_cpp/properties.rst:553-554` </location>
<code_context>
+* **Default value:** ``10000000``
+
+ Maximum number of entries allowed in the SSD cache. A value of 0 means no limit.
+ When the limit is reached, new entry writes will be skipped.
+
+ The default of 10 million entries keeps metadata memory usage around 500MB, as each
</code_context>
<issue_to_address>
**nitpick (typo):** Consider rephrasing "new entry writes" for clarity
You could instead say “writes of new entries” or “writes of additional entries” so it’s clearer that only writes of new entries are skipped once the limit is reached.
```suggestion
Maximum number of entries allowed in the SSD cache. A value of 0 means no limit.
When the limit is reached, writes of new entries will be skipped.
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.