fix: chunk store_load over RPC to avoid msgpack BufferFull.#9544
Closed
hiijoshi wants to merge 4 commits intoborgbackup:masterfrom
Closed
fix: chunk store_load over RPC to avoid msgpack BufferFull.#9544hiijoshi wants to merge 4 commits intoborgbackup:masterfrom
hiijoshi wants to merge 4 commits intoborgbackup:masterfrom
Conversation
hiijoshi
force-pushed
the
fix/store-load-chunked-rpc
branch
4 times, most recently
from
39cfeb9 to
899c9ec
Compare
Large repositories can have a chunks index exceeding the msgpack unpacker buffer size, causing BufferFull errors over SSH/RPC. Instead of loading the entire value in one RPC call, add two new server-side methods (store_get_size, store_load_chunk) and override store_load in RemoteRepository to fetch data in MAX_DATA_SIZE pieces, reassembling on the client side. Fixes borgbackup#8440
hiijoshi
force-pushed
the
fix/store-load-chunked-rpc
branch
from
899c9ec to
3531c82
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9544 +/- ##
==========================================
- Coverage 83.39% 83.00% -0.40%
==========================================
Files 87 87
Lines 15469 15489 +20
Branches 2321 2324 +3
==========================================
- Hits 12901 12857 -44
- Misses 1818 1877 +59
- Partials 750 755 +5 ☔ View full report in Codecov by Sentry. |
hiijoshi
force-pushed
the
fix/store-load-chunked-rpc
branch
from
bee6d62 to
ff1646a
Compare
hiijoshi
force-pushed
the
fix/store-load-chunked-rpc
branch
from
ff1646a to
36fdb56
Compare
hiijoshi
force-pushed
the
fix/store-load-chunked-rpc
branch
from
e623a84 to
a4e4412
Compare
Member
|
Your PRs are a mess and I am tired of reviewing them. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes a crash when loading large stores via RPC