Skip to content

resumable: introduce new append-granularity limit #3320

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
danielresnick wants to merge 1 commit into
base: main
Choose a base branch
from
Open

resumable: introduce new append-granularity limit #3320

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 41 additions & 2 deletions draft-ietf-httpbis-resumable-upload.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,9 @@ The following key-value pairs are defined:
`min-append-size`:
: Specifies a minimum size counted in bytes for the request content in a single upload append request ({{upload-appending}}). The server might reject requests below this limit. A client that is aware of this limit MUST NOT send smaller upload append requests. The value is an Integer. Requests completing the upload by including the `Upload-Complete: ?1` header field are exempt from this limit.

`append-granularity`:
: Specifies that the size of request content in bytes SHOULD be an integer multiple of this value for an upload append request ({{upload-appending}}), and for an upload creation request ({{upload-creation}}) that contains content. The server might reject requests that do not adhere to this limit (see {{inconsistent-granularity}}). A client that is aware of this limit MUST adhere to this requirement. The value is an Integer. Requests completing the upload by including the `Upload-Complete: ?1` header field are exempt from this limit.

`max-age`:
: Specifies the remaining lifetime of the upload resource in seconds counted from the generation of the response. After the resource's lifetime is reached, the server might make the upload resource inaccessible and a client SHOULD NOT attempt to access the upload resource as these requests will likely fail. The value is an Integer.

Expand Down Expand Up @@ -408,7 +411,7 @@ While the request content is being received, the server MAY send multiple interi

Where a response requires a `Location` header field to be included, all interim and final response messages for the same request MUST contain an identical `Location` value. However, final responses including the `Upload-Complete: ?1` header field are exempt from this requirement because they are the result of processing the transferred representation and the `Location` value does not necessarily represent the upload location. Where the `Location` value is expected to be identical across multiple messages, clients SHOULD verify this. If verification fails, clients SHOULD abort the current request and cancel the upload ({{upload-cancellation}}).

The server might not receive the entire request content when the upload is interrupted, for example because of dropped connection or canceled request. In this case, the server SHOULD append as much of the request content as possible to the upload resource, allowing the client to resume the upload from where it was interrupted. In addition, the upload resource MUST NOT be considered complete then.
The server might not receive or process the entire request content, for example because of a dropped connection, a canceled request, or a request that does not adhere to the append-granularity limit (see {{inconsistent-granularity}}). In this case, the server SHOULD append as much of the request content as possible to the upload resource, allowing the client to resume the upload from the end of the successfully processed data. In addition, the upload resource MUST NOT be considered complete then.

### Examples {#upload-creation-example}

Expand Down Expand Up @@ -590,6 +593,8 @@ The server might not receive the entire patch document when the upload is interr

If the `Upload-Offset` request header field value does not match the current offset ({{upload-offset}}), the upload resource MUST reject the request with a `409 (Conflict)` status code. The response MUST include the correct offset in the `Upload-Offset` header field. The response can use the problem type {{PROBLEM}} of "https://iana.org/assignments/http-problem-types#mismatching-upload-offset" ({{mismatching-offset}}).

If the request content size violates a server-advertised `append-granularity` limit (see {{upload-limit}}), the server MAY process and accept a portion of the request content that adheres to the granularity, discarding the remainder. In this case, the server SHOULD respond with a `400 (Bad Request)` status code and use the problem type "https://iana.org/assignments/http-problem-types#inconsistent-upload-granularity" ({{inconsistent-granularity}}). The response MUST include an `Upload-Offset` header field indicating the new offset after accepting the partial content, allowing the client to recover and resume the upload.
Copy link
Member

@Acconut Acconut Nov 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How should a client act when receiving this response? The document currently recommends client to stop the upload once they receive a 400.

Copy link
Author

@danielresnick danielresnick Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically the spec says "4xx (Client Error) status code, the client SHOULD NOT attempt to retry or resume the upload, unless the semantics of the response allow or recommend the client to retry the request."

I agree that's quite subtle though -- expecting a client dev to read this, and then trawl through all possible response_code+problem_type combinations to decide on resume vs abort behaviour. Particularly because even parsing out the problem type requires doing some json parsing. I'll have a think about the best path forward here.. the most obvious options in my mind are (in no particular order):

  1. Switch to a 5xx response code. As discussed on the corresponding issue this is likely to improve client interop, but is a hacky / less pure solution since this isn't actually a server error.

  2. Use a different / more specific 4xx response code and improve the wording to call this code out specifically. The problem is I don't see a super obvious pre-existing code in the 4xx space. Maybe 422 Unprocessable Content, but that feels slightly odd vs the suggestion that a server should partially process the provided response.

  3. Keep using 400 (Bad Request) and try to improving the wording in the client section to be clearer (e.g. with some combination of improving cross-referencing with problem-types, marking problem-types more explicitly as retryable vs not, rephrasing the 4xx status code handling piece quoted above to lead with a discussion on what's retryable vs not).

Copy link
Member

@Acconut Acconut Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 409 would be an option. If #3319 lands, clients should resume the upload when they get a 409, which I think it desired in that case.

3. Keep using 400 (Bad Request) and try to improving the wording in the client section to be clearer (e.g. with some combination of improving cross-referencing with problem-types, marking problem-types more explicitly as retryable vs not, rephrasing the 4xx status code handling piece quoted above to lead with a discussion on what's retryable vs not).

I wouldn't want to mandate clients to parse the response body. It would be great if we can keep using status codes as the definitive signal for clients.

Copy link
Author

@danielresnick danielresnick Nov 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's a good point. I'd sort of dismissed that option as I was finding it hard to justify granularity mismatch as a state conflict (though I agree that if that PR lands it would solve the problem from a pure behaviour standpoint), but thinking some more I guess an argument could be made that partial acceptance of a chunk (e.g., accepting 256 KiB of a 300 KiB request) creates a direct mismatch between the client's perceived state and the server's actual state. And yes agreed leveraging status codes only is much more robust & likely to get adoption in practice


If the upload is already complete ({{upload-complete}}), the server MUST NOT modify the upload resource and MUST reject the request. The choice of response depends on the nature of the upload request and server state, including but not limited to:

- If the client attempted to append a non-zero length document, the server MUST treat this as an inconsistent length failure. The server can use the problem type {{PROBLEM}} of "https://iana.org/assignments/http-problem-types#inconsistent-upload-length" ({{inconsistent-length}}) in the response.
Expand Down Expand Up @@ -766,6 +771,26 @@ Content-Type: application/problem+json
}
~~~

## Inconsistent Granularity {#inconsistent-granularity}

This section defines the "https://iana.org/assignments/http-problem-types#inconsistent-upload-granularity" problem type {{PROBLEM}}. A server SHOULD use this problem type when responding to a upload creation ({{upload-creation}}) or upload append request ({{upload-appending}}) to indicate that the request content size violates the server's advertised `append-granularity` limit. The server MAY still accept a portion of the request content.

The following example shows a response where the server accepted a portion of the request, resulting in a new offset of 262144, and signals that the client should resume from there:

~~~ http-message
# NOTE: '\' line wrapping per RFC 8792

HTTP/1.1 400 Bad Request
Content-Type: application/problem+json
Upload-Offset: 262144

{
"type":"https://iana.org/assignments/http-problem-types#\
inconsistent-upload-granularity",
"title": "Request content size violates append-granularity limit"
}
~~~

# Content Codings

Since the codings listed in `Content-Encoding` are a characteristic of the representation (see {{Section 8.4 of HTTP}}), both the client and the server always compute the values for `Upload-Offset` and optionally `Upload-Length` on the content coded data (that is, the representation data). Moreover, the content codings are retained throughout the entire upload, meaning that the server is not required to decode the representation data to support resumable uploads. See {{Appendix A of DIGEST-FIELDS}} for more information.
Expand Down Expand Up @@ -984,6 +1009,20 @@ Recommended HTTP status code:
Reference:
: {{inconsistent-length}} of this document

IANA is asked to register the following entry in the "HTTP Problem Types" registry:

Type URI:
: https://iana.org/assignments/http-problem-types#inconsistent-upload-granularity

Title:
: Inconsistent Upload Granularity

Recommended HTTP status code:
: 400

Reference:
: {{inconsistent-granularity}} of this document

--- back

# Changes
Expand All @@ -992,7 +1031,7 @@ Reference:
## Since draft-ietf-httpbis-resumable-upload-10
{:numbered="false"}

None yet.
* Introduced `append-granularity` limit and corresponding problem type.

## Since draft-ietf-httpbis-resumable-upload-09
{:numbered="false"}
Expand Down