Skip to content

feat: add S3 endpoint configuration for LocalStack compatibility #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jfrench9 merged 1 commit into from
Nov 27, 2025
Merged

feat: add S3 endpoint configuration for LocalStack compatibility #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions robosystems_client/extensions/extensions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class RoboSystemsExtensionConfig:
max_retries: int = 5
retry_delay: int = 1000
timeout: int = 30
s3_endpoint_url: Optional[str] = None # Override S3 endpoint (e.g., for LocalStack)


class RoboSystemsExtensions:
Expand All @@ -41,6 +42,7 @@ def __init__(self, config: RoboSystemsExtensionConfig = None):
"max_retries": config.max_retries,
"retry_delay": config.retry_delay,
"timeout": config.timeout,
"s3_endpoint_url": config.s3_endpoint_url,
}

# Extract token from headers if it was set by auth classes
Expand Down
24 changes: 20 additions & 4 deletions robosystems_client/extensions/file_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ class FileUploadOptions:
"""Options for file upload operations"""

on_progress: Optional[Callable[[str], None]] = None
fix_localstack_url: bool = True
ingest_to_graph: bool = False


Expand Down Expand Up @@ -78,6 +77,9 @@ def __init__(self, config: Dict[str, Any]):
self.base_url = config["base_url"]
self.headers = config.get("headers", {})
self.token = config.get("token")
self.s3_endpoint_url = config.get(
"s3_endpoint_url"
) # Optional S3 endpoint override
self._http_client = httpx.Client(timeout=120.0)

def upload(
Expand Down Expand Up @@ -171,9 +173,23 @@ def upload(
upload_url = upload_data.upload_url
file_id = upload_data.file_id

# Fix LocalStack URL if needed
if options.fix_localstack_url and "localstack:4566" in upload_url:
upload_url = upload_url.replace("localstack:4566", "localhost:4566")
# Override S3 endpoint if configured (e.g., for LocalStack)
if self.s3_endpoint_url:
from urllib.parse import urlparse, urlunparse

parsed_url = urlparse(upload_url)
override_parsed = urlparse(self.s3_endpoint_url)
# Replace scheme, host, and port with the override endpoint
upload_url = urlunparse(
(
override_parsed.scheme or parsed_url.scheme,
override_parsed.netloc,
parsed_url.path,
Comment on lines +183 to +187
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve bucket when overriding presigned S3 endpoint

When s3_endpoint_url is set, the upload URL is rebuilt with the override netloc but keeps only parsed_url.path. For common virtual-host–style presigned URLs (bucket encoded in the hostname), this drops the bucket name when swapping to a LocalStack/alternative endpoint (e.g., https://bucket.s3.amazonaws.com/key becomes http://localhost:4566/key), so the PUT targets the root rather than the bucket and the upload fails with 403/404. This affects any client using virtual-host presigned URLs with a custom endpoint—precisely the LocalStack scenario this change is meant to support.

Useful? React with 👍 / 👎.

parsed_url.params,
parsed_url.query,
parsed_url.fragment,
)
)

# Step 2: Upload file to S3
if options.on_progress:
Expand Down