Skip to content

Commit 9af2804

Browse files
committed
Add support for multiple push URLs in Git repositories
Enables pushing to multiple remotes (e.g., GitHub + GitLab mirrors) by configuring multiple push URLs using multiline syntax, consistent with version-overrides and ignores patterns. Implementation: - config.py: Add parse_multiline_list() to parse multiline pushurl values - config.py: Detect and store multiple pushurls as list with backward compat - vcs/git.py: Update git_set_pushurl() to use --add flag for additional URLs - First pushurl set without --add, subsequent ones with --add Configuration example: ```ini [my-package] url = https://github.com/org/repo.git pushurl = git@github.com:org/repo.git git@gitlab.com:org/repo.git git@bitbucket.org:org/repo.git ``` Backward compatibility: - Single pushurl: Works unchanged (no pushurls list created) - Multiple pushurls: New multiline syntax (pushurls list created) - First pushurl stored in pushurl key for backward compat - Smart threading logic unchanged (checks for pushurl presence) Tests: - test_config_parse_multiple_pushurls: Config parsing validation - test_git_set_pushurl_multiple: Git command sequence verification - test_git_checkout_with_multiple_pushurls: End-to-end integration All 197 tests pass (194 existing + 3 new). All linting checks pass. Documentation updated in README.md and CHANGES.md.
1 parent 5b819f7 commit 9af2804

File tree

6 files changed

+225
-5
lines changed

6 files changed

+225
-5
lines changed

CHANGES.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
## 5.0.2 (2025-10-23)
44

5+
- Feature: Git repositories can now specify multiple push URLs using multiline syntax in the `pushurl` configuration option. This enables pushing to multiple remotes (e.g., GitHub + GitLab mirrors) automatically. Syntax follows the same multiline pattern as `version-overrides` and `ignores`. Example: `pushurl =` followed by indented URLs on separate lines. When `git push` is run in the checked-out repository, it will push to all configured pushurls sequentially, mirroring Git's native multi-pushurl behavior. Backward compatible with single pushurl strings.
6+
[jensens]
57
- Feature: Added `--version` command-line option to display the current mxdev version. The version is automatically derived from git tags via hatch-vcs during build. Example: `mxdev --version` outputs "mxdev 5.0.1" for releases or "mxdev 5.0.1.dev27+g62877d7" for development versions.
68
[jensens]
79
- Fix #70: HTTP-referenced requirements/constraints files are now properly cached and respected in offline mode. Previously, offline mode only skipped VCS operations but still fetched HTTP URLs. Now mxdev caches all HTTP content in `.mxdev_cache/` during online mode and reuses it during offline mode, enabling true offline operation. This fixes the inconsistent behavior where `-o/--offline` didn't prevent all network activity.

README.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -239,7 +239,7 @@ For package sources, the section name is the package name: `[PACKAGENAME]`
239239
| `extras` | optional | Comma-separated package extras (e.g., `test,dev`) | empty |
240240
| `subdirectory` | optional | Path to Python package when not in repository root | empty |
241241
| `target` | optional | Custom target directory (overrides `default-target`) | `default-target` |
242-
| `pushurl` | optional | Writable URL for pushes (not applied after initial checkout) ||
242+
| `pushurl` | optional | Writable URL(s) for pushes. Supports single URL or multiline list for pushing to multiple remotes. Not applied after initial checkout. ||
243243

244244
**VCS Support Status:**
245245
- `git` (stable, tested)
@@ -266,6 +266,23 @@ For package sources, the section name is the package name: `[PACKAGENAME]`
266266
- **`checkout`**: Submodules only fetched during checkout, existing submodules stay untouched
267267
- **`recursive`**: Fetches submodules recursively, results in `git clone --recurse-submodules` on checkout and `submodule update --init --recursive` on update
268268

269+
##### Multiple Push URLs
270+
271+
You can configure a package to push to multiple remotes (e.g., mirroring to GitHub and GitLab):
272+
273+
```ini
274+
[my-package]
275+
url = https://github.com/org/repo.git
276+
pushurl =
277+
git@github.com:org/repo.git
278+
git@gitlab.com:org/repo.git
279+
git@bitbucket.org:org/repo.git
280+
```
281+
282+
When you run `git push` in the checked-out repository, Git will push to all configured pushurls sequentially.
283+
284+
**Note:** Multiple pushurls only work with the `git` VCS type. This mirrors Git's native behavior where a remote can have multiple push URLs.
285+
269286
### Usage
270287

271288
Run `mxdev` (for more options run `mxdev --help`).

src/mxdev/config.py

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,26 @@ def to_bool(value):
1616
return value.lower() in ("true", "on", "yes", "1")
1717

1818

19+
def parse_multiline_list(value: str) -> list[str]:
20+
"""Parse a multiline configuration value into a list of non-empty strings.
21+
22+
Handles multiline format where items are separated by newlines:
23+
value = "
24+
item1
25+
item2
26+
item3"
27+
28+
Returns a list of non-empty, stripped strings.
29+
"""
30+
if not value:
31+
return []
32+
33+
# Split by newlines and strip whitespace
34+
items = [line.strip() for line in value.strip().splitlines()]
35+
# Filter out empty lines
36+
return [item for item in items if item]
37+
38+
1939
class Configuration:
2040
settings: dict[str, str]
2141
overrides: dict[str, str]
@@ -125,6 +145,16 @@ def is_ns_member(name) -> bool:
125145
if not package.get("url"):
126146
raise ValueError(f"Section {name} has no URL set!")
127147

148+
# Special handling for pushurl to support multiple values
149+
if "pushurl" in package:
150+
pushurls = parse_multiline_list(package["pushurl"])
151+
if len(pushurls) > 1:
152+
# Store as list for multiple pushurls
153+
package["pushurls"] = pushurls
154+
# Keep first one in "pushurl" for backward compatibility
155+
package["pushurl"] = pushurls[0]
156+
# If single pushurl, leave as-is (no change to existing behavior)
157+
128158
# Handle deprecated "direct" mode for per-package install-mode
129159
pkg_mode = package.get("install-mode")
130160
if pkg_mode == "direct":

src/mxdev/vcs/git.py

Lines changed: 40 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -349,21 +349,57 @@ def update(self, **kwargs) -> str | None:
349349
return self.git_update(**kwargs)
350350

351351
def git_set_pushurl(self, stdout_in, stderr_in) -> tuple[str, str]:
352+
"""Set one or more push URLs for the remote.
353+
354+
Supports both single pushurl (backward compat) and multiple pushurls.
355+
"""
356+
# Check for multiple pushurls (new format)
357+
pushurls = self.source.get("pushurls", [])
358+
359+
# Fallback to single pushurl (backward compat)
360+
if not pushurls and "pushurl" in self.source:
361+
pushurls = [self.source["pushurl"]]
362+
363+
if not pushurls:
364+
return (stdout_in, stderr_in)
365+
366+
# Set first pushurl (without --add)
352367
cmd = self.run_git(
353368
[
354369
"config",
355370
f"remote.{self._upstream_name}.pushurl",
356-
self.source["pushurl"],
371+
pushurls[0],
357372
],
358373
cwd=self.source["path"],
359374
)
360375
stdout, stderr = cmd.communicate()
361376

362377
if cmd.returncode != 0:
363-
raise GitError(
364-
"git config remote.{}.pushurl {} \nfailed.\n".format(self._upstream_name, self.source["pushurl"])
378+
raise GitError(f"git config remote.{self._upstream_name}.pushurl {pushurls[0]} \nfailed.\n")
379+
380+
stdout_in += stdout
381+
stderr_in += stderr
382+
383+
# Add additional pushurls with --add flag
384+
for pushurl in pushurls[1:]:
385+
cmd = self.run_git(
386+
[
387+
"config",
388+
"--add",
389+
f"remote.{self._upstream_name}.pushurl",
390+
pushurl,
391+
],
392+
cwd=self.source["path"],
365393
)
366-
return (stdout_in + stdout, stderr_in + stderr)
394+
stdout, stderr = cmd.communicate()
395+
396+
if cmd.returncode != 0:
397+
raise GitError(f"git config --add remote.{self._upstream_name}.pushurl {pushurl} \nfailed.\n")
398+
399+
stdout_in += stdout
400+
stderr_in += stderr
401+
402+
return (stdout_in, stderr_in)
367403

368404
def git_init_submodules(self, stdout_in, stderr_in) -> tuple[str, str, list]:
369405
cmd = self.run_git(["submodule", "init"], cwd=self.source["path"])

tests/test_config.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -272,3 +272,42 @@ def test_per_package_target_override():
272272
pathlib.Path(pkg_interpolated["path"]).as_posix()
273273
== pathlib.Path(pkg_interpolated["target"]).joinpath("package.with.interpolated.target").as_posix()
274274
)
275+
276+
277+
def test_config_parse_multiple_pushurls(tmp_path):
278+
"""Test configuration parsing of multiple pushurls."""
279+
from mxdev.config import Configuration
280+
281+
config_content = """
282+
[settings]
283+
requirements-in = requirements.txt
284+
285+
[package1]
286+
url = https://github.com/test/repo.git
287+
pushurl =
288+
git@github.com:test/repo.git
289+
git@gitlab.com:test/repo.git
290+
git@bitbucket.org:test/repo.git
291+
292+
[package2]
293+
url = https://github.com/test/repo2.git
294+
pushurl = git@github.com:test/repo2.git
295+
"""
296+
297+
config_file = tmp_path / "mx.ini"
298+
config_file.write_text(config_content)
299+
300+
config = Configuration(str(config_file))
301+
302+
# package1 should have multiple pushurls
303+
assert "pushurls" in config.packages["package1"]
304+
assert len(config.packages["package1"]["pushurls"]) == 3
305+
assert config.packages["package1"]["pushurls"][0] == "git@github.com:test/repo.git"
306+
assert config.packages["package1"]["pushurls"][1] == "git@gitlab.com:test/repo.git"
307+
assert config.packages["package1"]["pushurls"][2] == "git@bitbucket.org:test/repo.git"
308+
# First pushurl should be kept for backward compatibility
309+
assert config.packages["package1"]["pushurl"] == "git@github.com:test/repo.git"
310+
311+
# package2 should have single pushurl (no pushurls list)
312+
assert "pushurls" not in config.packages["package2"]
313+
assert config.packages["package2"]["pushurl"] == "git@github.com:test/repo2.git"

tests/test_git_additional.py

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1007,3 +1007,99 @@ def test_smart_threading_separates_https_with_pushurl():
10071007

10081008
# HTTPS with pushurl, SSH, and fs should be in parallel queue
10091009
assert set(other_pkgs) == {"https-with-pushurl", "ssh-url", "fs-url"}
1010+
1011+
1012+
def test_git_set_pushurl_multiple():
1013+
"""Test git_set_pushurl with multiple URLs."""
1014+
from mxdev.vcs.git import GitWorkingCopy
1015+
from unittest.mock import Mock
1016+
from unittest.mock import patch
1017+
1018+
with patch("mxdev.vcs.common.which", return_value="/usr/bin/git"):
1019+
source = {
1020+
"name": "test-package",
1021+
"url": "https://github.com/test/repo.git",
1022+
"path": "/tmp/test",
1023+
"pushurls": [
1024+
"git@github.com:test/repo.git",
1025+
"git@gitlab.com:test/repo.git",
1026+
],
1027+
"pushurl": "git@github.com:test/repo.git",
1028+
}
1029+
1030+
wc = GitWorkingCopy(source)
1031+
1032+
mock_process = Mock()
1033+
mock_process.returncode = 0
1034+
mock_process.communicate.return_value = (b"output", b"")
1035+
1036+
with patch.object(wc, "run_git", return_value=mock_process) as mock_git:
1037+
stdout, stderr = wc.git_set_pushurl(b"", b"")
1038+
1039+
# Should be called twice
1040+
assert mock_git.call_count == 2
1041+
1042+
# First call: without --add
1043+
first_call_args = mock_git.call_args_list[0][0][0]
1044+
assert first_call_args == [
1045+
"config",
1046+
"remote.origin.pushurl",
1047+
"git@github.com:test/repo.git",
1048+
]
1049+
1050+
# Second call: with --add
1051+
second_call_args = mock_git.call_args_list[1][0][0]
1052+
assert second_call_args == [
1053+
"config",
1054+
"--add",
1055+
"remote.origin.pushurl",
1056+
"git@gitlab.com:test/repo.git",
1057+
]
1058+
1059+
1060+
def test_git_checkout_with_multiple_pushurls(tempdir):
1061+
"""Test git_checkout with multiple pushurls."""
1062+
from mxdev.vcs.git import GitWorkingCopy
1063+
from unittest.mock import Mock
1064+
from unittest.mock import patch
1065+
1066+
with patch("mxdev.vcs.common.which", return_value="/usr/bin/git"):
1067+
source = {
1068+
"name": "test-package",
1069+
"url": "https://github.com/test/repo.git",
1070+
"path": str(tempdir / "test-multi-pushurl"),
1071+
"pushurls": [
1072+
"git@github.com:test/repo.git",
1073+
"git@gitlab.com:test/repo.git",
1074+
"git@bitbucket.org:test/repo.git",
1075+
],
1076+
"pushurl": "git@github.com:test/repo.git", # First one for compat
1077+
}
1078+
1079+
wc = GitWorkingCopy(source)
1080+
1081+
mock_process = Mock()
1082+
mock_process.returncode = 0
1083+
mock_process.communicate.return_value = (b"", b"")
1084+
1085+
with patch.object(wc, "run_git", return_value=mock_process) as mock_git:
1086+
with patch("os.path.exists", return_value=False):
1087+
wc.git_checkout(submodules="never")
1088+
1089+
# Verify git config was called 3 times for pushurls
1090+
config_calls = [call for call in mock_git.call_args_list if "config" in call[0][0]]
1091+
1092+
# Should have 3 config calls for the 3 pushurls
1093+
pushurl_config_calls = [call for call in config_calls if "pushurl" in " ".join(call[0][0])]
1094+
assert len(pushurl_config_calls) == 3
1095+
1096+
# First call should be without --add
1097+
assert "--add" not in pushurl_config_calls[0][0][0]
1098+
assert "git@github.com:test/repo.git" in pushurl_config_calls[0][0][0]
1099+
1100+
# Second and third calls should have --add
1101+
assert "--add" in pushurl_config_calls[1][0][0]
1102+
assert "git@gitlab.com:test/repo.git" in pushurl_config_calls[1][0][0]
1103+
1104+
assert "--add" in pushurl_config_calls[2][0][0]
1105+
assert "git@bitbucket.org:test/repo.git" in pushurl_config_calls[2][0][0]

0 commit comments

Comments
 (0)