migrate to moby modules #13078

thaJeztah · 2025-07-21T18:51:11Z

depends on pkg/compose: align classic builder implementation with docker/cli #13278
testing Create github.com/moby/moby/api and github.com/moby/moby/client module moby/moby#50280
testing migrate to moby/moby/api and moby/moby/client modules cli#6196
testing vendor: github.com/moby/moby/api v1.52.0, moby/client v0.1.0 buildx#3326

What I did

Related issue

(not mandatory) A picture of a cute animal, if possible in relation to what you did

ndeloof · 2025-07-21T18:56:05Z

Nice to see moby API as a dedicated module, so we don't get transitive dependencies from engine inside Compose ! ❤️

cmd/compose/bridge.go

thaJeztah · 2025-07-21T18:58:32Z

pkg/compose/internal/urlutil/urlutil.go

@@ -0,0 +1,89 @@
+// Package urlutil provides helper function to check if a given build-context


Added this fork temporarily; didn't realise compose itself also uses it (not only through cli); the CLI fork is "internal", so probably needs to made public, or we keep a fork here 😞

thaJeztah · 2025-07-21T18:58:58Z

Remaining uses of docker/docker;

tree -d vendor/github.com/docker/docker
vendor/github.com/docker/docker
├── internal
│   └── lazyregexp
├── pkg
│   ├── jsonmessage
│   ├── namesgenerator
│   ├── pidfile
│   ├── process
│   ├── progress
│   ├── stdcopy
│   ├── streamformatter
│   └── stringid
└── registry

13 directories

thaJeztah · 2025-07-21T19:00:26Z

Build failure looks like it may be an incompatible change in buildx itself?

 > [build 1/1] RUN --mount=type=bind,target=.     --mount=type=cache,target=/root/.cache     --mount=type=cache,target=/go/pkg/mod     --mount=type=bind,from=osxcross,src=/osxsdk,target=/xx-sdk     xx-go --wrap &&     if [ "$(xx-info os)" == "darwin" ]; then export CGO_ENABLED=1; fi &&     make build GO_BUILDTAGS="e2e" DESTDIR=/out &&     xx-verify --static /out/docker-compose:
1.027 GO111MODULE=on go build  -trimpath -tags "e2e" -ldflags "-w -X github.com/docker/compose/v2/internal.Version=fa50b46" -o "/out/docker-compose" ./cmd
38.90 # github.com/docker/buildx/build
38.90 /go/pkg/mod/github.com/tha!jeztah/buildx@v0.2.1-0.20250721183044-80b40a761230/build/provenance.go:135:16: pred1.ConvertToSLSA02 undefined (type *"github.com/moby/buildkit/solver/llbsolver/provenance/types".ProvenancePredicateSLSA1 has no field or method ConvertToSLSA02)
67.83 make: *** [Makefile:61: build] Error 1

38.90 /go/pkg/mod/github.com/tha!jeztah/buildx@v0.2.1-0.20250721183044-80b40a761230/build/provenance.go:135:16: pred1.ConvertToSLSA02 undefined (type

thaJeztah · 2025-07-21T19:10:48Z

OK, it's because buildx depends on an unreleased version of BuildKit, so go modules considers it to be older than the latest release and won't update; https://github.com/docker/buildx/blob/3f4bf829d8c5b92a8f670609417d9aa4c0b6be98/go.mod#L32

github.com/moby/buildkit v0.23.0-rc1.0.20250618182037-9b91d20367db // master

ndeloof · 2025-07-22T06:04:18Z

jsonmessage
this one should be part of the API, don't you think ?

thaJeztah · 2025-07-22T06:09:46Z

Yes, or at least the type; the utilities could be "client" - those packages are messy.

Also looking at things that are in the api, but shouldn't (api/types/plugins/logdriver)

thaJeztah · 2025-07-23T20:32:59Z

Also included a (WIP) branch for docker/cli#6202

Before/After;

Details

Before:

tree -d vendor/github.com/docker/cli
vendor/github.com/docker/cli
├── cli
│   ├── command
│   │   ├── completion
│   │   ├── container
│   │   ├── formatter
│   │   │   └── tabwriter
│   │   ├── image
│   │   │   └── build
│   │   │       └── internal
│   │   │           ├── git
│   │   │           └── urlutil
│   │   └── inspect
│   ├── compose
│   │   ├── interpolation
│   │   ├── loader
│   │   ├── schema
│   │   │   └── data
│   │   ├── template
│   │   └── types
│   ├── config
│   │   ├── configfile
│   │   ├── credentials
│   │   ├── memorystore
│   │   └── types
│   ├── connhelper
│   │   ├── commandconn
│   │   ├── internal
│   │   │   └── syntax
│   │   └── ssh
│   ├── context
│   │   ├── docker
│   │   └── store
│   ├── debug
│   ├── flags
│   ├── hints
│   ├── streams
│   ├── trust
│   └── version
├── cli-plugins
│   ├── hooks
│   ├── manager
│   ├── metadata
│   ├── plugin
│   └── socket
├── internal
│   ├── jsonstream
│   ├── lazyregexp
│   ├── prompt
│   └── tui
├── opts
│   └── swarmopts
├── pkg
│   └── kvfile
└── templates

55 directories

After:

tree -d vendor/github.com/docker/cli
vendor/github.com/docker/cli
├── cli
│   ├── command
│   │   ├── completion
│   │   ├── container
│   │   ├── formatter
│   │   │   └── tabwriter
│   │   ├── image
│   │   │   └── build
│   │   │       └── internal
│   │   │           ├── git
│   │   │           └── urlutil
│   │   └── inspect
│   ├── config
│   │   ├── configfile
│   │   ├── credentials
│   │   ├── memorystore
│   │   └── types
│   ├── connhelper
│   │   ├── commandconn
│   │   ├── internal
│   │   │   └── syntax
│   │   └── ssh
│   ├── context
│   │   ├── docker
│   │   └── store
│   ├── debug
│   ├── flags
│   ├── hints
│   ├── streams
│   ├── trust
│   └── version
├── cli-plugins
│   ├── hooks
│   ├── manager
│   ├── metadata
│   ├── plugin
│   └── socket
├── internal
│   ├── jsonstream
│   ├── lazyregexp
│   ├── prompt
│   ├── tui
│   └── volumespec
├── opts
├── pkg
│   └── kvfile
└── templates

48 directories

thaJeztah · 2025-08-05T22:52:03Z

Down to two packages; both possibly candidates for github.com/moby/sys;

tree -d vendor/github.com/docker/docker
vendor/github.com/docker/docker
└── pkg
    ├── pidfile
    └── process

4 directories

CLI still needs sorting out;

Details

tree -d vendor/github.com/docker/cli
vendor/github.com/docker/cli
├── cli
│   ├── command
│   │   ├── completion
│   │   ├── container
│   │   ├── formatter
│   │   │   └── tabwriter
│   │   ├── image
│   │   │   └── build
│   │   │       └── internal
│   │   │           ├── git
│   │   │           └── urlutil
│   │   ├── inspect
│   │   └── system
│   │       └── pruner
│   ├── config
│   │   ├── configfile
│   │   ├── credentials
│   │   ├── memorystore
│   │   └── types
│   ├── connhelper
│   │   ├── commandconn
│   │   ├── internal
│   │   │   └── syntax
│   │   └── ssh
│   ├── context
│   │   ├── docker
│   │   └── store
│   ├── debug
│   ├── flags
│   ├── hints
│   ├── streams
│   ├── trust
│   └── version
├── cli-plugins
│   ├── hooks
│   ├── manager
│   ├── metadata
│   ├── plugin
│   └── socket
├── internal
│   ├── jsonstream
│   ├── lazyregexp
│   ├── prompt
│   ├── registry
│   ├── tui
│   └── volumespec
├── opts
├── pkg
│   └── kvfile
└── templates

51 directories

pkg/compose/push.go

pkg/compose/pull.go

codecov · 2026-01-26T08:56:27Z

Codecov Report

❌ Patch coverage is 47.45989% with 393 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/dryrun/dryrunclient.go	0.00%	218 Missing ⚠️
pkg/compose/create.go	65.89%	37 Missing and 22 partials ⚠️
pkg/compose/push.go	0.00%	19 Missing ⚠️
cmd/compose/list.go	0.00%	15 Missing and 2 partials ⚠️
pkg/bridge/transformers.go	17.64%	13 Missing and 1 partial ⚠️
pkg/compose/cp.go	56.66%	13 Missing ⚠️
pkg/compose/generate.go	0.00%	13 Missing ⚠️
pkg/compose/pull.go	54.54%	8 Missing and 2 partials ⚠️
pkg/compose/events.go	0.00%	5 Missing ⚠️
pkg/compose/watch.go	80.00%	5 Missing ⚠️
... and 12 more

📢 Thoughts on this report? Let us know!

pkg/compose/create_test.go

thaJeztah · 2026-02-10T15:49:17Z

go.mod

@@ -18,7 +18,6 @@ require (
 	github.com/docker/cli v29.2.0+incompatible


Oh! Thought I had updated to v29.2.1; let me do so as well (no significant changes I think though)

Also update TestDefaultNetworkSettings: Test that the network with the highest priority is returned as "primary" network, and other networks as extra networks. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Format layer progress details with minimal efforts as new UI does not render individual layers Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Format layer progress details with minimal efforts as new UI does not render individual layers. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

glours

Sounds good to me, just 2 points we need to check

glours · 2026-02-10T16:45:28Z

pkg/compose/watch.go

 func (s *composeService) pruneDanglingImagesOnRebuild(ctx context.Context, projectName string, imageNameToIdMap map[string]string) {
-	images, err := s.apiClient().ImageList(ctx, image.ListOptions{
-		Filters: filters.NewArgs(
-			filters.Arg("dangling", "true"),


We lost the projectLabel filter, no?

Let me check!

Ah, right, so it now uses projectFilter;

images, err := s.apiClient().ImageList(ctx, client.ImageListOptions{ Filters: projectFilter(projectName).Add("dangling", "true"), })

which should add it;

compose/pkg/compose/filters.go

Lines 27 to 29 in c75e494

func projectFilter(projectName string) client.Filters {

return make(client.Filters).Add("label", fmt.Sprintf("%s=%s", api.ProjectLabel, projectName))

}

glours · 2026-02-10T16:47:24Z

pkg/compose/watch.go

-	containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{
-		All: true,
-		Filters: filters.NewArgs(
-			filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, project.Name)),


Same here as well, so I think we should be good 🫶

thaJeztah requested a review from a team as a code owner July 21, 2025 18:51

thaJeztah requested review from glours and ndeloof July 21, 2025 18:51

thaJeztah marked this pull request as draft July 21, 2025 18:51

thaJeztah changed the title ~~Bump moby cli~~ WIP: migrate to moby modules Jul 21, 2025

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from a5b199b to 73bb946 Compare July 21, 2025 18:55

thaJeztah commented Jul 21, 2025

View reviewed changes

cmd/compose/bridge.go Outdated Show resolved Hide resolved

thaJeztah commented Jul 21, 2025

View reviewed changes

thaJeztah mentioned this pull request Jul 21, 2025

Create github.com/moby/moby/api and github.com/moby/moby/client module moby/moby#50280

Merged

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from 83ca48f to 13827c4 Compare July 21, 2025 19:20

thaJeztah force-pushed the bump_moby_cli branch from 13827c4 to 52f0ad8 Compare July 23, 2025 20:32

thaJeztah force-pushed the bump_moby_cli branch 3 times, most recently from d2146d4 to 3a7124d Compare July 24, 2025 09:24

thaJeztah force-pushed the bump_moby_cli branch 3 times, most recently from 3b1ec4c to 6cd56af Compare August 5, 2025 22:51

thaJeztah force-pushed the bump_moby_cli branch from 6cd56af to 8925b9c Compare August 5, 2025 23:13

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from d7c329f to 04ac165 Compare August 28, 2025 21:12

thaJeztah commented Jan 22, 2026

View reviewed changes

pkg/compose/push.go Outdated Show resolved Hide resolved

thaJeztah commented Jan 22, 2026

View reviewed changes

pkg/compose/pull.go Outdated Show resolved Hide resolved

thaJeztah force-pushed the bump_moby_cli branch 3 times, most recently from ce597b5 to 8797fa0 Compare January 26, 2026 08:45

thaJeztah force-pushed the bump_moby_cli branch from 8797fa0 to b6784a8 Compare January 27, 2026 10:25

thaJeztah force-pushed the bump_moby_cli branch from b6784a8 to 3dbbaa6 Compare February 4, 2026 14:24

thaJeztah commented Feb 4, 2026

View reviewed changes

pkg/compose/create_test.go Show resolved Hide resolved

thaJeztah force-pushed the bump_moby_cli branch 5 times, most recently from 6510727 to fae6d96 Compare February 4, 2026 16:48

thaJeztah closed this Feb 4, 2026

thaJeztah reopened this Feb 4, 2026

ndeloof mentioned this pull request Feb 5, 2026

Format layer progress details with minimal efforts as new UI does not render individual layers thaJeztah/compose#335

Merged

thaJeztah force-pushed the bump_moby_cli branch 2 times, most recently from f93c1f2 to 88078fb Compare February 10, 2026 12:53

thaJeztah mentioned this pull request Feb 10, 2026

pkg/compose: defaultNetworkSettings: refactor #13580

Draft

1 task

thaJeztah marked this pull request as ready for review February 10, 2026 12:57

thaJeztah commented Feb 10, 2026

View reviewed changes

thaJeztah and others added 5 commits February 10, 2026 16:51

go.mod: bump github.com/moby/moby/api v1.53.0, moby/client v0.2.2

c75e494

Also update TestDefaultNetworkSettings: Test that the network with the highest priority is returned as "primary" network, and other networks as extra networks. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/bridge: remove uses of go-connections

f168567

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/compose: remove unused consts

eb7130e

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/compose: format layer pull progress

e001ebe

Format layer progress details with minimal efforts as new UI does not render individual layers Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/compose: format layer push progress

47a2724

Format layer progress details with minimal efforts as new UI does not render individual layers. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah force-pushed the bump_moby_cli branch from 88078fb to 47a2724 Compare February 10, 2026 15:51

glours approved these changes Feb 10, 2026

View reviewed changes

vvoland approved these changes Feb 10, 2026

View reviewed changes

		@@ -0,0 +1,89 @@
		// Package urlutil provides helper function to check if a given build-context

		@@ -18,7 +18,6 @@ require (
		github.com/docker/cli v29.2.0+incompatible

	func projectFilter(projectName string) client.Filters {
	return make(client.Filters).Add("label", fmt.Sprintf("%s=%s", api.ProjectLabel, projectName))
	}

migrate to moby modules #13078

Are you sure you want to change the base?

migrate to moby modules #13078

Conversation

thaJeztah commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ndeloof commented Jul 21, 2025

Uh oh!

Uh oh!

thaJeztah Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

thaJeztah commented Jul 21, 2025

Uh oh!

ndeloof commented Jul 22, 2025

Uh oh!

thaJeztah commented Jul 22, 2025

Uh oh!

thaJeztah commented Jul 23, 2025

Uh oh!

thaJeztah commented Aug 5, 2025

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

thaJeztah Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

glours left a comment

Choose a reason for hiding this comment

Uh oh!

glours Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

thaJeztah Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

thaJeztah Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

glours Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

thaJeztah Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

thaJeztah commented Jul 21, 2025 •

edited

Loading

codecov bot commented Jan 26, 2026 •

edited

Loading

thaJeztah Feb 10, 2026 •

edited

Loading