Add Compose hierarchy inspector (preview)#60
Open
actorFromCalifornia wants to merge 1 commit into
Open
Conversation
…mpose Hierarchy
Adds a new menu action that captures the current screen via `uiautomator dump`
and analyses it with a Compose-aware heuristic to produce a summary dialog.
The action is independent of the existing capture flow and does not modify
any code paths used by Launch YALI.
What the new action does:
- enumerates connected devices via the existing ConnectedDeviceInfoProvider;
- runs `adb shell uiautomator dump`, pulls the XML and parses it with a SAX handler;
- classifies nodes as "Compose candidates" by two rules:
1. any node whose ancestor is `AndroidComposeView` / `ComposeView`;
2. any `android.view.View` with empty resource-id and a non-empty
text or content-description (typical for Compose semantics leaves);
- displays a Swing dialog with the summary stats (total / visible /
Compose roots / Compose candidates / nodes with text / nodes with
content-desc / nodes with resource-id), a tree of Compose candidates
and three side tables (top classes, top texts, top content-descriptions);
- `Copy summary to clipboard` button serialises stats as JSON-like text.
Files added:
- docs/COMPOSE_INSPECTOR_SPEC.md — technical spec covering the proper
App Inspection / compose-ui-inspection.jar approach used by Android
Studio and explaining why this iteration ships the UI Automator-based
Layer 0 (with a roadmap for Layers 1 and 2).
- proto/compose_layout_inspection.proto — vendored protocol file
(Apache 2.0, originates from AOSP / Android Studio source).
Not used at build time yet, kept for the next iteration.
- src/main/kotlin/com/github/grishberg/android/li/compose/* — implementation:
- InspectComposeHierarchyAction.kt — the action;
- ComposeHierarchyAnalyzer.kt — SAX parser + heuristic classifier;
- ComposeInspectDialog.kt — Swing UI;
- model/ComposeCandidateNode.kt — TreeNode for the Compose candidate tree;
- model/ComposeHierarchySummary.kt — summary data class.
- src/main/resources/META-INF/plugin.xml — registers the new action under
the existing `YALI.TopMenu` group, no other changes.
Why UI Automator first:
The full Compose tree (with @composable names, source file / line, parameter
values, recomposition counts) requires Android Studio's App Inspection
framework: a transport daemon pushed to the device deploys
`compose-ui-inspection.jar` into the target process via JVMTI, and the IDE
talks to it over RSocket-on-UDS using the protobuf in
`proto/compose_layout_inspection.proto`. That path involves Studio internal
APIs in `com.android.tools.idea.appinspection.*`, which are unstable across
Studio releases and require non-trivial wiring.
UI Automator is a much simpler proxy: it walks the accessibility tree, which
exposes Compose nodes that opt in via `Modifier.semantics { ... }`. We lose
internal Compose details but get bounds, text, content-desc, resource-id
and clickable/focusable info — enough to ship a useful summary today and
to validate the UI surface before committing to the full agent integration.
See docs/COMPOSE_INSPECTOR_SPEC.md for the staged roadmap.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new menu action Tools → YALI → Inspect Compose Hierarchy that captures the on-screen UI via
uiautomator dumpand surfaces a Compose-focused summary in a Swing dialog. The change is purely additive — the existing Launch YALI flow is untouched.The dialog reports:
AndroidComposeView/ComposeView)AndroidComposeView, plusandroid.view.Viewleaves with emptyresource-idbut non-emptytext/content-desc— the typical shape of Compose semantics nodes exposed through accessibility)text, withcontent-desc, withresource-idPlus a tree view of the Compose candidate subtree and a "Copy summary to clipboard" action.
Why UI Automator for now
The complete Compose tree (with
@Composablenames, source location, parameters, recomposition counts) requires Android Studio's App Inspection framework: a transport daemon pushed to the device, which deployscompose-ui-inspection.jarinto the target process via JVMTI, and a protobuf RPC channel between the IDE and the agent. That path touchescom.android.tools.idea.appinspection.*internals which are unstable across Studio releases. UI Automator gives a stable subset (semantics-merged tree) immediately and lets us ship the UI without committing to the full agent integration yet.The full plan, including the layered roadmap, is in
docs/COMPOSE_INSPECTOR_SPEC.md. The protobuf used by the App Inspection agent (taken from the existingcomposebranch / AOSPandroidx.compose.ui:ui-inspection) is checked in atproto/compose_layout_inspection.protofor the next iteration.Files
src/main/kotlin/com/github/grishberg/android/li/compose/InspectComposeHierarchyAction.ktsrc/main/kotlin/com/github/grishberg/android/li/compose/ComposeHierarchyAnalyzer.ktsrc/main/kotlin/com/github/grishberg/android/li/compose/ComposeInspectDialog.ktsrc/main/kotlin/com/github/grishberg/android/li/compose/model/{ComposeCandidateNode,ComposeHierarchySummary}.ktsrc/main/resources/META-INF/plugin.xml(registers the new action inside the existingYALI.TopMenu)docs/COMPOSE_INSPECTOR_SPEC.md,proto/compose_layout_inspection.protoTest plan
./gradlew buildPlugin. (I was unable to run this in my sandbox — the build needs to download the Android Studio target platform and a JDK 21 toolchain, both blocked behind my proxy. Please confirm the build is green.)Compose rootsandCompose candidates, plus a populated content-description list).Open as a draft if a reviewer prefers — happy to iterate. The next iteration (Layer 1 in the spec) plugs in the proto-based agent path and yields real
@Composablenames / source positions.