Virtual/on-screen joystick support by riccardobl · Pull Request #2803 · jMonkeyEngine/jmonkeyengine

riccardobl · 2026-05-24T23:54:29Z

This PR makes joystick support available everywhere by adding an on-screen virtual joystick integrated directly into the input backend.

The advantage of this approach is that a jME application sees the virtual joystick just like any other Xbox-style joystick supported by jME. No special handling is needed: applications can use the usual joystick bindings.

The controls are rendered as spatials attached to the node passed to the joystick update method. When using SimpleApplication, this is already handled correctly and the controls end up in the guiNode, so normal jME apps do not need any additional setup.

(wip needs rebase)

Since it now conflicts with this, the AndroidSensorJoyInput (that is used to simulate android joysticks using accelerometer), is now disabled by default and gated behind the UseAndroidSensorJoystick app setting.

Other changes to AppSettings:

DisableJoysticks is now off by default, meaning gamepads are always supported when available
VirtualJoystick is set to VIRTUAL_JOYSTICK_AUTO_MINIMIZED, this will display the button to toggle the virtual gamepad on mobile, unless an hardware gamepad is connected, other values are:
- VIRTUAL_JOYSTICK_DISABLED : disable the virtual gamepad entirely
- VIRTUAL_JOYSTICK_ENABLED_MINIMIZED always display the "toggle gamepad button" even on desktop and even if an hardware gamepad is detected
- VIRTUAL_JOYSTICK_ENABLED same as VIRTUAL_JOYSTICK_ENABLED_MINIMIZED but will actually show the virtual gamepad, not only the toggle button
- VIRTUAL_JOYSTICK_AUTO will show the virtual gamepad automatically on mobile if no hardware gamepad is found

riccardobl · 2026-05-25T12:15:23Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a comprehensive on-screen virtual joystick system for jMonkeyEngine, enabling gamepad-like controls on mobile and desktop platforms. The implementation includes new classes for joystick logic, layout management, and theming, as well as integration with existing camera controllers and application settings. Review feedback identifies critical thread-safety issues in the Android, iOS, and LWJGL3 backends, specifically regarding the cross-thread visibility of the virtual joystick instance and synchronization of shared layout and theme data. A regression was also noted in the iOS implementation where POV Y axis support for physical joysticks was removed.

gemini-code-assist · 2026-05-25T12:19:49Z

+    private boolean useAndroidSensorJoystick = false;
+    private boolean physicalJoystickAvailable = false;
+    private boolean keyboardSuppressedAutoJoystick = false;
+    private VirtualJoystick virtualJoystick;


The virtualJoystick field should be declared as volatile. It is initialized in loadJoysticks (which runs on the JME thread) and accessed in onTouch (which runs on the Android UI thread). Without volatile, the input thread might see a stale null value or a partially initialized object.

Suggested change

private VirtualJoystick virtualJoystick;

private volatile VirtualJoystick virtualJoystick;

gemini-code-assist · 2026-05-25T12:19:49Z

@@ -67,12 +70,15 @@
 public final class IosJoyInput implements JoyInput {
    private static IosJoyInput active;
    private static final int POV_X_AXIS_ID = 0x4000;


The POV Y axis support for physical joysticks seems to have been removed in this file (previously POV_Y_AXIS_ID = 0x4001). This is a regression for physical gamepads on iOS, as the D-pad vertical axis will no longer be available via getPovYAxis(). Please restore the POV Y axis handling to match the LWJGL3 implementation.

gemini-code-assist · 2026-05-25T12:19:49Z

+    private String virtualJoystickMode = AppSettings.VIRTUAL_JOYSTICK_AUTO_MINIMIZED;
+    private boolean useJoysticks = true;
+    private boolean keyboardSuppressedAutoJoystick;
+    private VirtualJoystick virtualJoystick;


The virtualJoystick field should be declared as volatile to ensure thread-safe publication, as it is assigned in loadJoysticks and read in onPointerDown/Move/Up from different threads.

Suggested change

private VirtualJoystick virtualJoystick;

private volatile VirtualJoystick virtualJoystick;

gemini-code-assist · 2026-05-25T12:19:49Z

    private float globalJitterThreshold;
    private boolean loadGamepads;
    private boolean loadRaw;
+    private VirtualJoystick virtualJoystick;


The virtualJoystick field should be declared as volatile to ensure visibility across threads, as it is initialized in loadJoysticks (JME thread) and accessed in onPointerDown/Move/Up (input thread).

Suggested change

private VirtualJoystick virtualJoystick;

private volatile VirtualJoystick virtualJoystick;

gemini-code-assist · 2026-05-25T12:19:49Z

+    public Element getButtonElement(String logicalId) {
+        return buttons.get(logicalId);
+    }
+
+    public Element getAxisElement(String logicalId) {
+        return findAxisElement(logicalId);
+    }


The getButtonElement and getAxisElement methods access internal maps (buttons, axisElements) that are cleared and repopulated in the read method. Since read is synchronized, these getters should also be synchronized to avoid race conditions or ConcurrentModificationException if the layout is being reloaded while being accessed from another thread.

gemini-code-assist · 2026-05-25T12:19:49Z

+        void sync(int width, int height, float scale, boolean pressed) {
+            float shortSide = Math.min(width, height);
+            float scaledShortSide = shortSide * scale;
+            pixelSize = Math.max(scaledShortSide * size, 1f);
+            pixelWidth = pixelSize * aspect;
+            pixelHeight = pixelSize;
+            pixelX = positionX * width + shortOffsetX * scaledShortSide;
+            pixelY = positionY * height + shortOffsetY * scaledShortSide;
+            if (pixelWidth < width) {
+                pixelX = FastMath.clamp(pixelX, pixelWidth * 0.5f, width - pixelWidth * 0.5f);
+            } else {
+                pixelX = width * 0.5f;
+            }
+            if (pixelHeight < height) {
+                pixelY = FastMath.clamp(pixelY, pixelHeight * 0.5f, height - pixelHeight * 0.5f);
+            } else {
+                pixelY = height * 0.5f;
+            }
+            float baseX = -pixelWidth * 0.5f;
+            float baseY = -pixelHeight * 0.5f;
+            if (!baseGeometrySynced || lastBaseWidth != pixelWidth) {
+                base.setWidth(pixelWidth);
+                lastBaseWidth = pixelWidth;
+            }
+            if (!baseGeometrySynced || lastBaseHeight != pixelHeight) {
+                base.setHeight(pixelHeight);
+                lastBaseHeight = pixelHeight;
+            }
+            if (!baseGeometrySynced || lastBaseX != baseX || lastBaseY != baseY) {
+                base.setPosition(baseX, baseY);
+                lastBaseX = baseX;
+                lastBaseY = baseY;
+            }
+            baseGeometrySynced = true;


The sync method updates multiple volatile fields (pixelX, pixelY, pixelWidth, pixelHeight, etc.) sequentially. The input thread calling contains(x, y) might see an inconsistent state where some fields are updated for the current frame and others are still from the previous frame.

It is recommended to calculate all values using local variables and assign them to the volatile fields at the end of the method to minimize the window for race conditions.

gemini-code-assist · 2026-05-25T12:19:49Z

+        return textures.get(key);
+    }
+
+    public void setTexture(TextureKey key, String texturePath) {
+        if (key == null) {
+            throw new IllegalArgumentException("Texture key cannot be null.");
+        }
+        if (texturePath == null) {
+            textures.remove(key);
+        } else {
+            textures.put(key, texturePath);
+        }
+        markUpdateNeeded();
+    }


The textures map (an EnumMap) is not thread-safe. Since VirtualJoystickTheme can be modified from the UI thread (via setTexture) while being read by the JME thread in VirtualJoystick.updateVisuals, access to this map should be synchronized.

riccardobl changed the title ~~Virtual joystick support~~ Virtual/on-screen joystick support May 24, 2026

This comment was marked as outdated.

Sign in to view

riccardobl force-pushed the onscreengamepad branch 2 times, most recently from 1c6fb91 to b81c49d Compare May 25, 2026 09:48

riccardobl added 3 commits May 25, 2026 13:41

Virtual/on-screen gamepad

b1d277a

documentation

8725dad

disable AndroidSensorJoyInput

ab54490

riccardobl force-pushed the onscreengamepad branch from b81c49d to ab54490 Compare May 25, 2026 12:01

riccardobl marked this pull request as ready for review May 25, 2026 12:02

This comment was marked as outdated.

Sign in to view

reduce objects allocations

0d5efb5

gemini-code-assist Bot reviewed May 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Virtual/on-screen joystick support#2803

Virtual/on-screen joystick support#2803
riccardobl wants to merge 4 commits into
jMonkeyEngine:masterfrom
riccardobl:onscreengamepad

riccardobl commented May 24, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

riccardobl commented May 25, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 25, 2026

Uh oh!

gemini-code-assist Bot May 25, 2026

Uh oh!

gemini-code-assist Bot May 25, 2026

Uh oh!

gemini-code-assist Bot May 25, 2026

Uh oh!

gemini-code-assist Bot May 25, 2026

Uh oh!

gemini-code-assist Bot May 25, 2026

Uh oh!

gemini-code-assist Bot May 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	private VirtualJoystick virtualJoystick;
	private volatile VirtualJoystick virtualJoystick;

Uh oh!

Conversation

riccardobl commented May 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

riccardobl commented May 25, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 25, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 25, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 25, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 25, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 25, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 25, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

riccardobl commented May 24, 2026 •

edited

Loading