Replace Playwright with Kernel native API in OpenAI CUA templates (#124)

## Summary

- Replace Playwright (over CDP) with Kernel's native computer control
API in both TypeScript and Python OpenAI CUA templates
- Add `batch_computer_actions` function tool that executes multiple
browser actions in a single API call, reducing latency
- Add local test scripts (`test.local.ts` / `test_local.py`) that create
remote Kernel browsers for testing without deploying a Kernel app

## Details

**New `KernelComputer` class** (TS + Python) wraps the Kernel SDK for
all computer actions:
- `captureScreenshot`, `clickMouse`, `typeText`, `pressKey`, `scroll`,
`moveMouse`, `dragMouse`
- `batch` endpoint for batched actions
- `playwright.execute` for navigation (`goto`, `back`, `forward`,
`getCurrentUrl`)
- CUA key name to X11 keysym translation map (ported from Go reference
implementation)
- Button normalization (CUA model sends numeric button values
`1`/`2`/`3` in batch calls)

**Batch tool**: System instructions guide the model to prefer
`batch_computer_actions` for predictable sequences (e.g., click + type +
enter).

**Removed dependencies**: `playwright-core`, `sharp` (TS), `playwright`
(Python). Bumped `@onkernel/sdk` to `^0.38.0` and `kernel` to
`>=0.38.0`.

## Test plan

- [x] TypeScript `test.local.ts` E2E: created remote Kernel browser, ran
CUA agent (eBay search task), batch tool used successfully, browser
cleaned up
- [x] Python `test_local.py` E2E: same test, batch tool used on first
action (type + enter), agent completed successfully
- [x] TypeScript compiles cleanly (`tsc --noEmit`)

Made with [Cursor](https://cursor.com)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Moderate risk because it replaces the core browser automation layer
(Playwright/CDP) with new Kernel API wrappers and batching semantics,
which could change runtime behavior and failure modes. No auth/data
model changes beyond new optional replay recording and improved OpenAI
request retry handling.
> 
> **Overview**
> These templates now use **Kernel’s native computer control API**
(screenshot/click/type/scroll/batch) instead of Playwright-over-CDP, via
new `KernelComputer` wrappers in both Python and TypeScript.
> 
> The agent loop is updated to support a new `batch_computer_actions`
function tool (plus `computer_use_extra` for `goto`/`back`/`url`) with
explicit model instructions, post-action screenshot settling, and
structured `on_event` streaming for prompts, reasoning, actions,
screenshots, and errors.
> 
> Adds local runners (`run_local.py`, `run_local.ts`) to test against a
remote Kernel browser without deployment, optional browser replay
recording (`replay.py` / `lib/replay.ts`) surfaced as `replay_url`, and
updates dependencies/config/docs (Kernel SDK bump,
Playwright/sharp/Pillow removals, `.env.example` includes
`KERNEL_API_KEY`, `.gitignore` adds caches).
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
9f0f9b5. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: rgarcia

.gitignore

@@ -44,3 +44,7 @@ kernel
 
 # QA testing directories
 qa-*
+
+
+__pycache__
+.dmux/

pkg/templates/python/openai-computer-use/.env.example

@@ -1,2 +1,3 @@
-# Copy this file to .env and fill in your API key
+# Copy this file to .env and fill in your API keys
 OPENAI_API_KEY=your_openai_api_key_here
+KERNEL_API_KEY=your_kernel_api_key_here

pkg/templates/python/openai-computer-use/README.md

@@ -1,7 +1,27 @@
 # Kernel Python Sample App - OpenAI Computer Use
 
-This is a Kernel application that demonstrates using the Computer Use Agent (CUA) from OpenAI.
+This is a Kernel application that demonstrates using the Computer Use Agent (CUA) from OpenAI with Kernel's native browser control API.
 
-It generally follows the [OpenAI CUA Sample App Reference](https://github.com/openai/openai-cua-sample-app) and uses Playwright via Kernel for browser automation.
+It uses Kernel's computer control endpoints (screenshot, click, type, scroll, batch, etc.) and includes a `batch_computer_actions` tool that executes multiple actions in a single API call for lower latency.
 
-See the [docs](https://www.kernel.sh/docs/quickstart) for more information.
+## Local testing
+
+You can test against a remote Kernel browser without deploying:
+
+```bash
+cp .env.example .env
+# Fill in OPENAI_API_KEY and KERNEL_API_KEY in .env
+uv run run_local.py
+uv run run_local.py --task "go to https://news.ycombinator.com and get the top 5 articles"
+```
+
+The local runner defaults to a built-in sample task. Pass `--task "..."` to run a custom prompt locally, and add `--debug` to include verbose in-flight events.
+
+## Deploy to Kernel
+
+```bash
+kernel deploy main.py --env-file .env
+kernel invoke python-openai-cua cua-task -p '{"task":"go to https://news.ycombinator.com and list top 5 articles"}'
+```
+
+See the [docs](https://www.kernel.sh/docs/quickstart) for more information.