feat: add human-like smooth typing with optional typo injection#201
feat: add human-like smooth typing with optional typo injection#201ulziibay-kernel wants to merge 3 commits intomainfrom
Conversation
Adds smooth typing mode to POST /computer/type that types text in word-sized chunks with variable intra-word delays and natural inter-word pauses via xdotool, following the same Go-side sleep pattern as doMoveMouseSmooth. Optionally injects realistic typos (adjacent-key, doubling, transpose) using geometric gap sampling (O(typos) random calls, not O(chars)) with QWERTY adjacency lookup, then corrects them with backspace after a "realization" pause. New API fields on TypeTextRequest: - smooth: boolean (default false) - enable human-like timing - typo_chance: number 0.0-0.10 (default 0) - per-char typo rate New package: server/lib/typinghumanizer with word chunking, QWERTY adjacency map, and typo position generation. Made-with: Cursor
- typing_demo.html: Dark-themed page with a textarea and keystroke timing visualization (bar chart of inter-key intervals) - demo_smooth_typing.py: Records three phases (instant, smooth, smooth+typos) via the kernel API for comparison GIF/MP4 Made-with: Cursor
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| sh.options.RequestErrorHandlerFunc(w, r, fmt.Errorf("can't decode JSON body: %w", err)) | ||
| return | ||
| } | ||
| request.Body = &body |
There was a problem hiding this comment.
Codegen downgrade breaks endpoints accepting empty request bodies
High Severity
The oapi-codegen version downgrade from v2.6.0 to v2.5.1 removed the io.EOF graceful handling for empty request bodies in the generated middleware for TakeScreenshot, DeleteRecording, StartRecording, and StopRecording. Previously, empty bodies were allowed and request.Body was left nil. Now, any empty body triggers an error response. The TakeScreenshot handler explicitly handles request.Body == nil to allow bodyless calls, so this is a breaking regression — clients calling the screenshot endpoint without a JSON body will get an error instead of a full-screen capture.
jarugupj
left a comment
jarugupj
left a comment
There was a problem hiding this comment.
Really cool stuff, the typinghumanizer package is super clean. The realization pause before backspace is a great detail, and the typo types cover the common cases really well -- adjacent key, doubling, transpose, extra char feels like it captures how people actually mess up.
One small thing I noticed -- in typeChunkWithTypo, the transpose branch and the else branch do the same thing:
if typo.Kind == typinghumanizer.TypoTranspose && typoLocalPos+1 < len(chunkRunes) {
correctText = string(chunkRunes[typoLocalPos:])
} else {
correctText = string(chunkRunes[typoLocalPos:])
}Could probably just be correctText = string(chunkRunes[typoLocalPos:]) without the if/else?
Both branches produced identical correctText. Simplified to a single expression as suggested by jarugupj. Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
There are 3 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 99e2e79. Configure here.
|
|
||
| // ExitCode Exit code if the process has exited. | ||
| ExitCode *int `json:"exit_code,omitempty"` | ||
| ExitCode *int `json:"exit_code"` |
There was a problem hiding this comment.
Removed omitempty changes API JSON response format
Medium Severity
The regenerated oapi.go dropped omitempty from several pointer-typed JSON struct tags (e.g., ExitCode, AsUser, Cwd, TimeoutSec, FinishedAt, StartedAt). These fields will now serialize as null in JSON responses when nil, instead of being omitted entirely. This is a breaking change for API consumers that don't expect null values for these fields.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 99e2e79. Configure here.
| chunkTypo = &t | ||
| break | ||
| } | ||
| } |
There was a problem hiding this comment.
Only first typo per chunk is applied
Medium Severity
doTypeTextSmooth generates typo positions across the full text via GenerateTypoPositions, but the per-chunk loop breaks after finding the first typo in each chunk. Any additional typos falling within the same chunk are silently dropped. For long words or text without spaces (single chunk), this means only one typo is ever injected regardless of how many were generated, making the effective typo rate lower than requested.
Reviewed by Cursor Bugbot for commit 99e2e79. Configure here.
| type: number | ||
| description: | | ||
| Probability (0.0-0.10) of a typo per character, which is then | ||
| corrected with backspace. Requires smooth to be true. Set to 0 |
There was a problem hiding this comment.
nit: "Requires smooth to be true" — this wording implies the server would return an error, but it actually silently ignores it. consider "Ignored when smooth is false" to match the actual behavior. no schema-level constraint enforces this either, so aligning the docs avoids confusion.
| return adj | ||
| } | ||
|
|
||
| // GenerateTypoPositions computes typo positions using geometric gap sampling. |
There was a problem hiding this comment.
nit: comment says "geometric gap sampling" but the implementation uses uniform gap sampling — halfGap + Intn(avgGap) produces a uniform distribution over [halfGap, halfGap+avgGap). geometric sampling would use exponentially distributed gaps (e.g. rng.ExpFloat64()). the behavior is fine, just the label is inaccurate.
| } | ||
| total := counts[TypoAdjacentKey] + counts[TypoDoubling] + counts[TypoTranspose] + counts[TypoExtraChar] | ||
| require.Greater(t, total, 0) | ||
| adjPct := float64(counts[TypoAdjacentKey]) / float64(total) |
There was a problem hiding this comment.
this only asserts TypoAdjacentKey (~60%). the other three kinds (TypoDoubling ~20%, TypoTranspose ~15%, TypoExtraChar ~5%) are counted but never checked — if two kinds got accidentally merged the test would still pass. consider asserting all four buckets.
masnwilliams
left a comment
masnwilliams
left a comment
There was a problem hiding this comment.
solid feature — clean separation between the humanizer library and the integration layer. main notes are nits on docs wording and test coverage. the identical if/else branches in typeChunkWithTypo were already flagged.
3 participants
Summary
smoothandtypo_chancefields toPOST /computer/typefor human-like typing via xdotoolsmooth=true, text is typed in word-sized chunks with variable intra-word delays ([50, 120]ms) and inter-word pauses ([80, 200]ms, 1.5x at sentence boundaries)typo_chanceis set (0.0-0.10), realistic typos are injected using geometric gap sampling (O(typos) random calls, not O(chars)) and corrected with backspace after a "realization" pauseNew package:
server/lib/typinghumanizerSplitWordChunks-- word chunking with trailing delimiters attached to preceding wordUniformJitter-- random duration in a range, clamped to minimumIsSentenceEnd-- sentence boundary detection for longer pausesAdjacentKey-- QWERTY neighbor lookup from static[26][]bytearray, O(1)GenerateTypoPositions-- geometric gap sampling for typo placementTypo types (weighted distribution)
Related: #169 (plan document)
Demo
Test plan
smooth: trueon a running instancesmooth: true, typo_chance: 0.03to verify typo/correction behaviorMade with Cursor
Note
Medium Risk
Medium risk: changes
POST /computer/typebehavior and adds non-trivial timing/typo logic aroundxdotool, plus regenerates OpenAPI client/server bindings which can affect request parsing/optional fields.Overview
Adds a new human-like typing mode to
POST /computer/typeviasmoothandtypo_chance, typing in word chunks with jittered delays, optional typo injection, and backspace-based correction.Introduces
server/lib/typinghumanizer(with tests) to handle chunking, timing jitter, QWERTY-adjacent key selection, and typo position generation;computer.gonow routes to a new smooth-typing path whensmooth=true.Updates the OpenAPI spec and regenerates
server/lib/oapi/oapi.go/go.sumaccordingly, and adds a demo HTML page plus a Python script to record an instant-vs-smooth typing comparison video.Reviewed by Cursor Bugbot for commit 99e2e79. Bugbot is set up for automated code reviews on this repo. Configure here.