Fixes C# benchmark test failures caused by table naming convention mismatches #4059
+8,323
−1,903
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In StrictMode, React unmounts and remounts components synchronously to detect side effects. The previous cleanup would disconnect immediately, killing the in-flight WebSocket connection. Fix: defer disconnect with setTimeout(..., 0). Since StrictMode remounts happen in the same JavaScript task, the remount cancels the pending timeout, preserving the connection. Real unmounts proceed normally since no remount occurs to cancel the timeout.
The error now hints that the table may be private, helping users diagnose visibility issues.
For consistency with other CLI commands (publish, call, logs, sql), the database name is now a positional argument instead of a flag. The --database flag is deprecated but still works with a warning.
Add "Understand tables and reducers" step showing template code and "Test with the CLI" step demonstrating spacetime call, sql, and logs commands with example output. Also simplify basic-c-sharp template to match Rust/TypeScript templates (person table with name field only).
- Split "Column Types and Constraints" into three pages: - Column Types: primitive, structured, and special types - Primary Keys: rules, implications, multi-column workarounds - Constraints: unique, auto-increment, default values - Add table styling with border radius and row separators - Add Check component for green checkmark badges in tables - Update functions comparison table to use Check component - Add note about view transaction isolation
Schedule tables store schedules; the reducers they trigger are "scheduled reducers". This naming clarifies that the table itself is a schedule, not something that is scheduled. - Rename file and update slug from /tables/scheduled-tables to /tables/schedule-tables - Update all references throughout docs - Add note explaining why code uses "scheduled" (refers to the scheduled reducer)
- Explain tables as SpacetimeDB's fundamental unit (like files in Unix) - Describe self-describing nature via system tables (st_table, st_column) - Connect to data-oriented design philosophy - Recommend table decomposition by access pattern - Provide concrete game schema example showing decomposed approach - Fix stale link to schedule-tables
…ucture - Add section explaining how tables separate logical queries from physical representation - Explain how indexes improve performance without changing queries - Update Table Visibility to mention view functions and link to access permissions - Add brief Constraints section linking to primary keys and constraints pages - Add brief Schedule Tables section linking to schedule tables page - Remove em dashes throughout
- Combine primitive, structured, and special types into single table per language - Add "Representing Collections" section on Vec/Array vs table tradeoffs - Add "Binary Data and Files" section on storing blobs with Vec<u8> - Add "Type Performance" section covering smaller types, fixed-size types, and column ordering for alignment/padding optimization
Column Types: - Rename "Structured" category to "Composite" for clarity - Add complete example showing all type categories in one table Indexes: - Add "When to Use Indexes" section with practical guidance - Document B-tree index type and its capabilities - Cover single-column indexes with field-level and table-level syntax - Explain multi-column indexes with prefix matching semantics - Add comprehensive query examples: equality, range, and multi-column - Document deletion via indexes - Include index design guidelines for column selection and ordering - Fix TypeScript examples to use correct Range class with Bound objects
- /tables/columns -> /tables/column-types (page was renamed) - /tables/scheduled-tables -> /tables/schedule-tables (page was renamed)
…ult values - Create dedicated auto-increment page with sequence semantics - Document trigger value (zero activates auto-increment) - Explain sequence parameters, wrapping, and crash recovery - Add concurrency section: gaps can occur, no sequential guarantee - Include manual counter pattern for strict sequential numbering - Merge primary keys into constraints page - Primary key is technically a constraint - Consolidate all constraint documentation in one place - Create dedicated default values page - Extract from constraints page (not actually a constraint) - Document schema evolution use case - Update all cross-references and Next Steps sections
- Document direct indexes as O(1) alternative to B-tree for dense integer keys - Add index types comparison table (B-tree vs Direct) - Expand access permissions with public/private table visibility - Add view examples for filtering rows by caller and hiding sensitive columns - All view examples use index lookups (not table scans) per performance requirements
Cover all CRUD operations with code samples in all three languages: insert, find, filter, update, delete, iter, and count.
…and examples - Explain why anonymous views scale better (shared materialization) - Add per-user view example (my_player) - Add shared leaderboard example - Add region-based design pattern showing how to structure data for sharing
- Note that Rust requires importing the Table trait for methods like try_insert - Expand ViewContext vs AnonymousViewContext with performance guidance - Add examples: per-user view, shared leaderboard, region-based design
- Add TypeScript examples throughout subscriptions documentation - Fix tab groupId from "server-language" to "client-language" for client SDK docs - Fix reducer-context groupId to "server-language" for consistency - Strengthen RLS deprecation warning, recommend views instead
Fix most issues from John's suggestions. Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com> Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
- Add spacetimedb-rust.mdc with Rust-specific patterns and common mistakes - Add spacetimedb-csharp.mdc with C#-specific patterns and common mistakes - Update spacetimedb-typescript.mdc with views and procedures sections - Update CLI build.rs to embed all language-specific AI rules - Update CLI init.rs to install language-appropriate rules to projects - Update quickstarts to use interactive project naming (no hardcoded name) - Use <database-name> placeholder for CLI command examples
- Add comprehensive sum type (TaggedEnum) documentation to C# AI rules
- Add hallucinated sum type APIs and common mistakes to avoid
- Add Index attribute ambiguity warning (SpacetimeDB.Index vs System.Index)
- Add collection expression warning for attributes (use new[] {})
- Strip ANSI color codes from benchmark stderr/stdout output
Update all C# examples in core docs and quickstarts to use PascalCase table names (e.g., Name = "User" instead of Name = "user") and ensure ctx.Db accessors match the table names exactly.
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
- Add ci-comment command to generate PR comment markdown locally - Add tab filtering for language-specific context (server-language, client-language) - Improve analyze command with language-grouped output and typed failure extraction - Update GitHub workflow to use generated comment file and show diffs vs master - Add ScoreDetails.failure_reason() for typed extraction of failure info - Document context construction and tab filtering in DEVELOP.md
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com> Signed-off-by: Tyler Cloutier <cloutiertyler@users.noreply.github.com>
Added missing C# tab examples for: - Filtering Rows by Caller (Message table, MyMessages view) - Hiding Sensitive Columns (UserAccount table, PublicUserProfile type, MyProfile view) - Combining Both Techniques (Employee table, TeamMember type, MyTeam view)
- Removed the leaderboard example that used .iter() to scan all rows - Added "Why Views Cannot Use .iter()" section explaining: - Views are black boxes that can't be incrementally evaluated - Full table scans create pessimistic read sets requiring full re-evaluation - Index lookups enable targeted invalidation - SQL subscriptions can scan because the query engine can compute incremental updates - The tradeoff is acceptable for indexed access patterns
Signed-off-by: John Detter <4099508+jdetter@users.noreply.github.com>
Changed from .iter() to filtering on a btree-indexed score column to demonstrate how to get high scorers without full table scans.
All 21 C# task prompts now use PascalCase table names matching their corresponding golden answers. This fixes the mismatch where prompts used lowercase (e.g., "users", "primitives") but golden answers used PascalCase (e.g., "User", "Primitive").
Anthropic:
- Add anthropic-beta: prompt-caching-2024-07-31 header
- Add cache_control: {"type": "ephemeral"} to static docs prefix
- This reduces costs by ~90% for cached tokens after the first request
OpenAI:
- Added documentation noting automatic caching behavior
- OpenAI's Responses API caches repeated prefixes automatically
- Static docs prefix placed first to maximize cache hits
- Update table_name() to convert lowercase singular names to appropriate case per language (C#: PascalCase, Rust: snake_case) - Update all spec.rs files to use table_name() instead of hardcoded names - Update Rust task prompts to use singular table names (users → user) - Update Rust golden answers to use singular table/struct names and accessor methods (ctx.db.users() → ctx.db.user()) This fixes the C# benchmark test failures caused by table name mismatches where the LLM generates "User" but tests query for "users".
Contributor
Author
|
/update-llm-benchmark |
- Add compute_processed_context_hash() for language-specific hash computation after tab filtering is applied - Update CI check to verify both rustdoc_json and docs modes for Rust - Fix hash-only mode to skip golden builds - Update benchmark analysis with latest results
Contributor
Author
|
/update-llm-benchmark |
cloutiertyler
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
Fixes C# benchmark test failures caused by table naming convention mismatches.
Problem: The LLM generates
User(PascalCase singular) as instructed by prompts, but the test harness was querying forusers(lowercase plural), resulting in "no such table: users" errors.Solution: Standardize on singular table names for both languages:
user(snake_case singular)User(PascalCase singular)Changes:
table_name()helper to accept lowercase singular names and convert to appropriate case per languagetable_name("user", lang)instead of hardcoded"users"compute_processed_context_hash()for language-specific hash computation after tab filteringrustdoc_jsonanddocsmodes for Rust--hash-onlymode to skip golden buildsAPI and ABI breaking changes
None - these are internal benchmark tooling changes only.
Expected complexity level and risk
Complexity: 2
The changes are straightforward find-and-replace style updates across many files, with a simple helper function modification. Low risk since this only affects the benchmark tooling, not the core SpacetimeDB codebase.
Testing
cargo build -p xtask-llm-benchmarkcompiles successfullycargo llm ci-quickfixto verify C# tests pass with new table names