Skip to content

EMBERDOCS-TECHNICAL-SPEC.md

Latest commit

 

History

History
1130 lines (921 loc) · 26.9 KB

EMBERDOCS-TECHNICAL-SPEC.md

File metadata and controls

1130 lines (921 loc) · 26.9 KB

EmberDocs Technical Specification

Version: 1.0
Last Updated: December 2025
Status: Ready for Development

Executive Summary

EmberDocs is a source-available documentation framework designed for indie developers who need simple, extensible, and beautiful documentation without the complexity of enterprise solutions. The product addresses two core problems: documentation is annoying to maintain for developers and difficult to navigate for users.

Mission: Provide drop-in documentation that works in 30 seconds, grows with your needs, and respects user privacy.

Target Market: Indie developers, OSS maintainers, small teams who want free, self-hosted documentation with optional premium hosting.

Business Model:

  • Self-hosted (Free): Full-featured framework, unlimited usage, self-hosted
  • Premium ($15/mo): Hosted version with zero-deployment, AI-powered search, advanced analytics

Product Overview

Core Value Propositions

  1. Zero-Config to Start: (Planned) npx emberdocs init will create working documentation in 30 seconds. Currently, users clone the repo and run npm install.
  2. True Portability: Works as standalone site, deployable anywhere Next.js runs
  3. Privacy-First: Client-side search, no tracking, privacy-first architecture (no external services)
  4. Git-Native: Version detection from git tags (routing planned for Phase 02), no manual version management
  5. Developer Experience: Hot reload, broken link detection, markdown-only workflow

Positioning

Feature EmberDocs Docusaurus GitBook VitePress
Setup time Typically 2 minutes (with Node.js 18+) 15 minutes 5 minutes 10 minutes
Easy deployment ✅ ❌ ❌ ❌
Self-hosted (free) ✅ ✅ ❌ ✅
Zero config ✅ ❌ Partial Partial
Privacy-first analytics ✅ ❌ ❌ ❌
Semantic AI search ✅ (opt-in) ❌ ❌ ❌

Technical Architecture

Technology Stack

Frontend:

  • Next.js 16+ (App Router)
  • React 18+
  • TypeScript 5+
  • Tailwind CSS 3+

Search:

  • FlexSearch (client-side, default)
  • OpenAI/Anthropic (semantic search, opt-in)

Content Processing:

  • Remark/Rehype (markdown processing)
  • Shiki (syntax highlighting)
  • Gray-matter (frontmatter parsing)

Deployment:

  • Vercel (primary, edge functions)
  • Netlify (alternative)
  • GitHub Pages (static export)
  • Docker (self-hosted)

Analytics:

  • Firefly Stats integration (privacy-friendly)
  • Custom events for doc effectiveness

System Architecture

┌─────────────────────────────────────────────────────────────┐
│                     User Interface                           │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐   │
│  │ Sidebar  │  │  Search  │  │ Content  │  │ Version  │   │
│  │   Nav    │  │   Box    │  │  Area    │  │ Switcher │   │
│  └──────────┘  └──────────┘  └──────────┘  └──────────┘   │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                  Core Processing Layer                       │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │   Markdown   │  │  Navigation  │  │   Version    │      │
│  │   Parser     │  │  Generator   │  │   Manager    │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                      Data Layer                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │   Markdown   │  │    Search    │  │     Git      │      │
│  │    Files     │  │    Index     │  │    Metadata  │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘

File Structure

emberdocs/
├── packages/
│   ├── core/                    # Main Next.js application
│   │   ├── src/
│   │   │   ├── app/            # Next.js app router
│   │   │   ├── components/     # React components
│   │   │   │   ├── DocLayout.tsx
│   │   │   │   ├── Sidebar.tsx
│   │   │   │   ├── Search.tsx
│   │   │   │   ├── ContentArea.tsx
│   │   │   │   ├── VersionSwitcher.tsx
│   │   │   │   └── CodeBlock.tsx
│   │   │   ├── lib/            # Core utilities
│   │   │   │   ├── markdown.ts
│   │   │   │   ├── navigation.ts
│   │   │   │   ├── search.ts
│   │   │   │   ├── versioning.ts
│   │   │   │   └── config.ts
│   │   │   ├── styles/         # Global styles
│   │   │   └── types/          # TypeScript types
│   │   ├── public/
│   │   │   └── docs/           # User's markdown files
│   │   ├── emberdocs.config.js
│   │   └── package.json
│   │
│   ├── cli/                     # CLI tool
│   │   ├── src/
│   │   │   ├── commands/
│   │   │   │   ├── init.ts     # Initialize new project
│   │   │   │   ├── build.ts    # Build static site
│   │   │   │   ├── dev.ts      # Development server
│   │   │   │   └── deploy.ts   # Deploy helpers
│   │   │   ├── templates/      # Project scaffolding
│   │   │   └── utils/
│   │   └── package.json
│   │
├── examples/
│   ├── standalone/              # Full standalone site
│   └── nextjs-integration/      # Next.js integration
│
├── docs/                        # EmberDocs own documentation
│   ├── index.md
│   ├── getting-started.md
│   ├── configuration.md
│   ├── theming.md
│   └── api/
│
└── tools/
    ├── migrate/                 # Migration scripts
    └── scripts/                 # Build/deploy scripts

Component Specifications

1. Markdown Parser (lib/markdown.ts)

Purpose: Transform markdown files into structured HTML with metadata.

Features:

  • Frontmatter parsing (title, description, category, etc.)
  • GitHub Flavored Markdown support
  • Syntax highlighting with Shiki
  • Auto-linking headings
  • Table of contents generation
  • Code block enhancements (copy button, filename, line numbers)

API:

interface ParsedMarkdown {
  content: string;           // Rendered HTML
  metadata: MarkdownMetadata;
  toc: TableOfContents[];
  excerpt: string;           // First paragraph
  readingTime: number;       // Estimated minutes
}

interface MarkdownMetadata {
  title: string;
  description?: string;
  category?: string;
  tags?: string[];
  date?: string;
  author?: string;
  version?: string;
}

async function parseMarkdown(
  filepath: string,
  options?: ParseOptions
): Promise<ParsedMarkdown>

Implementation Details:

import { unified } from 'unified';
import remarkParse from 'remark-parse';
import remarkGfm from 'remark-gfm';
import remarkRehype from 'remark-rehype';
import rehypeShiki from '@shikijs/rehype';
import rehypeStringify from 'rehype-stringify';
import matter from 'gray-matter';

export async function parseMarkdown(
  filepath: string,
  options: ParseOptions = {}
): Promise<ParsedMarkdown> {
  const rawContent = await fs.readFile(filepath, 'utf-8');
  const { data: metadata, content } = matter(rawContent);

  const processor = unified()
    .use(remarkParse)
    .use(remarkGfm)
    .use(remarkRehype)
    .use(rehypeShiki, {
      themes: {
        light: 'github-light',
        dark: 'github-dark',
      },
    })
    .use(rehypeStringify);

  const result = await processor.process(content);

  return {
    content: String(result),
    metadata: metadata as MarkdownMetadata,
    toc: extractTOC(result),
    excerpt: extractExcerpt(content),
    readingTime: calculateReadingTime(content),
  };
}

2. Navigation Generator (lib/navigation.ts)

Purpose: Auto-generate sidebar navigation from folder structure.

Features:

  • Recursive directory scanning
  • Alphabetical or custom ordering
  • Collapsible sections
  • Active page highlighting
  • Breadcrumb generation

API:

interface NavItem {
  title: string;
  path: string;
  children?: NavItem[];
  icon?: string;
  order?: number;
}

interface NavigationStructure {
  items: NavItem[];
  breadcrumbs: NavItem[];
  prev?: NavItem;
  next?: NavItem;
}

async function generateNavigation(
  docsPath: string,
  currentPath?: string
): Promise<NavigationStructure>

Folder Structure Parsing:

docs/
├── index.md                    → Home
├── getting-started.md          → Getting Started
├── 01-installation.md          → Installation (ordered)
├── 02-configuration.md         → Configuration (ordered)
├── api/
│   ├── index.md               → API / Overview
│   ├── authentication.md      → API / Authentication
│   └── endpoints.md           → API / Endpoints
└── guides/
    ├── quickstart.md          → Guides / Quickstart
    └── advanced.md            → Guides / Advanced

Ordering Rules:

  1. Items with order field in frontmatter are sorted numerically (lower numbers first)
  2. Items without order field are sorted alphabetically after ordered items
  3. Folders always appear before documents in the same parent
  4. Within each group (ordered vs. unordered), the respective sorting applies

Custom Ordering via Frontmatter:

Add an order field to your markdown file's frontmatter:

---
title: Getting Started
order: 1
---

# Getting Started
---
title: Installation
order: 2
---

# Installation

Files without an order field will be sorted alphabetically after all ordered items. This allows you to control sidebar order without renaming files.

3. Search System (lib/search.ts)

Purpose: Provide instant, client-side search with optional AI enhancement.

Default: FlexSearch (Client-Side)

interface SearchIndex {
  id: string;
  title: string;
  content: string;
  path: string;
  category?: string;
}

interface SearchResult {
  item: SearchIndex;
  score: number;
  matches: string[];
}

class SearchClient {
  private index: FlexSearch.Index;

  async buildIndex(docs: ParsedMarkdown[]): Promise<void>
  async search(query: string, limit?: number): Promise<SearchResult[]>
}

Build Process:

// Build search index at compile time
export async function buildSearchIndex(
  docsPath: string
): Promise<SearchIndex[]> {
  const files = await glob(`${docsPath}/**/*.md`);
  const docs = await Promise.all(
    files.map(async (file) => {
      const parsed = await parseMarkdown(file);
      return {
        id: fileToPath(file),
        title: parsed.metadata.title || fileToTitle(file),
        content: stripHTML(parsed.content),
        path: fileToPath(file),
        category: parsed.metadata.category,
      };
    })
  );

  // Write to public/search-index.json
  await fs.writeFile(
    'public/search-index.json',
    JSON.stringify(docs)
  );

  return docs;
}

Optional: AI Semantic Search

interface AISearchConfig {
  provider: 'openai' | 'anthropic';
  apiKey: string;
  model: string;
  embeddingDimensions?: number;
}

class AISearchClient {
  async buildEmbeddings(docs: ParsedMarkdown[]): Promise<void>
  async semanticSearch(query: string): Promise<SearchResult[]>
}

Embedding Generation:

// Only runs when AI enabled in config
export async function buildAIIndex(
  docs: SearchIndex[],
  config: AISearchConfig
): Promise<void> {
  const embeddings = await Promise.all(
    docs.map(async (doc) => {
      const embedding = await generateEmbedding(
        `${doc.title}\n${doc.content}`,
        config
      );
      return {
        id: doc.id,
        embedding,
      };
    })
  );

  // Store embeddings for runtime search
  await fs.writeFile(
    'public/embeddings.json',
    JSON.stringify(embeddings)
  );
}

async function generateEmbedding(
  text: string,
  config: AISearchConfig
): Promise<number[]> {
  if (config.provider === 'openai') {
    const response = await fetch(
      'https://api.openai.com/v1/embeddings',
      {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${config.apiKey}`,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          model: 'text-embedding-3-small',
          input: text,
        }),
      }
    );
    const data = await response.json();
    return data.data[0].embedding;
  }
  // Similar for Anthropic
}

4. Version Manager (lib/versioning.ts)

Purpose: Auto-detect versions from git tags and manage version switching.

Features:

  • Git tag detection (semver pattern)
  • Version comparison
  • Changelog generation
  • Version switcher UI data

API:

interface Version {
  tag: string;
  date: string;
  isLatest: boolean;
  docsPath: string;
}

interface VersionComparison {
  from: string;
  to: string;
  added: string[];
  modified: string[];
  removed: string[];
}

async function detectVersions(): Promise<Version[]>
async function compareVersions(
  from: string,
  to: string
): Promise<VersionComparison>

Git Tag Detection:

import { execSync } from 'child_process';

export async function detectVersions(): Promise<Version[]> {
  try {
    // Get all tags matching semver pattern
    const tags = execSync('git tag -l "v*.*.*"', {
      encoding: 'utf-8',
    })
      .trim()
      .split('\n')
      .filter(Boolean);

    const versions = await Promise.all(
      tags.map(async (tag) => {
        const date = execSync(`git log -1 --format=%ai ${tag}`, {
          encoding: 'utf-8',
        }).trim();

        return {
          tag,
          date,
          isLatest: false,
          docsPath: `/versions/${tag}`,
        };
      })
    );

    // Sort by semver
    versions.sort((a, b) =>
      semver.rcompare(a.tag, b.tag)
    );

    // Mark latest
    if (versions.length > 0) {
      versions[0].isLatest = true;
    }

    return versions;
  } catch (error) {
    // No git or no tags
    return [];
  }
}

Build Multi-Version Docs:

export async function buildVersionedDocs(
  versions: Version[]
): Promise<void> {
  for (const version of versions) {
    // Checkout tag
    execSync(`git checkout ${version.tag}`);

    // Build docs for this version
    await buildDocs({
      outputDir: `public/versions/${version.tag}`,
    });
  }

  // Return to main branch
  execSync('git checkout main');

  // Create symlink for latest
  await fs.symlink(
    `versions/${versions[0].tag}`,
    'public/latest'
  );
}

5. Configuration System (lib/config.ts)

Purpose: Load and validate user configuration.

Config File Schema:

interface EmberDocsConfig {
  // Site metadata
  site: {
    title: string;
    description?: string;
    url?: string;
    logo?: string;
  };

  // Content
  docs: {
    path: string;              // Default: './docs'
    exclude?: string[];        // Files/folders to ignore
  };

  // Theming
  theme: {
    colors?: {
      primary?: string;
      background?: string;
      text?: string;
      code?: string;
    };
    fonts?: {
      sans?: string;
      mono?: string;
    };
    borderRadius?: string;
  };

  // Search
  search: {
    provider: 'flexsearch' | 'ai';
    aiConfig?: {
      provider: 'openai' | 'anthropic';
      apiKey: string;
      model: string;
    };
  };

  // Versioning
  versions?: {
    strategy: 'git-tags' | 'manual' | 'folders';
    versions?: string[];       // For manual strategy
    changelog?: {
      enabled: boolean;
      compareStrategy: 'git-diff' | 'manual';
    };
  };

  // Analytics
  analytics?: {
    firefly?: {
      siteId: string;
    };
  };

  // Navigation
  navigation?: {
    customOrder?: NavItem[];
    showBreadcrumbs?: boolean;
    collapsible?: boolean;
  };
}

Default Config:

// emberdocs.config.js
module.exports = {
  site: {
    title: 'Documentation',
  },
  docs: {
    path: './docs',
  },
  theme: {
    colors: {
      primary: '#8B5CF6',
    },
    fonts: {
      sans: 'Inter, system-ui, sans-serif',
      mono: 'JetBrains Mono, monospace',
    },
  },
  search: {
    provider: 'flexsearch',
  },
  versions: {
    strategy: 'git-tags',
    changelog: {
      enabled: true,
      compareStrategy: 'git-diff',
    },
  },
};

UI Components

DocLayout Component

Purpose: Main layout wrapper with sidebar, content area, and search.

interface DocLayoutProps {
  navigation: NavigationStructure;
  currentPath: string;
  children: React.ReactNode;
  config: EmberDocsConfig;
}

export function DocLayout({
  navigation,
  currentPath,
  children,
  config,
}: DocLayoutProps) {
  return (
    <div className="ed-layout">
      <header className="ed-header">
        <Logo config={config} />
        <Search />
        <VersionSwitcher />
        <ThemeToggle />
      </header>

      <div className="ed-main">
        <Sidebar
          navigation={navigation}
          currentPath={currentPath}
        />

        <main className="ed-content">
          {children}
        </main>

        <aside className="ed-toc">
          <TableOfContents />
        </aside>
      </div>
    </div>
  );
}

Search Component

Purpose: Instant search with keyboard shortcuts and smart results.

interface SearchProps {
  searchIndex: SearchIndex[];
  onSelect?: (result: SearchResult) => void;
}

export function Search({ searchIndex, onSelect }: SearchProps) {
  const [query, setQuery] = useState('');
  const [results, setResults] = useState<SearchResult[]>([]);
  const [isOpen, setIsOpen] = useState(false);

  // Keyboard shortcut: Cmd+K or Ctrl+K
  useEffect(() => {
    const handleKeyDown = (e: KeyboardEvent) => {
      if ((e.metaKey || e.ctrlKey) && e.key === 'k') {
        e.preventDefault();
        setIsOpen(true);
      }
    };
    window.addEventListener('keydown', handleKeyDown);
    return () => window.removeEventListener('keydown', handleKeyDown);
  }, []);

  const handleSearch = async (q: string) => {
    setQuery(q);
    if (q.length < 2) {
      setResults([]);
      return;
    }

    const searchResults = await searchClient.search(q, 10);
    setResults(searchResults);
  };

  return (
    <div className="ed-search">
      <button
        onClick={() => setIsOpen(true)}
        className="ed-search-trigger"
      >
        <SearchIcon />
        <span>Search...</span>
        <kbd>⌘K</kbd>
      </button>

      {isOpen && (
        <SearchModal
          query={query}
          results={results}
          onQueryChange={handleSearch}
          onClose={() => setIsOpen(false)}
          onSelect={onSelect}
        />
      )}
    </div>
  );
}

CodeBlock Component

Purpose: Enhanced code blocks with copy, filename, and line numbers.

interface CodeBlockProps {
  code: string;
  language: string;
  filename?: string;
  showLineNumbers?: boolean;
  highlightLines?: number[];
}

export function CodeBlock({
  code,
  language,
  filename,
  showLineNumbers = false,
  highlightLines = [],
}: CodeBlockProps) {
  const [copied, setCopied] = useState(false);

  const handleCopy = async () => {
    await navigator.clipboard.writeText(code);
    setCopied(true);
    setTimeout(() => setCopied(false), 2000);
  };

  return (
    <div className="ed-code-block">
      {filename && (
        <div className="ed-code-header">
          <span className="ed-code-filename">{filename}</span>
          <button
            onClick={handleCopy}
            className="ed-code-copy"
          >
            {copied ? <CheckIcon /> : <CopyIcon />}
          </button>
        </div>
      )}

      <pre className={`language-${language}`}>
        <code>{code}</code>
      </pre>
    </div>
  );
}

CLI Tool

Status: Planned for Phase 02 / Future Version (see dev-docs/planning/cli_tool_implementation.md)

The CLI tool is not yet implemented. Users currently need to clone the repository and set up manually. The CLI functionality described below is planned for future releases.

Commands

init - Initialize New Project

npx emberdocs init [directory]

Options:
  --minimal         Create minimal setup (no examples)
  --integrate       Add to existing Next.js project
  --template        Use specific template

Implementation:

export async function init(
  directory: string,
  options: InitOptions
): Promise<void> {
  const targetDir = path.resolve(directory);

  if (options.integrate) {
    // Add to existing project
    await addToExistingProject(targetDir);
    return;
  }

  // Create new project
  await createDirectory(targetDir);

  if (options.minimal) {
    await scaffoldMinimal(targetDir);
  } else {
    await scaffoldFull(targetDir);
  }

  await installDependencies(targetDir);

  console.log(`
✓ EmberDocs initialized in ${directory}

Get started:
  cd ${directory}
  npm run dev

Then open http://localhost:3000
  `);
}

build - Build Static Site

npx emberdocs build

Options:
  --output <dir>    Output directory (default: out)
  --static          Static export only

dev - Development Server

npx emberdocs dev

Options:
  --port <number>   Port number (default: 3000)
  --host <string>   Host address

deploy - Deploy Helpers

npx emberdocs deploy <platform>

Platforms:
  vercel            Deploy to Vercel
  netlify           Deploy to Netlify
  github-pages      Deploy to GitHub Pages

Deployment Configurations

Static Export (GitHub Pages, Netlify)

// next.config.js
module.exports = {
  output: 'export',
  images: {
    unoptimized: true,
  },
  trailingSlash: true,
};

Vercel Edge

// vercel.json
{
  "buildCommand": "npx emberdocs build",
  "framework": "nextjs",
  "regions": ["iad1", "sfo1", "cdg1"],
  "functions": {
    "app/**/*": {
      "runtime": "edge"
    }
  }
}

Docker

# Dockerfile
FROM node:20-alpine AS builder

WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS runner
WORKDIR /app

ENV NODE_ENV production

COPY --from=builder /app/public ./public
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static

EXPOSE 3000
CMD ["node", "server.js"]

Performance Targets

Global Performance:

  • Time to First Byte (TTFB): < 50ms (edge deployment)
  • First Contentful Paint (FCP): < 500ms
  • Largest Contentful Paint (LCP): < 1s
  • Time to Interactive (TTI): < 1.5s

Search Performance:

  • Index build time: < 5s for 1000 pages
  • Search latency: < 50ms for client-side
  • AI search latency: < 500ms with caching

Build Performance:

  • 100 pages: < 10s
  • 1000 pages: < 60s
  • Incremental builds: < 2s per changed file

Asset Sizes:

  • Initial bundle: < 100KB (gzipped)
  • Search index: < 1KB per page
  • AI embeddings: ~2KB per page (opt-in)

Security Considerations

Content Security:

  • Sanitize markdown output (XSS prevention)
  • CSP headers for static assets
  • No inline scripts in production

API Security:

  • Rate limiting on search endpoints
  • API key validation for AI features

User Privacy:

  • No tracking by default
  • Analytics opt-in only
  • GDPR-compliant data handling
  • Client-side search (no query logging)

Browser Support

Modern Browsers:

  • Chrome/Edge 90+
  • Firefox 88+
  • Safari 14+
  • Mobile Safari 14+
  • Chrome Android 90+

Graceful Degradation:

  • JavaScript disabled: Static HTML navigation works
  • No CSS Grid: Fallback to flexbox
  • Old browsers: Basic styled content accessible

Accessibility Standards

WCAG 2.1 Level AA Compliance:

  • Semantic HTML structure
  • Keyboard navigation throughout
  • Screen reader optimized
  • Color contrast ratios > 4.5:1
  • Focus indicators visible
  • ARIA labels where needed
  • Skip navigation links

Keyboard Shortcuts:

  • Cmd/Ctrl + K: Open search
  • /: Focus search
  • Esc: Close modals
  • Arrow keys: Navigate results
  • Tab: Navigate sections

Testing Strategy

Unit Tests:

  • Markdown parser
  • Navigation generator
  • Search algorithms
  • Version detection

Integration Tests:

  • Build process
  • Search indexing
  • Version switching
  • Component interactions

E2E Tests:

  • Full user workflows
  • Cross-browser testing
  • Mobile responsive
  • Accessibility audit

Performance Tests:

  • Lighthouse CI
  • Bundle size monitoring
  • Search performance
  • Build time tracking

Migration Support

From Docusaurus:

npx emberdocs migrate --from docusaurus

# Converts:
# - docusaurus.config.js → emberdocs.config.js
# - sidebars.js → auto-generated
# - Moves docs/ to docs/

From VitePress:

npx emberdocs migrate --from vitepress

# Converts:
# - .vitepress/config.js → emberdocs.config.js
# - Adjusts markdown frontmatter
# - Converts custom components

From GitBook:

npx emberdocs migrate --from gitbook

# Converts:
# - SUMMARY.md → navigation structure
# - .gitbook.yaml → emberdocs.config.js

This specification provides the foundation for v1.0 development. Additional detailed specifications for database schema, brand guidelines, and development roadmap follow in separate documents.