Add full-featured example with person detection, AI prompts, and file upload

README.md

@@ -2,13 +2,14 @@
 
 Virtual try-on lets shoppers see how clothing looks on them in real time, using just a webcam. Decart's `lucy_2_rt` model takes a live camera feed and a garment reference image, then streams back video of the person wearing that garment - all through a WebRTC connection with no server-side rendering. This repo provides drop-in examples so you can add try-on to your own app in minutes.
 
-Three production-ready Next.js examples that show how to integrate Decart's realtime virtual try-on. Each example is self-contained and runs independently.
+Four production-ready Next.js examples that show how to integrate Decart's realtime virtual try-on. Each example is self-contained and runs independently.
 
 | Example | Use case | Integration style |
 |---------|----------|-------------------|
 | [**E-commerce**](examples/ecommerce/) | "Try it on" button on product pages | Simple - hardcoded prompts, modal overlay |
 | [**Standalone**](examples/standalone/) | Dedicated try-on experience | Advanced - AI-generated prompts via LLM |
 | [**Person Detection**](examples/person-detection/) | Kiosks and unattended displays | Auto-connect - only uses credits when someone is in frame |
+| [**Full-Featured**](examples/full-featured/) | Complete try-on experience | All features - person detection, AI prompts, file upload |
 
 ---
 
@@ -141,7 +142,7 @@ This is a good fallback, but for the best results we still recommend writing det
 
 ### Generating prompts with an LLM
 
-For garments where you don't have a pre-written prompt (e.g. user-uploaded images), you can use any vision LLM to auto-generate one from the garment image. The [standalone example](examples/standalone/) demonstrates this using OpenAI's GPT-4o-mini, but any vision-capable LLM works.
+For garments where you don't have a pre-written prompt (e.g. user-uploaded images), you can use any vision LLM to auto-generate one from the garment image. The [standalone example](examples/standalone/) demonstrates this using OpenAI's GPT-4o-mini, but any vision-capable LLM works (Claude, Gemini, etc.). A built-in Decart solution is coming soon.
 
 The example sends both the garment image and a camera frame of the person to `/api/enhance-prompt`:
 
@@ -201,7 +202,7 @@ Your permanent `DECART_API_KEY` never leaves the server. The browser receives a
 | Variable | Required | Purpose |
 |----------|----------|---------|
 | `DECART_API_KEY` | Yes | Creates client tokens for realtime WebRTC connections |
-| `OPENAI_API_KEY` | Standalone only | Powers `/api/enhance-prompt` - auto-generates prompts from garment images |
+| `OPENAI_API_KEY` | Standalone / Full-Featured | Powers `/api/enhance-prompt` - auto-generates prompts from garment images. Can be swapped for any vision-capable LLM. |
 
 ---
 

examples/full-featured/.env.example

@@ -0,0 +1,2 @@
+DECART_API_KEY=sk_your_key_here
+OPENAI_API_KEY=sk_your_openai_key_here

examples/full-featured/README.md

@@ -0,0 +1,126 @@
+# Full-Featured Virtual Try-On
+
+> Combines all features: person detection, AI-generated prompts, file upload, and a product catalog. The most complete integration example.
+
+Auto-detects when a person is in the camera frame, generates try-on prompts with GPT-4o-mini, lets users upload their own garment images, and includes a product catalog sidebar — all in one app.
+
+
+---
+
+## Quick start
+
+### 1. Install dependencies
+
+```bash
+cd examples/full-featured
+npm install
+```
+
+### 2. Set your API keys
+
+```bash
+cp .env.example .env.local
+```
+
+Open `.env.local` and add both keys:
+
+```env
+DECART_API_KEY=sk_your_key_here
+OPENAI_API_KEY=sk_your_openai_key_here
+```
+
+> **Note:** This example requires both keys. The Decart key powers the realtime try-on, and the OpenAI key powers the prompt generation.
+
+### 3. Start the dev server
+
+```bash
+npm run dev
+```
+
+Open [http://localhost:3000](http://localhost:3000). Your camera will start and MediaPipe will begin scanning. Step in front of the camera to trigger a connection, then click a product or upload your own garment.
+
+---
+
+## How it works
+
+```
+Page loads
+  → Camera starts automatically (getUserMedia)
+    → MediaPipe PoseLandmarker loads (WASM + WebGL, runs in browser)
+      → Every 1s: detectForVideo() checks for body landmarks
+        → Person detected (landmarks found)
+          → Fetch client token from /api/tokens
+            → Connect to Decart's lucy_2_rt model (WebRTC)
+              → User clicks a product OR uploads a garment image
+                → Capture a frame from the camera
+                → Send garment image + person frame to /api/enhance-prompt
+                → GPT-4o-mini generates a try-on prompt
+                → setImage(garment, prompt) sends the garment to the model
+                  → AI video stream shows the user wearing the garment
+        → Person leaves (3 consecutive misses, ~3s)
+          → Disconnect from Decart (stops billing)
+            → Person returns → fresh token → reconnect → re-apply last garment
+```
+
+---
+
+## Features
+
+| Feature | Description |
+|---------|-------------|
+| **Person detection** | Auto-connects when someone is in frame, disconnects when they leave (saves credits) |
+| **AI-generated prompts** | Generates descriptive try-on prompts from the garment image + camera frame. This example uses GPT-4o-mini, but any vision-capable LLM works (Claude, Gemini, etc.). A built-in Decart solution is coming soon. |
+| **File upload** | Users can upload their own garment images and get AI-generated prompts automatically |
+| **Product catalog** | Sidebar with product grid, click to try on |
+| **Editable prompts** | View and edit the generated prompt, re-apply with one click |
+| **Auto-reconnect** | When a person leaves and returns, the last garment is automatically re-applied |
+
+---
+
+## Key files
+
+| File | Purpose |
+|------|---------|
+| `app/page.tsx` | Main orchestration — detection-driven connect/disconnect with AI prompt generation |
+| `components/CombinedView.tsx` | Video display with detection status badges, processing overlay, and prompt editor |
+| `components/ProductSidebar.tsx` | Product grid with file upload section |
+| `hooks/usePersonDetection.ts` | MediaPipe PoseLandmarker polling, exposes `personPresent` state |
+| `hooks/useDecartRealtime.ts` | Decart WebRTC connection management |
+| `hooks/useCamera.ts` | Camera access via getUserMedia |
+| `lib/enhance-prompt.ts` | Client-side helper for calling `/api/enhance-prompt` |
+| `app/api/enhance-prompt/route.ts` | Server-side GPT-4o-mini prompt generation |
+| `app/api/tokens/route.ts` | Server-side Decart client token creation |
+
+---
+
+## Customization
+
+### Tuning detection sensitivity
+
+In `hooks/usePersonDetection.ts`:
+
+- **`DETECTION_INTERVAL_MS`** (default: 1000) — how often to check for a person
+- **`MISS_THRESHOLD`** (default: 3) — consecutive missed detections before disconnecting
+
+### Add your own products
+
+Edit `lib/products.ts`. Each product just needs a name, image path, and price — prompts are generated automatically:
+
+```typescript
+{
+  name: "Striped Polo",
+  image: "/products/striped-polo.jpg",
+  price: 45,
+}
+```
+
+Place the garment image in `public/products/`.
+
+---
+
+## Environment variables
+
+| Variable | Required | Purpose |
+|----------|----------|---------|
+| `DECART_API_KEY` | Yes | Creates client tokens for realtime WebRTC connections |
+| `OPENAI_API_KEY` | Yes | Powers `/api/enhance-prompt` for auto-generating prompts. Can be swapped for any vision-capable LLM. |

examples/full-featured/app/api/enhance-prompt/route.ts

@@ -0,0 +1,118 @@
+import { NextRequest, NextResponse } from "next/server";
+
+const SYSTEM_PROMPT = `You write prompts for a virtual try-on model. You receive a reference clothing image, and optionally a camera frame showing the person.
+
+Follow these steps:
+
+Step 1 - Examine the person's camera frame (if provided):
+Identify what the person is currently wearing on their upper body, lower body, head, etc. Note the specific garment (e.g. "a plain white t-shirt", "dark blue jeans", "a grey hoodie").
+
+Step 2 - Examine the reference clothing image:
+Describe it with material, texture, pattern, fit, and colors. Be specific (e.g. "a red plaid flannel shirt with a relaxed fit" not just "a shirt").
+
+Step 3 - Choose the action:
+- If the person is ALREADY WEARING something in the same slot as the reference item (e.g. they wear a t-shirt and the reference is a blouse), use SUBSTITUTE:
+  "Substitute the [description of current clothing] with [description of reference clothing]"
+  Example: "Substitute the plain white t-shirt with a red plaid flannel shirt with a relaxed fit and chest pockets"
+
+- If the person is NOT wearing anything in that slot (e.g. no hat, no jacket over their shirt), use ADD:
+  "Add [description of reference clothing] to the person's [body part]"
+  Example: "Add a wide-brimmed natural straw hat with a chin strap to the person's head"
+
+Fallback: If no person frame is provided or the relevant body part is not visible, use "the current top" for upper-body items or "the current bottoms" for lower-body items.
+Example: "Substitute the current top with a navy cable-knit sweater with a crew neck"
+
+Keep the total prompt between 20-30 words. Include colors, textures, and patterns. Return only the final prompt, nothing else.`;
+
+interface ChatMessage {
+  type: string;
+  text?: string;
+  image_url?: { url: string; detail: string };
+}
+
+export async function POST(req: NextRequest) {
+  const apiKey = process.env.OPENAI_API_KEY;
+  if (!apiKey) {
+    return NextResponse.json(
+      { error: "OPENAI_API_KEY not set" },
+      { status: 500 }
+    );
+  }
+
+  try {
+    const formData = await req.formData();
+    const file = formData.get("image") as File | null;
+    if (!file) {
+      return NextResponse.json(
+        { error: "No image provided" },
+        { status: 400 }
+      );
+    }
+
+    const buffer = await file.arrayBuffer();
+    const base64 = Buffer.from(buffer).toString("base64");
+    const mimeType = file.type || "image/png";
+    const clothingDataUri = `data:${mimeType};base64,${base64}`;
+
+    const userContent: ChatMessage[] = [
+      {
+        type: "text",
+        text: "Generate a try-on prompt for this clothing item:",
+      },
+      {
+        type: "image_url",
+        image_url: { url: clothingDataUri, detail: "low" },
+      },
+    ];
+
+    const personFrame = formData.get("personFrame") as File | null;
+    if (personFrame) {
+      const personBuffer = await personFrame.arrayBuffer();
+      const personBase64 = Buffer.from(personBuffer).toString("base64");
+      const personMime = personFrame.type || "image/jpeg";
+      const personDataUri = `data:${personMime};base64,${personBase64}`;
+      userContent.push(
+        { type: "text", text: "Here is the person from the camera:" },
+        {
+          type: "image_url",
+          image_url: { url: personDataUri, detail: "low" },
+        }
+      );
+    }
+
+    const res = await fetch("https://api.openai.com/v1/chat/completions", {
+      method: "POST",
+      headers: {
+        Authorization: `Bearer ${apiKey}`,
+        "Content-Type": "application/json",
+      },
+      body: JSON.stringify({
+        model: "gpt-4o-mini",
+        max_tokens: 200,
+        messages: [
+          { role: "system", content: SYSTEM_PROMPT },
+          { role: "user", content: userContent },
+        ],
+      }),
+    });
+
+    if (!res.ok) {
+      const err = await res.text();
+      console.error("OpenAI API error:", err);
+      return NextResponse.json(
+        { error: "Failed to generate prompt" },
+        { status: 500 }
+      );
+    }
+
+    const data = await res.json();
+    const prompt = data.choices[0]?.message?.content?.trim() || "";
+    return NextResponse.json({ prompt });
+  } catch (error) {
+    console.error("Prompt generation failed:", error);
+    return NextResponse.json(
+      { error: "Prompt generation failed" },
+      { status: 500 }
+    );
+  }
+}

examples/full-featured/app/api/tokens/route.ts

@@ -0,0 +1,24 @@
+import { createDecartClient } from "@decartai/sdk";
+import { NextResponse } from "next/server";
+
+export async function POST() {
+  const apiKey = process.env.DECART_API_KEY;
+  if (!apiKey) {
+    return NextResponse.json(
+      { error: "DECART_API_KEY is not set" },
+      { status: 500 }
+    );
+  }
+
+  try {
+    const client = createDecartClient({ apiKey });
+    const token = await client.tokens.create();
+    return NextResponse.json(token);
+  } catch (error) {
+    console.error("Failed to create token:", error);
+    return NextResponse.json(
+      { error: "Failed to create token" },
+      { status: 500 }
+    );
+  }
+}

examples/full-featured/app/globals.css

@@ -0,0 +1,9 @@
+@import "tailwindcss";
+
+@theme inline {
+  --font-sans: system-ui, -apple-system, sans-serif;
+}
+
+button {
+  cursor: pointer;
+}

examples/full-featured/app/layout.tsx

@@ -0,0 +1,20 @@
+import type { Metadata } from "next";
+import "./globals.css";
+
+export const metadata: Metadata = {
+  title: "Virtual Try-On - Full Featured Example",
+  description:
+    "Combines person detection, AI-generated prompts, and file upload",
+};
+
+export default function RootLayout({
+  children,
+}: {
+  children: React.ReactNode;
+}) {
+  return (
+    <html lang="en">
+      <body className="antialiased bg-gray-950 text-white">{children}</body>
+    </html>
+  );
+}