Skip to content

Commit 7dc7f7d

Browse files
committed
docs: add scoring docs
1 parent 3451929 commit 7dc7f7d

File tree

1 file changed

+288
-0
lines changed

1 file changed

+288
-0
lines changed

packages/cli/docs/scoring.md

Lines changed: 288 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,288 @@
1+
# Scoring
2+
3+
**Scoring strategies** are mathematical approaches used to convert raw metrics into normalized scores
4+
(typically 0-1 range) that provide meaningful feedback about performance against targets or budgets. These
5+
strategies are particularly valuable in software development contexts where you need to track and visualize how
6+
well various metrics (like bundle sizes, test coverage, API latency, or build times) are performing relative to
7+
established thresholds or baselines. Each strategy offers different characteristics - some provide linear
8+
relationships between the metric and score, while others offer more nuanced behaviors like soft caps or
9+
logarithmic scaling.
10+
11+
## Value Scoring Strategies
12+
13+
### Overview Table
14+
15+
| **Strategy Name** | **Formula** | **Description** |
16+
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
17+
| **Percent Used** | $$\text{score}= \max\!\left(0,\,1 - \frac{S}{M}\right)$$ | Linearly falls from 1→0 as **S** goes 0→**M**, then clamps at 0. |
18+
| **Linear Overshoot** | $$\text{score}= \begin{cases}1,&S\le M\\[4pt]1 - \dfrac{S - M}{M},&S>M\end{cases}$$ | Holds 1 until **S**=**M**, then falls linearly to 0 by **S**=2·**M**. |
19+
| **Relative Baseline** | $$\text{score}= \mathrm{clamp}_{0..1}\!\Bigl(0.5 + \frac{\text{baseline} - S}{2\,\text{baseline}}\Bigr)$$ | Centers at 0.5 when **S** matches baseline; >0.5 improvement, <0.5 regression. |
20+
| **Sigmoid Soft-Cap** | $$\text{score}= \frac{1}{1 + e^{\,k\,(S - M)}}$$ | Remains near 1 before **M**, then gently decays, steep after threshold. |
21+
| **Logarithmic Decay** | $$\text{score}= \max\!\Bigl(0,\,1 - \frac{\log_{10}(1+S)}{\log_{10}(1 + M\,k)}\Bigr)$$ | Penalises large **S** sharply, small overruns have mild impact. |
22+
| **Tiered Grading** | $$\text{score}= \begin{cases}1,&P\ge0.9\\[2pt]0.75,&P\ge0.75\\[2pt]0.5,&P\ge0.5\\[2pt]0,&\text{otherwise}\end{cases},\;P=\frac{S}{M}$$ | Maps **S/M** into A/B/C/F bands for clear grade visualization. |
23+
24+
---
25+
26+
### Percent Used Details
27+
28+
Directly reflects remaining budget fraction.
29+
30+
**Pros:**
31+
32+
- Intuitive (% headroom).
33+
- Simple to compute.
34+
**Cons:**
35+
- Flatlines at 0 once budget is exceeded.
36+
37+
**Implementation:**
38+
39+
```ts
40+
function percentUsed(current: number, max: number): number {
41+
return Math.max(0, 1 - current / max);
42+
}
43+
```
44+
45+
**When to use it:**
46+
Ideal for any strict size or time budget (bundle size, build time) where you care about how much headroom remains. Simple dashboards and alerts benefit from a direct “% remaining” metric.
47+
48+
**Examples:**
49+
50+
- **Bundle Stats:** Track JS bundle headroom under a 500 KB limit.
51+
- **Test Coverage:** % lines covered vs 80 % threshold.
52+
- **Page Load (LCP):** Remaining ms under a 2 s budget.
53+
54+
**Chart:**
55+
56+
```mermaid
57+
xychart-beta
58+
title "Percent Used (M=10)"
59+
x-axis [0,1,2,3,4,5,6,7,8,9,10]
60+
y-axis "Score" 0 --> 1
61+
line Percent [1,0.9,0.8,0.7,0.6,0.5,0.4,0.3,0.2,0.1,0]
62+
```
63+
64+
---
65+
66+
### Linear Overshoot Details
67+
68+
Penalty grows linearly with overshoot.
69+
70+
**Pros:**
71+
72+
- Proportional and predictable.
73+
- Easy to explain to stakeholders.
74+
**Cons:**
75+
- Can go negative if not clamped.
76+
77+
**Implementation:**
78+
79+
```ts
80+
function linearOvershoot(current: number, max: number): number {
81+
if (current <= max) {
82+
return 1;
83+
}
84+
return Math.max(0, 1 - (current - max) / max);
85+
}
86+
```
87+
88+
**When to use it:**
89+
When you want to maintain proportional visibility into how far a metric has exceeded its limit (e.g., memory usage, API latency). Works well if you clamp at 0 to avoid negatives.
90+
91+
**Examples:**
92+
93+
- **API Latency:** Overshoot above 200 ms threshold.
94+
- **Memory Usage:** RAM above configured max.
95+
- **CSS Sizes:** Style sheets above a 50 KB cap.
96+
97+
**Chart:**
98+
99+
```mermaid
100+
xychart-beta
101+
title "Linear Overshoot (M=10)"
102+
x-axis [0,1,2,3,4,5,6,7,8,9,10]
103+
y-axis "Score" 0 --> 1
104+
line Linear [1,0.9,0.8,0.7,0.6,0.5,0.4,0.3,0.2,0.1,0]
105+
```
106+
107+
---
108+
109+
### Relative Baseline Details
110+
111+
Compares current value to a historical baseline.
112+
113+
**Pros:**
114+
115+
- Symmetric: 0.5 = “no change.”
116+
- Highlights improvements/regressions.
117+
**Cons:**
118+
- Requires reliable baseline data.
119+
120+
**Implementation:**
121+
122+
```ts
123+
function relativeBaseline(current: number, baseline: number): number {
124+
const score = 0.5 + (baseline - current) / (2 * baseline);
125+
return Math.max(0, Math.min(1, score));
126+
}
127+
```
128+
129+
**When to use it:**
130+
Perfect for tracking trends across releases (bundle size, test performance, code complexity). Shifts focus from absolute budgets to continuous improvement.
131+
132+
**Examples:**
133+
134+
- **Bundle Stats:** Compare today’s bundle size vs last release.
135+
- **Test Duration:** Measure speedup/regression of test suite.
136+
- **Doc Coverage:** Track improvements in storybook coverage.
137+
138+
**Chart:**
139+
140+
```mermaid
141+
xychart-beta
142+
title "Relative Baseline (baseline=10)"
143+
x-axis [0,1,2,3,4,5,6,7,8,9,10]
144+
y-axis "Score" 0 --> 1
145+
line Delta [1,0.95,0.9,0.85,0.8,0.75,0.7,0.65,0.6,0.55,0.5]
146+
```
147+
148+
---
149+
150+
### Sigmoid Soft-Cap Details
151+
152+
Soft threshold with tunable steepness.
153+
154+
**Pros:**
155+
156+
- Gentle near the cap; harsh after.
157+
- Never negative or >1.
158+
**Cons:**
159+
- Requires selecting a constant **k**.
160+
- Less intuitive without plotting.
161+
162+
**Implementation:**
163+
164+
```ts
165+
function sigmoidSoftCap(current: number, max: number, k: number = 0.5): number {
166+
return 1 / (1 + Math.exp(k * (current - max)));
167+
}
168+
```
169+
170+
**Parameter Details:**
171+
172+
- **k** (steepness factor): Controls how quickly the score drops after exceeding the threshold
173+
- `k = 0.1`: Very gentle decline, scores remain high even with significant overshoot
174+
- `k = 0.5`: Moderate decline (default), balanced between tolerance and penalty
175+
- `k = 1.0`: Steep decline, harsh penalty for any overshoot
176+
- `k = 2.0`: Very steep decline, almost binary behavior near the threshold
177+
178+
**When to use it:**
179+
Use when minor breaches are acceptable but you want strong pressure against large overruns (e.g. accessibility violations, deprecation warnings).
180+
181+
**Examples:**
182+
183+
- **ESLint Errors:** Small numbers of lint errors tolerated.
184+
- **TS Diagnostics:** Minor type mismatches vs critical errors.
185+
- **Security Audits:** Few low-severe alerts OK; many not.
186+
187+
**Chart:**
188+
189+
```mermaid
190+
xychart-beta
191+
title "Sigmoid Soft-Cap (M=5, k=0.5)"
192+
x-axis [0,1,2,3,4,5,6,7,8,9,10]
193+
y-axis "Score" 0 --> 1
194+
line Sigmoid [0.92,0.88,0.82,0.73,0.62,0.50,0.38,0.27,0.18,0.12,0.08]
195+
```
196+
197+
---
198+
199+
### Logarithmic Decay Details
200+
201+
Log-scale penalty for wide-range data.
202+
203+
**Pros:**
204+
205+
- Highlights massive overruns.
206+
- Small ones barely noticed.
207+
**Cons:**
208+
- Formula less intuitive.
209+
210+
**Implementation:**
211+
212+
```ts
213+
function logarithmicDecay(current: number, max: number, k: number = 2): number {
214+
const score = 1 - Math.log10(1 + current) / Math.log10(1 + max * k);
215+
return Math.max(0, score);
216+
}
217+
```
218+
219+
**Parameter Details:**
220+
221+
- **k** (scaling factor): Controls the logarithmic compression range
222+
- `k = 1`: Minimal compression, behaves more like linear decay
223+
- `k = 2`: Moderate compression (default), good balance for most metrics
224+
- `k = 5`: Strong compression, very tolerant of small overruns but penalizes large ones
225+
- `k = 10`: Maximum compression, almost flat until very large overruns
226+
227+
**When to use it:**
228+
Ideal for metrics spanning orders of magnitude (e.g. network RTT, bundle gzipped vs uncompressed sizes).
229+
230+
**Examples:**
231+
232+
- **CWV (FID):** First Input Delay distribution.
233+
- **Bundle Stats:** Large vs small chunks on log scale.
234+
- **API Throttling:** High‐tail latency events.
235+
236+
**Chart:**
237+
238+
```mermaid
239+
xychart-beta
240+
title "Logarithmic Decay (M=5, k=2)"
241+
x-axis [0,1,2,3,4,5,6,7,8,9,10]
242+
y-axis "Score" 0 --> 1
243+
line Log [1,0.71,0.54,0.42,0.33,0.25,0.19,0.13,0.08,0.04,0]
244+
```
245+
246+
---
247+
248+
### Tiered Grading Details
249+
250+
Discrete grade bands (A/B/C/F).
251+
252+
**Pros:**
253+
254+
- Very readable.
255+
- No continuous math needed for viewers.
256+
**Cons:**
257+
- Abrupt jumps hide nuance.
258+
259+
**Implementation:**
260+
261+
```ts
262+
function tieredGrading(current: number, max: number): number {
263+
const ratio = current / max;
264+
if (ratio >= 0.9) return 1;
265+
if (ratio >= 0.75) return 0.75;
266+
if (ratio >= 0.5) return 0.5;
267+
return 0;
268+
}
269+
```
270+
271+
**When to use it:**
272+
Best for executive dashboards or compliance checks where letter‐grade is preferred over percentages.
273+
274+
**Examples:**
275+
276+
- **Test Coverage:** A: ≥90 %, B: 75 – 89 %, C: 50 – 74 %.
277+
- **Doc Coverage:** Storybook or JSDoc rates.
278+
- **Security Score:** CVSS banding.
279+
280+
**Chart:**
281+
282+
```mermaid
283+
xychart-beta
284+
title "Tiered Grading"
285+
x-axis [0,1,2,3,4,5,6,7,8,9,10]
286+
y-axis "Score" 0 --> 1
287+
line Grade [0,0,0,0,0.5,0.5,0.5,0.75,1,1,1]
288+
```

0 commit comments

Comments
 (0)