Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Multi-language toolkit for ad-blocking and AdGuard DNS management with **identic
- `adguard-validate` (CLI tool)

### Compiler Equivalence
All four compilers (TypeScript, .NET, Python, Rust) wrap `@adguard/hostlist-compiler` and **must**:
All four compilers (TypeScript, .NET, Python, Rust) use `@jk-com/adblock-compiler` and **must**:
- Support JSON, YAML, TOML config formats (except PowerShell: JSON only)
- Count rules identically (exclude empty lines and `!`/`#` comments)
- Compute SHA-384 hash of output (96 hex chars)
Expand Down Expand Up @@ -402,7 +402,7 @@ All compilers return/output:
```

### Configuration Schema
Supports 3 formats (JSON/YAML/TOML), mirrors `@adguard/hostlist-compiler`:
Supports 3 formats (JSON/YAML/TOML), based on `@jk-com/adblock-compiler` schema:
```json
{
"output": "data/output/adguard_user_filter.txt",
Expand Down Expand Up @@ -493,11 +493,13 @@ cd src/rules-compiler-rust && cargo test
- **Auth**: Bearer token in `Authorization` header
- **Retry logic**: 3 attempts with exponential backoff (408, 429, 5xx)

### @adguard/hostlist-compiler
- **All compilers depend on this**: npm package must be globally installed
- **Installation**: `npm install -g @adguard/hostlist-compiler`
### @jk-com/adblock-compiler
- **All compilers depend on this**: JSR package
- **Installation**: `deno add @jk-com/adblock-compiler` or via JSR
- **Source**: https://github.com/jaypatrick/hostlistcompiler
- **Documentation**: https://jsr.io/@jk-com/adblock-compiler
- Provides 11 transformations (RemoveComments, Compress, Validate, etc.)
- Compilers wrap this, handling config parsing and result formatting
- Compilers use this, handling config parsing and result formatting

### Docker Development
- **Dockerfile**: `Dockerfile.warp` (multi-stage with .NET 8 SDK + Node 20 + PowerShell 7)
Expand Down
14 changes: 7 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ This repository is a comprehensive multi-language toolkit for ad-blocking, netwo
- **Linear Import Tool** (`src/linear/`) - TypeScript tool with Deno support

### Configuration Support
All compilers support JSON, YAML, and TOML configuration formats with full @adguard/hostlist-compiler compatibility.
All compilers support JSON, YAML, and TOML configuration formats with full @jk-com/adblock-compiler compatibility.

## Docker Development Environment

Expand Down Expand Up @@ -294,7 +294,7 @@ cargo test config:: # Tests in module
- `data/output/adguard_user_filter.txt` - Main tracked filter list consumed by AdGuard DNS

### Rules Compiler - TypeScript (`src/rules-compiler-typescript/`)
- TypeScript wrapper around @adguard/hostlist-compiler
- TypeScript compiler using @jk-com/adblock-compiler
- Deno 2.0+ runtime with npm compatibility
- Supports JSON, YAML, and TOML configuration formats
- **Library API** (`src/lib/`):
Expand Down Expand Up @@ -324,7 +324,7 @@ cargo test config:: # Tests in module
- Supports JSON, YAML, TOML via external tools (yq, Python)

### Rules Compiler - .NET (`src/rules-compiler-dotnet/`)
- .NET 10 library wrapping @adguard/hostlist-compiler
- .NET 10 library for filter compilation
- Supports JSON, YAML, and TOML configuration formats
- `RulesCompiler` - Core library with abstractions, models, and services
- `RulesCompiler.Console` - Spectre.Console interactive and CLI frontend
Expand All @@ -333,7 +333,7 @@ cargo test config:: # Tests in module
- Features: Configuration validation, verbose mode, dependency injection

### Rules Compiler - Python (`src/rules-compiler-python/`)
- Python 3.9+ package wrapping @adguard/hostlist-compiler
- Python 3.9+ package for filter compilation
- Supports JSON, YAML, and TOML configuration formats
- `rules_compiler/config.py` - Multi-format configuration reader
- `rules_compiler/compiler.py` - Core `RulesCompiler` class and `compile_rules()` function
Expand All @@ -343,7 +343,7 @@ cargo test config:: # Tests in module
- Tools: pytest, mypy, ruff

### Rules Compiler - Rust (`src/rules-compiler-rust/`)
- High-performance Rust library and CLI wrapping @adguard/hostlist-compiler
- High-performance Rust library and CLI for filter compilation
- Supports JSON, YAML, and TOML configuration formats
- `src/config.rs` - Configuration structs and parsing
- `src/compiler.rs` - `RulesCompiler` struct and `compile_rules()` function
Expand Down Expand Up @@ -400,7 +400,7 @@ cargo test config:: # Tests in module

## Configuration Schema

All compilers support the same @adguard/hostlist-compiler configuration schema:
All compilers support the same @jk-com/adblock-compiler configuration schema:

### Root-Level Properties
| Property | Type | Required | Description |
Expand Down Expand Up @@ -463,7 +463,7 @@ GitHub Actions workflows validate:
| PowerShell | 7+ | PowerShell scripts |
| Python | 3.9+ | Python compiler |
| Rust | 1.85+ | Rust compiler (install via rustup) |
| hostlist-compiler | Latest | All compilers (via Deno: `deno run npm:@adguard/hostlist-compiler`) |
| adblock-compiler | 0.6.0 | TypeScript compiler (via JSR: `deno add @jk-com/adblock-compiler`) |
| Docker | 24.0+ | Container development (optional but recommended) |

## Key File Locations
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ A comprehensive multi-language toolkit for ad-blocking, network protection, and
| **Rust** | Native binary | Cargo/Binary | Zero-runtime deps, LTO optimization |
| **PowerShell** | PowerShell 7+ | Module | Pipeline-friendly, Pester tests |

All compilers wrap [@adguard/hostlist-compiler](https://github.com/AdguardTeam/HostlistCompiler) with **built-in security validation** and support:
All compilers use [@jk-com/adblock-compiler](https://github.com/jaypatrick/hostlistcompiler) with **built-in security validation** and support:
- **All 11 transformations**: Deduplicate, Validate, RemoveComments, Compress, RemoveModifiers, etc.
- **Multi-format config**: JSON, YAML, and TOML configuration files
- **Source-specific settings**: Per-source transformations, inclusions, exclusions
Expand Down Expand Up @@ -216,7 +216,7 @@ curl -fsSL https://deno.land/install.sh | sh
irm https://deno.land/install.ps1 | iex
```

The `@adguard/hostlist-compiler` package is accessed via Deno's npm compatibility.
The `@jk-com/adblock-compiler` package is accessed via Deno's JSR integration.

### Clone and Setup

Expand Down Expand Up @@ -536,7 +536,7 @@ See [`data/archive/README.md`](data/archive/README.md) for detailed usage and re
┌─────────────────────────────────────────────────────┐
│ 4. Compile with @adguard/hostlist-compiler │
│ 4. Compile with @jk-com/adblock-compiler │
│ - Merge all sources │
│ - Apply transformations (dedupe, validate, etc) │
│ - Convert hosts format to adblock if needed │
Expand All @@ -562,7 +562,7 @@ See [`data/archive/README.md`](data/archive/README.md) for detailed usage and re

## Rules Compilers

All compilers wrap [@adguard/hostlist-compiler](https://github.com/AdguardTeam/HostlistCompiler) and support:
All compilers use [@jk-com/adblock-compiler](https://github.com/jaypatrick/hostlistcompiler) and support:

- **Multi-format config**: JSON, YAML, TOML
- **All 11 transformations**: Deduplicate, Validate, RemoveComments, Compress, etc.
Expand Down Expand Up @@ -744,7 +744,7 @@ println!("Compiled {} rules", result.rule_count);

**Location**: `src/shell/`

Cross-platform shell scripts that wrap `@adguard/hostlist-compiler` for simple automation and CI/CD pipelines.
Cross-platform shell scripts that use `@jk-com/adblock-compiler` for simple automation and CI/CD pipelines.

| Script | Platform | Shell | Features |
|--------|----------|-------|----------|
Expand Down
2 changes: 1 addition & 1 deletion data/input/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ data/input/
2. **Validation**: Lint and verify syntax of each file
3. **Hashing**: Compute SHA-384 hash for integrity verification
4. **Remote fetch** (if applicable): Download internet lists with hash verification
5. **Compilation**: Merge all sources using `@adguard/hostlist-compiler`
5. **Compilation**: Merge all sources using `@jk-com/adblock-compiler`
6. **Output**: Write final adblock-format list to `data/output/adguard_user_filter.txt`

## Security
Expand Down
2 changes: 1 addition & 1 deletion docs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

- `data/` contains the tracked filter list (`data/output/adguard_user_filter.txt`) and compiler configuration files.
- `src/` contains the multi-language toolchain:
- `src/rules-compiler-*` (TypeScript/Deno, .NET, Python, Rust, shell) compilers that wrap `@adguard/hostlist-compiler`.
- `src/rules-compiler-*` (TypeScript/Deno, .NET, Python, Rust, shell) compilers that use `@jk-com/adblock-compiler`.
- `src/adguard-api-dotnet/`, `src/adguard-api-typescript/`, and `src/adguard-api-rust/` SDKs + interactive clients for the AdGuard DNS API.
- `src/adguard-api-powershell/` PowerShell modules and Pester tests.
- `docs/` holds guides and reference documentation.
Expand Down
4 changes: 2 additions & 2 deletions docs/LINEAR_DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ export ADGUARD_ARCHIVE_RETENTION_DAYS=90

**Technology Stack:**
- TypeScript 5.4.5
- @adguard/hostlist-compiler v1.0.39
- @jk-com/adblock-compiler v0.6.0
- Deno test (testing)
- Deno 2.0+

Expand Down Expand Up @@ -284,7 +284,7 @@ ad-blocking/
| Technology | Version | Purpose |
|------------|---------|---------|
| TypeScript | 5.4.5 | Strongly-typed JavaScript |
| @adguard/hostlist-compiler | 1.0.39 | Core compilation engine |
| @jk-com/adblock-compiler | 0.6.0 | Core compilation engine |
| Deno | 2.0+ | TypeScript/JavaScript runtime |
| Deno test | built-in | Testing framework |

Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,6 @@ All URIs are relative to `https://api.adguard-dns.io`

- [AdGuard DNS](https://adguard-dns.io/)
- [AdGuard DNS API Documentation](https://api.adguard-dns.io/static/swagger/swagger.json)
- [@adguard/hostlist-compiler](https://github.com/AdguardTeam/HostlistCompiler)
- [@jk-com/adblock-compiler](https://github.com/jaypatrick/hostlistcompiler)
- [AdBlock Tester](https://adblock-tester.com/)
- [AdGuard Tester](https://d3ward.github.io/toolz/adblock.html)
7 changes: 4 additions & 3 deletions docs/RUNTIME_ENFORCEMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,16 @@
┌─────────────────────────────────────────┐
│ @adguard/hostlist-compiler │
│ @jk-com/adblock-compiler │
│ (JSR package) │
└─────────────────────────────────────────┘
```

## How It Works

### 1. Mandatory Wrapper Function

All compilers **MUST** use `compile_with_validation()` instead of calling `@adguard/hostlist-compiler` directly:
All compilers **MUST** use `compile_with_validation()` instead of calling `@jk-com/adblock-compiler` directly:

**❌ FORBIDDEN - Direct compilation bypass:**
```typescript
Expand Down Expand Up @@ -280,7 +281,7 @@ module.exports = {
rules: {
'no-restricted-imports': ['error', {
patterns: [{
group: ['@adguard/hostlist-compiler'],
group: ['@jk-com/adblock-compiler'],
message: 'Do not import hostlist-compiler directly. Use compile_with_validation() from @adguard/validation instead.'
}]
}]
Expand Down
2 changes: 1 addition & 1 deletion docs/RUST_WORKSPACE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Rust implementation of the AdGuard DNS API client.
- **adguard-api-cli**: Interactive CLI for AdGuard DNS API

### 3. **rules-compiler-rust** (`src/rules-compiler-rust/`)
Rust compiler for AdGuard filter rules using `@adguard/hostlist-compiler`.
Rust compiler for AdGuard filter rules using `@jk-com/adblock-compiler`.

- Library and CLI for compiling filter rules
- Supports JSON, YAML, and TOML configurations
Expand Down
2 changes: 1 addition & 1 deletion docs/VALIDATION_ENFORCEMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ async function compile(config: CompilerConfig): Promise<CompilerResult> {
}
}

// 3. Proceed with compilation using @adguard/hostlist-compiler
// 3. Proceed with compilation using @jk-com/adblock-compiler
const output = await hostlistCompiler.compile(config);

// 4. MANDATORY: Handle file conflicts
Expand Down
2 changes: 1 addition & 1 deletion docs/WARP.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ High-level architecture and structure
- Filter rules (data/output/)
- data/output/adguard_user_filter.txt is the tracked output list consumed by AdGuard DNS.
- Filter compiler (src/rules-compiler-typescript/)
- Deno/TypeScript wrapper around @adguard/hostlist-compiler. Reads configuration, compiles sources, and writes compiled rules. Deno tests cover config parsing and output writing.
- Deno/TypeScript wrapper around @jk-com/adblock-compiler. Reads configuration, compiles sources, and writes compiled rules. Deno tests cover config parsing and output writing.
- API clients
- src/adguard-api-dotnet/: Auto-generated C# SDK for AdGuard DNS API v1.11. Targets net10.0; uses Newtonsoft.Json and JsonSubTypes. Includes Helpers for configuration and Polly-based retry policies. Console UI uses Spectre.Console.
- src/adguard-api-typescript/: TypeScript/Deno SDK with full API coverage, repository pattern, and interactive CLI using inquirer/ora.
Expand Down
2 changes: 1 addition & 1 deletion docs/chunking-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This guide explains the parallel chunking feature available in the rules compile

## Overview

When compiling filter lists with many sources or millions of rules, the single-threaded nature of `@adguard/hostlist-compiler` can become a bottleneck. Chunking addresses this by:
When compiling filter lists with many sources or millions of rules, chunking addresses this by:

1. **Splitting sources into chunks** - Distributes sources across multiple parallel workers
2. **Compiling chunks in parallel** - Uses multiple CPU cores simultaneously
Expand Down
4 changes: 2 additions & 2 deletions docs/configuration-reference.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Configuration Reference

All rules compilers in this repository use the same configuration schema based on [@adguard/hostlist-compiler](https://github.com/AdguardTeam/HostlistCompiler).
All rules compilers in this repository use the same configuration schema based on [@jk-com/adblock-compiler](https://github.com/jaypatrick/hostlistcompiler).

## Supported Formats

Expand Down Expand Up @@ -376,7 +376,7 @@ data/input/ → Compiler → data/output/
3. Validate syntax of each source
4. Compute SHA-384 hashes for tampering detection
5. Fetch internet sources with hash verification
6. Merge all sources using `@adguard/hostlist-compiler`
6. Merge all sources using `@jk-com/adblock-compiler`
7. Apply transformations (deduplicate, validate, etc.)
8. Convert hosts format to adblock if needed
9. Write to `data/output/adguard_user_filter.txt`
Expand Down
2 changes: 1 addition & 1 deletion docs/docker-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ WORKDIR /workspace

### Pre-installed Tools

- **Deno packages**: `@adguard/hostlist-compiler` (via npm: specifier)
- **Deno packages**: `@jk-com/adblock-compiler` (via JSR)
- **Python packages**: `pytest`, `pytest-cov`, `mypy`, `ruff`, `pyyaml`, `tomlkit`
- **Rust components**: `clippy`, `rustfmt`
- **PowerShell modules**: `Pester`, `PSScriptAnalyzer`
Expand Down
6 changes: 3 additions & 3 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ This guide will help you get up and running with the ad-blocking toolkit.
| Requirement | Version | Purpose | Installation |
|-------------|---------|---------|--------------|
| Deno | 2.0+ | TypeScript compilers and tools | [deno.land](https://deno.land/) |
| hostlist-compiler | Latest | Filter compilation | `deno run npm:@adguard/hostlist-compiler` |
| adblock-compiler | 0.6.0 | Filter compilation | `deno run jsr:@jk-com/adblock-compiler` |

### Language-Specific Requirements

Expand Down Expand Up @@ -48,7 +48,7 @@ Verify installation:
deno --version
```

The `@adguard/hostlist-compiler` package is accessed via Deno's npm compatibility.
The `@jk-com/adblock-compiler` package is accessed via Deno's JSR integration.

### 3. Choose Your Compiler

Expand Down Expand Up @@ -281,7 +281,7 @@ deno --version
You can run hostlist-compiler directly with:

```bash
deno run --allow-all npm:@adguard/hostlist-compiler --version
deno run --allow-all jsr:@jk-com/adblock-compiler --version
```

### Permission denied on Linux/macOS
Expand Down
6 changes: 3 additions & 3 deletions docs/guides/migration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ dotnet run --project src/RulesCompiler.Console
"compile": "node compile.js"
},
"dependencies": {
"@adguard/hostlist-compiler": "^1.0.0"
"@jk-com/adblock-compiler": "^0.6.0"
}
}
```
Expand All @@ -126,7 +126,7 @@ dotnet run --project src/RulesCompiler.Console
"compile": "deno run --allow-all src/mod.ts"
},
"imports": {
"@adguard/hostlist-compiler": "npm:@adguard/hostlist-compiler"
"@jk-com/adblock-compiler": "jsr:@jk-com/adblock-compiler@^0.6.0"
}
}
```
Expand Down Expand Up @@ -739,7 +739,7 @@ jsonlint config.json # For JSON
# Transformations are applied in fixed order regardless of config

# Check for version differences in hostlist-compiler
deno run npm:@adguard/hostlist-compiler --version
deno run jsr:@jk-com/adblock-compiler --version

# Enable debug output
new-compiler -c config.yaml -d
Expand Down
10 changes: 5 additions & 5 deletions docs/guides/troubleshooting-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,18 +85,18 @@ deno run -A src/mod.ts
}
```

### hostlist-compiler Not Found
### adblock-compiler Not Found

**Symptom:** `npm:@adguard/hostlist-compiler not found`
**Symptom:** `jsr:@jk-com/adblock-compiler not found`

**Solution:**

```bash
# Test npm: specifier
deno run --allow-all npm:@adguard/hostlist-compiler --version
# Test JSR specifier
deno run --allow-all jsr:@jk-com/adblock-compiler --version

# If that works, the issue is with your configuration
# Check deno.json for correct npm imports
# Check deno.json for correct JSR imports

# Ensure Deno version is 2.0+
deno upgrade
Expand Down
6 changes: 3 additions & 3 deletions docs/guides/typescript-rules-compiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ A comprehensive guide to using the TypeScript rules compiler with Deno 2.0+.

## Overview

The TypeScript rules compiler is a Deno-based implementation that wraps [@adguard/hostlist-compiler](https://github.com/AdguardTeam/HostlistCompiler) to compile filter lists from multiple sources with transformations, inclusions, and exclusions.
The TypeScript rules compiler is a Deno-based implementation that uses [@jk-com/adblock-compiler](https://github.com/jaypatrick/hostlistcompiler) to compile filter lists from multiple sources with transformations, inclusions, and exclusions.

## Features

Expand All @@ -20,7 +20,7 @@ The TypeScript rules compiler is a Deno-based implementation that wraps [@adguar
| Requirement | Version | Installation |
|-------------|---------|--------------|
| Deno | 2.0+ | [deno.land](https://deno.land/) |
| @adguard/hostlist-compiler | Latest | Via Deno's npm compatibility |
| @jk-com/adblock-compiler | 0.6.0 | Via JSR |

## Installation

Expand Down Expand Up @@ -349,7 +349,7 @@ deno --version
Test hostlist-compiler access:

```bash
deno run --allow-all npm:@adguard/hostlist-compiler --version
deno run --allow-all jsr:@jk-com/adblock-compiler --version
```

### Permission Denied
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@

<ItemGroup>
<ProjectReference Include="..\AdGuard.ApiClient\AdGuard.ApiClient.csproj" />
<ProjectReference Include="..\AdGuard.ConsoleUI\AdGuard.ConsoleUI.csproj" />
</ItemGroup>

</Project>
Loading
Loading