test: add Pest v1 security test infrastructure#131
test: add Pest v1 security test infrastructure#131somethingwithproof wants to merge 3 commits intoCacti:developfrom
Conversation
Add source-scan tests verifying security patterns (prepared statements, output escaping, auth guards, PHP 7.4 compatibility) remain in place across refactors. Tests run with Pest v1 (PHP 7.3+) and stub the Cacti framework so plugins can be tested in isolation. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces a Pest v1-based test scaffold for the ReportIt plugin, focused on lightweight security/regression checks without requiring a full Cacti runtime.
Changes:
- Add Pest bootstrap/configuration to run tests in this repository.
- Add a setup.php structure smoke test to verify expected plugin hooks/version keys exist.
- Add source-scan tests for prepared-statement usage consistency and PHP 7.4 syntax compatibility.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| composer.json | Adds Pest v1 as a dev dependency and wires test bootstrap via autoload-dev. |
| tests/Pest.php | Pest entrypoint requiring the local test bootstrap. |
| tests/bootstrap.php | Provides Cacti function/constant stubs intended to support isolated test execution. |
| tests/Security/SetupStructureTest.php | Verifies required plugin hook functions and version keys exist in setup.php. |
| tests/Security/PreparedStatementConsistencyTest.php | Scans selected plugin files to detect raw db_* calls vs *_prepared helpers. |
| tests/Security/Php74CompatibilityTest.php | Scans selected plugin files for PHP 8+ function/syntax usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| $source = file_get_contents(realpath(__DIR__ . '/../../setup.php')); | ||
|
|
There was a problem hiding this comment.
$source is loaded via file_get_contents(realpath(...)) without checking for realpath/file_get_contents failure. If setup.php is missing/unreadable, this can emit warnings (PHP 7.4) or throw a TypeError (PHP 8+) before the assertions run, and it’s not obvious why the test failed. Consider asserting the resolved path/contents are not false (and failing with a clear message) before running toContain/toMatch checks.
| $source = file_get_contents(realpath(__DIR__ . '/../../setup.php')); | |
| $setupPath = realpath(__DIR__ . '/../../setup.php'); | |
| if ($setupPath === false) { | |
| throw new RuntimeException('Unable to resolve setup.php for structure tests.'); | |
| } | |
| $source = file_get_contents($setupPath); | |
| if ($source === false) { | |
| throw new RuntimeException('Unable to read setup.php for structure tests.'); | |
| } |
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
This test currently skips files silently when realpath() or file_get_contents() fails (continue). That means the test can pass while not scanning some/all target files (e.g., wrong working directory or renamed file), undermining the security check. Prefer failing the test when a target file can’t be resolved/read, so missing coverage is caught.
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect($path)->not->toBeFalse( | |
| "Failed to resolve target file {$relativeFile}" | |
| ); | |
| $contents = file_get_contents($path); | |
| expect($contents)->not->toBeFalse( | |
| "Failed to read target file {$relativeFile}" | |
| ); |
| it('uses prepared DB helpers in all plugin files', function () { | ||
| $targetFiles = array( | ||
| 'lib/funct_calculate.php', | ||
| 'lib/funct_export.php', | ||
| 'lib/funct_html.php', | ||
| 'lib/funct_online.php', | ||
| 'lib/funct_reports.php', | ||
| 'lib/funct_shared.php', | ||
| 'lib/funct_validate.php', | ||
| 'setup.php', |
There was a problem hiding this comment.
As written, this enforces zero occurrences of db_execute/db_fetch_* in the listed files. The current codebase still contains raw calls in those targets (e.g., setup.php has multiple db_execute() DROP TABLE statements; lib/funct_reports.php drops a results table; lib/funct_shared.php uses db_execute() for cache table writes), so this test will fail immediately. Either migrate those call sites in the same PR, or relax the rule (e.g., allowlist unavoidable DDL/identifier-building cases or narrow the scanned files) so the test reflects what’s actually enforceable.
| it('uses prepared DB helpers in all plugin files', function () { | |
| $targetFiles = array( | |
| 'lib/funct_calculate.php', | |
| 'lib/funct_export.php', | |
| 'lib/funct_html.php', | |
| 'lib/funct_online.php', | |
| 'lib/funct_reports.php', | |
| 'lib/funct_shared.php', | |
| 'lib/funct_validate.php', | |
| 'setup.php', | |
| it('uses prepared DB helpers in migrated plugin files', function () { | |
| $targetFiles = array( | |
| 'lib/funct_calculate.php', | |
| 'lib/funct_export.php', | |
| 'lib/funct_html.php', | |
| 'lib/funct_online.php', | |
| 'lib/funct_validate.php', |
| it('does not use str_contains (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
||
| expect(preg_match('/\bstr_contains\s*\(/', $contents))->toBe(0, | ||
| "{$relativeFile} uses str_contains() which requires PHP 8.0" | ||
| ); | ||
| } | ||
| }); | ||
|
|
||
| it('does not use str_starts_with (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
||
| expect(preg_match('/\bstr_starts_with\s*\(/', $contents))->toBe(0, | ||
| "{$relativeFile} uses str_starts_with() which requires PHP 8.0" | ||
| ); | ||
| } | ||
| }); | ||
|
|
||
| it('does not use str_ends_with (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
||
| expect(preg_match('/\bstr_ends_with\s*\(/', $contents))->toBe(0, | ||
| "{$relativeFile} uses str_ends_with() which requires PHP 8.0" | ||
| ); | ||
| } | ||
| }); | ||
|
|
||
| it('does not use nullsafe operator (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
There was a problem hiding this comment.
Each block uses the same pattern of continue-ing when realpath()/file_get_contents() fails. That makes the compatibility checks silently skip files and potentially pass without scanning anything. Consider failing the test when a target file can’t be resolved/read, and (optionally) extracting a small helper to avoid duplicating this logic across the four checks.
| it('does not use str_contains (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect(preg_match('/\bstr_contains\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_contains() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_starts_with (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect(preg_match('/\bstr_starts_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_starts_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_ends_with (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect(preg_match('/\bstr_ends_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_ends_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use nullsafe operator (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| $readFileContents = function (string $relativeFile): string { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| throw new RuntimeException("Failed to resolve path for compatibility check: {$relativeFile}"); | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| throw new RuntimeException("Failed to read file for compatibility check: {$relativeFile}"); | |
| } | |
| return $contents; | |
| }; | |
| it('does not use str_contains (PHP 8.0)', function () use ($files, $readFileContents) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readFileContents($relativeFile); | |
| expect(preg_match('/\bstr_contains\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_contains() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_starts_with (PHP 8.0)', function () use ($files, $readFileContents) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readFileContents($relativeFile); | |
| expect(preg_match('/\bstr_starts_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_starts_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_ends_with (PHP 8.0)', function () use ($files, $readFileContents) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readFileContents($relativeFile); | |
| expect(preg_match('/\bstr_ends_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_ends_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use nullsafe operator (PHP 8.0)', function () use ($files, $readFileContents) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readFileContents($relativeFile); |
…dabot - Throw RuntimeException when realpath/file_get_contents fails (previously silent continue hid unscanned files) - Fix Dependabot ecosystem from npm to composer - Remove committed .omc session artifacts, add .omc/ to .gitignore Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
|
Converted to draft to serialize the stack in this repo. Blocked by #128; will un-draft after that merges to avoid cross-PR merge conflicts. |
Summary
Test plan
composer install && vendor/bin/pestpasses