first commit
This commit is contained in:
A R R R Associates
@@ -0,0 +1,267 @@
|
|||||||
|
# Full ERP UAT + VAPT Suite v2.3
|
||||||
|
|
||||||
|
This is a **separate Playwright testing project** for Audit Firm ERP. It can run from your local system or from a separate Coolify/Node container against the ERP URL.
|
||||||
|
|
||||||
|
## What v2.3 includes
|
||||||
|
|
||||||
|
- Updated checklist workbook with v2.0.4 security/FY additions.
|
||||||
|
- 272 source use cases in `data/checklist-rows.json`.
|
||||||
|
- 1,296 generated runnable/traceable variants in `data/generated-test-matrix.json`.
|
||||||
|
- Existing v2.2 coverage for UAT, RBAC, validation, boundary, CSRF, IDOR, upload and security header checks.
|
||||||
|
- New targeted v2.0.4 checks for:
|
||||||
|
- Forgot-password reset token exposure
|
||||||
|
- Reset-token rejection
|
||||||
|
- API login brute-force safety
|
||||||
|
- `X-Tenant-Code`, `X-Branch-Code`, `X-Year-Code` spoofing lockdown
|
||||||
|
- Trusted context-header secret mode
|
||||||
|
- Active FY selector/session visibility
|
||||||
|
- Active FY transactional screen smoke test
|
||||||
|
- Locked FY write protection
|
||||||
|
- Backup export access protection
|
||||||
|
- Executable/oversized upload rejection
|
||||||
|
- Storage-agent authentication
|
||||||
|
- Cookie/security header checks
|
||||||
|
- Optional strict CSP check
|
||||||
|
- Seed data helper: `seed/seed_data.py`.
|
||||||
|
- OTP auto-read support when this suite starts ERP itself.
|
||||||
|
- Excel result update support.
|
||||||
|
|
||||||
|
## Important truth
|
||||||
|
|
||||||
|
This suite gives strong repeatable UAT/VAPT regression coverage, but it does not replace a professional manual VAPT. Some tests are intentionally defensive and skip when the corresponding seed IDs or optional production flags are not configured.
|
||||||
|
|
||||||
|
For strongest results:
|
||||||
|
|
||||||
|
1. Run ERP migrations.
|
||||||
|
2. Run `seed/seed_data.py`.
|
||||||
|
3. Paste emitted IDs into `.env`.
|
||||||
|
4. Run targeted v2.0.4 checks using `npm run test:v204`.
|
||||||
|
5. Run the full matrix using `npm run test:excel`.
|
||||||
|
|
||||||
|
## Folder layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
playwright_full_erp_uat_vapt_v22
|
||||||
|
├─ Audit_Firm_ERP_Master_UAT_VAPT_Checklist.xlsx
|
||||||
|
├─ data/
|
||||||
|
│ ├─ checklist-rows.json
|
||||||
|
│ ├─ generated-test-matrix.json
|
||||||
|
│ └─ summary.json
|
||||||
|
├─ fixtures/
|
||||||
|
│ ├─ auth.js
|
||||||
|
│ ├─ assertions.js
|
||||||
|
│ └─ v204-helpers.js
|
||||||
|
├─ scripts/
|
||||||
|
├─ seed/
|
||||||
|
│ └─ seed_data.py
|
||||||
|
├─ tests/
|
||||||
|
│ ├─ full-excel-variants.spec.js
|
||||||
|
│ ├─ vapt-targeted.spec.js
|
||||||
|
│ └─ v204-security-additions.spec.js
|
||||||
|
└─ test-data/uploads/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 1 — Install
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm install
|
||||||
|
npx playwright install chromium
|
||||||
|
copy .env.example .env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 2 — Seed ERP data
|
||||||
|
|
||||||
|
From your ERP project root, after migrations:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
python D:\path\to\playwright_full_erp_uat_vapt_v22\seed\seed_data.py
|
||||||
|
```
|
||||||
|
|
||||||
|
The script prints IDs like:
|
||||||
|
|
||||||
|
```env
|
||||||
|
TENANT_A_ID=...
|
||||||
|
TENANT_B_ID=...
|
||||||
|
CLIENT_A_ID=...
|
||||||
|
NOTICE_CASE_A_ID=...
|
||||||
|
```
|
||||||
|
|
||||||
|
Paste those into `.env` of this Playwright suite.
|
||||||
|
|
||||||
|
## Step 3 — Configure `.env`
|
||||||
|
|
||||||
|
Minimum:
|
||||||
|
|
||||||
|
```env
|
||||||
|
BASE_URL=http://127.0.0.1:8000
|
||||||
|
SYSTEM_ADMIN_EMAIL=admin@auditfirm.local
|
||||||
|
SYSTEM_ADMIN_PASSWORD=your-password
|
||||||
|
FIRM_ADMIN_EMAIL=uat.firmadmin@tenant-a.test
|
||||||
|
FIRM_ADMIN_PASSWORD=Password@123
|
||||||
|
STAFF_EMAIL=uat.staff@tenant-a.test
|
||||||
|
STAFF_PASSWORD=Password@123
|
||||||
|
ACTIVE_FY=2025-26
|
||||||
|
PREVIOUS_FY=2024-25
|
||||||
|
TRUST_CONTEXT_HEADERS=false
|
||||||
|
```
|
||||||
|
|
||||||
|
For locked-year tests, first lock a FY in ERP, then set:
|
||||||
|
|
||||||
|
```env
|
||||||
|
LOCKED_FY=2024-25
|
||||||
|
```
|
||||||
|
|
||||||
|
For production cookie assertion on HTTPS:
|
||||||
|
|
||||||
|
```env
|
||||||
|
EXPECT_SECURE_COOKIES=true
|
||||||
|
```
|
||||||
|
|
||||||
|
For strict CSP assertion only after inline JS/CSS is removed:
|
||||||
|
|
||||||
|
```env
|
||||||
|
EXPECT_STRICT_CSP=true
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 4 — Run ERP
|
||||||
|
|
||||||
|
Option A: run ERP separately:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
uvicorn app.main:app --reload
|
||||||
|
```
|
||||||
|
|
||||||
|
Then use:
|
||||||
|
|
||||||
|
```env
|
||||||
|
START_ERP=false
|
||||||
|
BASE_URL=http://127.0.0.1:8000
|
||||||
|
STATIC_OTP=123456
|
||||||
|
```
|
||||||
|
|
||||||
|
Option B: let Playwright start ERP and read OTP from console:
|
||||||
|
|
||||||
|
```env
|
||||||
|
START_ERP=true
|
||||||
|
OTP_FROM_CONSOLE=true
|
||||||
|
ERP_WORKDIR=D:\01 Project\versions\version 2.0\audit_firm_v2_baseline_with_alembic\Audit_Firm_v2.0.3.6\Audit_Firm_v2.0.3.6
|
||||||
|
ERP_COMMAND=uvicorn app.main:app --reload
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 5 — Check matrix and seed config
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm run matrix
|
||||||
|
npm run test:seed-check
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected v2.3 summary:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Source Excel rows: 272
|
||||||
|
Generated variants: 1296
|
||||||
|
v204AdditionalChecks: 16
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 6 — Run tests
|
||||||
|
|
||||||
|
Full Excel mapped suite:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm run test:excel
|
||||||
|
```
|
||||||
|
|
||||||
|
Original targeted VAPT checks:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm run test:vapt
|
||||||
|
```
|
||||||
|
|
||||||
|
New v2.0.4 targeted hardening checks:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm run test:v204
|
||||||
|
```
|
||||||
|
|
||||||
|
All tests:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm run test:all
|
||||||
|
```
|
||||||
|
|
||||||
|
Show report:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm run show-report
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 7 — Update Excel result status
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
npm run report:json
|
||||||
|
npm run update:excel
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
|
||||||
|
```text
|
||||||
|
results/Audit_Firm_ERP_Master_UAT_VAPT_Checklist_v2_2_Results.xlsx
|
||||||
|
```
|
||||||
|
|
||||||
|
## Coolify test-runner usage
|
||||||
|
|
||||||
|
You can run this suite as a separate Node/Playwright container in Coolify. Set:
|
||||||
|
|
||||||
|
```env
|
||||||
|
BASE_URL=https://uat-erp.yourdomain.com
|
||||||
|
START_ERP=false
|
||||||
|
TRUST_CONTEXT_HEADERS=false
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not run heavy load tests from the same VPS that hosts the ERP. Use this suite for functional UAT, regression, and security flow checks.
|
||||||
|
|
||||||
|
## Recommended ERP UI improvement for stronger automation
|
||||||
|
|
||||||
|
Add stable selectors over time:
|
||||||
|
|
||||||
|
```html
|
||||||
|
<select data-testid="active-fy-select"></select>
|
||||||
|
<button data-testid="client-create-submit">Save</button>
|
||||||
|
<input data-testid="client-pan">
|
||||||
|
<a data-testid="notice-case-new">New Case</a>
|
||||||
|
```
|
||||||
|
|
||||||
|
Without stable selectors, Playwright uses defensive generic selectors, which is useful but less precise.
|
||||||
|
|
||||||
|
## Security note
|
||||||
|
|
||||||
|
Keep public deployment with:
|
||||||
|
|
||||||
|
```env
|
||||||
|
TRUST_CONTEXT_HEADERS=false
|
||||||
|
```
|
||||||
|
|
||||||
|
Trusted internal context headers should only be used for backend-to-backend automation and must include `X-AuditFirm-Context-Secret` when enabled.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v2.4.0 changelog
|
||||||
|
|
||||||
|
- **Checklist reconciled to automation.** The Excel checklist is now regenerated
|
||||||
|
from the Playwright source matrix (`data/checklist-rows.json`, 272 source cases
|
||||||
|
/ 1296 variants) so every automated case has exactly one tracking row.
|
||||||
|
New file: `Audit_Firm_ERP_Master_UAT_VAPT_Checklist_v2.4.xlsx` (302 rows total).
|
||||||
|
- **Added `UAT_Work`, `UAT_Domains`, `VAPT_Domains`, `V204_Additional_Checks`** to
|
||||||
|
the checklist — previously automated but missing from the tracker.
|
||||||
|
- **`UAT_FY_Lock_Backup` (30 cases) carried in as a MANUAL block** and given
|
||||||
|
scaffolding in `tests/fy-lock-backup.spec.js`.
|
||||||
|
⚠️ This scaffolding is **UNTESTED**: every case is `test.fixme()` and must be
|
||||||
|
verified against a running ERP (routes/selectors/assertions) before it counts
|
||||||
|
as coverage. Run with `npm run test:fy`.
|
||||||
|
- 9 API-level security checks were recorded from a local SQLite run
|
||||||
|
(6 Pass, 1 Partial in the v204 block, plus 2 more). All UI/browser and manual
|
||||||
|
cases remain **Not Started** pending a full run against the deployed environment.
|
||||||
|
|
||||||
|
### Known source-of-truth note
|
||||||
|
The older `Audit_Firm_ERP_Master_UAT_VAPT_Checklist.xlsx` (218 rows) is superseded
|
||||||
|
by the v2.4 file. Use the v2.4 file going forward.
|
||||||
Reference in New Issue
Block a user