first commit
This commit is contained in:
A R R R Associates
@@ -0,0 +1,267 @@
|
||||
# Full ERP UAT + VAPT Suite v2.3
|
||||
|
||||
This is a **separate Playwright testing project** for Audit Firm ERP. It can run from your local system or from a separate Coolify/Node container against the ERP URL.
|
||||
|
||||
## What v2.3 includes
|
||||
|
||||
- Updated checklist workbook with v2.0.4 security/FY additions.
|
||||
- 272 source use cases in `data/checklist-rows.json`.
|
||||
- 1,296 generated runnable/traceable variants in `data/generated-test-matrix.json`.
|
||||
- Existing v2.2 coverage for UAT, RBAC, validation, boundary, CSRF, IDOR, upload and security header checks.
|
||||
- New targeted v2.0.4 checks for:
|
||||
- Forgot-password reset token exposure
|
||||
- Reset-token rejection
|
||||
- API login brute-force safety
|
||||
- `X-Tenant-Code`, `X-Branch-Code`, `X-Year-Code` spoofing lockdown
|
||||
- Trusted context-header secret mode
|
||||
- Active FY selector/session visibility
|
||||
- Active FY transactional screen smoke test
|
||||
- Locked FY write protection
|
||||
- Backup export access protection
|
||||
- Executable/oversized upload rejection
|
||||
- Storage-agent authentication
|
||||
- Cookie/security header checks
|
||||
- Optional strict CSP check
|
||||
- Seed data helper: `seed/seed_data.py`.
|
||||
- OTP auto-read support when this suite starts ERP itself.
|
||||
- Excel result update support.
|
||||
|
||||
## Important truth
|
||||
|
||||
This suite gives strong repeatable UAT/VAPT regression coverage, but it does not replace a professional manual VAPT. Some tests are intentionally defensive and skip when the corresponding seed IDs or optional production flags are not configured.
|
||||
|
||||
For strongest results:
|
||||
|
||||
1. Run ERP migrations.
|
||||
2. Run `seed/seed_data.py`.
|
||||
3. Paste emitted IDs into `.env`.
|
||||
4. Run targeted v2.0.4 checks using `npm run test:v204`.
|
||||
5. Run the full matrix using `npm run test:excel`.
|
||||
|
||||
## Folder layout
|
||||
|
||||
```text
|
||||
playwright_full_erp_uat_vapt_v22
|
||||
├─ Audit_Firm_ERP_Master_UAT_VAPT_Checklist.xlsx
|
||||
├─ data/
|
||||
│ ├─ checklist-rows.json
|
||||
│ ├─ generated-test-matrix.json
|
||||
│ └─ summary.json
|
||||
├─ fixtures/
|
||||
│ ├─ auth.js
|
||||
│ ├─ assertions.js
|
||||
│ └─ v204-helpers.js
|
||||
├─ scripts/
|
||||
├─ seed/
|
||||
│ └─ seed_data.py
|
||||
├─ tests/
|
||||
│ ├─ full-excel-variants.spec.js
|
||||
│ ├─ vapt-targeted.spec.js
|
||||
│ └─ v204-security-additions.spec.js
|
||||
└─ test-data/uploads/
|
||||
```
|
||||
|
||||
## Step 1 — Install
|
||||
|
||||
```cmd
|
||||
npm install
|
||||
npx playwright install chromium
|
||||
copy .env.example .env
|
||||
```
|
||||
|
||||
## Step 2 — Seed ERP data
|
||||
|
||||
From your ERP project root, after migrations:
|
||||
|
||||
```cmd
|
||||
python D:\path\to\playwright_full_erp_uat_vapt_v22\seed\seed_data.py
|
||||
```
|
||||
|
||||
The script prints IDs like:
|
||||
|
||||
```env
|
||||
TENANT_A_ID=...
|
||||
TENANT_B_ID=...
|
||||
CLIENT_A_ID=...
|
||||
NOTICE_CASE_A_ID=...
|
||||
```
|
||||
|
||||
Paste those into `.env` of this Playwright suite.
|
||||
|
||||
## Step 3 — Configure `.env`
|
||||
|
||||
Minimum:
|
||||
|
||||
```env
|
||||
BASE_URL=http://127.0.0.1:8000
|
||||
SYSTEM_ADMIN_EMAIL=admin@auditfirm.local
|
||||
SYSTEM_ADMIN_PASSWORD=your-password
|
||||
FIRM_ADMIN_EMAIL=uat.firmadmin@tenant-a.test
|
||||
FIRM_ADMIN_PASSWORD=Password@123
|
||||
STAFF_EMAIL=uat.staff@tenant-a.test
|
||||
STAFF_PASSWORD=Password@123
|
||||
ACTIVE_FY=2025-26
|
||||
PREVIOUS_FY=2024-25
|
||||
TRUST_CONTEXT_HEADERS=false
|
||||
```
|
||||
|
||||
For locked-year tests, first lock a FY in ERP, then set:
|
||||
|
||||
```env
|
||||
LOCKED_FY=2024-25
|
||||
```
|
||||
|
||||
For production cookie assertion on HTTPS:
|
||||
|
||||
```env
|
||||
EXPECT_SECURE_COOKIES=true
|
||||
```
|
||||
|
||||
For strict CSP assertion only after inline JS/CSS is removed:
|
||||
|
||||
```env
|
||||
EXPECT_STRICT_CSP=true
|
||||
```
|
||||
|
||||
## Step 4 — Run ERP
|
||||
|
||||
Option A: run ERP separately:
|
||||
|
||||
```cmd
|
||||
uvicorn app.main:app --reload
|
||||
```
|
||||
|
||||
Then use:
|
||||
|
||||
```env
|
||||
START_ERP=false
|
||||
BASE_URL=http://127.0.0.1:8000
|
||||
STATIC_OTP=123456
|
||||
```
|
||||
|
||||
Option B: let Playwright start ERP and read OTP from console:
|
||||
|
||||
```env
|
||||
START_ERP=true
|
||||
OTP_FROM_CONSOLE=true
|
||||
ERP_WORKDIR=D:\01 Project\versions\version 2.0\audit_firm_v2_baseline_with_alembic\Audit_Firm_v2.0.3.6\Audit_Firm_v2.0.3.6
|
||||
ERP_COMMAND=uvicorn app.main:app --reload
|
||||
```
|
||||
|
||||
## Step 5 — Check matrix and seed config
|
||||
|
||||
```cmd
|
||||
npm run matrix
|
||||
npm run test:seed-check
|
||||
```
|
||||
|
||||
Expected v2.3 summary:
|
||||
|
||||
```text
|
||||
Source Excel rows: 272
|
||||
Generated variants: 1296
|
||||
v204AdditionalChecks: 16
|
||||
```
|
||||
|
||||
## Step 6 — Run tests
|
||||
|
||||
Full Excel mapped suite:
|
||||
|
||||
```cmd
|
||||
npm run test:excel
|
||||
```
|
||||
|
||||
Original targeted VAPT checks:
|
||||
|
||||
```cmd
|
||||
npm run test:vapt
|
||||
```
|
||||
|
||||
New v2.0.4 targeted hardening checks:
|
||||
|
||||
```cmd
|
||||
npm run test:v204
|
||||
```
|
||||
|
||||
All tests:
|
||||
|
||||
```cmd
|
||||
npm run test:all
|
||||
```
|
||||
|
||||
Show report:
|
||||
|
||||
```cmd
|
||||
npm run show-report
|
||||
```
|
||||
|
||||
## Step 7 — Update Excel result status
|
||||
|
||||
```cmd
|
||||
npm run report:json
|
||||
npm run update:excel
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```text
|
||||
results/Audit_Firm_ERP_Master_UAT_VAPT_Checklist_v2_2_Results.xlsx
|
||||
```
|
||||
|
||||
## Coolify test-runner usage
|
||||
|
||||
You can run this suite as a separate Node/Playwright container in Coolify. Set:
|
||||
|
||||
```env
|
||||
BASE_URL=https://uat-erp.yourdomain.com
|
||||
START_ERP=false
|
||||
TRUST_CONTEXT_HEADERS=false
|
||||
```
|
||||
|
||||
Do not run heavy load tests from the same VPS that hosts the ERP. Use this suite for functional UAT, regression, and security flow checks.
|
||||
|
||||
## Recommended ERP UI improvement for stronger automation
|
||||
|
||||
Add stable selectors over time:
|
||||
|
||||
```html
|
||||
<select data-testid="active-fy-select"></select>
|
||||
<button data-testid="client-create-submit">Save</button>
|
||||
<input data-testid="client-pan">
|
||||
<a data-testid="notice-case-new">New Case</a>
|
||||
```
|
||||
|
||||
Without stable selectors, Playwright uses defensive generic selectors, which is useful but less precise.
|
||||
|
||||
## Security note
|
||||
|
||||
Keep public deployment with:
|
||||
|
||||
```env
|
||||
TRUST_CONTEXT_HEADERS=false
|
||||
```
|
||||
|
||||
Trusted internal context headers should only be used for backend-to-backend automation and must include `X-AuditFirm-Context-Secret` when enabled.
|
||||
|
||||
---
|
||||
|
||||
## v2.4.0 changelog
|
||||
|
||||
- **Checklist reconciled to automation.** The Excel checklist is now regenerated
|
||||
from the Playwright source matrix (`data/checklist-rows.json`, 272 source cases
|
||||
/ 1296 variants) so every automated case has exactly one tracking row.
|
||||
New file: `Audit_Firm_ERP_Master_UAT_VAPT_Checklist_v2.4.xlsx` (302 rows total).
|
||||
- **Added `UAT_Work`, `UAT_Domains`, `VAPT_Domains`, `V204_Additional_Checks`** to
|
||||
the checklist — previously automated but missing from the tracker.
|
||||
- **`UAT_FY_Lock_Backup` (30 cases) carried in as a MANUAL block** and given
|
||||
scaffolding in `tests/fy-lock-backup.spec.js`.
|
||||
⚠️ This scaffolding is **UNTESTED**: every case is `test.fixme()` and must be
|
||||
verified against a running ERP (routes/selectors/assertions) before it counts
|
||||
as coverage. Run with `npm run test:fy`.
|
||||
- 9 API-level security checks were recorded from a local SQLite run
|
||||
(6 Pass, 1 Partial in the v204 block, plus 2 more). All UI/browser and manual
|
||||
cases remain **Not Started** pending a full run against the deployed environment.
|
||||
|
||||
### Known source-of-truth note
|
||||
The older `Audit_Firm_ERP_Master_UAT_VAPT_Checklist.xlsx` (218 rows) is superseded
|
||||
by the v2.4 file. Use the v2.4 file going forward.
|
||||
Reference in New Issue
Block a user