Recording Pentest Engagement

Overview

A penetration test produces a lot of artifacts: scan outputs in multiple formats, screenshots showing the state of vulnerable pages, raw tool logs (nmap, Burp, custom scripts), the ROE and any amendments, exec-summary docs, and the findings themselves. Six months after the engagement closes, a question arises — sometimes benign ("can you remind me what we found?"), sometimes adversarial ("the customer claims we accessed an out-of-scope system; show us the logs"). The answer needs to be: "yes, here's the archive, and here's cryptographic proof it hasn't been touched since the engagement closed."

This skill hashes supported, non-excluded regular files and optionally packages the engagement directory with a SHA-256 manifest. An existing manifest is a verification baseline: mismatches preserve its bytes and block signing and packaging. Choose a distinct --manifest path to create a deliberate new snapshot; retain the original for investigation. Freeze evidence writers during hashing and packaging: this is not an atomic filesystem snapshot. Review excluded files and symlink findings separately; the tarball may contain entries the manifest does not cover.

The skill also surfaces inconsistencies that would weaken the chain-of-custody claim: out-of-tree paths referenced in findings (meaning the finding refers to a file not actually in the archive), files in the directory not listed in the manifest, manifest entries whose hashes don't match. These are HIGH findings — they mean the archive is incomplete or has been modified after the fact, neither of which is acceptable for evidence purposes.

When the skill produces findings

Prerequisites

• Python 3.9+
• An engagement directory laid out as recommended (see structure below)
• Optional GPG installed for manifest signing

Recommended engagement directory structure

engagements/acme-2026-q2/
├── roe.yaml
├── roe.amendments/
│   └── amendment-001-20260615.yaml
├── scope/
│   ├── allowed-ips.txt
│   └── normalized-targets.json
├── findings/
│   ├── cluster1-tls-2026-06-05.json
│   ├── cluster1-headers-2026-06-05.json
│   ├── cluster3-secrets-2026-06-07.json
│   └── ...
├── evidence/
│   ├── screenshots/
│   ├── tool-logs/
│   └── raw-scan-output/
├── reports/
│   ├── vulnerability-report.md
│   ├── owasp-mapping.json
│   └── executive-summary.md
└── manifest.sha256        # produced by this skill

Instructions

Step 1 — Identify the engagement directory

python3 ./scripts/record_engagement.py engagements/acme-2026-q2/

The skill walks the directory recursively, builds a SHA-256 manifest, and produces a chain-of-custody report.

Step 2 — Run the packager

Options:

Usage: record_engagement.py PATH [OPTIONS]

Options:
  --output FILE              Findings output
  --format FMT               json | jsonl | markdown (default: markdown)
  --min-severity SEV         default info
  --manifest FILE            Manifest output path (default: PATH/manifest.sha256)
  --tar FILE                 Also create a .tar.gz archive at this path
  --sign                     Sign the manifest with GPG (uses default identity)
  --signer KEY               GPG key ID to sign with
  --exclude GLOB             Skip files matching glob (repeatable)

Step 3 — Verify the manifest

The skill emits a SHA-256 manifest in standard sha256sum format (one line per file: hash + two-space + path). The manifest is verifiable independent of the skill:

(cd engagements/acme-2026-q2 && sha256sum -c manifest.sha256)

Run the command from the engagement directory because paths are relative to that directory. A successful check verifies the listed files only; it does not prove completeness, collection time, authorship or custody.

Step 4 — Sign the manifest (optional)

python3 ./scripts/record_engagement.py engagements/acme-2026-q2/ --sign

The skill shells out to gpg --detach-sign against the manifest, producing manifest.sha256.asc. Later verification:

gpg --verify manifest.sha256.asc manifest.sha256
sha256sum -c manifest.sha256

Both checks together: the manifest's contents match the archive, AND the manifest itself is signed by an identifiable party.

Step 5 — Create the portable archive (optional)

python3 ./scripts/record_engagement.py engagements/acme-2026-q2/ \
    --tar engagements/archives/acme-2026-q2.tar.gz

The tarball contains the engagement directory + manifest + signature. Hand the tarball to legal / archive / customer.

Examples

Example 1 — End-of-engagement closeout

python3 ./scripts/record_engagement.py engagements/acme-2026-q2/ \
    --sign \
    --tar engagements/archives/acme-2026-q2.tar.gz \
    --output engagements/acme-2026-q2/chain-of-custody.md

Example 2 — Daily snapshot during a long engagement

DATE=$(date +%Y%m%d)
python3 ./scripts/record_engagement.py engagements/acme-2026-q2/ \
    --manifest engagements/acme-2026-q2/manifest-$DATE.sha256 \
    --output engagements/acme-2026-q2/snapshot-$DATE.md

Daily snapshots build a time-series of the engagement's state which can be useful in incident-investigation contexts.

Example 3 — Pre-handoff integrity check

# Customer claims the archive is incomplete; verify before disputing
python3 ./scripts/record_engagement.py engagements/acme-2026-q2/ --min-severity high

A zero exit status means no unfiltered HIGH/CRITICAL findings. --min-severity controls display only. Missing, malformed or unreadable existing manifests fail closed; symlink and other MEDIUM findings still need operator review. No cryptographic verification is performed by this command. Use the independent manifest/signature checks before handoff.

Output

JSON / JSONL / Markdown per lib/report.py. Exit codes: 0 clean, 1 high/critical, 2 error.

Each shared Finding includes skill_id, title, severity, target, detail, remediation, fingerprint and optional evidence. Existing manifests are not rewritten on a successful verification either. Report, manifest, signature and archive destinations must be distinct; archives must be outside the engagement tree. Existing signatures are preserved when --sign is requested again; verify them separately or use a new snapshot path. A requested output must not replace collected evidence.

Error Handling

• Path doesn't exist → nonzero with operational finding.
• Existing manifest differs or is malformed → nonzero; preserve baseline and prior signature/archive. A new snapshot needs a distinct path.
• Unparseable JSON/JSONL findings → HIGH; no new manifest or package.
• Permission denied reading a file → emits HIGH finding, continues walking other files.
• GPG not installed with --sign requested → emits HIGH finding, manifest written unsigned, exits 1.
• GPG signing fails (no default identity, etc.) → emits HIGH finding, manifest written unsigned, exits 1.
• tar command fails with --tar requested → manifest still written; archive creation failure surfaces as HIGH finding.

Resources

• references/THEORY.md — Chain of custody as a legal concept, evidence-integrity standards (NIST SP 800-86), SHA-256 vs SHA-3 vs SHA-512 tradeoffs, GPG detached-signature semantics, archive format choices (tar vs zip vs WORM), retention horizons per jurisdiction
• references/PLAYBOOK.md — Engagement directory templates per engagement type, daily-snapshot cron pattern, customer-handoff protocol, dispute-resolution playbook (when the customer challenges the archive), long-term storage and access-control patterns