Statement Parser¶

Status: Live Runs as: heezy-statement-scanner CronJob in the heezy k8s namespace Parser: heezy-containers/dockerfiles/heezy-finance/parse.py Scanner: heezy-containers/dockerfiles/heezy-finance/scan_statements.py Manifest: heezy-k8s/apps/heezy-finance/cronjob-scanner.yaml Last Updated: 2026-08-11

This no longer runs on big-boi

The parser used to be /opt/statement-parser/parse.py on big-boi (192.168.1.21), driven by a 5-minute host cron and logging to /var/log/statement-ingest.log. It is not. That log file does not exist, the ansible-heezy/roles/statement-parser/ role is vestigial, and the code now lives inside the heezy-finance container image. The NFS drop directory is the only part of the old design that survived.

Ingest Pipeline¶

file lands in /nfs/heezy/ingest/raw/statements/new/
  → heezy-statement-scanner CronJob, hourly at :30
  → scan_statements.py walks /ingest/new (nfs-heezy-ingest PVC)
  → detect_bank() identifies the institution
  → parse() extracts statements + transactions
  → category + merchant_normalized applied by rule
  → bank_statements, bank_transactions, account_balances written
  → file moves to processed/ or error/

Files arrive in the drop directory two ways: the statements.heezy.info web upload, or an scp straight to the NFS path.

There is a second, independent path: POST /api/statements/upload on the finance dashboard calls the same parse.py synchronously in-process and returns the result immediately. Nothing touches NFS. See the Statements Tab.

The scanner is scheduled at :30 to stay clear of heezy-finance-sync at :00.

Supported Formats¶

Institution	Types	Formats
Bank of America	checking, credit cards	PDF
Capital One	checking, savings, multi-account PDFs	PDF
Chase	Amazon Visa	PDF, CSV, OFX/QFX
Edward Jones	brokerage, 529, IRA, Roth IRA, money market	PDF
Fifth Third	mortgage (writes to `mortgage_statements`)	PDF

BoA year-end summary PDFs are detected and moved to processed/ without parsing — they contain both "bank of america" and "credit card" and would otherwise be mis-parsed as a CC statement.

Detection order¶

detect_bank() order is load-bearing, because issuers print each other's names inside transaction descriptions:

year-end summary → Capital One → Bank of America (checking markers before CC markers) → Edward Jones → Chase Amazon → Fifth Third

Anything unrecognized returns None and the file goes to error/.

Statement ID Format¶

{bank_slug}{type_slug}{last4}_{YYYY-MM-DD} — e.g. boachk1897_2026-01-27, chasecrd2609_2026-06-19

The date is the statement closing date.

Current Data¶

80 statements, 1,582 transactions, 9 mortgage statements as of 2026-08-11.

Institution	Type	Statements	Coverage
Bank of America	checking	7	2026-01 to 2026-07
Bank of America	credit_card	14	2026-01 to 2026-07
Capital One	checking	10	2025-11 to 2026-06
Capital One	savings	10	2025-11 to 2026-06
Chase	credit_card	14	2025-10 to 2026-07
Edward Jones	5 account types	24	2026-05 to 2026-07

Upload Workflow¶

Preferred: statements.heezy.info, which lands the file on NFS for the next scanner run.

Direct to NFS:

scp statement.pdf mcp-admin@192.168.1.21:/nfs/heezy/ingest/raw/statements/new/

Immediate parse with a response: upload from the dashboard's Statements page, which is synchronous.

Why not Gmail auto-fetch¶

BoA and Capital One send notification-only emails with no PDF attachment. The PDF has to be downloaded from the bank portal by hand.

Categorization¶

category and merchant_normalized are applied on insert by a rule table in scan_statements.py. Ollama was tried and is too slow on CPU for this volume.

A merchant rule is (pattern, merchant, category), and merchant is the display name written to bank_transactions.merchant_normalized and rendered as the Vendor column — not a category label. A rule spanning several chains must be split one rule per chain, otherwise every Meijer, Costco and Aldi debit shows up as a vendor literally named "grocery", which is what it did until 2026-08-11.

merchant = DERIVE marks a pattern that deliberately covers an open-ended set of independent vendors (every TST* restaurant, every SQ* small business) where no fixed label is right. Those fall through to derive_merchant().

To recategorize existing rows after a rule change:

kubectl exec -n heezy deploy/heezy-finance -- python3 backfill_categories.py
kubectl exec -n heezy deploy/heezy-finance -- python3 backfill_merchants.py

Troubleshooting¶

kubectl get cronjob heezy-statement-scanner -n heezy
kubectl logs -n heezy -l job-name=heezy-statement-scanner --tail=100
ssh mcp-admin@192.168.1.21 ls /nfs/heezy/ingest/raw/statements/error/

The /ingest mount exists only inside the scanner Job pod, not the dashboard pod, so inspect the directories over NFS on big-boi rather than with kubectl exec into deploy/heezy-finance.

A file in error/ was either unrecognized by detect_bank() or threw during parsing. Move it back to new/ after fixing the parser to reprocess. statement_files_processed dedupes by file hash, so a re-upload of an already-ingested file is a no-op.