diff --git a/docs/superpowers/specs/2026-08-23-accounting-agent-architecture-design.md b/docs/superpowers/specs/2026-08-23-accounting-agent-architecture-design.md new file mode 100644 index 0000000..0c643f9 --- /dev/null +++ b/docs/superpowers/specs/2026-08-23-accounting-agent-architecture-design.md @@ -0,0 +1,151 @@ +# Personal Accounting Agent — Architecture Design + +**Date:** 2026-08-23 +**Status:** Approved (Jake, 2026-08-23) +**Scope:** Master architecture and phase decomposition. Each phase gets its own spec and implementation plan; this document is the frame they hang on. + +## Purpose + +Build a personal accounting agent around the existing Firefly III stack at +`https://firefly.jbnel.dev`. The agent: + +1. Monitors account balances across all institutions. +2. Assesses spending trends, tracks budgets, and forecasts cashflow. +3. Prepares utility and other bill payments, executing only with Jake's explicit approval. +4. Alerts Jake to pertinent information through Telegram, ntfy, and email. +5. Maintains account records (statements, bill PDFs) in a structured local folder, with cloud sync later. +6. Keeps all record data queryable — ledger data in Firefly, everything else in a dedicated agent database. + +## Decisions + +These were settled in the design conversation and bind all phases: + +| Decision | Choice | +|---|---| +| Primary bank feed | SimpleFIN Bridge (read-only, ~$1.50/mo) | +| Coverage strategy | Connector layer: SimpleFIN plus per-institution connectors (CSV drop folder, email parsing, browser automation) until NFCU, TFCU, Cash App, Venmo, and PayPal are all covered. Verify real coverage per institution early; assume nothing. | +| Bill-pay autonomy | Prep + approval. The agent tracks, verifies funding, and asks. Money never moves without an explicit approval from Jake. | +| Runtime | Hybrid. One Python worker container does deterministic work on schedules; scheduled Claude sessions do judgment work (analysis, forecast narrative, anomaly triage). | +| Notification channels | Telegram bot (interactive: alerts, approvals), ntfy (urgent push), email (digests, receipts). Routed by message class. | +| Document acquisition | Dedicated finance email inbox via IMAP first; browser-automation downloads later for billers that never email PDFs. | +| Business accounts | Separate Firefly instance when a venture opens. This instance stays purely personal. The connector layer is account-agnostic so it can serve both. | +| Agent state | `agentdb`: a separate database in the existing Postgres container. | + +## Architecture + +One new container, `agent`, joins the existing compose stack (Firefly III core, +Data Importer, Postgres 16, Redis, cron) behind Traefik. The worker is a Python +3.12 application under `agent/` in this repo, structured as modules with hard +boundaries so any module can become its own service during the planned k8s +migration: + +- **Connectors** — pull data from sources on schedules. Each connector + implements one interface: fetch → emit canonical records. Initial set: + SimpleFIN poller, CSV drop-folder watcher, IMAP inbox poller. Later: browser + automation jobs. +- **Sync engine** — normalizes connector output into canonical models + (transaction, balance, document, bill), deduplicates, and pushes to Firefly + via its REST API using a personal access token. +- **Notifier** — one interface, three sinks (Telegram, ntfy, SMTP). Message + class determines routing: interactive requests and real-time alerts go to + Telegram, urgent one-way pushes to ntfy, digests and receipts to email. +- **Rules engine** — deterministic checks after each sync: low balance, large + or unusual transaction, failed sync, upcoming bill without positioned funds. +- **Bill module** — maintains the bill calendar in `agentdb` from Firefly + bills and parsed bill emails; runs the approval workflow. +- **Scheduler** — APScheduler; every job idempotent and individually + triggerable for testing. + +Claude sessions run on schedules outside the container (Claude Code +routines/cron). They read the Firefly API and `agentdb`, write analysis and +narratives out through the notifier, and never write directly to the ledger. +The agent container exposes the notifier to them as one token-authenticated +HTTP endpoint (`POST /notify`), reachable only on the internal Docker network +or via SSH tunnel — never through Traefik. + +### Data flow + +``` +SimpleFIN ─┐ +CSV drop ─┼→ Connectors → canonical models → dedup → Firefly API (ledger) +IMAP ─┤ │ +Browser ─┘ └→ agentdb (documents, bills, + sync runs, approvals) +Firefly API + agentdb → Claude sessions → Notifier → Telegram / ntfy / email +``` + +Deduplication: use the source's external ID where one exists (SimpleFIN +provides one); otherwise a content hash of date + amount + account + normalized +description. Firefly's own duplicate detection is the backstop, never the +primary mechanism. + +### agentdb schema (v1) + +- `sync_runs` — one row per connector execution: connector, started, finished, status, counts, error. +- `documents` — index of filed records: path, sha256, institution, account, doc type, period, source, received date. +- `bills` — calendar: payee, expected amount, due date, funding account, autopay flag, status. +- `approvals` — bill-pay approvals: bill, amount, requested at, approved/denied at, Telegram message ref, executed at. + +## Safety and error handling + +- **Idempotent imports.** Dedup keys make every retry safe. +- **Escalating failure alerts.** Every connector run logs to `sync_runs`; three + consecutive failures for a connector escalate to ntfy. +- **Secrets stay server-side.** `.env` on the host only, never in the repo. + Aggregator access is read-only. +- **Approval integrity.** An approval exists only as an `approvals` row created + by Jake's own Telegram interaction. The executor re-checks the approval + immediately before acting and refuses stale or amount-mismatched approvals. +- **Payment receipts.** Every browser-automation payment run captures + screenshots and files them as documents. + +## Documents + +Local folder structure (exact layout specified in the Phase 5 spec): +`records////` with normalized filenames +(`2026-08-nfcu-checking-statement.pdf`). Every filed document gets a +`documents` row. Cloud sync via rclone; Jake picks the target during Phase 5. + +## Testing + +TDD throughout (superpowers test-driven-development skill). Connectors test +against recorded fixtures: captured SimpleFIN JSON, real Venmo/Cash App/PayPal +CSV exports, sample bill emails. The Firefly client tests against a throwaway +local Firefly via a compose test profile. One end-to-end smoke test proves +fixture → dedup → Firefly. + +## Deployment + +The stack runs on a remote host behind Traefik; this repo is the config source. +Assumed workflow: SSH to the host, `git pull`, `docker compose up -d`. Confirm +the exact workflow during Phase 1 planning and record it in the Phase 1 spec. + +## Phases + +Each phase is its own sub-project: spec → plan → implementation. Later phases +may not start until the prior phase runs in production. + +1. **Foundation** — Firefly configured for API use (personal access token, + asset accounts created); `agent/` skeleton: Dockerfile, config loading, + scheduler with a heartbeat job, `agentdb` migrations, compose service, + deployed to the host. +2. **Ingestion** — SimpleFIN → Firefly sync with dedup; verified coverage for + NFCU and TFCU; CSV drop-folder connector with mappers for Venmo, Cash App, + and PayPal exports. +3. **Alerts & monitoring** — notifier with all three sinks; rules engine: + low balance, large/unusual transaction, sync failure, negative trend. +4. **Analysis & forecasting** — scheduled Claude sessions: weekly spending + review, budget tracking, recurring-transaction detection, 30/60/90-day + cashflow forecast, monthly digest. +5. **Documents** — finance inbox via IMAP, PDF extraction and filing, document + index, cloud sync target chosen and wired. +6. **Bills** — bill calendar, funding checks, Telegram approve/deny flow, + browser automation for approved payments and statement downloads. +7. **Later** — business Firefly instance when a venture opens; k8s migration + per `k8s/README.md`. + +## Out of scope + +- Investment portfolio management and tax preparation. +- Multi-user support; this system serves Jake alone. +- Modifying Firefly III itself; the agent is strictly an API consumer.