firefly/docs/superpowers/specs/2026-08-23-accounting-agent-architecture-design.md

# Personal Accounting Agent — Architecture Design

**Date:** 2026-08-23
**Status:** Approved (Jake, 2026-08-23)
**Scope:** Master architecture and phase decomposition. Each phase gets its own spec and implementation plan; this document is the frame they hang on.

## Purpose

Build a personal accounting agent around the existing Firefly III stack at
`https://firefly.jbnel.dev`. The agent:

1. Monitors account balances across all institutions.
2. Assesses spending trends, tracks budgets, and forecasts cashflow.
3. Prepares utility and other bill payments, executing only with Jake's explicit approval.
4. Alerts Jake to pertinent information through Telegram, ntfy, and email.
5. Maintains account records (statements, bill PDFs) in a structured local folder, with cloud sync later.
6. Keeps all record data queryable — ledger data in Firefly, everything else in a dedicated agent database.

## Decisions

These were settled in the design conversation and bind all phases:

| Decision | Choice |
|---|---|
| Primary bank feed | SimpleFIN Bridge (read-only, ~$1.50/mo) |
| Coverage strategy | Connector layer: SimpleFIN plus per-institution connectors (CSV drop folder, email parsing, browser automation) until NFCU, TFCU, Cash App, Venmo, and PayPal are all covered. Verify real coverage per institution early; assume nothing. |
| Bill-pay autonomy | Prep + approval. The agent tracks, verifies funding, and asks. Money never moves without an explicit approval from Jake. |
| Runtime | Hybrid. One Python worker container does deterministic work on schedules; scheduled Claude sessions do judgment work (analysis, forecast narrative, anomaly triage). |
| Notification channels | Telegram bot (interactive: alerts, approvals), ntfy (urgent push), email (digests, receipts). Routed by message class. |
| Document acquisition | Dedicated finance email inbox via IMAP first; browser-automation downloads later for billers that never email PDFs. |
| Business accounts | Separate Firefly instance when a venture opens. This instance stays purely personal. The connector layer is account-agnostic so it can serve both. |
| Agent state | `agentdb`: a separate database in the existing Postgres container. |

## Architecture

One new container, `agent`, joins the existing compose stack (Firefly III core,
Data Importer, Postgres 16, Redis, cron) behind Traefik. The worker is a Python
3.12 application under `agent/` in this repo, structured as modules with hard
boundaries so any module can become its own service during the planned k8s
migration:

- **Connectors** — pull data from sources on schedules. Each connector
  implements one interface: fetch → emit canonical records. Initial set:
  SimpleFIN poller, CSV drop-folder watcher, IMAP inbox poller. Later: browser
  automation jobs.
- **Sync engine** — normalizes connector output into canonical models
  (transaction, balance, document, bill), deduplicates, and pushes to Firefly
  via its REST API using a personal access token.
- **Notifier** — one interface, three sinks (Telegram, ntfy, SMTP). Message
  class determines routing: interactive requests and real-time alerts go to
  Telegram, urgent one-way pushes to ntfy, digests and receipts to email.
- **Rules engine** — deterministic checks after each sync: low balance, large
  or unusual transaction, failed sync, upcoming bill without positioned funds.
- **Bill module** — maintains the bill calendar in `agentdb` from Firefly
  bills and parsed bill emails; runs the approval workflow.
- **Scheduler** — APScheduler; every job idempotent and individually
  triggerable for testing.

Claude sessions run on schedules outside the container (Claude Code
routines/cron). They read the Firefly API and `agentdb`, write analysis and
narratives out through the notifier, and never write directly to the ledger.
The agent container exposes the notifier to them as one token-authenticated
HTTP endpoint (`POST /notify`), reachable only on the internal Docker network
or via SSH tunnel — never through Traefik.

### Data flow

```
SimpleFIN ─┐
CSV drop  ─┼→ Connectors → canonical models → dedup → Firefly API (ledger)
IMAP      ─┤                                    │
Browser   ─┘                                    └→ agentdb (documents, bills,
                                                    sync runs, approvals)
Firefly API + agentdb → Claude sessions → Notifier → Telegram / ntfy / email
```

Deduplication: use the source's external ID where one exists (SimpleFIN
provides one); otherwise a content hash of date + amount + account + normalized
description. Firefly's own duplicate detection is the backstop, never the
primary mechanism.

### agentdb schema (v1)

- `sync_runs` — one row per connector execution: connector, started, finished, status, counts, error.
- `documents` — index of filed records: path, sha256, institution, account, doc type, period, source, received date.
- `bills` — calendar: payee, expected amount, due date, funding account, autopay flag, status.
- `approvals` — bill-pay approvals: bill, amount, requested at, approved/denied at, Telegram message ref, executed at.

## Safety and error handling

- **Idempotent imports.** Dedup keys make every retry safe.
- **Escalating failure alerts.** Every connector run logs to `sync_runs`; three
  consecutive failures for a connector escalate to ntfy.
- **Secrets stay server-side.** `.env` on the host only, never in the repo.
  Aggregator access is read-only.
- **Approval integrity.** An approval exists only as an `approvals` row created
  by Jake's own Telegram interaction. The executor re-checks the approval
  immediately before acting and refuses stale or amount-mismatched approvals.
- **Payment receipts.** Every browser-automation payment run captures
  screenshots and files them as documents.

## Documents

Local folder structure (exact layout specified in the Phase 5 spec):
`records/<institution>/<account>/<year>/` with normalized filenames
(`2026-08-nfcu-checking-statement.pdf`). Every filed document gets a
`documents` row. Cloud sync via rclone; Jake picks the target during Phase 5.

## Testing

TDD throughout (superpowers test-driven-development skill). Connectors test
against recorded fixtures: captured SimpleFIN JSON, real Venmo/Cash App/PayPal
CSV exports, sample bill emails. The Firefly client tests against a throwaway
local Firefly via a compose test profile. One end-to-end smoke test proves
fixture → dedup → Firefly.

## Deployment

The stack runs on a remote host behind Traefik; this repo is the config source.
Assumed workflow: SSH to the host, `git pull`, `docker compose up -d`. Confirm
the exact workflow during Phase 1 planning and record it in the Phase 1 spec.

## Phases

Each phase is its own sub-project: spec → plan → implementation. Later phases
may not start until the prior phase runs in production.

1. **Foundation** — Firefly configured for API use (personal access token,
   asset accounts created); `agent/` skeleton: Dockerfile, config loading,
   scheduler with a heartbeat job, `agentdb` migrations, compose service,
   deployed to the host.
2. **Ingestion** — SimpleFIN → Firefly sync with dedup; verified coverage for
   NFCU and TFCU; CSV drop-folder connector with mappers for Venmo, Cash App,
   and PayPal exports.
3. **Alerts & monitoring** — notifier with all three sinks; rules engine:
   low balance, large/unusual transaction, sync failure, negative trend.
4. **Analysis & forecasting** — scheduled Claude sessions: weekly spending
   review, budget tracking, recurring-transaction detection, 30/60/90-day
   cashflow forecast, monthly digest.
5. **Documents** — finance inbox via IMAP, PDF extraction and filing, document
   index, cloud sync target chosen and wired.
6. **Bills** — bill calendar, funding checks, Telegram approve/deny flow,
   browser automation for approved payments and statement downloads.
7. **Later** — business Firefly instance when a venture opens; k8s migration
   per `k8s/README.md`.

## Out of scope

- Investment portfolio management and tax preparation.
- Multi-user support; this system serves Jake alone.
- Modifying Firefly III itself; the agent is strictly an API consumer.