CLAUDE.md — Tasmanian Planning Scheme Assistant

tasplanning.report

This file gives Claude Code full context on the project architecture, conventions, and deployment workflow. Read this before making any changes.

Project Overview

tasplanning.report is an AI-powered assistant for the Tasmanian Planning Scheme (TPS). It allows planners, building designers, architects, and consultants to ask natural-language questions about SPPs and LPS documents and receive clause-cited answers.

Core capability: Retrieval-Augmented Generation (RAG) — user queries are embedded, matched against a Qdrant vector database of official planning documents, and the retrieved context is injected into an LLM prompt. The LLM generates a cited answer using only the retrieved context — it never invents clause numbers.

Repository Layout

/home/modulos_llm/
├── pdfs/
│   ├── as/                    # Australian Standards in PDF
│   ├── lps/                   # Local Planning Scheme in PDF
│   ├── ncc/                   # National Construction Code in PDF
│   ├── tps/                   # Tasmanian Planning Scheme in PDF
├── backend/                   # FastAPI Python backend
│   ├── app.py                 # Main API — /ask, /feedback, /councils, /readyz, admin endpoints
│   ├── telemetry.py           # SQLite telemetry DB, /telemetry POST endpoint, schema init
│   ├── ingest.py              # PDF ingestion into Qdrant (run manually)
│   ├── ingest_ollama.py       # Ollama-based ingestion variant
│   └── requirements.txt       # Python dependencies
├── public/                    # PHP/Apache web frontend (bind-mounted into web container)
│   ├── index.php              # Landing page with demo modal and waitlist
│   ├── local_state-planning-scheme.php   # Main chat assistant UI
│   ├── site-report.php        # Property lookup (LIST/ArcGIS + Google Maps)
│   ├── section-builder.php    # Planning report section builder
│   ├── byok-settings.php      # API key management (BYOK feature)
│   ├── dashboard.php          # Internal query monitoring dashboard
│   ├── waitlist.php           # PHPMailer waitlist signup endpoint
│   ├── gmaps-key.php          # Google Maps API key proxy
│   ├── faq.php                # FAQ page
│   ├── privacy.php            # Privacy policy
│   ├── terms.php              # Terms of use
│   ├── js/
│   │   └── api-status.js      # Shared live API health indicator
│   ├── css/
│   │   └── report.css         # Shared stylesheet for index.php
│   ├── vendor/                # Composer PHP dependencies (PHPMailer, phpword, google/apiclient)
│   └── composer.json          # PHP dependency manifest
├── web/
│   └── Dockerfile             # Custom PHP 8.3 + Apache image
├── cache/
├── creds/
│   ├── oauth-client.json
│   ├── service-account.json
├── images/
├── models/
├── qdrant_storage/            # Qdrant vector DB persistent storage
│   ├── aliases/               #
│   ├── collections/           #
│   ├── raft_state.json        #
├── telemetry_data/
│   └── telemetry.db           # SQLite telemetry database
├── venv/
│   ├── include
│   │   └── python3.1.2
├── docker-compose.yml         # Full stack orchestration
└── .env                       # Secrets (SMTP_PASS, GMAPS_API_KEY, TPR_IP_SECRET, etc.)
└── CHANGELOG.md
└── CLAUDE.md
└── README.md

Docker Services

All services defined in docker-compose.yml. Run from /home/modulos_llm/.

Service	Container	Port	Purpose
`qdrant`	`modulos-qdrant`	6333, 6334	Vector database
`backend`	`modulos-backend`	2281→8000	FastAPI API
`web`	`modulos-web`	2380→80	PHP/Apache frontend
`sqliteweb`	`modulos-sqliteweb`	8091→8080	SQLite Web UI (internal)
`composer`	(run-once)	—	PHP dependency installer

Common commands

# Start everything
docker compose up -d

# Restart a single service (backend changes need restart, web changes do not)
docker compose restart backend

# View logs
docker logs modulos-backend --tail 50
docker logs modulos-web --tail 50

# Force recreate (picks up docker-compose.yml changes)
docker compose up -d --force-recreate web

# Run composer to install/update PHP dependencies
docker compose run --rm composer
# Or directly:
docker run --rm -v /home/modulos_llm/public:/app composer:2 \
  require phpmailer/phpmailer --no-interaction --no-progress --ignore-platform-reqs

# One-off Python command in backend container
docker exec modulos-backend python3 -c "..."

# Check API health
curl -s https://api.modulos.com.au/readyz

Important: bind mounts

./backend:/app — backend Python files are live. Edit files on host, restart backend to pick up changes.
./public:/var/www/html — PHP files are live. Edit files on host, no restart needed.
/home/modulos_llm/telemetry_data:/data — telemetry DB shared between backend and web containers.

Backend (`app.py`)

Key environment variables

OLLAMA_URL=http://192.168.8.73:11434      # Windows machine running Ollama
QDRANT_URL=http://qdrant:6333             # Internal Docker network
OLLAMA_KEEP_ALIVE=-1                      # Keep model loaded permanently (-1 = forever)
CHAT_MODEL=llama3.1:8b-instruct-q4_K_M   # Current model (Q4 quantization)
EMBED_MODEL=nomic-embed-text              # Embedding model
QDRANT_COLLECTION=planning_docs           # Vector collection name
CORS_ORIGINS=https://tasplanning.report,http://localhost:3000,...
TPR_DB=/data/telemetry.db
TPR_IP_SECRET=<secret>                    # HMAC secret for IP hashing

API endpoints

Method	Path	Purpose
GET	`/readyz`	Health check — returns `{"ok":true}`
GET	`/councils`	List indexed council names
POST	`/ask`	Main RAG query endpoint
GET	`/ask`	Same, query string params
POST	`/feedback`	Store thumbs up/down with query+answer
POST	`/telemetry`	Generic event logging
GET	`/admin/stats`	Collection statistics
GET	`/admin/files`	List indexed documents
GET	`/admin/sample`	Sample random chunks
GET	`/admin/export`	NDJSON export of all chunks

`/ask` request body (`AskBody`)

{
  "query": "What are the setbacks in the Village Zone?",
  "council": "launceston",     # optional — filters to that council's LPS
  "top_k": 8,                  # number of chunks to retrieve
  "scope": "state_plus_local", # state_plus_local | state_only | local_only | any
  "section_id": null,          # optional — triggers specific output format guide
  "include_ncc": false,
  "include_standards": false,
  "context_only": false        # BYOK mode — returns context+prompt without calling Ollama
}

`/ask` response

{
  "answer": "...",
  "sources": [
    {"source_file": "tasmanian-planning-scheme.pdf", "page": 42, "chunk_index": 3, "score": 0.87}
  ]
}

When context_only: true (BYOK mode):

{
  "context_only": true,
  "context": "...",
  "prompt": "...",   // full prompt ready to send to external LLM
  "sources": [...],
  "sections": [...]
}

Ollama settings (in `ollama_chat()`)

"keep_alive": -1,          # top-level param, NOT inside options
"options": {
  "num_ctx": 6144,          # context window — don't change between requests or model reloads
  "num_predict": 1024,      # max output tokens
  "temperature": 0.15,      # low = more deterministic
  "top_p": 0.85,
  "top_k": 40,
  "repeat_penalty": 1.15,
  "stop": ["QUESTION:", "---END---"],
}

Important: keep_alive must be a top-level JSON key, not inside options. Putting it inside options causes Ollama to silently ignore it.

Prompt structure

The system prompt uses ## AUTHORITY ORDER, ## STRICT RULES, ## OUTPUT FORMAT headers. llama3.1 responds well to Markdown section headers as instruction anchors. The fallback phrase when context is insufficient: "The provided context does not cover this — check the TPSO viewer directly at tpso.planning.tas.gov.au"

Qdrant document schema

Each chunk in the vector DB has this payload:

{
  "corpus": "tps",                          // "tps" | "lps" | "ncc" | "as"
  "council": "launceston",                  // only for LPS chunks
  "source_file": "launceston-lps.pdf",
  "page": 42,
  "chunk_index": 3,
  "text": "..."
}

Scope filtering uses corpus and council exact-match filters in Qdrant.

Frontend Pages

Design system

All pages use a consistent dark design system. Never use Bootstrap on redesigned pages.

--bg:            #0b0f0e;    /* page background */
--bg-1:          #111614;    /* slightly lighter panels */
--bg-2:          #181e1b;    /* card interiors */
--bg-card:       #141a17;
--border:        rgba(255,255,255,0.07);
--accent:        #2ddc8a;    /* green accent */
--accent-dim:    rgba(45,220,138,0.10);
--text-primary:  #eaf0ec;
--text-secondary:#8fa899;
--text-muted:    #4f6459;
--danger:        #f08080;
--serif:         'DM Serif Display', Georgia, serif;
--sans:          'DM Sans', system-ui, sans-serif;
--mono:          ui-monospace, 'Cascadia Code', Menlo, monospace;

Fonts loaded from Google Fonts: DM Serif Display (headings/display) + DM Sans (body). Bootstrap Icons CDN used for icons: https://cdn.jsdelivr.net/npm/bootstrap-icons@1.11.3/font/bootstrap-icons.css

Page inventory

File	Purpose	Notes
`index.php`	Landing page	Has demo modal, waitlist form, hero search
`local_state-planning-scheme.php`	Main chat assistant	Two-panel: sidebar + chat thread
`site-report.php`	Property lookup	Google Maps PlaceAutocomplete, LIST/ArcGIS API
`section-builder.php`	Report builder	3-panel: context / sections / preview
`byok-settings.php`	API key management	Keys in localStorage only, never server
`dashboard.php`	Query monitoring	Restricted by IP via .htaccess
`waitlist.php`	Waitlist endpoint	PHPMailer, reads SMTP_* from environment
`gmaps-key.php`	Maps key proxy	Serves GMAPS_API_KEY from env, checks HTTP_HOST
`faq.php`	FAQ	Accordion, sticky sidebar, FAQPage schema
`privacy.php`	Privacy policy
`terms.php`	Terms of use

Cross-tab data passing

Property data passes from site-report.php → section-builder.php via localStorage:

// site-report.php writes:
localStorage.setItem('tpr_builder_ctx', JSON.stringify({ ctx, written_at: Date.now() }));

// section-builder.php reads (30-minute TTL):
const { ctx, written_at } = JSON.parse(localStorage.getItem('tpr_builder_ctx'));
if (Date.now() - written_at > 30 * 60 * 1000) { /* expired */ }

Never use sessionStorage for cross-tab data — each tab gets its own isolated sessionStorage.

BYOK (Bring Your Own Key)

localStorage keys:

tpr_byok_active — active provider: 'internal' | 'anthropic' | 'openai' | 'grok' | 'ollama'
tpr_byok_key_{provider} — API key for each provider
tpr_byok_model_{provider} — selected model for each provider

BYOK flow in local_state-planning-scheme.php:

POST to /ask with context_only: true → get RAG context + pre-built prompt
Call provider API directly from browser with the prompt
Render answer with provider badge (purple for external, green for internal)

Supported providers: Anthropic (claude-sonnet-4-5), OpenAI (gpt-4o-mini), xAI Grok (grok-3-mini), local Ollama.

API status indicator

/js/api-status.js is included on every page:

<script src="/js/api-status.js" data-api="https://api.modulos.com.au"></script>

Polls /readyz every 30s (10s when down). Updates .status-dot and .nav-status-text. Nav HTML must include:

<div class="nav-status">
  <span class="status-dot"></span>
  <span class="nav-status-text">API live</span>
</div>

Feedback system

Each assistant message stores context as data-* attributes for scope-safe feedback:

div.dataset.query    = query;
div.dataset.scope    = scope;
div.dataset.provider = provider;
div.dataset.answer   = answer.replace(/<[^>]*>/g, '').substring(0, 4000);

window.feedback(msgId, verdict, btn) reads these attributes and POSTs to /feedback. Never rely on closure variables — use data-* attributes or localStorage reads.

Telemetry Database

SQLite at /home/modulos_llm/telemetry_data/telemetry.db

Tables

ask_logs — every query:

id, ts, sid, ip_hash, query, normalized, scope, allow_tps,
latency_ms, model, ok, topk_json, tokens_in, tokens_out, answer

feedback — thumbs up/down:

id, ts, sid, ip_hash, verdict, query, answer, note, model, scope, sources_json

events — generic telemetry events:

id, ts, type, sid, ua, ip_hash, data_json

Adding missing columns (migration pattern)

docker exec modulos-backend python3 - << 'EOF'
import sqlite3, os
db = sqlite3.connect(os.getenv('TPR_DB', '/data/telemetry.db'))
cols = [r[1] for r in db.execute("PRAGMA table_info(ask_logs)").fetchall()]
if 'answer' not in cols:
    db.execute("ALTER TABLE ask_logs ADD COLUMN answer TEXT DEFAULT ''")
    db.commit()
    print("✓ added")
db.close()
EOF

Useful queries

-- Today's queries with answers
SELECT ts, query, answer, latency_ms, scope, model
FROM ask_logs WHERE ts LIKE '2026-03-26%' ORDER BY ts DESC;

-- Thumbs-down for prompt improvement
SELECT query, answer, note, model, scope
FROM feedback WHERE verdict='down' ORDER BY ts DESC;

-- Satisfaction rate
SELECT
  SUM(CASE WHEN verdict='up' THEN 1 ELSE 0 END) as up,
  SUM(CASE WHEN verdict='down' THEN 1 ELSE 0 END) as down,
  ROUND(SUM(CASE WHEN verdict='up' THEN 1.0 ELSE 0 END) / COUNT(*) * 100) as pct
FROM feedback;

-- Repeated queries (cache candidates)
SELECT normalized, COUNT(*) as hits, ROUND(AVG(latency_ms)) as avg_ms
FROM ask_logs WHERE ok=1 GROUP BY normalized ORDER BY hits DESC LIMIT 20;

PHP/Composer

Installed packages (composer.json)

{
  "require": {
    "phpoffice/phpword": "^1.3",
    "phpmailer/phpmailer": "^6.9",
    "erusev/parsedown": "^1.7",
    "google/apiclient": "^2.15"
  }
}

Installing/updating dependencies

# Add a new package (use docker run to avoid platform issues)
docker run --rm -v /home/modulos_llm/public:/app composer:2 \
  require vendor/package --no-interaction --no-progress --ignore-platform-reqs

# Update all (run from project root)
docker compose run --rm composer update --no-interaction --ignore-platform-req=ext-gd

Known issue: phpoffice/phpword 1.4.0 requires ext-gd which isn't in the Composer container — always use --ignore-platform-req=ext-gd or --ignore-platform-reqs.

PHPMailer environment variables

SMTP_HOST=mail.modulos.com.au
SMTP_PORT=465           # 465=SSL, 587=TLS
SMTP_USER=no-reply@modulos.com.au
SMTP_PASS=<from .env>
SMTP_FROM=no-reply@modulos.com.au
SMTP_FROM_NAME=Tas Planning Assistant
NOTIFY_EMAIL=ben@modulos.com.au

Infrastructure

Reverse proxy

Nginx/Cloudflare sits in front. The web container is on port 2380, backend on 2281. Public URLs:

https://tasplanning.report → web container (port 2380)
https://api.modulos.com.au → backend container (port 2281)

Google Maps

gmaps-key.php serves the API key from env — never put it in page source. site-report.php loads Maps via:

const keyRes = await fetch('/gmaps-key.php');
const { key } = await keyRes.json();
await google.maps.importLibrary('places');

PassEnv GMAPS_API_KEY is written to Apache config in the web container startup command.

Google Docs export

OAuth2 credentials at /var/www/secure/service-account.json inside the web container. GDOC_PARENT_ID and GDOC_SHARE_EMAIL set in docker-compose environment.

Dashboard access control

/home/modulos_llm/public/.htaccess restricts dashboard.php to specific IPs:

<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteCond %{REMOTE_ADDR} ^YOUR\.IP\.HERE$
  RewriteRule ^dashboard\.php$ - [L]
  RewriteCond %{HTTP:X-Forwarded-For} ^YOUR\.IP\.HERE
  RewriteRule ^dashboard\.php$ - [L]
  RewriteRule ^dashboard\.php$ https://tasplanning.report/ [R=302,L]
</IfModule>

Key Conventions

Never do these

Never use sessionStorage for cross-tab data — use localStorage with a TTL
Never put API keys in PHP page output — use env vars + proxy endpoints
Never add Bootstrap to redesigned pages — use the custom dark design system
Never change num_ctx between requests — Ollama reloads the model on every change
Never put keep_alive inside Ollama options{} — it must be a top-level key
Never rely on closure variables in window.* functions — use data-* attributes
Never run composer install when lock file is out of sync — run composer update first
Never commit .env — it contains SMTP_PASS, API keys, secrets

Always do these

Always restart the backend after changing app.py — it's not hot-reloaded
Always use --ignore-platform-reqs with composer when adding packages
Always run apachectl -t after changing Apache config or .htaccess
Always escape output in PHP with htmlspecialchars($s, ENT_QUOTES, 'UTF-8')
Always use credentials: 'omit' for cross-origin fetch to the API
Always add new DB columns via ALTER TABLE migration, not by dropping/recreating
Always test context_only: true after prompt changes to verify RAG retrieval before LLM issues

Scope values

Value	Searches
`state_plus_local`	SPPs + selected council's LPS
`state_only`	SPPs only
`local_only`	Selected council's LPS only
`any`	All indexed documents

Prompt Engineering Notes

The current prompt structure (in do_ask() in app.py):

## AUTHORITY ORDER
1. SPP (baseline)
2. LPS for selected council (overrides SPP)
3. NCC (optional, building control only)
4. Australian Standards (optional)

## STRICT RULES
- Use ONLY information in CONTEXT
- Fallback phrase: "The provided context does not cover this..."
- Every claim must cite (filename, p.N)
- Distinguish A (Acceptable Solutions) vs P (Performance Criteria)

## OUTPUT FORMAT
- Markdown with ## headings
- Tables for setbacks, parking rates, multiple standards
- ## Sources section at end of every response

## CONTEXT
{retrieved chunks}

{format_guide}  # section-specific formatting instructions

## QUESTION
{query}

## ANSWER:

Debugging RAG vs prompt issues:

Use context_only: true to inspect what context was retrieved
If context contains the answer but model missed it → prompt issue
If context doesn't contain the answer → RAG/retrieval issue (increase top_k, check chunk boundaries)

Development Workflow

Making backend changes

# Edit the file
nano /home/modulos_llm/backend/app.py

# Restart to pick up changes
docker compose restart backend

# Watch logs
docker logs modulos-backend -f

Making frontend changes

# Edit directly — changes are live immediately
nano /home/modulos_llm/public/local_state-planning-scheme.php
# No restart needed

Adding a new PHP page

Create pagename.php in /home/modulos_llm/public/
Use the dark design system CSS variables
Include nav with .status-dot and .nav-status-text
Add <script src="/js/api-status.js" data-api="https://api.modulos.com.au"></script>
Add link to nav on other pages

Adding a new API endpoint

Add Pydantic model for request body if needed
Add endpoint function with @app.post("/path") and @limiter.limit("20/minute")
Add telemetry insert inside a try/except — never let logging break the endpoint
Restart backend: docker compose restart backend

Ingesting new planning documents

# Copy PDFs to backend folder
cp new-lps.pdf /home/modulos_llm/backend/

# Run ingestion (adjust args for corpus/council)
docker exec modulos-backend python3 ingest.py \
  --pdf new-lps.pdf \
  --corpus lps \
  --council launceston

External Services

Service	URL	Purpose
Ollama	`http://192.168.8.73:11434`	LLM inference (Windows machine, RTX 4070 Super)
Qdrant	`http://qdrant:6333` (internal)	Vector search
TPSO Viewer	`https://tpso.planning.tas.gov.au/tpso/external/planning-scheme-viewer/30`	SPP viewer (iframe)
LIST GIS	`https://services.thelist.tas.gov.au/arcgis/rest/services`	Property/parcel data
Google Maps	`https://maps.googleapis.com`	Address autocomplete in site-report
Anthropic API	`https://api.anthropic.com/v1/messages`	BYOK Claude
OpenAI API	`https://api.openai.com/v1/chat/completions`	BYOK GPT
xAI API	`https://api.x.ai/v1/chat/completions`	BYOK Grok

Current Model

Model: llama3.1:8b-instruct-q4_K_M
Hardware: NVIDIA GeForce RTX 4070 Super (12GB VRAM), Windows host
VRAM usage: ~4.5GB model + ~1GB KV cache at num_ctx=6144
Typical latency: 4-12s depending on answer length
Memory bandwidth: saturated at ~95-98% during inference (bandwidth-bound, not compute-bound)
Upgrade path: RTX 4090 (24GB) would allow llama3.1:70B-Q4 and roughly 2× throughput

Security Notes

IP hashes in telemetry use HMAC-SHA256 with TPR_IP_SECRET — not reversible
BYOK keys stored in browser localStorage only — never sent to our servers
gmaps-key.php checks HTTP_HOST/HTTP_ORIGIN before serving the Maps key
dashboard.php restricted by .htaccess IP allowlist
DEMO_TOKEN in docker-compose is for optional API gating (currently disabled)
.env file contains all secrets — never commit to version control

CLAUDE.md 22 KB Histórico Raw