initial commit

1 files changed, 325 insertions, 0 deletions

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..d9baf54
--- /dev/null
+++ b/README.md
@@ -0,0 +1,325 @@
+# srht-contrib
+
+`srht-contrib` is a small Python service that polls SourceHut activity, normalizes it into one internal event model, stores it in SQLite, and exposes a contribution-calendar JSON API that an iOS app can render directly.
+
+The current V1 is intentionally narrow and production-oriented:
+
+- FastAPI JSON API only
+- SQLite-backed persistence
+- polling-based ingestion
+- complete `todo.sr.ht` ingestion path
+- practical `git.sr.ht` commit ingestion for tracked repositories
+- API-key protection for `/api/*`
+- Alembic-managed schema migrations
+
+## What It Does
+
+The service collects SourceHut activity from one or more sr.ht GraphQL services, turns those records into a canonical event shape, aggregates activity by day, and returns zero-filled calendar ranges so the client never has to patch missing dates.
+
+Example use cases:
+
+- render a GitHub-style contribution grid in an iOS app
+- show total score and streak stats for a SourceHut user
+- poll recent activity on a schedule or trigger polling manually
+
+## Architecture Overview
+
+The code is split into small, testable layers:
+
+- `src/srht_contrib/config.py`: environment-driven settings and event weights
+- `src/srht_contrib/db.py`: SQLAlchemy engine/session setup and app-scoped DB access
+- `src/srht_contrib/models.py`: ORM models for normalized events, sync state, aliases, and tracked repos
+- `src/srht_contrib/services/srht_client.py`: generic SourceHut GraphQL client with error handling and simple retries
+- `src/srht_contrib/services/todo.py`: `todo.sr.ht` ingestion and normalization
+- `src/srht_contrib/services/git.py`: `git.sr.ht` tracked-repository commit ingestion
+- `src/srht_contrib/services/aggregator.py`: per-day aggregation and streak/stat calculations
+- `src/srht_contrib/jobs/poller.py`: repeated-safe polling and idempotent persistence
+- `src/srht_contrib/api/`: FastAPI routes, auth dependencies, and repository management
+- `alembic/`: schema migration environment and versioned migrations
+
+## Supported sr.ht Services
+
+### Implemented
+
+- `todo.sr.ht`
+- `git.sr.ht`
+
+Current normalized event types:
+
+- `ticket_created`
+- `ticket_comment`
+- `ticket_closed`
+- `commit`
+
+`todo.sr.ht` uses a feed-first strategy and falls back to crawling the authenticated user’s trackers, tickets, and ticket events when the top-level activity feed is empty. `git.sr.ht` polls tracked repositories for recent commits on the default branch.
+
+## Canonical Event Model
+
+All ingestion services normalize external activity into this shape:
+
+- `service`
+- `event_type`
+- `actor`
+- `repo_name`
+- `resource_id`
+- `external_uid`
+- `occurred_at`
+- `weight`
+- `raw_payload_json`
+
+The database enforces uniqueness on `(service, external_uid)` so polling is safe to repeat.
+
+Tracked git repositories are persisted in the `tracked_repositories` table and stored in canonical `~owner/repo` form. The poller seeds that table from `GIT_TRACKED_REPOSITORIES`, and repositories can also be created, updated, and deleted through the API.
+
+## Configuration
+
+Environment variables:
+
+- `API_KEY`: required header token for all `/api/*` routes via `X-API-Key`
+- `ENABLE_SCHEDULER`: defaults to `false`; enables in-process polling when set to `true`
+- `SRHT_TOKEN`: bearer token for SourceHut GraphQL
+- `TODO_SRHT_ENDPOINT`: defaults to `https://todo.sr.ht/query`
+- `GIT_SRHT_ENDPOINT`: defaults to `https://git.sr.ht/query`
+- `DATABASE_URL`: defaults to `sqlite:///./srht_contrib.db`
+- `DEFAULT_ACTOR`: actor used by the scheduled poll job
+- `POLL_INTERVAL_SECONDS`: scheduler interval in seconds
+- `ACTOR_ALIASES_JSON`: optional JSON object for actor/email/display-name alias mapping
+- `GIT_TRACKED_REPOSITORIES`: optional JSON array of repository names or `owner/repo` strings for git polling
+
+Example `.env`:
+
+```env
+API_KEY=replace-me
+ENABLE_SCHEDULER=false
+SRHT_TOKEN=replace-me
+TODO_SRHT_ENDPOINT=https://todo.sr.ht/query
+GIT_SRHT_ENDPOINT=https://git.sr.ht/query
+DATABASE_URL=sqlite:///./srht_contrib.db
+DEFAULT_ACTOR=~ccleberg
+POLL_INTERVAL_SECONDS=900
+ACTOR_ALIASES_JSON={"~ccleberg":["[email protected]","Chris Cleberg"]}
+GIT_TRACKED_REPOSITORIES=["Hutch","~ccleberg/cleberg.net"]
+```
+
+## Local Run Instructions
+
+### 1. Create a virtual environment and install dependencies
+
+Using `uv`:
+
+```bash
+uv venv
+source .venv/bin/activate
+uv pip install -e ".[dev]"
+```
+
+Using `pip`:
+
+```bash
+python3.12 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+### 2. Configure environment
+
+```bash
+cp .env.example .env
+```
+
+Set at least:
+
+- `API_KEY`
+- `SRHT_TOKEN`
+- `DEFAULT_ACTOR`
+- `GIT_TRACKED_REPOSITORIES` if you want git commit ingestion
+
+### 3. Run database migrations
+
+```bash
+alembic upgrade head
+```
+
+### 4. Run the API
+
+```bash
+uvicorn srht_contrib.main:app --reload
+```
+
+## Manual Polling
+
+Manual polling is exposed as an API endpoint:
+
+```bash
+curl -X POST "http://127.0.0.1:8000/api/contributions/poll?actor=~ccleberg" \
+  -H "X-API-Key: replace-me"
+```
+
+Example response:
+
+```json
+{
+  "actor": "~ccleberg",
+  "inserted_events": 3,
+  "services": ["todo", "git"]
+}
+```
+
+Scheduled polling only runs when `ENABLE_SCHEDULER=true` and uses `DEFAULT_ACTOR`.
+
+For `git.sr.ht`, tracked repositories are configured via `GIT_TRACKED_REPOSITORIES`. Entries may be either:
+
+- `"Hutch"` for a repository owned by `DEFAULT_ACTOR`
+- `"~ccleberg/cleberg.net"` for an explicit owner/repository pair
+
+## API Endpoints
+
+### Health
+
+```bash
+curl "http://127.0.0.1:8000/health"
+```
+
+Response:
+
+```json
+{"status":"ok"}
+```
+
+### Contribution Calendar by Year
+
+```bash
+curl "http://127.0.0.1:8000/api/contributions/~ccleberg?year=2026" \
+  -H "X-API-Key: replace-me"
+```
+
+### Contribution Calendar by Date Range
+
+```bash
+curl "http://127.0.0.1:8000/api/contributions/~ccleberg?from=2026-01-01&to=2026-03-30" \
+  -H "X-API-Key: replace-me"
+```
+
+Example response:
+
+```json
+{
+  "actor": "~ccleberg",
+  "from": "2026-01-01",
+  "to": "2026-03-30",
+  "days": [
+    {"date": "2026-03-28", "count": 3, "score": 3.5},
+    {"date": "2026-03-29", "count": 0, "score": 0.0},
+    {"date": "2026-03-30", "count": 7, "score": 8.25}
+  ]
+}
+```
+
+### Contribution Stats
+
+```bash
+curl "http://127.0.0.1:8000/api/contributions/~ccleberg/stats?year=2026" \
+  -H "X-API-Key: replace-me"
+```
+
+Example response:
+
+```json
+{
+  "actor": "~ccleberg",
+  "from": "2026-01-01",
+  "to": "2026-12-31",
+  "total_events": 42,
+  "total_score": 37.5,
+  "active_days": 18,
+  "longest_streak": 5,
+  "current_streak": 2
+}
+```
+
+### Tracked Repositories
+
+List tracked repositories:
+
+```bash
+curl "http://127.0.0.1:8000/api/repositories?actor=~ccleberg" \
+  -H "X-API-Key: replace-me"
+```
+
+Create a tracked repository:
+
+```bash
+curl -X POST "http://127.0.0.1:8000/api/repositories" \
+  -H "X-API-Key: replace-me" \
+  -H "Content-Type: application/json" \
+  -d '{"actor":"~ccleberg","repo_name":"Hutch"}'
+```
+
+Get, update, and delete a tracked repository:
+
+```bash
+curl "http://127.0.0.1:8000/api/repositories/1" \
+  -H "X-API-Key: replace-me"
+
+curl -X PATCH "http://127.0.0.1:8000/api/repositories/1" \
+  -H "X-API-Key: replace-me" \
+  -H "Content-Type: application/json" \
+  -d '{"repo_name":"~ccleberg/cleberg.net"}'
+
+curl -X DELETE "http://127.0.0.1:8000/api/repositories/1" \
+  -H "X-API-Key: replace-me"
+```
+
+## Event Weighting
+
+Weights live in `src/srht_contrib/config.py` so they are easy to tune without touching aggregation code:
+
+- `commit`: `1.0`
+- `ticket_created`: `1.0`
+- `ticket_comment`: `0.5`
+- `ticket_closed`: `0.75`
+- `build_started`: `0.25`
+- `build_passed`: `0.25`
+
+## Testing
+
+Run the test suite with:
+
+```bash
+pytest
+```
+
+Covered areas:
+
+- health endpoint
+- API key enforcement
+- calendar aggregation
+- zero-filled ranges
+- stats calculations
+- invalid date handling
+- idempotent ingestion
+- todo feed fallback traversal
+- repository CRUD and normalization
+- SourceHut error mapping
+- git commit alias normalization
+- Alembic upgrade path
+
+## SourceHut Schema Assumptions
+
+The SourceHut-specific assumptions are isolated to the service modules:
+
+- `src/srht_contrib/services/todo.py` uses the authenticated `events(cursor)` feed first, then falls back to tracker/ticket event traversal for reliable contribution discovery.
+- `src/srht_contrib/services/git.py` uses the documented repository `log(cursor)` query against tracked repositories and attributes commits through the configured alias map.
+
+## Known Limitations
+
+- `git.sr.ht` polling is limited to repositories listed in `GIT_TRACKED_REPOSITORIES`
+- scheduled polling runs in-process, so it is not a distributed scheduler
+- alias management is config-driven; there is no alias CRUD API yet
+- current deployment model is trusted-operator V1, not a public multi-tenant service
+
+## Recommended Next Steps
+
+1. Add alias-management APIs or seed files for stronger actor identity mapping.
+2. Add more SourceHut services such as `builds.sr.ht` and `lists.sr.ht`.
+3. Move scheduled polling into an external worker if the deployment grows past a single process.