add durable actor queueing and staggered discovery imports

1 files changed, 18 insertions, 0 deletions

diff --git a/README.md b/README.md
index 656f1c7..695d222 100644
--- a/README.md
+++ b/README.md
@@ -83,6 +83,9 @@ Environment variables:
 - `DATABASE_URL`: defaults to `sqlite:///./srht_contrib.db`
 - `DEFAULT_ACTOR`: actor used by the scheduled poll job
 - `POLL_INTERVAL_SECONDS`: scheduler interval in seconds
+- `DISCOVERY_BATCH_SIZE`: max number of due actors to process per scheduler pass
+- `INDEXED_ACTOR_REPOLL_SECONDS`: how long to wait before re-polling an already indexed actor
+- `DISCOVERY_ERROR_BACKOFF_SECONDS`: base retry delay after a failed scheduled poll
 - `ACTOR_ALIASES_JSON`: optional JSON object for actor/email/display-name alias mapping
 - `GIT_TRACKED_REPOSITORIES`: optional JSON array of repository names or `owner/repo` strings to union into git polling
 
@@ -97,6 +100,9 @@ GIT_SRHT_ENDPOINT=https://git.sr.ht/query
 DATABASE_URL=sqlite:///./srht_contrib.db
 DEFAULT_ACTOR=~your-user
 POLL_INTERVAL_SECONDS=900
+DISCOVERY_BATCH_SIZE=5
+INDEXED_ACTOR_REPOLL_SECONDS=21600
+DISCOVERY_ERROR_BACKOFF_SECONDS=3600
 ACTOR_ALIASES_JSON={"~your-user":["[email protected]","Your Name"]}
 GIT_TRACKED_REPOSITORIES=["your-repo","~your-user/your-site"]
 ```
@@ -167,6 +173,18 @@ Example response:
 
 Scheduled polling only runs when `ENABLE_SCHEDULER=true`. The scheduler seeds `DEFAULT_ACTOR` as an initial known actor, runs one poll immediately at startup, and public contribution reads register additional actors for later background polling and one-year backfill.
 
+The scheduler now drains actors gradually instead of polling every tracked actor on every pass. It only claims due actors, up to `DISCOVERY_BATCH_SIZE` per run, then reschedules indexed actors with `INDEXED_ACTOR_REPOLL_SECONDS` and failed actors with backoff based on `DISCOVERY_ERROR_BACKOFF_SECONDS`.
+
+## Bulk Enqueue Without Immediate Indexing
+
+To durably queue a large username list without polling it immediately:
+
+```bash
+srht-enqueue-actors srht_usernames.txt --stagger-seconds 300
+```
+
+This command stores usernames in `tracked_actors`, marks them queued, and spaces out their first eligible poll time. With `--stagger-seconds 300`, a file of 15,771 users will be spread across roughly 54.8 days before becoming due for first poll.
+
 For `git.sr.ht`, owned repositories are auto-discovered for the actor. `GIT_TRACKED_REPOSITORIES` can still be used to union in extra repositories. Entries may be either:
 
 - `"Hutch"` for a repository owned by `DEFAULT_ACTOR`