From 5763dc3cfeadf491d6eda3b6a91623f86396a134 Mon Sep 17 00:00:00 2001 From: Christian Cleberg Date: Tue, 14 Jul 2026 20:22:26 -0500 Subject: feat: throttle and time out outbound DeviantArt requests DeviantArt fronts its API with CloudFront + WAF, which bans egress IPs that hit it too hard. devianter issues requests with a bare http.Client, so unbounded concurrent handlers each pulled ~150-200 KB of JSON, which both risked a ban and could exhaust the process under a bot flood. Wrap the default transport to bound rate and concurrency for deviantart.com and add timeouts. Other hosts (wixmp image CDN) pass straight through, so media stays fast, and ProxyFromEnvironment is preserved so HTTPS_PROXY egress still works. --- main.go | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'main.go') diff --git a/main.go b/main.go index 55622d0..c57ad7a 100755 --- a/main.go +++ b/main.go @@ -17,6 +17,10 @@ func main() { app.ExecuteConfig() static.CopyTemplatesToMemory() + // Rate/concurrency-limit + time-out outbound DeviantArt requests so bot floods + // can't exhaust the process or get our egress IP banned by CloudFront/WAF. + app.InstallDAThrottle() + go func() { for { err := devianter.UpdateCSRF() -- cgit v1.2.3