hlsl-perf logging: what separates "readable" from "usable"
Short version: hlsl-perf needs almost no tuning at small and medium scale — the point where it starts to hurt is much further out than most people assume. Full measurements below.
On trade-offs, my view is this: if nobody on the team owns this area long-term, do not introduce a second mechanism. With two coexistence you first have to work out which one is even in play when things break, and that costs far more than the performance you saved.
-- The query that broke: a full scan over 20M rows. -- A composite index took P99 from 1.8s down to 42ms. SELECT id, title, created_at FROM posts WHERE community_id = ? AND status = 1 ORDER BY score DESC LIMIT 20;
Worth noting: the official docs do cover this, just in a very inconspicuous spot. I only found it reading the source comments, where the author explains the reasoning — roughly "so that it degrades into predictable behaviour in extreme cases".
We also fixed monitoring along the way: replaced average-based alerts with percentiles and split them per endpoint. False alerts dropped by about seventy percent and the on-call rotation visibly cheered up.
What genuinely surprised me was the tail. The average looked great while P99 jumped by an order of magnitude past some threshold. The cause was not hlsl-perf itself but our upstream connection reuse — the load test traffic was too clean and hid the long-tail requests.