When the fastest PHP option is the wrong choice
Five cases from the ASAMB dataset where semantics, safety or failure behavior matter more than the lowest timing number.
A benchmark table makes one row visually attractive: the row with the smallest median. That row is evidence, but it is not yet a recommendation.
Across the current ASAMB suite, several comparisons change behavior as well as speed. Choosing a variant without naming that behavioral difference can turn a technically correct measurement into poor engineering advice.
Escaping is a security boundary, not a cosmetic formatting step
The HTML escaping study compares output modes that do not provide identical protection. A faster row cannot be promoted when it leaves quote characters unescaped in a context where quotes delimit attributes. The relevant question is whether the chosen flags match the exact HTML context.
The useful result is therefore conditional: measure the cost after the safe output rule has been selected. Performance cannot be used to negotiate away a required encoding boundary.
Correctness gate: select the safe output behavior first; compare timing only among variants that satisfy it.
Atomic publication is about what readers can observe during failure
The file-publication experiment recorded Temporary file + rename as the lower-median path on this host, with an observed spread of 43.63%. Even had direct overwrite been faster, timing alone would not settle the design.
A temporary file followed by rename is intended to prevent readers from observing a partly written JSON document. The engineering value appears during interruption, concurrent reads or process failure—not in the happy-path timing loop. That failure behavior is the reason to treat atomic replacement as a correctness property.
Token representation changes interfaces and storage
Hex and base64url can encode the same random bytes, yet they create different string lengths and interoperability expectations. The shortest or fastest representation is not automatically compatible with an existing database column, URL format, log parser or external protocol.
Entropy must come from the random bytes. Encoding only changes the representation. A sound decision documents the required alphabet, padding policy, fixed length and decoding behavior before comparing isolated encoding cost.
Protocol compatibility outranks a local query-string winner
RFC 1738 and RFC 3986 query encoding differ in how spaces and selected characters are represented. A local timing difference does not authorize changing the encoding expected by a signing algorithm, OAuth flow, cache key or receiving API.
When the protocol fixes an encoding convention, compatibility has already made the decision. The benchmark remains useful for capacity planning, but not for changing a contract.
A smaller payload can cost far more CPU for very few remaining bytes
In the current gzip run, level 1 produced 9,532 bytes bytes at a median of 380.10 µs, while level 9 produced 6,369 bytes bytes at 2.49 ms. The last 3,163 bytes saved came with a large increase in compression work.
That trade-off can be acceptable for an asset compressed once and served many times. It can be wasteful for dynamic output compressed on every request. The correct answer depends on cacheability and where the CPU cost is paid.
A practical decision rule
Before selecting a benchmark winner, write down the invariant the code must preserve: output safety, protocol compatibility, atomic visibility, entropy, data fidelity or maintainability. Eliminate any variant that violates that invariant. Only then compare the surviving rows on latency, throughput, memory and size.
This ordering is deliberately conservative. It prevents a microbenchmark from turning a local speed difference into a recommendation that weakens the system it was meant to improve.
- Define required behavior and failure semantics.
- Confirm the variants perform sufficiently equivalent work.
- Compare complete costs, including output size and downstream processing.
- Re-test in the application path before changing production code.