Posts by Huan Li

An AITER Pin Keyed by Kernel Text: Faster Glue Than the Inductor Default

GEMM and attention kernels already have offline tuning paths. The kernels around them often do not. A compiled inference graph still launches a long tail of small Triton kernels for casts, concatenations, normalizations, activations, gathers, and histogram-style updates. TorchInductor generates those kernels. With autotune on, it times a candidate list and writes the winner into a cache keyed by a fusion number. Figure 1, below, is why that cache is not what ships.

Read more ...