Buck comments on Sam Marks’s Shortform

[Buck]

If you have kv caching between inference calls, this shouldn’t be a big cost.