Inference Cost Optimization

Techniques like quantization, caching, batching, or model selection to reduce compute resource consumption and cost per generate