Quantization Quantization trades off model precision for smaller memory footprint, allowing large models to be run on a wider range of devices. Contents: Supported_Hardware Auto_Awq Bnb Bitblas Gguf Gptqmodel Int4 Int8 Fp8 Modelopt Quark Quantized_Kvcache Torchao