ai/granite-4.0-h-small repository overview

⁠Granite-4.0-h-Small

⁠Description

Granite-4.0-H-Small is a 32B parameter long-context instruct model finetuned from Granite-4.0-H-Small-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging. Granite 4.0 instruct models feature improved instruction following (IF) and tool-calling capabilities, making them more effective in enterprise applications.

⁠Characteristics

⁠Available model variants

¹: VRAM estimated based on model characteristics.

⁠Use this AI model with Docker Model Runner

docker model run ai/granite-4.0-h-small

⁠Considerations

• Optimized for instruction following, tool/function calling, and long-context (up to 128K tokens) scenarios.
• Strong generalist capabilities: summarization, classification, extraction, QA/RAG, coding, function-calling, and multilingual dialogue.
• Multilingual: best performance in English; a few-shot approach or light finetuning can help close gaps for other languages.
• Safety & reliability: despite alignment, the model can still produce inaccurate or biased outputs—apply domain-specific evaluation and guardrails.
• Infrastructure note: trained on NVIDIA GB200 NVL72 at CoreWeave; use acceleration libraries (e.g., accelerate, optimized attention/KV cache settings) for efficient inference.

⁠Benchmark performance

⁠Links

• https://www.ibm.com/granite⁠
• https://www.ibm.com/granite/docs/⁠
• https://ibm.biz/granite-learning-resources⁠