IBM Granite 4.2: A Detailed Look at How Agentic LLMs Are Built

IBM has updated its Granite models to version 4.2, with the focus on reasoning and agentic work. The point is not a bigger parameter count. It is post-training (SFT and multi-stage RL on top of the Granite 4.1 base models) to build models that can use tools, work in a terminal and even browse the web. The update shows how LLMs are built that can do more than produce text: they can act in the digital world.

Agency as the foundation of the architecture

The main difference in Granite 4.2 is that the models (especially the 8B and 30B versions) go through additional reinforcement learning (agentic RL). This training lets them act as agents instead of following instructions step by step: they call tools, edit and run code, drive terminals and search the internet. This paradigm moves the LLM from the role of a passive conversation partner to an active problem solver capable of autonomous work.

graph TD
  A[Запрос пользователя] --> B{"LLM (Thinking Mode)"}
  B --> C{Tool Calling / Planning}
  C --> D[Инструменты: Terminal, API, Web Search]
  D --> E[Наблюдение и результат]
  E --> B

In practice: thinking modes and tool calling

Granite 4.2 supports several "thinking" modes: `thinking`, `non-thinking` and `low-effort`. These modes let the model adjust how it reasons depending on the task, saving resources and time. On top of that, the models natively support tool calling in the OpenAI Function Calling format, which makes integration with external systems and APIs much simpler. The model can "think", plan actions and interact with the outside environment.

Architecture and training

Granite 4.2 models are built on a decoder-only dense transformer architecture with a number of optimizations:

• Attention: Grouped Query Attention (GQA): 8B and 30B have 32 attention heads, 3B has 40, with 8 KV heads.
• Positional embedding: Rotary Position Embedding (RoPE) with θ = 10,000,000.
• Feed-Forward: MLP with SwiGLU activation.
• Normalization: RMSNorm (ε = 1e-5).
• Embeddings: Separate input/output embeddings (untied).
• Precision: bfloat16.

graph LR
  subgraph G["Модели IBM Granite 4.2"]
    A["3B dense<br/>embedding 2560, 40 слоев<br/>контекст 131072"]
    B["8B dense<br/>embedding 4096, 40 слоев<br/>контекст 131072"]
    C["30B dense<br/>embedding 4096, 64 слоя<br/>контекст 131072"]
    A --- B --- C
  end

Pre-training and Fine-tuning

The Granite 4.1 base models that 4.2 is built on were trained from scratch on about 15 trillion tokens using a five-phase strategy. This approach gradually extends the context window to 512K tokens and shifts from broad web data to curated high-quality sources.

Supervised fine-tuning (SFT) turns the base model into an assistant that can follow instructions, reason and use tools. The SFT data mix contains 7.2 million samples (~100B tokens). 31.6% of it is agentic data covering software development (69%), tool calling (12.1%), terminal use (8.0%), math (3.5%), search (0.8%) and actions (0.2%). A separate SFT phase for the 30B model further strengthens agentic coding.

pie title SFT Data Mixture
    "Agentic Corpus (31.6%)" : 31.6
    "Non-Agentic Corpus (68.4%)" : 68.4

Why this matters for business

For a business, adopting models like these means you can build more autonomous and reliable AI systems. Models that can reason and work with tools reduce the need to hard-code logic and let you automate more complex tasks, which cuts operating costs and speeds up decision-making. This is the shift from AI as an advisor to AI as the one doing the work.

My take

IBM shows that billions of parameters are not enough to build useful LLM agents. You also need targeted reinforcement learning on data that imitates real interaction with tools. This supports my position that the harness and the pipelines matter more than the model itself, and that LLM "thinking" is basically a sequence of tool calls and observations. It's important to understand that "agency" does not appear on its own. It takes deliberate training and engineering work.

What to test in practice

• How the `thinking`/`non-thinking` modes affect latency and accuracy in real agentic pipelines. I want to understand when it is worth letting the model "think" longer and when a fast answer is enough.
• How well `agentic RL` holds up on complex tasks that need many tool calls and iterations (for example, multi-step problem solving or business process automation).
• Compare the ability to "self-reflect" and correct errors in the 8B and 30B models when they hit unexpected results or tool failures.
• Test `tool calling` against different APIs that imitate internal accounting systems or external services, to assess how flexible and reliable the integration is.

How to tell it worked

Success will be measured by output quality, and also by less human intervention in the agents' work, fewer errors when using external tools, and the model's ability to adapt to new tasks without retraining. My success criterion: the agent consistently handles a task that needs 3+ tool calls, with minimal "hallucinations" in the action sequence that would require a human to step in. That means completing the task without constant supervision.

Sources

• Granite 4.2 LLMs: How They're Built - Hugging Face