Llama 3 Groq 8B Tool Use

Technical Overview

What "Tool Use" Means

When you describe available functions (tools) to this model, it can decide when to call them and generate properly formatted JSON arguments. For example:

{"name": "get_weather",
 "arguments": {
   "location": "Tokyo",
   "unit": "celsius"
 }}

Your application then executes the function and feeds the result back to the model for a natural language response.

Despite the "Groq" Name...

This model does NOT require Groq hardware. "Groq" in the name means Groq Inc. created the fine-tune, not that it only runs on their hardware. It works on any CPU/GPU that can run Llama 3 8B — MacBooks, gaming PCs, or cloud servers. You can optionally use the Groq Cloud API for faster inference, but it's not required.

Berkeley Function-Calling Leaderboard (BFCL)

BFCL is the standard benchmark for evaluating function-calling ability. It tests whether models can correctly select functions, format arguments, and handle edge cases.

Source for the Groq rows: Groq's model card, reporting the BFCL version current at release. For every other model, use the Berkeley Function-Calling Leaderboard directly: it is re-scored with each BFCL version, so copied numbers go stale. Simple FC = single function call, Multiple FC = selecting among functions, Parallel FC = calling multiple functions at once.

General Capabilities (Same as Llama 3 8B)

Since this is a fine-tune, general knowledge is close to Llama 3 8B. Meta's model card figures for Llama 3 8B (not re-measured for the Groq fine-tune):

Tool-use fine-tuning may slightly affect general benchmarks. Use Llama 3 8B Instruct for general-purpose tasks.

VRAM by Quantization

Sizes are the Ollama registry downloads; the VRAM column is file + 0.5 GB of cache at 4K + about 1 GB of buffers, worked out rather than measured. The default llama3-groq-tool-use:8b is Q4_0, 4.66GB. For tool-use models, Q4_K_M or higher is recommended. Lower quantizations can cause JSON formatting errors in function calls, making the output unparseable.

Limitations to Know

• - Parallel function calls (calling 2+ tools at once) are less reliable than single calls; test with your own tools
• - Complex nested arguments (objects inside objects) can malform
• - Long multi-step chains (> 5 tool calls) may lose context or hallucinate tool names
• - JSON formatting occasionally breaks at low quantizations (use Q4_K_M+)

Groq API Example

# pip install groq
from groq import Groq

client = Groq(api_key="your_groq_api_key")

response = client.chat.completions.create(
    model="llama3-groq-8b-8192-tool-use-preview",
    messages=[
        {"role": "system", "content": "You are a helpful assistant with tool access."},
        {"role": "user", "content": "What's the weather in London?"}
    ],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
                },
                "required": ["location"]
            }
        }
    }],
    tool_choice="auto"
)

print(response.choices[0].message)

Get a free API key at console.groq.com. Free tier includes generous rate limits for development.

Local vs Cloud: When to Use Each

When to Choose Each