Dolphin 2.6 Mixtral 8x7B:
Ollama Tags, VRAM and Setup
Updated: November 2, 2026
Dolphin 2.6 Mixtral 8x7B is Eric Hartford's uncensored fine-tune of Mistral's Mixtral 8x7B. On Ollama it is dolphin-mixtral:8x7b-v2.6, a 26.4 GB 4-bit download that needs about 28 GB of memory, so it fits a 32 GB GPU or a 48 GB Mac, and runs split across GPU and RAM on a 24 GB card. Plain dolphin-mixtral:8x7b pulls the later v2.7 build, the same size.
🔧 Mixture-of-Experts Architecture
🎯 Expert Network Design
⚡ Performance Advantages
The mixture-of-experts architecture represents a significant advancement in transformer model design. Unlike traditional dense models where all parameters participate in processing every token, MoE models activate only a subset of experts for each input, achieving better computational efficiency.
Dolphin 2.6 keeps the base Mixtral 8x7B architecture unchanged and changes only the behaviour, through a qLoRA fine-tune on a dataset its author filtered to remove refusals and alignment. The result answers more requests than Mixtral Instruct does, with no safety layer of its own.
In Mistral's own evaluation (arXiv:2401.04088), the base Mixtral 8x7B matched or beat Llama 2 70B on most benchmarks while using about a fifth of the active parameters per token. The catch for local use is memory, not compute: the 4-bit file is 26.4 GB, against 38.9 GB for Llama 2 70B.
🎓 Training Methodology & Fine-Tuning
Base model scores (Mixtral 8x7B)
Mistral's figures for the base model, Table 2 of arXiv:2401.04088. The Dolphin model card publishes no scores of its own.
Performance Metrics
🔬 Fine-Tuning Process
📊 Training Data & Methodology
The model card describes a qLoRA fine-tune of Mixtral 8x7B run with Axolotl: 1.5 epochs, about three days on four A100 GPUs, at a 16K sequence length. Its author notes it was trained with a lot of coding data and is obedient but not DPO-tuned, so the system prompt matters.
The model uses the ChatML prompt format (system, user and assistant turns wrapped in im_start and im_end markers), which Ollama's template applies for you. The weights are released under Apache 2.0.
Uncensored means the refusals were filtered out of the training data, not that some other safety system replaces them. The card is explicit: “You are advised to implement your own alignment layer before exposing the model as a service.”
📈 Performance Benchmarks
MMLU 5-shot, as published by each vendor
🎯 Technical Performance Analysis
No benchmark scores have been published for Dolphin 2.6 Mixtral. The model card gives none, and this site has not run any. What can be said: it starts from Mixtral 8x7B, which Mistral reports at 70.6 MMLU, 40.2 HumanEval, 60.7 MBPP and 74.4 GSM8K.
Fine-tunes like this one mainly change behaviour (fewer refusals, a ChatML chat style, more coding data) rather than raw knowledge, so expect broadly base-Mixtral capability. If a score matters for your use, run your own evaluation on your own prompts.
| Model | Ollama tag | Download | Memory at 4K | Parameters | Licence | Note |
|---|---|---|---|---|---|---|
| Dolphin 2.6 Mixtral 8x7B | dolphin-mixtral:8x7b-v2.6 | 26.4 GB | ~28 GB | 46.7B total, 12.9B active | Apache 2.0 (model card) | Uncensored; ChatML |
| Nous Hermes 2 Mixtral 8x7B | nous-hermes2-mixtral:8x7b | 26.4 GB | ~28 GB | 46.7B total, 12.9B active | Apache 2.0 | Nous Research fine-tune; ChatML |
| Mixtral 8x7B Instruct | mixtral:8x7b | 26.4 GB | ~28 GB | 46.7B total, 12.9B active | Apache 2.0 | MMLU 70.6 (base, Mistral) |
| Llama 2 70B | llama2:70b | 38.9 GB | ~41 GB | 70B dense | Llama 2 Community | MMLU 68.9 (Meta) |
| Claude 3 Haiku | cloud API only | n/a | n/a | not published | Paid API | MMLU 75.2 (Anthropic) |
Nous Hermes 2 Mixtral 8x7B is the other well-known Mixtral fine-tune of the same era, from Nous Research, also in ChatML. It is the same 26.4 GB on Ollama, so the hardware advice on this page applies to it unchanged. Mixture-of-experts models get no speed figure on this site: their speed depends on how many experts each token touches and where they sit in memory, so the simple bandwidth ceiling does not apply.
Dolphin 2.6 quantisations on Ollama
| Tag | Download | Memory at 4K |
|---|---|---|
dolphin-mixtral:8x7b-v2.6-q3_K_M | 22.6 GB | ~24 GB |
dolphin-mixtral:8x7b-v2.6 (q4_0) | 26.4 GB | ~28 GB |
dolphin-mixtral:8x7b-v2.6-q4_K_M | 28.5 GB | ~30 GB |
dolphin-mixtral:8x7b-v2.6-q5_K_M | 33.2 GB | ~35 GB |
dolphin-mixtral:8x7b-v2.6-q8_0 | 49.6 GB | ~51 GB |
Sizes from the Ollama registry. Memory = file + 0.5 GB key-value cache at 4K + about 1 GB of buffers; worked out, not measured.
🔬 Benchmark Analysis
📊 Performance Metrics
- • No published scores for Dolphin 2.6 itself
- • 70.6 MMLU for the base Mixtral 8x7B (Mistral)
- • 40.2 HumanEval / 60.7 MBPP for the base model (Mistral)
- • 74.4 GSM8K for the base model (Mistral)
⚡ Efficiency Metrics
- • 12.9B active parameters per token
- • 46.7B total parameters in model
- • 26.4GB Ollama download at 4-bit
- • ~28 GB of memory at a 4K context
⚡ Installation & Deployment
System Requirements
Install Ollama
Download and install Ollama for local model deployment
Download Model
Pull Dolphin 2.6 (26.4 GB, 4-bit). dolphin-mixtral:8x7b without a suffix pulls the later v2.7
Test Installation
Verify the model is working correctly
Keep Memory in Check
One loaded model and one request at a time leaves the most memory for this 26 GB model
🔧 Deployment Configuration
🚀 Performance Settings
- • In Ollama, set
num_ctx(for example 8192) to raise the 4K default - • Set
num_gputo control how many layers go on the GPU when it does not all fit - • In llama.cpp the equivalents are
--ctx-sizeand--n-gpu-layers - • Set
num_threadto your physical core count for the CPU share
⚙️ Model Configuration
- • Temperature 0.7 for balanced creativity
- • Top-p 0.9 for diverse responses
- • Repeat penalty 1.1 for natural flow
- • Context: fine-tuned at 16K (base Mixtral supports 32K)
💻 Terminal Interface
🎯 Use Cases & Applications
💻 Software Development
Code generation, debugging assistance, and technical documentation with enhanced reasoning capabilities for complex programming challenges.
> Generate Python functions for data analysis
> Debug complex algorithm implementations
> Create comprehensive API documentation📊 Business Analytics
Data analysis, market research, and strategic planning with comprehensive insights without content restrictions.
> Analyze market trends and patterns
> Generate business intelligence reports
> Create strategic planning frameworks🔬 Research & Analysis
Academic research, technical analysis, and comprehensive exploration of complex topics without artificial limitations.
> Conduct comprehensive literature reviews
> Analyze complex technical concepts
> Generate research methodologies📝 Content Creation
Technical writing, educational content, and detailed documentation with comprehensive coverage of complex topics.
> Create detailed technical guides
> Generate educational materials
> Develop comprehensive documentation⚡ Performance Optimization
🎯 Memory Management
Expert routing cuts compute per token, not memory: all eight experts stay loaded. To save memory, pick a smaller quantisation or a shorter context.
⚡ GPU Acceleration
Put as many layers on the GPU as fit; Ollama does this automatically and runs the rest from system RAM, which is much slower.
🔧 Quantization
Apply precision reduction techniques to decrease model size and memory usage while preserving reasoning capabilities and response quality.
📊 Batch Processing
Optimize throughput for multiple concurrent requests through efficient batching and expert allocation strategies.
🔄 Caching Strategies
Implement intelligent caching for frequently accessed expert networks and routing patterns to reduce computational overhead.
⚙️ Configuration Tuning
Fine-tune model parameters for specific use cases and hardware configurations to achieve optimal performance-to-resource ratios.
🔬 Research Background & Development
Dolphin 2.6 Mixtral 8x7B was released in late December 2023 by Eric Hartford (Cognitive Computations), one of a series of Dolphin fine-tunes built to follow instructions without the refusals of the original chat models.
The Dolphin dataset is modelled on Microsoft's Orca approach: instructions answered by stronger models, then filtered. Version 2.6 adds the Magicoder coding sets and Nous Research's Capybara conversations, per the model card.
The fine-tune does not change Mixtral's architecture or routing; it is a qLoRA adapter merged into the base weights. Anything you read about Mixtral's hardware needs applies to Dolphin unchanged.
Newer open models have since overtaken the whole Mixtral generation at smaller sizes. Dolphin 2.6 is still pulled for its permissive behaviour; for current options see the best uncensored local LLMs.
For how mixture-of-experts routing works, see mixture of experts explained.
📚 Authoritative Research Sources
Technical Research Papers:
- • Mixtral of Experts (arXiv:2401.04088) - Mistral AI
- • Orca: Progressive Learning from Complex Explanation Traces - the approach the Dolphin dataset follows
- • Self-Instruct: Aligning Language Models with Self-Generated Instructions - Synthetic Data Methods
Model Documentation:
- • Mistral AI GitHub Repository - Official Source
- • Dolphin 2.6 Model Page - Hugging Face
- • Dolphin Dataset - Hugging Face
- • dolphin-mixtral tags - Ollama
❓ Frequently Asked Questions
🔧 What makes Dolphin 2.6 Mixtral 8x7B different from the base Mixtral model?
Dolphin 2.6 is a qLoRA fine-tune of Mixtral 8x7B on the Dolphin dataset plus coding and conversation data, filtered to remove refusals and alignment. It uses ChatML and is uncensored. The architecture, size and hardware needs are the same as Mixtral's.
⚡ What are the hardware requirements for running this model locally?
The 4-bit download (dolphin-mixtral:8x7b-v2.6) is 26.4 GB and needs about 28 GB at a 4K context. A 32 GB GPU or a Mac with 48 GB or more holds it entirely. On a 24 GB card such as the RTX 4090 Ollama puts part of it in system RAM, so you need 32 GB+ of RAM as well and it runs more slowly. The smallest tag, q3_K_M, is 22.6 GB. 16 GB of RAM is not enough.
🎯 How does the mixture-of-experts architecture work?
Each of Mixtral's 32 layers has eight feed-forward experts. For every token a router picks two of them, so about 12.9B of the 46.7B parameters are used per token. Attention layers are shared. The saving is in compute per token; memory still has to hold all 46.7B.
🔬 What types of tasks is this model best suited for?
Chat and writing tasks where refusals get in the way, coding help (its author trained it on a lot of coding data), and experiments with system prompts. For hard maths or the strongest code, newer models do better.
🛡️ Is the uncensored nature safe for professional use?
Not without your own safeguards. The model card says the dataset was filtered to remove alignment and that you are “advised to implement your own alignment layer before exposing the model as a service.” For private, single-user use that is your call; for anything customer-facing, add filtering.
📈 How does it compare to other models in its size class?
No head-to-head scores have been published for Dolphin 2.6. Its base, Mixtral 8x7B, scores 70.6 MMLU against 69.9 for Llama 2 70B in Mistral's evaluation, from a smaller file (26.4 GB against 38.9 GB at 4-bit). Nous Hermes 2 Mixtral is the closest alternative fine-tune, at the same size.
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.
Related Guides
Continue your local AI journey with these comprehensive guides
Mixture of Experts Explained
How Mixtral routes each token to two of eight experts, and why memory still follows the total.
Dolphin 2.6 Mistral 7B: Compact Power
The smaller version with focused capabilities.
Best Uncensored Local LLMs
Current uncensored models for Ollama, with sizes and hardware needs.
Dolphin 2.6 Mixtral 8x7B Architecture
Dolphin 2.6 is a qLoRA fine-tune of Mixtral 8x7B: 32 layers, eight experts per layer, two used per token
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.

Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
- PILLARLocal AI Hardware Requirements (2026): Complete Guide
- Best Mac for Local AI: Every Apple Silicon Chip Ranked M1–M6
- Ollama System Requirements: Minimum RAM, GPU, Disk Size
- Run an LLM on Your Phone: Offline AI on Android & iPhone
- Gemma 2 27B Hardware Requirements: VRAM, Ollama, Specs
- Best Local Model for a 24GB GPU Right Now
- Llama 3.2 11B Vision: Requirements, VRAM & Ollama Setup
- Phi-4 14B Local Setup: Ollama, GGUF, 12GB GPUs and Mac
- RTX Laptop GPU VRAM for Local AI: Every Card Listed
- Mistral Medium 3.5: 128B Dense, 4-GPU Setup (2026)
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.
Found your model? Now build something with it.
25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.