★ Reading this for free? Get 25 structured AI courses + per-chapter AI tutor — the first chapter of every course free, no card.Start free in 30 secondsOr own every course: $149 once

Dolphin 2.6 Mixtral 8x7B:
Ollama Tags, VRAM and Setup

Updated: November 2, 2026

Dolphin 2.6 Mixtral 8x7B is Eric Hartford's uncensored fine-tune of Mistral's Mixtral 8x7B. On Ollama it is dolphin-mixtral:8x7b-v2.6, a 26.4 GB 4-bit download that needs about 28 GB of memory, so it fits a 32 GB GPU or a 48 GB Mac, and runs split across GPU and RAM on a 24 GB card. Plain dolphin-mixtral:8x7b pulls the later v2.7 build, the same size.

🔧 8x7B
Mixture-of-Experts
8 expert networks
⚡ 26.4GB
Ollama download (4-bit)
~28 GB in memory at 4K
🎯 16K
Fine-tune Context
Base Mixtral: 32K
💻 Local
Private Deployment
No cloud dependency

🔧 Mixture-of-Experts Architecture

Dolphin 2.6 Mixtral 8x7B builds upon Mistral's innovative mixture-of-experts (MoE) architecture, with eight experts per layer and a router that sends each token to two of them. Mistral gives the totals as 46.7B parameters, of which 12.9B are used for any one token. All 46.7B still have to sit in memory.

🎯 Expert Network Design

Selective Activation:Only 2 experts active per token
Router Network:Intelligent expert selection
Load Balancing:Distributed computational load
Efficiency:12.9B active parameters vs 46.7B total

⚡ Performance Advantages

Computational Efficiency
Reduced FLOPs per token while maintaining quality
Memory Is Not Reduced
Every expert must be loaded, so memory follows the 46.7B total
Inherited Context
32K in base Mixtral; Dolphin 2.6 was fine-tuned at 16K

The mixture-of-experts architecture represents a significant advancement in transformer model design. Unlike traditional dense models where all parameters participate in processing every token, MoE models activate only a subset of experts for each input, achieving better computational efficiency.

Dolphin 2.6 keeps the base Mixtral 8x7B architecture unchanged and changes only the behaviour, through a qLoRA fine-tune on a dataset its author filtered to remove refusals and alignment. The result answers more requests than Mixtral Instruct does, with no safety layer of its own.

In Mistral's own evaluation (arXiv:2401.04088), the base Mixtral 8x7B matched or beat Llama 2 70B on most benchmarks while using about a fifth of the active parameters per token. The catch for local use is memory, not compute: the 4-bit file is 26.4 GB, against 38.9 GB for Llama 2 70B.

🎓 Training Methodology & Fine-Tuning

Base model scores (Mixtral 8x7B)

Mistral's figures for the base model, Table 2 of arXiv:2401.04088. The Dolphin model card publishes no scores of its own.

Performance Metrics

MMLU
70.6
GSM8K
74.4
MBPP
60.7
HumanEval
40.2
MATH
28.4

🔬 Fine-Tuning Process

Base Model
Built on Mistral's Mixtral 8x7B architecture
Synthetic Data
Dolphin dataset plus Magicoder OSS-Instruct and Evol-Instruct coding data
Uncensored Approach
Dataset filtered to remove alignment; the card says to add your own alignment layer
New in 2.6
A training-config fix, Samantha empathy data restored, Capybara in place of Synthia and Pure-Dove

📊 Training Data & Methodology

4x A100
Training Hardware
About 3 days, qLoRA with Axolotl
1.5
Training Epochs
Per the model card
16K
Fine-tune Sequence Length
Base model supports 32K

The model card describes a qLoRA fine-tune of Mixtral 8x7B run with Axolotl: 1.5 epochs, about three days on four A100 GPUs, at a 16K sequence length. Its author notes it was trained with a lot of coding data and is obedient but not DPO-tuned, so the system prompt matters.

The model uses the ChatML prompt format (system, user and assistant turns wrapped in im_start and im_end markers), which Ollama's template applies for you. The weights are released under Apache 2.0.

Uncensored means the refusals were filtered out of the training data, not that some other safety system replaces them. The card is explicit: “You are advised to implement your own alignment layer before exposing the model as a service.”

📈 Performance Benchmarks

MMLU 5-shot, as published by each vendor

Mixtral 8x7B (Mistral, arXiv:2401.04088)70.6 % accuracy
70.6
Llama 2 70B (as evaluated by Mistral)69.9 % accuracy
69.9
Claude 3 Haiku (Anthropic model card)75.2 % accuracy
75.2

🎯 Technical Performance Analysis

No benchmark scores have been published for Dolphin 2.6 Mixtral. The model card gives none, and this site has not run any. What can be said: it starts from Mixtral 8x7B, which Mistral reports at 70.6 MMLU, 40.2 HumanEval, 60.7 MBPP and 74.4 GSM8K.

Fine-tunes like this one mainly change behaviour (fewer refusals, a ChatML chat style, more coding data) rather than raw knowledge, so expect broadly base-Mixtral capability. If a score matters for your use, run your own evaluation on your own prompts.

ModelOllama tagDownloadMemory at 4KParametersLicenceNote
Dolphin 2.6 Mixtral 8x7Bdolphin-mixtral:8x7b-v2.626.4 GB~28 GB46.7B total, 12.9B activeApache 2.0 (model card)Uncensored; ChatML
Nous Hermes 2 Mixtral 8x7Bnous-hermes2-mixtral:8x7b26.4 GB~28 GB46.7B total, 12.9B activeApache 2.0Nous Research fine-tune; ChatML
Mixtral 8x7B Instructmixtral:8x7b26.4 GB~28 GB46.7B total, 12.9B activeApache 2.0MMLU 70.6 (base, Mistral)
Llama 2 70Bllama2:70b38.9 GB~41 GB70B denseLlama 2 CommunityMMLU 68.9 (Meta)
Claude 3 Haikucloud API onlyn/an/anot publishedPaid APIMMLU 75.2 (Anthropic)

Nous Hermes 2 Mixtral 8x7B is the other well-known Mixtral fine-tune of the same era, from Nous Research, also in ChatML. It is the same 26.4 GB on Ollama, so the hardware advice on this page applies to it unchanged. Mixture-of-experts models get no speed figure on this site: their speed depends on how many experts each token touches and where they sit in memory, so the simple bandwidth ceiling does not apply.

Dolphin 2.6 quantisations on Ollama

TagDownloadMemory at 4K
dolphin-mixtral:8x7b-v2.6-q3_K_M22.6 GB~24 GB
dolphin-mixtral:8x7b-v2.6 (q4_0)26.4 GB~28 GB
dolphin-mixtral:8x7b-v2.6-q4_K_M28.5 GB~30 GB
dolphin-mixtral:8x7b-v2.6-q5_K_M33.2 GB~35 GB
dolphin-mixtral:8x7b-v2.6-q8_049.6 GB~51 GB

Sizes from the Ollama registry. Memory = file + 0.5 GB key-value cache at 4K + about 1 GB of buffers; worked out, not measured.

🔬 Benchmark Analysis

📊 Performance Metrics

  • • No published scores for Dolphin 2.6 itself
  • • 70.6 MMLU for the base Mixtral 8x7B (Mistral)
  • • 40.2 HumanEval / 60.7 MBPP for the base model (Mistral)
  • • 74.4 GSM8K for the base model (Mistral)

⚡ Efficiency Metrics

  • • 12.9B active parameters per token
  • • 46.7B total parameters in model
  • • 26.4GB Ollama download at 4-bit
  • • ~28 GB of memory at a 4K context

⚡ Installation & Deployment

System Requirements

▸
Operating System
Windows 10+, macOS 12+, Ubuntu 20.04+
▸
RAM
32GB minimum for CPU or split GPU+CPU use; 48GB+ is comfortable
▸
Storage
27GB per 4-bit tag (dolphin-mixtral:8x7b-v2.6 is 26.4GB)
▸
GPU
32GB VRAM to hold the 4-bit file entirely (e.g. RTX 5090 or 2 x 24GB); a 24GB card runs it split with system RAM
▸
CPU
Any modern 8-core CPU; MoE models are bound by memory bandwidth
1

Install Ollama

Download and install Ollama for local model deployment

$ curl -fsSL https://ollama.com/install.sh | sh
2

Download Model

Pull Dolphin 2.6 (26.4 GB, 4-bit). dolphin-mixtral:8x7b without a suffix pulls the later v2.7

$ ollama pull dolphin-mixtral:8x7b-v2.6
3

Test Installation

Verify the model is working correctly

$ ollama run dolphin-mixtral:8x7b-v2.6 "Hello, can you introduce yourself?"
4

Keep Memory in Check

One loaded model and one request at a time leaves the most memory for this 26 GB model

$ export OLLAMA_NUM_PARALLEL=1 && export OLLAMA_MAX_LOADED_MODELS=1 && ollama serve

🔧 Deployment Configuration

🚀 Performance Settings

  • • In Ollama, set num_ctx (for example 8192) to raise the 4K default
  • • Set num_gpu to control how many layers go on the GPU when it does not all fit
  • • In llama.cpp the equivalents are --ctx-size and --n-gpu-layers
  • • Set num_thread to your physical core count for the CPU share

⚙️ Model Configuration

  • • Temperature 0.7 for balanced creativity
  • • Top-p 0.9 for diverse responses
  • • Repeat penalty 1.1 for natural flow
  • • Context: fine-tuned at 16K (base Mixtral supports 32K)

💻 Terminal Interface

Terminal
$ollama pull dolphin-mixtral:8x7b-v2.6
pulling manifest pulling 91536457ed20... 100% 26 GB verifying sha256 digest writing manifest success
$ollama run dolphin-mixtral:8x7b-v2.6 "Explain the key differences between supervised and unsupervised learning in machine learning"
# Supervised vs Unsupervised Learning: Key Differences ## Supervised Learning Supervised learning uses labeled data to train models. The algorithm learns from examples with known correct outputs, making it ideal for tasks like: - Classification (spam detection, image recognition) - Regression (price prediction, weather forecasting) **Advantages**: Clear objectives, measurable accuracy, faster convergence **Limitations**: Requires labeled data, expensive annotation process ## Unsupervised Learning Unsupervised learning discovers patterns in unlabeled data without predefined outputs: - Clustering (customer segmentation, anomaly detection) - Dimensionality reduction (feature extraction, visualization) **Advantages**: No labeling required, can discover hidden patterns **Limitations**: Less predictable results, harder to evaluate performance ## Key Differences 1. **Data Requirements**: Supervised needs labeled data, unsupervised works with raw data 2. **Objective Function**: Supervised minimizes prediction error, unsupervised maximizes pattern discovery 3. **Evaluation**: Supervised has clear accuracy metrics, unsupervised requires domain expertise 4. **Applications**: Supervised for prediction tasks, unsupervised for exploration and discovery The choice depends on your data availability and specific use case requirements.
$_

🎯 Use Cases & Applications

💻 Software Development

Code generation, debugging assistance, and technical documentation with enhanced reasoning capabilities for complex programming challenges.

> Generate Python functions for data analysis
> Debug complex algorithm implementations
> Create comprehensive API documentation

📊 Business Analytics

Data analysis, market research, and strategic planning with comprehensive insights without content restrictions.

> Analyze market trends and patterns
> Generate business intelligence reports
> Create strategic planning frameworks

🔬 Research & Analysis

Academic research, technical analysis, and comprehensive exploration of complex topics without artificial limitations.

> Conduct comprehensive literature reviews
> Analyze complex technical concepts
> Generate research methodologies

📝 Content Creation

Technical writing, educational content, and detailed documentation with comprehensive coverage of complex topics.

> Create detailed technical guides
> Generate educational materials
> Develop comprehensive documentation

⚡ Performance Optimization

🎯 Memory Management

Expert routing cuts compute per token, not memory: all eight experts stay loaded. To save memory, pick a smaller quantisation or a shorter context.

⚡ GPU Acceleration

Put as many layers on the GPU as fit; Ollama does this automatically and runs the rest from system RAM, which is much slower.

🔧 Quantization

Apply precision reduction techniques to decrease model size and memory usage while preserving reasoning capabilities and response quality.

📊 Batch Processing

Optimize throughput for multiple concurrent requests through efficient batching and expert allocation strategies.

🔄 Caching Strategies

Implement intelligent caching for frequently accessed expert networks and routing patterns to reduce computational overhead.

⚙️ Configuration Tuning

Fine-tune model parameters for specific use cases and hardware configurations to achieve optimal performance-to-resource ratios.

🔬 Research Background & Development

Dolphin 2.6 Mixtral 8x7B was released in late December 2023 by Eric Hartford (Cognitive Computations), one of a series of Dolphin fine-tunes built to follow instructions without the refusals of the original chat models.

The Dolphin dataset is modelled on Microsoft's Orca approach: instructions answered by stronger models, then filtered. Version 2.6 adds the Magicoder coding sets and Nous Research's Capybara conversations, per the model card.

The fine-tune does not change Mixtral's architecture or routing; it is a qLoRA adapter merged into the base weights. Anything you read about Mixtral's hardware needs applies to Dolphin unchanged.

Newer open models have since overtaken the whole Mixtral generation at smaller sizes. Dolphin 2.6 is still pulled for its permissive behaviour; for current options see the best uncensored local LLMs.

For how mixture-of-experts routing works, see mixture of experts explained.

📚 Authoritative Research Sources

Technical Research Papers:

Model Documentation:

❓ Frequently Asked Questions

🔧 What makes Dolphin 2.6 Mixtral 8x7B different from the base Mixtral model?

Dolphin 2.6 is a qLoRA fine-tune of Mixtral 8x7B on the Dolphin dataset plus coding and conversation data, filtered to remove refusals and alignment. It uses ChatML and is uncensored. The architecture, size and hardware needs are the same as Mixtral's.

⚡ What are the hardware requirements for running this model locally?

The 4-bit download (dolphin-mixtral:8x7b-v2.6) is 26.4 GB and needs about 28 GB at a 4K context. A 32 GB GPU or a Mac with 48 GB or more holds it entirely. On a 24 GB card such as the RTX 4090 Ollama puts part of it in system RAM, so you need 32 GB+ of RAM as well and it runs more slowly. The smallest tag, q3_K_M, is 22.6 GB. 16 GB of RAM is not enough.

🎯 How does the mixture-of-experts architecture work?

Each of Mixtral's 32 layers has eight feed-forward experts. For every token a router picks two of them, so about 12.9B of the 46.7B parameters are used per token. Attention layers are shared. The saving is in compute per token; memory still has to hold all 46.7B.

🔬 What types of tasks is this model best suited for?

Chat and writing tasks where refusals get in the way, coding help (its author trained it on a lot of coding data), and experiments with system prompts. For hard maths or the strongest code, newer models do better.

🛡️ Is the uncensored nature safe for professional use?

Not without your own safeguards. The model card says the dataset was filtered to remove alignment and that you are “advised to implement your own alignment layer before exposing the model as a service.” For private, single-user use that is your call; for anything customer-facing, add filtering.

📈 How does it compare to other models in its size class?

No head-to-head scores have been published for Dolphin 2.6. Its base, Mixtral 8x7B, scores 70.6 MMLU against 69.9 for Llama 2 70B in Mistral's evaluation, from a smaller file (26.4 GB against 38.9 GB at 4-bit). Nous Hermes 2 Mixtral is the closest alternative fine-tune, at the same size.

Build Real AI on Your Machine

RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.

Reading now
Join the discussion

Dolphin 2.6 Mixtral 8x7B Architecture

Dolphin 2.6 is a qLoRA fine-tune of Mixtral 8x7B: 32 layers, eight experts per layer, two used per token

👤
You
💻
Your ComputerAI Processing
👤
🌐
🏢
Cloud AI: You → Internet → Company Servers
🎯
AI Learning Path

Go from reading about AI to building with AI

25 structured courses. Hands-on projects. Runs on your machine. Start free.

Or own it for life — Lifetime $149 $599, pay once
Barsha Mohapatra

Written by

Barsha Mohapatra

Founder and Editor, Local AI Master · Founder, Little AI Master

Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.

📅 Published: October 28, 2025🔄 Last Updated: November 2, 2026✓ Manually Reviewed
More on Local AI Hardware
See the full AI Hardware Guide 2026 guide.
📚
Free · no account required

Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide

No spam. Unsubscribe with one click.

🎯
AI Learning Path

Found your model? Now build something with it.

25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.

Or own it for life — Lifetime $149 $599, pay once
Free Tools & Calculators