Open-Source LLMs Explained

The LLM ecosystem includes both proprietary services and open-source alternatives. Services like ChatGPT, Claude, and Google’s Gemini keep their model weights private. Open-source LLMs take the opposite approach.

With open-source models, the weights are publicly available for anyone to download. You can run them on your own hardware, modify them for specific tasks, and inspect how they work. This transparency has fueled rapid growth across the open-source AI community.

But open source does not mean simple. Running your own model demands technical knowledge, hardware investment, and ongoing maintenance. This guide covers the major open-source models, their strengths and drawbacks, and how to decide whether open source fits your needs.

What Makes an LLM Open Source

The term “open source” in AI is more nuanced than in traditional software. In regular software, open source means the full source code is available under an approved license. With LLMs, the situation is messier.

Some models release weights but restrict commercial use. Others release weights and training code but keep the training data private. A few release everything, including data, code, and weights.

The Open-Source Spectrum

The AI community generally recognizes three levels of openness:

• Fully open: Weights, training code, and training data all public. Example: some older EleutherAI models.
• Open weights: Model weights available for download, but training data or code withheld. Most major “open-source” LLMs fall here, including Llama and Mistral.
• Open API only: The model can be accessed through an API, but weights are not downloadable. This is not open source, even if the API is free.

When most people say “open-source LLM,” they mean open-weight models. The weights are the core asset. They contain everything the model learned during training.

With the weights, you can run the model independently without relying on anyone else’s servers. If a provider discontinues an API or raises prices, you lose access. Downloaded weights keep working regardless.

Major Open-Source Models

The performance gap between open-source and proprietary models has narrowed significantly in recent years. Several open-source options now compete with commercial offerings on specific tasks.

Model	Developer	Parameters	Context Window	License	Best For
Llama 4 Scout	Meta	109B total / 17B active (MoE)	10M tokens	Llama 4 Community License	Long-context tasks
Llama 4 Maverick	Meta	400B total / 17B active (MoE)	1M tokens	Llama 4 Community License	General purpose
Llama 3.1	Meta	8B / 70B / 405B	128K tokens	Llama 3.1 Community License	Broad range of tasks
Muse Glimmer 30B	Meta	29.6B	131K tokens	Apache 2.0	Local agent workflows
gpt-oss-120b	OpenAI	117B total / 5.1B active (MoE)	128K tokens	Apache 2.0	General purpose on one 80GB GPU
gpt-oss-20b	OpenAI	21B total / 3.6B active (MoE)	128K tokens	Apache 2.0	Runs within 16GB of memory
Mistral Large 3	Mistral AI	675B total / 41B active (MoE)	256K tokens	Apache 2.0	Complex reasoning, multimodal
Mistral Small 4	Mistral AI	119B total / 6.5B active (MoE)	256K tokens	Apache 2.0	Instruct, reasoning, and coding in one model
DeepSeek V4-Pro	DeepSeek	1.6T total / 49B active (MoE)	1M tokens	MIT	Frontier-scale reasoning and agents
DeepSeek V4.1-Flash	DeepSeek	552B total / 8B-16B active (MoE)	1M tokens	MIT	Fast multimodal reasoning
Qwen3.8	Alibaba	27B dense, up to 2.4T / 95B active (MoE)	262K tokens, extendable	Apache 2.0 (27B), custom (2.4T)	Coding, agents, vision
Phi-4 / Phi-4-reasoning-vision	Microsoft	14B / 15B	16K tokens	MIT	Small-footprint tasks
Gemma 4	Google	E2B, E4B, 12B, 26B MoE, 31B	128K-256K tokens	Apache 2.0	Multimodal, on-device

Note: Specifications reflect available data as of September 2026. Verify current details on provider sites before making decisions.

Meta (Llama and Muse)

Meta’s Llama family is the most widely adopted open-source LLM series. Llama’s official page hosts downloads and documentation for all versions.

Llama 4 introduced mixture-of-experts (MoE) architecture. Scout offers a 10-million-token context window, the largest of any model in this guide. Maverick targets general-purpose tasks with 400 billion total parameters.

Llama 3.1 remains popular for its range of sizes. Its 8B model runs on consumer hardware. The 405-billion-parameter version competes with proprietary flagships on benchmarks.

Meta also ships Muse Glimmer 30B, released in August 2026 under Apache 2.0. It is a multimodal model built for local agent workflows, and it fits on a 24GB consumer GPU when quantized. Meta’s larger Muse Spark model is API-only, so Glimmer is the open-weight option.

OpenAI (gpt-oss)

OpenAI entered the open-weight space in August 2025 with gpt-oss, its first publicly released model weights. This marked a significant shift for a company that had been exclusively proprietary since GPT-2.

The gpt-oss-120b uses MoE architecture with 117 billion parameters and runs on a single H100 GPU. The smaller gpt-oss-20b needs only 16GB of memory, putting it within reach of high-end consumer GPUs. Both models ship under the Apache 2.0 license with no commercial restrictions.

Mistral AI

Mistral AI, based in Paris, has become one of the most prolific open-weight model producers. Their developer documentation covers all available models.

Mistral Small 4, released in March 2026, packs 119 billion parameters with only 6.5 billion active per token. It accepts text and images, and you can switch reasoning on per request. Mixtral 8x7B, the model that made Mistral’s name, now sits in the deprecated list.

Mistral Large 3, released in December 2025, represents a major shift. It uses a 675-billion-parameter MoE architecture with 41 billion active parameters and a 256K context window. Critically, Mistral changed its flagship license from commercial to Apache 2.0, removing the restrictions that had limited Large 2’s adoption.

DeepSeek

DeepSeek, a Chinese AI lab, first drew wide attention with V3 and R1 in late 2024 and early 2025. V3 was a general-purpose model with 671 billion parameters using MoE architecture. It demonstrated that open-source models could approach frontier performance at a fraction of the training cost reported by larger labs.

DeepSeek V4 is the current generation, previewed in April 2026 and fully released on August 13. V4-Pro carries 1.6 trillion total parameters with 49 billion active, and both V4 models handle a 1-million-token context window.

Reasoning is built into the V4 models. On V4-Pro you set reasoning effort to low, high, or max, and the model works through problems step by step before answering. V4.1-Flash, released in September 2026, adds image understanding while activating only 8 to 16 billion parameters per token.

The V4 models use the MIT license, which is among the most permissive available. Their open-source repositories include model weights and technical reports.

Other Notable Models

Alibaba’s Qwen3.8, released in August 2026, is the current generation. Its 27B model reads images and video as well as text, and thinking is on by default but can be turned off per request.

The flagship Qwen3.8-2.4T-A95B holds 2.4 trillion parameters with 95 billion active, and is Alibaba’s first open release of a Max-class model. It ships under a custom Qwen license rather than Apache 2.0, so read the terms before building on it.

Microsoft’s Phi-4 delivers surprising performance for its small 14B parameter size. Phi-4-reasoning-vision-15B, released in March 2026, adds image input and step-by-step reasoning. Phi-4 runs comfortably on consumer GPUs, making it practical for individual developers.

Google’s Gemma 4, released in April 2026, comes in E2B, E4B, 12B, 26B mixture-of-experts, and 31B sizes. Google moved Gemma to the Apache 2.0 license with this release, and the models are pre-trained on over 140 languages.

The E2B and E4B models support 128K tokens, while the 12B, 26B, and 31B models reach 256K. The three smallest sizes accept text, images, and audio, and the two largest accept text and images.

Benefits of Open-Source LLMs

Open-source models offer advantages that no API-based service can match. These benefits fall into four areas.

Data Privacy and Control

When you run an open-source model locally, your data never leaves your servers. Every prompt and response stays on hardware you control. For industries handling sensitive information like healthcare, legal, or finance, this eliminates third-party data exposure entirely.

API services process your prompts on external servers. Even with privacy policies in place, sending proprietary data to a third party introduces risk. Self-hosted models remove that concern.

Customization Through Fine-Tuning

Open weights mean you can fine-tune the model on your own data. A law firm can train a model on legal documents. A customer support team can train it on past tickets.

The result is a model that understands your specific domain better than any general-purpose alternative. Fine-tuning adjusts the model’s behavior at a fundamental level. This goes deeper than prompt engineering or temperature settings.

Cost Control at Scale

Open-source models charge no per-token API fees. Once you have the hardware, you can run unlimited queries. For workloads that keep a GPU busy around the clock, this can undercut LLM pricing through API services.

The trade-off is upfront hardware investment. But when demand is steady enough to keep that hardware working, the math can favor self-hosting.

Transparency and Reproducibility

Researchers and developers can examine exactly how an open-source model produces its outputs. They can study its behavior, test for biases, and verify claims about performance. This transparency supports reproducible research and matters for applications where explainability is a requirement.

Limitations and Challenges

Open-source LLMs come with real constraints. Understanding these helps you avoid costly surprises.

Hardware Requirements

Large open-source models require significant computing hardware. Running Llama 3.1 70B at full precision needs roughly 140GB of GPU memory. That exceeds what any single consumer GPU provides.

Smaller models are more accessible. Llama 3.1 8B fits on a GPU with 8GB of VRAM as a minimum when quantized.

Setup and Maintenance

API services handle infrastructure, updates, and optimization for you. With open-source models, those responsibilities shift to your team.

The tooling has improved significantly. Projects like Ollama and vLLM simplify local deployment. But troubleshooting still requires comfort with command-line tools.

Performance Gaps

While open-source models have closed much of the gap, proprietary models still lead on several fronts. Common LLM limitations apply to all models. Open-source versions tend to produce more hallucinations on complex reasoning tasks compared to frontier proprietary options.

The best proprietary models also update more frequently. When OpenAI or Anthropic releases an improvement, API users get it immediately. Open-source users wait for new weight releases, which can take months.

No Built-in Safety Rails

Proprietary services include content filtering and safety mechanisms. Open-source models typically ship without these guardrails, leaving you responsible for implementing safety measures in production.

How to Run Open-Source LLMs

You have two main options for running open-source models. Each involves different trade-offs in cost, complexity, and performance.

Running Models Locally

Local deployment means the model runs entirely on your own hardware. This gives you complete control over the inference environment and maximum data privacy.

Quantization is the key technique that makes local deployment practical. This process reduces a model’s memory requirements by 50-75% using lower-precision numbers to represent the weights. A 70B model that needs 140GB at full precision might need only 35-40GB after 4-bit quantization, with modest quality loss.

The quality trade-off from quantization varies by task. Conversational use and general writing see minimal impact at 4-bit precision, while complex reasoning and math show a wider gap. Testing on your specific use case is the only reliable way to assess it.

Popular tools for local deployment include:

• Ollama: The simplest option. One command downloads and runs models. Best for quick experimentation and personal use.
• llama.cpp: Highly optimized C++ inference engine. Runs models on CPUs and GPUs with excellent quantization support.
• vLLM: Production-grade inference server. Handles multiple concurrent users with efficient memory management.

For a single user, a desktop with a modern GPU works well. Team use requires a dedicated server with one or more data center GPUs.

Cloud Hosting

Cloud hosting combines open-source flexibility with managed infrastructure.

• Together AI: Their hosting platform runs popular open-source models with API access. Pay per token, but often cheaper than proprietary APIs.
• Hugging Face: Their Inference Endpoints let you deploy any model from the Hugging Face hub on dedicated infrastructure. You choose the GPU type and model, and Hugging Face handles the rest.
• AWS, GCP, Azure: All major cloud providers offer GPU instances for self-managed model deployment. This approach requires more setup but gives you the most control over your environment.

Cloud hosting eliminates hardware procurement but adds ongoing compute costs. Hugging Face lists $2.50 per hour for an 80GB A100 and $4.50 for an H100, and Together AI lists $5.49 per hour for an H100. At $4.50 an hour, a month of continuous use works out to roughly $3,300.

Pricing Overview

Open-source model weights cost nothing to download. The real expenses come from infrastructure.

Self-Hosting Hardware Costs

Running models locally requires GPU hardware, and VRAM is the number that matters. A 24GB or 32GB consumer card handles models up to about 30B parameters after quantization, enough for Muse Glimmer or gpt-oss-20b. Larger models need 80GB data center cards like the A100 or H100, which are sold through resellers rather than at a list price.

Electricity, cooling, and rack space add ongoing costs.

Cloud Hosting Costs

Cloud GPU instances offer a middle ground, with costs covered in the hosting section above. Preemptible instances cut those rates by about half, and Together AI lists an H100 cluster at $1.99 per hour preemptible against $3.99 on demand.

Total Cost of Ownership Framework

Compare open source against API pricing using three factors. First, calculate your monthly token volume and multiply by the API rate. Second, estimate your infrastructure cost for the same throughput.

Third, add staff time for setup and maintenance. One H100 at $4.50 an hour costs about $3,300 a month, which buys billions of tokens at serverless rates near $0.60 per million. Self-hosting pays off on price only when you keep the GPU busy, so privacy and control are usually the stronger reasons.

Open-Source LLM Licensing Explained

Not all open-source licenses are identical, and the differences affect what you can build.

License	Commercial Use	Modification	Key Restriction	Example Models
Apache 2.0	Yes	Yes	None significant	Mistral Large 3, Mistral Small 4, gpt-oss, Gemma 4, Muse Glimmer
MIT	Yes	Yes	None significant	DeepSeek V4, Phi-4
Llama License	Conditional	Yes	700M MAU threshold	Llama 3.1, Llama 4
Custom model licenses	Varies	Varies	Terms differ per model	Qwen3.8-2.4T-A95B

The Apache 2.0 license allows full commercial use with no meaningful restrictions. MIT is similarly permissive. These are the safest choices for business applications.

Meta’s Llama license permits commercial use but includes a threshold. If your product exceeds 700 million monthly active users, you need a separate agreement with Meta. For the vast majority of companies, this restriction is irrelevant.

Some models use custom licenses that restrict specific use cases. Always read the license before building a product on any open-source model.

When to Choose Open Source vs API Services

The decision between open-source and proprietary models depends on your priorities. Neither option is universally better. Choosing the right LLM requires matching the model to your actual requirements.

Choose Open Source When

1. Data privacy is non-negotiable. Regulated industries or sensitive applications benefit most from self-hosted models where data stays internal.
2. You need domain-specific fine-tuning. If your task requires specialized knowledge not covered by general models, open-source fine-tuning delivers better results than prompt engineering alone.
3. You run sustained, high-volume workloads. Self-hosting beats API pricing when demand keeps a GPU busy most of the day. The break-even point depends on your hardware and the token volume.
4. You want to avoid vendor lock-in. Open weights ensure you can switch infrastructure providers or run the model indefinitely without external dependencies.

Choose API Services When

1. You want the best available quality. Proprietary models from OpenAI, Anthropic, and Google still lead on most benchmarks, particularly for complex reasoning. Comparing free vs paid LLM options helps frame this trade-off.
2. Your team lacks ML infrastructure experience. API services require zero infrastructure management. If using LLMs effectively through an API already meets your goals, self-hosting adds unnecessary complexity.
3. You need rapid iteration. API models update frequently. Open-source models update on the developer’s schedule.
4. Your volume is low to moderate. For occasional use, total cost of ownership can exceed API pricing. Factor in hardware, electricity, and maintenance time before committing.

The Hybrid Approach

A growing number of teams combine both strategies. They route simple, high-volume requests to a self-hosted open-source model. Complex queries that require top-tier reasoning go to a proprietary API.

This hybrid approach balances cost, quality, and privacy. It works especially well for software development teams, where routine code completion runs locally while complex architecture questions go to a stronger model. Teams evaluating their options can also review the best LLM for coding to compare specific model strengths.

Simple routing rules based on query length, task type, or user tier can distribute requests effectively between the two.

Related Guides

Understanding how LLMs work helps when evaluating which open-source model fits your use case. If you are comparing specific proprietary alternatives, the ChatGPT vs Claude comparison covers the leading closed-source options side by side. For task-specific recommendations across both open and closed models, the best LLM for research guide breaks down performance by use case.

Conclusion

Open-source LLMs have evolved from research curiosities into production-ready tools. Models like Llama 4, Mistral Large 3, DeepSeek V4, Qwen3.8, Gemma 4, and OpenAI’s gpt-oss offer real alternatives to proprietary services. Each new release closes more of the capability gap.

The choice comes down to your specific needs. Privacy, customization, and cost control at scale favor open source. Top-tier quality with zero infrastructure overhead favors API services.

Start with API services to understand what LLMs can do for your workflow. Then evaluate open-source options for tasks where the benefits justify the added complexity.