Skip to main content

Open-Weight Models and Running One Locally

info

This content was developed with AI assistance and is regularly reviewed for accuracy.

Snapshot: October 2026. Model names, prices, and product features change monthly; What's New in AI is the maintained chronology.

"Open source AI" used to mean Meta's Llama. In 2026 it mostly means models from Chinese labs plus Google's Gemma and OpenAI's gpt-oss, and the practical question has changed from "which benchmark does it top" to "can I run something useful on my own machine, and what does the licence let me do with it." This page answers both. It replaces the older datasets-and-benchmarks reference, because the thing most readers now want from "models" is a private, cheap assistant they control.

Learning focus:

  • Know which open-weight families are current and what "open weights" does and does not mean
  • Read a model licence well enough to spot the three clauses that matter
  • Run a model locally with Ollama or LM Studio and connect a tool to it
  • Decide when local beats cloud, and when it does not

Open weights, not open source​

A model is "open weight" when its creator publishes the trained parameters so you can download and run it. That is different from open source in the software sense: the training data and training code are usually not published, and the licence may restrict commercial use, redistribution, or use above a revenue threshold. The phrase to look for on a model card is the licence name. Apache 2.0 and MIT are permissive and allow commercial use; "community" or "custom" licences need reading.

The frontier gap is real but narrower than the 2024 picture. Stanford's 2026 AI Index reports that the gap between the top US and Chinese models had effectively closed (about 2.7 percentage points as of March 2026, reported), and open-weight models from DeepSeek, Alibaba and Moonshot sit near the top of public leaderboards. The best closed models from Anthropic, OpenAI and Google still lead the Artificial Analysis Intelligence Index, so local models suit privacy, cost, and offline use rather than peak capability.

The 2026 open-weight families​

All entries below come from aggregator coverage rather than the labs' own model cards, and sizes and dates conflict across sources; treat the details as reported and check the model card before you build on one.

FamilyLatest reported (2026)LicenceWhat it is good for
DeepSeek V4V4 family through 2026; V4.1-Flash weights released September 10, 2026MITStrong general reasoning and coding at low cost; large versions need serious hardware
Qwen 3.8 (Alibaba)Qwen3.8-27B dense (August 2026), called the most downloaded new open model of 2026 by one sourceApache 2.0 for the 27B; the largest variant (2.4T total, 95B active) under a revenue-gated custom licenceThe default mid-size local model for many developers
Kimi K3 (Moonshot)About 2.8T parameters, July 2026 (weights date disputed); reported first on LMArena's Frontend Code Arena on October 12, 2026Open weights (check terms)Frontier-class coding when you can host it; not a laptop model
Gemma 4 (Google)Sizes from small on-device models up to 31B denseApache 2.0 (a change from earlier Gemma terms)The easiest high-quality family for laptops; the 12B is reported to run in 16GB of RAM
gpt-oss (OpenAI)Still the August 2025 release: 120B mixture-of-experts (4 of 128 experts active) and a 20B sibling, 131K contextApache 2.0Reasoning on a workstation (120B) or a laptop (20B)
MistralSmall 4 (March 2026), Medium 3.5 (April 30, 2026), Large 3 as the largest public model as of August 2026Small 4 is Apache 2.0; Medium 3.5 is a custom, non-OSI licenceEuropean option with a permissive small model
Llama 4 (Meta)Llama 4 Maverick (April 2025); no 2026 Llama was foundLlama 4 Community License (not OSI-approved)Still widely supported by tools, but Meta moved its frontier effort to the closed Muse line

Two things in that table matter more than the parameter counts. First, Meta shipped no 2026 Llama and released its first frontier model, Muse Wonder, as a closed API in October 2026; a lesson that equates open models with Llama is out of date. Second, "open" now spans three licence regimes in one family, so the same lab's small and large models can carry different rights.

Licence nuance: the three clauses to read​

You do not need a lawyer to use an open-weight model, but you do need to read three parts of the licence before putting one in a product.

Commercial use and thresholds. Apache 2.0 and MIT allow commercial use with attribution. Community and custom licences often allow it only below a revenue or monthly-active-user threshold, or require a separate agreement above it. Qwen's largest 3.8 model is reported to be revenue-gated while its 27B is Apache 2.0; Mistral Medium 3.5 carries custom terms while Small 4 does not.

Redistribution and derivatives. Fine-tuning a model and publishing the result is redistribution. Permissive licences allow it; some community licences require you to carry the original licence and naming forward, and a few restrict using outputs to train competing models.

Acceptable-use policies. Many model licences attach a use policy that prohibits specific applications. This is separate from the licence grant and still binds you.

"Open weights" also tells you nothing about the training data. If provenance matters for your use (publishing, regulated industries), look for a model card that documents data sources, and treat silence as a risk rather than a reassurance.

Running a model locally​

Local running moved from hobbyist to mainstream-adjacent in 2026. Two tools cover most people.

Ollama is the developer default: MIT-licensed, roughly 179,000 GitHub stars (reported), a command-line tool that now ships a desktop app alongside it, and a local API on localhost:11434 with OpenAI-compatible routes so existing code can point at it. On Apple Silicon it runs on Apple's MLX framework (reported as the default in a pre-release). LM Studio is the friendlier graphical option for people who do not live in a terminal: closed-source, free for personal use, with a model browser and a local server on localhost:1234.

The flow is the same in both:

  1. Install the app for your operating system (macOS, Windows, Linux).
  2. Pull a model that fits your hardware. Start with a small Gemma 4, Qwen 3.8, or gpt-oss-20b build; the app lists quantized versions (compressed weights that trade a little quality for a lot less memory).
  3. Chat to check speed and quality on a task you actually do, such as summarizing a document you cannot send to a cloud service.
  4. Connect a tool. Point an editor or agent at the local API: VS Code's bring-your-own-key setting works with Ollama as a built-in provider and LM Studio as a custom endpoint, and GitHub Copilot's CLI has been able to run against Ollama and other local servers since April 2026 (reported).

Hardware guidance from current local-LLM guides is organized by memory tier. With 8 to 16GB of RAM, stay with small models (a few billion parameters) and expect usable but limited quality. With 16 to 32GB, the 12B to 27B quantized models (Gemma 4 12B, Qwen3.8-27B) become comfortable. A 24GB GPU or 32GB of unified memory runs 27B to 31B dense models well. Workstations with 48GB to 128GB can run the larger mixture-of-experts releases such as gpt-oss-120b. These are rules of thumb from secondary guides, not vendor specifications; the app will tell you if a model does not fit.

Try this as your first local test:
"Summarize the attached meeting notes into decisions, owners and deadlines.
Flag anything that reads like a commitment without an owner."

Paste in something you would hesitate to upload, and compare the result with your cloud assistant. The difference is usually noticeable on nuance and length, and invisible on a straightforward summary.

On-device AI in phones and operating systems​

The other way "local" arrives is baked into the device. Apple's rebuilt Siri, announced at WWDC in June 2026 and shipping with iOS 27, routes simpler requests to Apple's on-device models and complex ones to a custom Gemini variant (reported); one guide says Apple's Foundation Models framework opened to third-party providers at the same event, but that rests on a single source. Windows on-device AI specifics for 2026 were not verified for this snapshot. The practical point for most readers is that on-device features are chosen for you by the OS vendor; running a model yourself is the only way to pick the model and keep the data on the machine by design.

When local beats cloud, and when it does not​

Local wins when the data cannot leave the building (client files, health records, unreleased code), when volume makes per-token pricing painful, when you need offline operation, or when you want to experiment with fine-tuning and tool integration without a bill. It also wins on predictability: the model does not change under you overnight.

Cloud wins on peak capability and on long context. A 2026 study of 35 open-weight models found the best fabricated about 1.2% of answers at 32K tokens of context but could not stay under 10% at 200K, so the 1M-token windows of the frontier APIs are not something a laptop model matches in practice. Agent workloads that browse, call tools and run for an hour also favor the closed models' reliability and safety tuning. Many teams land on a hybrid: a local model for private or bulk work and a frontier API for the hard 10%.

One safety note. Open-weight models vary widely in safety tuning, and some community fine-tunes remove it. Local does not mean harmless, and it does not remove the need to verify output; it only removes the cloud from the picture.

Key takeaways​

  • The open-weight field in 2026 is led by DeepSeek, Qwen, Kimi, Gemma and gpt-oss; Meta shipped no 2026 Llama and went closed at the frontier.
  • "Open weights" is not "open source": read the commercial-use, redistribution and acceptable-use clauses, and expect different licences within one family.
  • Ollama (CLI plus desktop app) and LM Studio (GUI) make a private assistant realistic on a 16GB laptop with a small Gemma 4, Qwen 3.8, or gpt-oss model.
  • Local models suit privacy, cost, offline use and experimentation; frontier APIs still win on peak capability, long context and agent reliability.

Next steps​

SponsoredYour product hereReach people actively learning AI — flat monthly rates, no ad networks