On-device AI has long forced a tradeoff between model capability and what the hardware can run. Larger models demand more memory, bandwidth, and power; smaller models can struggle with complex work. Moving that work to the cloud adds model-provider fees, data-handling constraints, and a dependence on connectivity.

This release addresses those constraints in four ways:

We're releasing the full stack so developers can reproduce the results, evaluate or fine-tune the models, and integrate them into their own products.

Get the model weights, explore the inference framework, or try the Edge0 App.

How We Make 100B+ Parameter AI Run on a Personal Computer

Edge0-145B Laptop Performance: Memory, Speed, and Power

Edge0-145B makes a 145-billion-parameter model usable on a personal laptop. A useful local model has to fit in memory, process prompts promptly, generate fast enough to use, and keep power consumption under control. It also needs to sustain longer tasks.

Our earlier Edge0-35B phone release reported peak memory below 2.5GB. On an iPhone 18 Pro Max, Edge0-35B delivers 37+ tokens/s using approximately 2.5GB of memory. Edge0-145B has roughly four times as many parameters; its laptop comparison reports 80+ tokens/s and 28GB peak memory. These measurements apply to different models and hardware.

On-device model scale over time. The chart covers 67 open models of at least 1B parameters from 14 labs, intended for phones, laptops, and AI PCs. Same-size base, instruct, and reasoning variants count once; each line tracks a lab's largest model, including releases before July 2025. Filled points mark new highs and hollow points other releases. The scale is logarithmic below 40B and linear above it. Apple is excluded because its weights are not released. Sources: vendor announcements and model cards.

Why On-Device Models Have Stayed Small

Memory limits kept most on-device models below 10B parameters before September 2026. Apple Intelligence's on-device foundation model had approximately 3B parameters, while Google's Gemini Nano came in 1.8B and 3.25B versions. Even a 7B model could consume approximately 4GB of memory.

Larger models faced a memory wall. Model weights occupy storage and must be read during generation, competing for limited memory and bandwidth with the operating system and other apps. Even an aggressively compressed 35B model could need approximately 20GB of memory, far more than a phone could make available.

Compute, bandwidth, and power created competing constraints. Larger models offered more capability, but strained memory capacity, generation speed, and battery life. Making models smaller helped them fit, often at the cost of capability.

Privacy alone could not close that gap. Apple Intelligence protected user data, but Apple acknowledged that its on-device model was not designed for agentic applications requiring general world knowledge. The prevailing assumption was that on-device intelligence could not match the cloud.

Under the Hood of Edge0: Four Layers of Model and Inference Design

We designed Edge0-145B and its inference engine together to reduce the memory and data movement that make large models difficult to run locally. The design spans architecture, training, loading, and scheduling.

Architecture: share large matrices across sparse experts. Sharing matrices substantially reduces the weights that must be stored and moved while keeping the expert count unchanged. Each group of 64 experts shares two large matrices, while each expert retains a small set of parameters that captures its differences. Previously, every expert stored a complete set of matrices. Of the model's 145B total parameters, only approximately 10B participate in computation on each inference step.

Training: build for local execution from day one. We train the model in its on-device architecture from the outset, with deployment constraints built into its design. This avoids treating local deployment as a compression or pruning step after training a cloud model.

Loading: pipeline GPU memory, system memory, and SSD access. Reading and computation overlap across three storage tiers, avoiding serial waits. Frequently used “hot experts” remain in GPU memory. An expert library of approximately 40GB sits in system memory, with experts brought into GPU memory as needed. Approximately 48GB of vocabulary data resides on the SSD and is read on demand.

Scheduling: use the CPU and GPU together. The GPU computes cached experts while the CPU handles the remaining work in parallel. Speculative decoding verifies multiple candidate tokens at once, spreading the cost of reading weights across more output tokens. These choices support the reported 80+ tokens/s decode rate.

Model and inference-engine design work together to make 100B+ parameter inference practical on personal computers. Source: internal testing.

Together, these changes let parameter counts grow from tens of billions to more than a hundred billion without proportional increases in memory use, power consumption, and latency.

What Edge0 Can Do: Anything From Chat to Complex Work on Your Computer

Edge0-145B can carry a complex task through to a usable result. Its coding and reasoning capabilities support local workflows that combine planning, tool use, execution, debugging, and visual checks. The benchmarks establish its capabilities; the demonstrations show how it puts them to work.

Coding and reasoning that rival frontier cloud models

Edge0-145B is competitive with specific frontier cloud models across coding, reasoning, knowledge, and instruction following:

• Coding: It scores 64.6 on DeepSWE 1.1, ahead of Qwen3.8-Flash-Next at 58.7 and DeepSeek-V4-Flash-0731 at 54.4.
• Reasoning and knowledge: It scores 90.4 on GPQA-Diamond, close to DeepSeek-V4-Flash-0731 at 90.8 and Claude-Opus-4.6 \(Max\) at 91.3.
• Instruction following: It scores 80.6 on IFBench, ahead of DeepSeek-V4-Flash-0731 at 79.2 and Claude-Opus-4.6 \(Max\) at 62.5.

Its 86.52 on Terminal-Bench 2.1 also exceeds Qwen3.8-Flash-Next's 83.3, DeepSeek-V4-Flash-0731's 82.7, and Claude's 78.2. On AA-LCR, Edge0-145B scores 79.3 to Claude's 78.0. These comparisons apply to the named models and evaluations.

Scores are percentages; higher is better. A dash means no public score is available, rather than a score of zero.

Edge0, Qwen, DeepSeek, and Claude results come from Edge0's internal evaluations. Claude-Opus-4.6 has no public DeepSWE 1.1 score. For Flint, Muse Glimmer, and Gemma 4, Terminal-Bench 2.1 and IFBench results come from the Recursive Flint evaluation; HLE, GPQA, and AA-LCR results come from Artificial Analysis.

The largest gains over Qwen3.8-27B, a mainstream on-device model baseline, are in agent and coding work: 18.5% on Terminal-Bench and 53.1% on DeepSWE. The relative gains on HLE, GPQA, and IFBench are 14.9%, 1.3%, and 1.4%, respectively.

DeepSWE 1.1 and Terminal-Bench 2.1 evaluate agent and coding work. HLE, GPQA-Diamond, and IFBench cover reasoning, knowledge, and instruction following. The complete evaluation setup, hardware environment, and reproduction instructions are being released with our technical report.

Competitive capability with a smaller memory footprint

Edge0-145B combines a 72.8 mean score across six benchmarks with 29GB measured peak memory. Qwen3.8-Flash-Next averages 71.7, with an estimated 70GB footprint for standard 4-bit deployment. That puts Edge0-145B at about 40% of the estimated baseline memory while delivering comparable or higher capability, enabling local use on a 48GB laptop.

The means cover DeepSWE 1.1, Terminal-Bench 2.1, HLE, GPQA-Diamond, IFBench, and AA-LCR. Edge0 memory figures are measured internal peaks. Asterisked baseline figures are estimates for standard 4-bit deployment and remain to be measured. Claude is excluded because it is cloud-only and has no DeepSWE score.

[Figure: Quality versus peak memory, with 48GB laptop and 64GB workstation capacities marked. Lower memory and higher scores are better; the chart's vertical axis does not start at zero.]

Local agents that plan, code, run, and check their work

A single natural-language request can start an entire local workflow. Edge0-145B breaks down the requirements, plans the steps, writes code, and calls tools to run it. When something fails, it debugs and repairs the code. It then takes screenshots to inspect the result and keeps iterating until the work is complete.

These projects often take dozens of rounds of planning, execution, and review. Local inference means those additional conversations, code revisions, and visual analyses incur no corresponding cloud-model token or API charges. Code and data can stay on the computer, and core model capabilities work offline.

Video production: generating Tesla's Roadster explainer within an hour

Edge0-145B produced an explainer about how Roadster could fly, handling technical research, scriptwriting, asset generation, and editing. We asked it to make the principles understandable to a general audience. The model ran locally without cloud-model calls or cloud token consumption, avoiding network round trips for inference.

The video was generated within an hour. Producing comparable work manually would have taken a director and scriptwriter, a 3D modeler, an editor, and a physics expert a month. An AI video workflow could previously cost more than hundreds of dollars; local inference eliminates the corresponding cloud-model token and API fees.

https://www.youtube.com/watch?v=TefX0rfGNkk

3D modeling: a Manhattan city scene in Blender

Edge0-145B built a 3D scene of Manhattan within an hour. Working in Blender from text and reference images, it constructed the scene, adjusted materials and lighting, and iterated on the rendered output.

The workflow brings visual inspection, code generation, and revision together inside Blender. The model works with objects, spatial relationships, and the creative goal, going beyond the pixels in a reference image. It inspects the result, writes code, and makes changes without handing those steps back to a person. The entire process stays local.

Creating a Manhattan-scale city scene manually would typically cost approximately $12,000-18,000 and take at least a month. A leading cloud model would still incur more than $150 in cloud token charges. Edge0-145B completes the local inference without those cloud-model token fees.

https://www.youtube.com/watch?v=YR9KeFT1p68

Game development: playable games and a medieval city

Edge0-145B can create playable games locally. We also used its 3D modeling capabilities to build an original medieval capital for a fantasy RPG and render a 20-second fly-through at game-trailer quality.

The model planned the city independently and built it from scratch in Blender: city walls, more than a thousand old-town homes, a plaza fountain, a Gothic twin-tower facade, a rib-vaulted nave, and stained-glass windows.

The fixed camera path made errors visible. Geometry clipping, occlusion, or incorrect proportions would appear along the route, so the model inspected the renders and corrected its work as it built. The workflow ran entirely locally.

A leading cloud model required hundreds of tool calls and hundreds of millions of tokens for a single completion, using the same prompt, reference images, and camera route. Edge0-145B delivered it on a laptop without cloud-model token or API fees for local inference. An art team would typically spend weeks and thousands of dollars producing a city scene to this standard.

https://www.youtube.com/watch?v=6Rfuh7lBiqY

Hardware control: code that flies a real drone

Edge0-145B wrote the program for a real drone from one instruction: “Write a drone program that navigates through obstacles and finds a red backpack.”

It generated the flight-control and visual-recognition programs locally. The drone then took off indoors, navigated through a dense row of obstacle poles, searched autonomously, and located the red backpack beside a wall. The task required no human-written code and no cloud call.

Hardware developers can describe the behavior they want and have the model turn it into a program for a real device, without moving repeatedly between an editor, documentation, and debugging tools.

PCB design: from KiCad layout to an assembly animation

Edge0-145B took a PCB design from component placement through an assembly visualization. We asked it to design a board for itself. It started from scratch in KiCad, placed components, routed a two-layer board, and imported the design into Unreal Engine 5. It then set up cameras and lighting and rendered an animation of PCB fabrication and assembly.

A single computer delivered work that previously required a hardware engineer and a 3D designer to hand tasks back and forth over several days. Schematics and board layouts can stay on the team's own hardware, supporting trade-secret protection and data-handling requirements without uploading the designs to a cloud model.

Why Edge0: A Full Stack for On-Device AI

Useful on-device AI requires model development, inference engineering, and hardware adaptation to work together. Our progress rests on three capabilities: developing complementary models quickly, optimizing the full stack, and carrying lessons from each deployment into the next.

Our September release of Edge0-35B showed what that approach could deliver on a phone. Its launch post passed 1.2 million views on its first day without paid promotion in overseas communities. Researchers and technical experts at OpenAI, NVIDIA, Qualcomm, and other companies shared it, and the model reached No. 1 on Hugging Face's overall Trending chart.

We released the model weights and unified inference framework together, giving companies and researchers a standard approach they could reproduce. The release helped move the discussion of on-device AI from a few billion parameters to the 35B tier.

Four Model Families for On-Device AI

Over the past year, we released 200 models across four families. Each handles a different part of the work, from listening and speaking to fast decisions, memory, and complex agent tasks.

The portfolio spans tens of millions to more than a hundred billion parameters. Both Edge0 and Audio8 have reached No. 1 on Hugging Face's Trending chart.

Models and Inference Systems Designed Together

We develop the model and the inference framework together, building hardware constraints into training and optimizing the engine for different chips. This connects model capability with the software and hardware needed to use it locally.

That work requires expertise in model development, inference frameworks, and low-level hardware adaptation. The combination is difficult for a company focused only on model development to reproduce.

Hardware Optimizations That Carry into Each Deployment

Each deployment gives us reusable optimizations for the next chip, model, or product. This matters because on-device hardware is fragmented: NPU architectures differ, as does support for numerical formats such as INT4, INT8, and FP4. A general-purpose model cannot automatically make full use of every chip.

We adapt to individual platforms. Our portfolio now covers 95% of mainstream chip platforms, including Qualcomm, BES, Rockchip, Allwinner, D-Robotics, and NVIDIA, across headphones, glasses, phones, PCs, and robots. This is portfolio-wide coverage; model requirements vary, and it does not mean every device can run Edge0-145B.

That deployment work now spans more than 5 million production devices. Our open-source models passed 15 million downloads within three months. By October 2026, we had more than 110,000 customer organizations.

The portfolio figures come from company papers and internal testing. They describe the broader Edge0 portfolio; the 145B performance results in this article apply specifically to Edge0-145B.

For Edge0-145B, day-one support covers macOS, Windows, and Linux, including Apple M-series chips and NVIDIA platforms such as RTX Spark, RTX 4090, and RTX 5090. The release includes one-click deployment tools and hardware guides.

The Future of On-Device AI and the Cloud

More complex work can stay on the device as local models approach cloud-level capability. Privacy, cost control, and offline access have long favored local execution; the remaining question was whether an on-device model could do the work. Edge0-145B expands the range of tasks for which it can.

Why More AI Work Can Stay Local

Local execution addresses four practical obstacles to adopting cloud-only AI:

• Control cloud-model costs. Local inference adds no corresponding cloud-model token or API fees as usage grows.
• Keep sensitive data local. Code, documents, and business records can stay on the hardware, giving teams control over data handling.
• Sustain a responsive experience. Model and inference-engine design address response speed, power use, system responsiveness, and continuous operation together.
• Keep core capabilities available offline. Local tasks continue when connectivity drops; features that depend on online information or services still need a connection.

From Cloud First to More Complex Work On-Device

We see the division of work between devices and the cloud shifting through three stages.

Before September 2026: cloud first, with on-device support. Apple Intelligence reflected this stage: it protected privacy, but handled a limited range of tasks locally while more demanding work relied on cloud capabilities.

After Edge0-35B's September 10 release: everyday tasks locally, complex tasks in the cloud. Local models could take on frequent requests that needed immediate responses. For one leading headphone manufacturer, our hybrid solution now handles 92% of AI usage on-device and 8% in the cloud. Everyday conversation accounts for most use cases on consumer wearables.

With Edge0-145B: toward on-device first, with cloud support. Vibe coding and other complex tasks can now run on a personal computer, giving teams more scope to choose local execution. This extends the range of work that can stay on the device while preserving the cloud for tasks and hardware that need it.

The diagram shows the shift from cloud-first AI to everyday local tasks and then more complex local work. Its colored areas are schematic, not measured usage shares.

Where Cloud AI Still Fits

Hardware constraints still determine which model fits. Compute capacity, heat, and battery life matter: Edge0-35B runs on phones, while Edge0-145B runs on suitably equipped laptops. A very low-compute device cannot currently run a 145B model locally. It needs a smaller model suited to its hardware or cloud access to a model of that size.

Cross-device long-term memory and searches requiring highly current information still require cloud services. Local and cloud models can work together; the appropriate split depends on the task, the hardware, and the information it needs.

The aim is to make local execution a practical choice for demanding work, with a capable model that can keep working close to the person using it. That is the next step toward our goal of enabling intelligence at zero distance for everybody.

Get Started with Edge0

Download the models to run locally, evaluate or fine-tune them, and build them into your own products:

• Model weights: Edge0-145B and Edge0-120B on Hugging Face.
• Inference framework and deployment guides: Edge0 on GitHub.
• Try it without environment setup: Download the Edge0 App. Its core model capabilities work offline, without an account or usage quota.
• Community: Join us on Discord.

For hardware or product integrations, reach the Edge0 team through GitHub or contact@autoark.ai.

Take a voice-note app. It can transcribe your recording on your phone, then send the transcript to a server for summarization. The transcription is on-device; the summary is a separate inference step. That distinction tells you which parts can work offline and which data the app may need to send elsewhere.

On-Device AI vs. Cloud AI vs. Edge AI

Follow the input to the machine running the model. A result displayed on your phone may have been computed on the phone, on a nearby server, or in the cloud.

On-device AI is usually considered a form of edge AI. The edge is the broader category: the computation can run on the endpoint or on infrastructure near the data source. Nearby servers can serve devices with limited resources, while cloud services provide access to centralized compute over a network.

How Does On-Device AI Work?

Training adjusts the model's weights; inference uses those weights to process new inputs. A model can be trained in a data center and optimized for mobile use before it reaches the phone. Here is what happens when the phone uses it to transcribe a recording.

1. Make the model available on the device

The app needs the model files and the software to run them. The files can ship with the app, download when needed, or be managed by a platform service. Bundling can make the model available at installation, while downloading it separately can keep the initial app package smaller. Google's ML Kit installation guide shows how these choices work for different Android APIs.

2. Prepare the input

The app converts the recording to the form the speech model expects. Whisper's reference implementation, for example, represents audio as a log-Mel spectrogram, which tracks energy across frequency bands over time. A text feature has a different input pipeline: a tokenizer converts text into tokens the language model can process.

3. Run inference with a local runtime

The runtime loads the model and executes its operations on the device. Depending on the model and hardware, it may use a CPU, GPU, or an accelerator such as an NPU. An accelerator can only help with the operations and numerical formats it supports. As the ONNX Runtime mobile guide explains, splitting a model across execution providers can affect performance. Test the complete feature on the target device.

4. Turn the model output into a result

The app decodes the model's predictions into text and displays the transcript. What happens next is a separate design choice. The app can save the transcript locally, sync it to an account, or use it as input to another model.

Recorded audio → input preparation → local inference → transcript

What Makes an AI Model Practical on a Device?

A model needs space for its files and enough working memory for inference. Those are different budgets. In addition to any weights it keeps in memory, a runtime needs room for intermediate results, buffers, and caches. Many language models use a key-value (KV) cache to reuse earlier attention calculations. Longer sequences can make that cache larger.

Quantization uses fewer bits to represent model weights and, in some implementations, intermediate values. It reduces weight storage. Whether it also speeds up inference or reduces runtime memory depends on what the runtime and hardware can do with that format. Quality can change after quantization, so evaluate the model on the inputs and tasks that matter to the application.

Model choice also depends on the job. A model trained for speech recognition and a language model that summarizes long documents have different quality and performance targets. Measure what users will experience: time to the first result, sustained speed, peak memory, battery use, and output quality.

How Edge0 35B Runs Locally on an iPhone

Edge0 35B shows why storage and working memory are separate budgets. It is a 35B-class sparse mixture-of-experts (MoE) language model. Its MoE layers route each token to a small subset of their expert networks. That saves computation per token, but the weights for the other experts still have to be stored somewhere. Edge0 tackles the storage, loading, and quality problems together:

• SSD expert offload reduces how many weights need to stay in memory. The Edge0 runtime reads expert weights from local storage into a fixed set of memory slots as they are needed. Shared weights, the KV cache, and runtime buffers still need memory. The runtime can serve the model without keeping all its expert weights resident at once.
• Prerouter overlaps expert loading with decoding. A trained head predicts expert choices a token ahead. In predicted routing mode, the runtime uses those choices for inference and can begin preparing the weights while the current token is still being processed. In an earlier routing comparison, Edge0 reported up to 59% higher decode throughput versus native routing with the model, adapter, and workload held constant.
• Recover-LoRA helps preserve model quality at 4-bit precision. Edge0 freezes the INT4 base and trains LoRA adapters to follow an FP16 teacher. The adapters recover quality lost to quantization and the routing change. They stay separate from the base weights, so different adapter sets can reuse the same read-only checkpoint. See the Edge0 technical report for the training and deployment details.

In our iPhone 17 Pro Max benchmarks, Edge0 35B reached 70+ tokens/s in prefill on a 627-token prompt and about 14–16 tokens/s in decoding across two workloads. Both used iOS Metal, predicted routing, and the benchmark's default 4,096-token context. Both reported 2.80 GiB peak process memory (phys_footprint). Prefill measures how quickly the model processes the prompt; decode throughput measures how quickly it generates new tokens.

Common On-Device AI Examples

• Speech recognition: Dictation or a voice-note app can produce a transcript without uploading the audio for inference.
• Camera features: A camera can classify an image or flag an object in a video stream locally, then send an alert only when needed.
• Writing assistance: On supported devices, Google's ML Kit GenAI APIs offer local rewriting, proofreading, and summarization.
• Translation: ML Kit's translation API can translate on the device after the required language models have been downloaded.
• Industrial sensors: A sensor can flag an unusual vibration pattern locally and send the event rather than the entire raw stream.

Benefits and Limits of On-Device AI

Resource limits still matter. A phone can slow down as it heats up, and the app shares memory with the rest of the device. Results may differ across devices, model versions, and runtimes. For a production feature, also check who can access local data and how model and app updates are delivered.

When Should You Use On-Device, Cloud, or Hybrid AI?

Use on-device AI when a model can meet the feature's quality and performance targets on the device and local operation solves a real problem. Offline dictation, for example, benefits from keeping the entire transcription path available on the phone.

Use cloud AI when the computation is beyond the target devices' capacity or when you need centralized serving across a wide range of hardware. A task that needs live information requires access to that information, but the model itself can still run locally.

Use a hybrid design when different steps have different needs. A meeting app can transcribe locally and use a server model to analyze a long transcript. Decide what information the remote step needs and whether it can work with a smaller input.

Frequently Asked Questions

Do I need an NPU to run AI locally?

No. Models can run on a CPU or GPU if the runtime supports that combination. Some applications require a specific accelerator to deliver the intended speed or battery life.

Can I check whether an AI feature works offline?

After setup, turn off network access and give the feature a new input. A successful result shows that the operation can run offline in that setup. It does not tell you what the app may sync or upload once it reconnects.

Try Edge0 35B

To explore local text generation, start with the runtime and model card. If you’re starting on a desktop, see our guide to running Edge0 35B on Mac and Windows.

• Try Edge0 on GitHub
• View the Edge0-35B-A3B-preview model info on Hugging Face
• Join our Discord to get help with setup or share what you’re building.

Run Edge0-35B in Edge0's desktop app on Mac or Windows.

Requirements

Both platforms need additional space for development tools and build files. Windows also retains a converted model. The full setup footprint has not been measured here.

Run Edge0-35B on Mac

This follows the macOS app's build instructions. You will build the app in Terminal, then download the model and chat in its interface.

1. Install the tools

If Command Line Tools are missing, open Terminal and install them:

xcode-select --install

After installing the tools, reopen Terminal so it picks up their paths.

2. Download Edge0 and check the setup

Download the repository:

git clone https://github.com/Edge0-AI/Edge0.git "$HOME/Edge0"

Enter the macOS directory:

cd "$HOME/Edge0/macos"

Run the prerequisite check:

bash scripts/check-prereqs.sh

Continue when it prints check-prereqs: ok. If it reports that Yarn or Corepack is missing, install the required Yarn version and rerun the check:

npm install --global yarn@1.22.22

3. Build and open the app

make app

This installs the app's dependencies, builds the native engine, and packages the app in macos/dist. Open the current version:

open dist/edge0-0.1.0-arm64.app

If the version changes, use the filename produced by the build. Opening the app also starts its local service.

4. Download the model and chat

1. If the welcome screen appears, click Skip for now. Open Models in the sidebar and find edge0-35b.
2. Click Download and wait for the download to finish.
3. Click Load model and wait for the model to show as loaded.
4. Click New chat, type Explain SSD streaming in one sentence., and press Enter.

A generated response confirms that setup worked. Models and app data are stored under ~/.edge0.

Run it again later

In Finder, open the app inside Edge0/macos/dist. Load the downloaded 35B model if needed, then start a chat.

Run Edge0-35B on Windows

The Windows app currently requires building from source. Keep the repository and engine build in place afterward; the app uses them to run the engine and convert models.

1. Install the tools

Open PowerShell 7 and check that the command-line tools are available:

Get-Command git, pwsh, cmake, cargo, node, npm.cmd, npx.cmd, python

All eight commands should appear. If one is missing, install it and reopen PowerShell. Confirm Python's version:

python --version

Install NumPy for model conversion:

python -m pip install numpy

Check that it imports:

python -c "import numpy; print(numpy.__version__)"

The engine build also needs a Git name and email for its patches. Check:

git -C "$env:USERPROFILE" var GIT_COMMITTER_IDENT

If it reports an unknown identity, follow Git's identity setup before building.

2. Build the engine

Download the repository:

git clone https://github.com/Edge0-AI/Edge0.git "$env:USERPROFILE\Edge0"

Enter the Windows directory:

Set-Location -LiteralPath "$env:USERPROFILE\Edge0\windows"

Build the engine:

pwsh -File scripts/vendor-build.ps1

After a successful build, check the executable:

Test-Path -LiteralPath '..\wt\win\build-vk\bin\Release\llama-server.exe'

It should return True. If it returns False, resolve the build error before continuing.

3. Build and open the app

Enter the app directory:

Set-Location -LiteralPath "$env:USERPROFILE\Edge0\windows\app"

Install its dependencies:

npm.cmd install

Build the app with Tauri 2, Edge0's Windows desktop framework:

npx.cmd tauri build

After a successful build, check the executable:

Test-Path -LiteralPath '.\src-tauri\target\release\edge0-app.exe'

It should return True. Open the app:

.\src-tauri\target\release\edge0-app.exe

4. Download the model and chat

1. Open Models and find 35B.
2. Click Download. Wait for both the download and automatic conversion to finish.
3. Click Enable and wait for the model to show as resident.
4. Click New chat and select 35B if needed.
5. Type Explain SSD streaming in one sentence. Press Enter or click Send.

A generated response confirms that setup worked. Conversion is needed because the Windows engine uses GGUF model files.

Run it again later

Open PowerShell 7 and run:

& "$env:USERPROFILE\Edge0\windows\app\src-tauri\target\release\edge0-app.exe"

Enable the existing model in Models if needed, then start a chat.

Troubleshooting

FAQ

Can I use Edge0-35B offline?

Yes. Complete installation, model download, and Windows conversion first. Edge0 then runs locally using the files on your computer.

Do I need an NVIDIA GPU?

No. Windows supports compatible AMD, NVIDIA, and Intel GPUs through Vulkan. The Mac setup uses Apple Silicon's GPU.

Can I use these Mac instructions on an Intel Mac?

No. The macOS app's current instructions require Apple Silicon, M3 or later.

Is Edge0-35B free to use?

Yes. The model is released under Apache 2.0. This local setup does not require a paid cloud API.

Stay connected

Follow us on X (Twitter) or join our Discord for updates and setup help.