Running Coding Agents on Ollama: What Works and What Does Not

Running coding agents on Ollama has become a primary goal for developers seeking privacy and reduced latency. The promise of having an autonomous entity that can write code, run tests, and refactor functions entirely on your own hardware is compelling. However, moving from cloud-based providers to local inference involves significant trade-offs in reasoning capacity and token throughput. This post explores the current state of local execution, examining which parts of the agentic workflow are ready for production and which still require the heavy lifting of proprietary models.

The Architecture of Local Coding Agents

A coding agent differs from a standard chatbot because it functions in a loop. It observes a file or terminal output, thinks about the next step, and acts by executing a command or writing code. When you are running coding agents on Ollama, the bottleneck is rarely the speed of the text generation. The difficulty lies in instruction following and maintaining state over a long conversation.

Most local agents rely on a specific interaction pattern. The agent tool sends a system prompt to Ollama, receives a response, and then parses that response to trigger a tool call. If the model fails to output valid JSON or the specific markdown format required by the agent tool, the loop breaks. Larger models like Llama 3 or Mistral are generally reliable here, but smaller models often struggle to stay in character for more than a few turns.

Which Models Actually Work

Not all models available in the Ollama library are suitable for agentic tasks. To be effective, a model needs a high degree of spatial reasoning and the ability to understand complex diff formats.

• Llama 3.1 8B and 70B: These are currently the gold standard for local agents. The 70B version is highly capable at complex refactoring but requires significant VRAM. The 8B version is fast but can sometimes hallucinate file paths.
• DeepSeek Coder V2: This model is specifically tuned for programming. It excels at understanding repository structures and generating syntactically correct code in niche languages.
• Qwen 2.5 Coder: A strong contender for mid-range hardware, offering a balance between speed and reasoning that keeps the agent loop responsive.
• Command R: Designed specifically for RAG and tool use. It is particularly effective when the agent needs to search your local documentation or existing codebase to find context.

The Hardware Reality Check

Running these models locally means your machine is performing two heavy tasks simultaneously: compiling code and running inference. If you are using a unified memory architecture like Apple Silicon, this is manageable. On Windows or Linux machines with dedicated GPUs, you must ensure that your VRAM is sufficient to hold both the model and the context window.

When the context window fills up, Ollama must shift tokens or re-process the prompt. For a coding agent, which might be looking at ten different files, the context can grow to 32k tokens very quickly. If the hardware cannot handle this, the agent starts to "forget" the initial instructions, leading to repetitive errors or nonsensical code suggestions.

Integrating Local Agents into the Workspace

One of the frustrations of working with local agents is the fragmented interface. Switching between a terminal, a browser to check documentation, and the agent output creates cognitive load. This is where a unified environment becomes necessary.

Deska addresses this by providing an infinite canvas workspace where you can place different components side by side. Instead of hidden tabs, you can have your local terminal, a code editor based on Monaco, and your agent output visible all at once. For those running coding agents on Ollama, Deska allows you to run tools like Claude Code, Codex CLI, and OpenCode as dedicated panels within the same view.

Because Deska is local-first, your source code and session data never leave your machine. You can use your own API keys for proprietary models if you need a quick boost in reasoning, but the workspace itself is designed to support the local execution flow that Ollama users prefer. You can zoom out to see the entire system and zoom in to focus on specific terminals or notes.

Managing the Agent Loop

To make a local coding agent successful, you must restrict its scope. Asking an agent to "build a whole app" using a local 8B model will almost certainly fail. Instead, the workflow should be broken down into smaller, verifiable tasks:

1. Identification: Use the agent to find where a specific logic lives.
2. Execution: Ask the agent to write a single function or a unit test.
3. Verification: Have the agent run the test in a terminal panel and report the results.

This modular approach minimizes the impact of model "drift" and ensures that if the agent makes a mistake, it is easy to spot and correct. Tools that allow you to see the agent threads alongside the actual code help in maintaining this oversight.

Voice and Mobile Accessibility

A significant hurdle for local development is being tied to the workstation. If you are running a long-running agent task on your desktop, you might want to monitor it without sitting in your chair for hours.

Deska offers a mobile app that allows you to monitor your workspace through a secure relay. Since the devices pair directly and no ports are exposed to the internet, it maintains the security posture expected of a local tool. You can check the progress of an agent, run a quick command, or read a session log from your phone. Additionally, the Ask Deska assistant supports voice commands to drive the workspace, which can be useful when you need to open multiple panels or check sessions hands-free.

Common Pitfalls and Solutions

The most common issue when using Ollama for agents is the failure to parse tool calls. If your agent is stuck, check the raw output from Ollama. Often, the model adds conversational filler like "Sure, I can help with that," which the agent parser doesn't expect. Using a system prompt that strictly forbids conversational filler can solve this.

Another issue is file system access. Ensure the agent has the correct permissions to read your project folder but be cautious about giving it write access to your entire home directory. A restricted workspace environment is always safer.

FAQ

How to improve Ollama agent speed?

Speed mostly depends on your GPU or unified memory bandwidth. You can improve perceived speed by using quantized models (like Q4_K_M) or by reducing the number of files the agent reads into its context at one time.

Can Ollama agents write full features?

While capable, local models below 70B parameters often struggle with global architectural changes. They are best at writing isolated functions, fixing bugs, or generating unit tests rather than designing entire systems from scratch.

Is it safe to give agents terminal access?

If the agent is running locally on your machine, it has the same permissions as your user. It is recommended to run agents in a controlled environment or a dedicated folder, and always review the commands they propose before execution.

Elevate Your Local Development

Transitioning to local AI tools requires the right environment to be productive. If you are tired of juggling multiple terminal windows and web browser tabs while waiting for local models to respond, a unified canvas might be the solution. You can download Deska for Mac, Windows, and Linux to start organizing your agents, terminals, and code in a single, infinite workspace that puts privacy and local-first principles at the center of your workflow.