The Ollama CLI: Models, Modelfiles and Serving

Running Large Language Models locally has transitioned from a niche hobby to a core architectural choice for privacy-conscious developers. The Ollama CLI facilitates this transition by providing a streamlined interface for downloading, managing, and interacting with open weights models. By mastering the Ollama CLI, you can integrate sophisticated reasoning capabilities into your local-first applications without relying on external APIs. This guide explores the command structure, the creation of custom Modelfiles, and the mechanics of serving models for external consumption.

The Foundation of Ollama CLI

The command line interface is the primary way to interact with the Ollama daemon. Once the service is running, the ollama command allows you to orchestrate your entire local library. The syntax follows a standard pattern of command followed by arguments.

To verify your installation and see available commands, simply run ollama. The most frequent operations involve fetching new models from the library and running them in an interactive session. For example, ollama run llama3 will pull the weights if they are not present and initiate a chat prompt.

Key commands include:

• ollama list: Shows all models currently stored on your local disk.
• ollama pull: Downloads a model without starting an execution session.
• ollama rm: Removes a model to free up storage space.
• ollama cp: Creates a duplicate of an existing model, which is useful for testing different system prompts.

Managing Local Models Effectively

Managing local resources is critical because LLMs consume significant disk space and memory. When you use the Ollama CLI, you are interacting with a local manifest that maps model names to their underlying blobs.

When working in a complex environment like the Deska canvas, you might have multiple terminals open. Deska allows you to place these terminals anywhere in an infinite workspace, making it easier to monitor ollama list in one panel while running a model in another. This visibility is helpful when you are juggling different versions of Mistral or Llama to compare their output side by side.

Understanding Quantization and Tags

Ollama uses tags to identify specific versions of models. If you do not specify a tag, the CLI defaults to latest. However, technical users often need specific quantization levels to balance performance and accuracy.

Crafting Custom Modelfiles

The Modelfile is the configuration blueprint for Ollama. It allows you to define the behavior, parameters, and system instructions for a model. This is similar to how a Dockerfile defines a container image.

A typical Modelfile contains the FROM instruction to specify a base model. You then add layers such as PARAMETER to control temperature or SYSTEM to define the persona.

FROM llama3
PARAMETER temperature 0.7
SYSTEM "You are a senior software architect. Provide concise, performant code examples."

Once you have saved this file, you use the CLI to create the new model: ollama create my-architect -f Modelfile. This new model will now appear in your list and can be used like any other.

In the Deska code editor, you can write these Modelfiles and immediately test them in a terminal panel placed right next to your code. This layout minimizes context switching. Because Deska is local-first, your custom model configurations and your source code remain on your machine.

Serving and API Integration

The ollama serve command starts the server process that exposes a REST API. This is what allows other applications to talk to Ollama. By default, the server listens on 127.0.0.1:11434.

Integration with developer tools usually happens through this API. For instance, coding agents can use your local Ollama instance to provide completions or refactor code. In the Deska environment, you can run coding agents like Claude Code or Codex CLI. While some agents prefer specific cloud APIs, many can be configured to point to your local Ollama endpoint.

Using local models for agents provides several advantages:

• No latency from network round trips.
• Zero cost per token during long refactoring sessions.
• Enhanced privacy for sensitive proprietary codebases.

Monitoring and Remote Access

When your models are running, monitoring their resource usage is vital. You can use standard system tools or dedicated CLI flags to see how much VRAM is allocated.

If you need to check on a long running task while away from your desk, the Deska mobile app offers a secure way to monitor your workspace. It uses a secure relay to pair your phone directly to your computer without exposing any ports to the public internet. This allows you to check the status of a model pull or a long running generation from your phone.

Frequently Asked Questions

How to change the Ollama model storage location?

You can change where Ollama stores its models by setting the OLLAMA_MODELS environment variable. On Linux or macOS, you would export this variable in your shell profile. On Windows, you add it to your System Environment Variables. This is particularly useful if you want to move large model files to an external SSD.

Can I run Ollama models in parallel?

Ollama handles multiple requests by queuing them or loading multiple models if your VRAM allows. You can adjust the OLLAMA_NUM_PARALLEL environment variable to define how many concurrent requests a model should handle. Note that increasing this number will significantly increase the memory footprint of the service.

How to use Ollama with external coding agents?

To use Ollama with agents, you generally need to set the base URL to your local instance. Most tools expect the OpenAI compatible endpoint format. Ollama provides this at /v1 paths. You can configure the terminals in Deska to export the necessary environment variables so that your agents automatically find the local Ollama service.

Getting Started with Local AI Workspaces

The Ollama CLI provides the power, but your environment determines your productivity. By combining local models with a flexible workspace, you create a robust development pipeline that does not compromise on privacy or speed.

If you are looking for a way to organize your terminals, code, and AI agents in one place, you should explore the Deska workspace. The app is free for Mac, Windows, and Linux. You can download the installer to start building your local-first AI toolkit today.