Ollama review

Ollama is an open-source runtime and platform, not just a hosted inference API. Its CLI manages open-weight models, while a background service exposes a local HTTP REST API on port 11434. That puts it between model weights and the application that sends prompts: you can run models on a workstation, an air-gapped system, on-premises GPU infrastructure or a self-hosted cloud machine.

The runtime addresses the setup and model-management work involved in running inference yourself. It can pull and run models, and its Modelfile syntax configures base weights, prompts, inference settings, templates and LoRA adapters. Ollama also offers hosted cloud models through the same API. The vendor says local use has zero data retention, with prompts and outputs neither stored nor used for model training. Ollama runs on macOS, Windows and Linux, and provides an official Docker image for containerized deployments.

The local service exposes HTTP REST endpoints for completions, chat and embeddings on port 11434. Ollama also provides OpenAI-compatible and Anthropic-compatible endpoints, which can let applications using those API patterns point at Ollama instead. Official client SDKs are available for Python and JavaScript.

The CLI covers pulling, running, listing, copying, stopping and deleting models. For customization, Modelfiles set base weights, system prompts, inference parameters, templates and LoRA adapters. The runtime imports GGUF, PyTorch and Safetensors weights, and supports image inputs, structured JSON schema outputs, and function or tool calling. Hardware acceleration includes Apple Silicon Metal, NVIDIA CUDA, AMD ROCm and Vulkan.

Ollama's documentation catalogs more than 100 community integrations, including Open WebUI, AnythingLLM, LibreChat, Continue, Cline, Void, LangChain and LlamaIndex. That is a broad set of starting points, but check each integration's documentation for its supported API behavior and model features.

Running Ollama locally has no subscription price: the software is open-source under MIT, though you supply the hardware. For hosted models, Pay as you go has a $0 subscription, three concurrent requests and per-token billing. It lists no service fees, but the vendor doesn't publish token rates here, so estimate cloud cost against your expected prompt and output volume before shipping.

Pro costs $20 monthly or $200 billed annually, equivalent to $16.67 per month, and includes $60 in monthly usage credits and three concurrent requests. Max is $100 per month with $300 in credits and 10 concurrent requests. Team costs $500 per month and includes unlimited users, $1,000 in shared monthly credits, 10 concurrent requests and priority support. Usage beyond Team's included credits is billed as you go from the shared balance. Enterprise has custom pricing and includes model access controls, budgets, dedicated support via private Slack and custom security questionnaires. Extra usage credits can be purchased; their prices are not published here.

Ollama is open-source software under the MIT license, and the vendor supports self-hosting on local machines, air-gapped systems, on-premises GPU infrastructure and cloud environments such as AWS, Google Cloud and Azure. That gives teams a path to keep inference within infrastructure they control rather than depending on Ollama's hosted service.

Model imports include GGUF, PyTorch and Safetensors, and applications can connect through REST endpoints or compatible API endpoints. Those interfaces can reduce the work of changing inference backends, but custom Modelfiles and model-specific behavior may still need to be recreated elsewhere. Local models and prompts stay in the environment where they run; the vendor doesn't publish a cloud data-export policy here, so ask about export and deletion before storing production data in hosted workflows.

The hosted plans publish concurrency caps, not a throughput guarantee: Pay as you go and Pro allow three concurrent requests, while Max and Team allow 10. The local API's request limits, service-level guarantees and scaling guidance are not stated here. Before relying on hosted inference, confirm capacity, availability commitments and usage behavior with the vendor; for local deployment, plan GPU capacity and monitor the service yourself.

The vendor says local prompts and outputs are not stored or used for training. Treat that as a vendor claim and review your deployment configuration, network exposure and data-handling requirements before connecting sensitive workloads. In Reddit discussions, some users report poor performance or limited customization compared with lower-level alternatives; others point to Ollama's easier setup. Threads also revisit concerns about how the project credited llama.cpp. These are user reports and opinions, not independent performance or license findings.