Modal review

Modal is a hosted cloud compute platform, not a model library or a server you install in your own infrastructure. It provides container-based infrastructure for running AI and compute-heavy applications, including model inference, training, agent sandboxes and serverless functions.

In a typical stack, Modal sits behind application code that needs on-demand CPU or GPU resources. Its functions can run arbitrary code, launch async batch work and expose HTTPS endpoints. For inference, teams can deploy custom stacks or use OpenAI-compatible endpoints. Training workloads include fine-tuning and multi-node runs, while sandboxes provide isolated environments for agent tasks.

The platform handles container scheduling, capacity and autoscaling. It says workloads can scale up for demand and back to zero when idle, with billing based on active resource use. Modal also provides a distributed filesystem for model weights, datasets and container images. This makes it infrastructure for the execution layer of an AI application, rather than a complete application framework.

Modal's homepage describes a container-based platform for running functions, inference servers, training jobs and sandboxes. The integration surfaces it names include HTTPS endpoints for functions, OpenAI-compatible endpoints for inference, and OpenTelemetry providers for custom trace export. You can bring a custom inference stack and run arbitrary code; the platform also supports async batch jobs and long-running work.

The homepage does not name programming languages, SDKs or a CLI. Confirm supported languages, SDK installation and deployment workflows in Modal's developer docs before choosing it for a codebase. The same check applies to API authentication, local development and CI integration: the homepage does not spell out those details.

Modal says its container runtime and image system are built for large images and GPU scheduling. That infrastructure may suit workloads with custom dependencies, but teams should verify that their build and deployment process fits the documented integration path. For external monitoring, the stated option is exporting custom traces to an OpenTelemetry provider.

Starter costs $0 per month and includes $30 in monthly compute credits, 3 workspace seats, 100 containers, 10 GPU concurrency, 1 TiB of monthly network egress and 5 deployed cron jobs. Team costs $250 per month and includes $100 in compute credits, unlimited seats, 5,000 containers, 50 GPU concurrency, 10 TiB of egress and 30-day log retention. Enterprise has custom pricing and compute, unlimited seats, 100 TiB of monthly egress, HIPAA compatibility, audit logs and private Slack support.

Compute is billed by usage: CPU costs $0.0000131 per core-second and memory $0.00000222 per GiB-second. GPU prices vary by model, from $0.000164 per second for an Nvidia one thread to $0.001972 per second for an Nvidia B300. Volumes cost $0.09 per GiB per month, with 1 TiB per month free; network egress is $0.04 per GiB, subject to plan allowances. Region selection costs 1.15–1.75× base prices, while non-preemptible execution costs 3×. The pricing page lists monthly plan prices, not annual rates.

Modal describes a hosted cloud platform, but its homepage does not publish an open-source licence or say whether the service can be self-hosted. It also does not publish data-export details. Ask Modal about source availability, deployment options and how to retrieve stored data before treating the platform as portable infrastructure.

The likely migration work depends on how much of your application uses Modal-specific execution, sandbox and storage features. Moving application code may be straightforward if it stays independent of those interfaces; replacing the hosted compute, scheduling and distributed filesystem layer will require an alternative runtime and storage path. OpenAI-compatible endpoints and OpenTelemetry export provide named integration surfaces, but do not establish full workload portability. Confirm which data and artifacts can be exported, in what formats, and whether workloads can be moved without changing application code.

Modal says it can scale workloads to thousands of CPU and GPU resources and back to zero, and that it runs continuous hardware health checks. Those are useful operational claims, but production teams should validate capacity, startup behavior and failure handling for their own images, regions and traffic patterns. The homepage advertises sub-second sandbox starts, including for large custom images and at high concurrency; confirm the conditions and service commitments behind that claim with the vendor.

Limits vary by plan: Starter includes 10 GPU concurrency and 100 containers, while Team includes 50 GPU concurrency and 5,000 containers. Network egress allowances are 1 TiB, 10 TiB or 100 TiB monthly across the listed plans. Check workload peaks, logs retention and the cost of exceeding included resources before deployment.

Modal lists workload isolation, configurable network and endpoint access, granular RBAC, audit logs and data residency controls. It says it is SOC 2 Type II compliant and HIPAA ready; Enterprise lists HIPAA compatibility and audit logs. Confirm requirements and support arrangements, including private Slack support, before relying on them.