Docker has launched a suite of AI development tools designed to simplify building and deploying AI agents. The updates, announced in late 2025 and early 2026, allow developers to define LLMs, tools, and applications as declarative services within Docker Compose, bringing infrastructure-as-code principles to agent development.
Docker Model Runner
Docker Model Runner, now generally available, converts LLMs into OCI-compliant containers that can be pulled from Docker Hub or Hugging Face and run locally. The tool exposes an OpenAI-compatible API, allowing developers to interact with self-hosted models using existing SDKs and libraries without code changes.
The service supports three inference engines: llama.cpp (default, works on all platforms), vLLM (requires NVIDIA GPUs, Linux x86_64 or Windows with WSL2), and Diffusers. Models load into memory only at runtime when requests are made and unload when idle to optimize resources.
Docker Offload
Docker Offload provides transparent access to cloud-based GPU resources from local Docker environments. Rather than learning cloud APIs or managing remote servers, developers work with their standard Docker workflow while compute-intensive tasks execute on cloud infrastructure. This is designed for training or running large models that overwhelm local hardware.
Model Context Protocol Support
Docker has integrated support for the Model Context Protocol (MCP), which provides pre-packaged tool servers that give AI agents access to external capabilities. These include database connections, web search APIs, and other services that agents need to function beyond pure text generation. Docker’s MCP Catalog offers plug-and-play tools that can be added directly to Compose files.
GPU-Optimized Base Images
Docker provides official GPU-optimized base images for frameworks like TensorFlow and PyTorch. These images come with NVIDIA drivers, CUDA, and cuDNN pre-installed, offering a reproducible foundation for custom AI workloads that run identically from development to production.
Declarative Agent Development
The tools enable developers to define entire agent stacks as a single docker-compose.yml file. A typical configuration might include a custom application, a local model server, and a tool server as interconnected services. This approach makes AI infrastructure portable and version-controlled, aligning with modern DevOps practices.
Docker has partnered with Google, Microsoft, and agent SDK providers including LangGraph, CrewAI, and Spring AI to support the ecosystem. The company also introduced Gordon, an AI agent integrated into Docker Desktop that understands containers, images, and environments to help developers debug and generate Dockerfiles.
For implementation details, developers can refer to the official Docker Compose documentation and the GenAI Stack announcement.
Follow Hashlytics on Bluesky, LinkedIn , Telegram and X to Get Instant Updates
