Conduit

Conduit is an open-source framework for tuning, deploying, and building applications on open-source models. Application code declares the models, schemas, and compute it needs; Conduit calculates the resources, provisions the infrastructure, and keeps it healthy. The same code runs on any provider. Sovereignty over models, data, and compute stays with the owner.

PROPERTIES

1.0The problem

The models are open. The infrastructure to run them is not. Deploying an open-source model demands GPU allocation, container orchestration, runtime selection, and provider-specific configuration. Work that should take hours takes weeks, so most developers default to API calls to centralized providers, who see the data, control the access, and can change terms at will.

2.0Design

Application code defines what is needed; Conduit handles the rest. Models, schemas, and compute constraints are declared in Python. Conduit calculates VRAM and GPU requirements, validates that everything fits, provisions infrastructure on the target provider, and gates traffic behind health checks. Switching providers is a change of configuration, not of application code.

Pipelines are built by chaining runtime blocks. Each block conforms to Conduit's communication spec, so inference, network, and storage compose the same way.

Two components carry the design. LM Lite is the batching engine: configurable batch timeouts, round-robin load balancing across replicas, readiness gates, GPU-aware routing, and multiple models served from a single GPU. MDL is the type system: input and output schemas written as Python dataclasses, compiled into model-safe specifications, every payload validated on request and response. A block call returns a guaranteed valid instance of the declared output type, never a string to parse.

3.0Validation

• 3.1Automatic GPU and memory requirement calculation, validated before provisioning.
• 3.2Multi-model deployments with round-robin load balancing across replicas.
• 3.3Health-check readiness gates hold traffic until nodes are ready.
• 3.4Type-safe input and output with automatic payload validation.
• 3.5Released open source; installs against Python 3.12.6+.

4.0Position

Agent frameworks decide what an AI system does and assume the infrastructure exists; Conduit is the layer they stand on. Managed inference services trade control for convenience; Conduit gives the same convenience and leaves control with the owner. DevOps toolkits demand separate expertise and separate configuration; Conduit embeds infrastructure in the application itself. What Linux did for software across hardware, Conduit does for AI across compute. Write once, deploy anywhere.

CITE

@techreport{covenant2026conduit,
  title     = {Conduit: An AI Infrastructure Framework},
  author    = {Covenant Labs},
  year      = {2026},
  number    = {Output 003},
  url       = {https://covenantlabs.ai/research/output-003}
}