ZenML

ZenML is an MLOps pipeline framework that lets you build portable, production-ready ML pipelines that run on any infrastructure — local machines, Kubernetes, cloud services, or managed platforms — without changing your pipeline code.

Key use cases include:

• Building reproducible ML pipelines with versioned artifacts and metadata tracking
• Infrastructure portability — develop locally, deploy to any cloud without code changes
• Model training pipelines — orchestrate data loading, preprocessing, training, and evaluation steps
• LLMOps — pipelines for fine-tuning, RAG data preparation, and model evaluation
• Continuous training — automated retraining triggered by data drift or schedule
• Stack management — swap infrastructure components (orchestrators, artifact stores, deployers) via configuration

ZenML is used by ML teams who need production ML pipelines without vendor lock-in. Its stack abstraction lets you plug in different orchestrators (Airflow, Kubeflow, Vertex AI), artifact stores (S3, GCS), and experiment trackers (MLflow, W&B) through configuration.

1. Install ZenML:

   pip install zenml

2. Initialize a project:

   zenml init

3. Build a pipeline:

   from zenml import step, pipeline
   
   @step
   def load_data() -> dict:
       return {'data': [1, 2, 3]}
   
   @step
   def train_model(data: dict) -> str:
       return 'trained_model'
   
   @pipeline
   def my_pipeline():
       data = load_data()
       train_model(data)
   
   my_pipeline()

4. Register cloud stacks for production deployment:

   zenml stack register my-cloud-stack -o kubernetes -a s3

5. See docs.zenml.io for full documentation.

Pricing: ZenML is free and open source (Apache 2.0). ZenML Cloud (managed dashboard and deployment) has a free tier and paid plans starting at $49/month. You pay separately for your infrastructure.