All projects
Pet Segmentation MLOps Pipeline

Pet Segmentation MLOps Pipeline

A full MLOps rebuild of an earlier U-Net/MobileNetV2 pet segmentation system. A two-process PyTorch DistributedDataParallel job, orchestrated by Airflow across train, evaluate, and conditional register tasks, trains on the real Oxford-IIIT Pet Dataset with an ImageNet-pretrained encoder, reaching test IoU 0.828, up from 0.112 in the earlier single-model version. MLflow only promotes a candidate to staging when its test IoU beats the current model; a demo run through the DAG (IoU 0.059) was correctly rejected against a staged model (IoU 0.542), verifying the gate end to end. A custom Rust extension (PyO3, compiled with Maturin) handles mask post-processing, checked for exact parity against a NumPy/SciPy reference before benchmarking 2.17× faster. The service is built as a multi-stage Docker image and deployed to a local kind Kubernetes cluster as a two-replica FastAPI Deployment.

Technologies Used

Core

  • Python
  • PyTorch
  • FastAPI

Orchestration

  • Airflow
  • MLflow

Systems

  • Rust
  • PyO3
  • Maturin

Deployment

  • Docker
  • Kubernetes