Pet Segmentation MLOps Pipeline
A full MLOps rebuild of an earlier U-Net/MobileNetV2 pet segmentation system. A two-process PyTorch DistributedDataParallel job, orchestrated by Airflow across train, evaluate, and conditional register tasks, trains on the real Oxford-IIIT Pet Dataset with an ImageNet-pretrained encoder, reaching test IoU 0.828, up from 0.112 in the earlier single-model version. MLflow only promotes a candidate to staging when its test IoU beats the current model; a demo run through the DAG (IoU 0.059) was correctly rejected against a staged model (IoU 0.542), verifying the gate end to end. A custom Rust extension (PyO3, compiled with Maturin) handles mask post-processing, checked for exact parity against a NumPy/SciPy reference before benchmarking 2.17× faster. The service is built as a multi-stage Docker image and deployed to a local kind Kubernetes cluster as a two-replica FastAPI Deployment.
Technologies Used
Core
- Python
- PyTorch
- FastAPI
Orchestration
- Airflow
- MLflow
Systems
- Rust
- PyO3
- Maturin
Deployment
- Docker
- Kubernetes