A mission-critical reference for structuring, training, and deploying real-world ML systems. Every domain from Computer Vision to Generative AI, each with pipeline diagrams, production folder structures, artifact checkpoints, file extension glossary, and expert engineering tips.
Computer Vision teaches
machines to interpret and understand visual information from images and video. It is the backbone of
self-driving cars, medical imaging diagnostics, quality control in manufacturing, and real-time security
systems. The core challenge is converting raw pixel arrays — tensors of shape (H, W, C) — into structured predictions: bounding boxes, segmentation masks,
or class labels.
The lifecycle of a CV project is highly iterative. Labeling is expensive and annotation quality is the single largest determinant of model performance. A poorly labelled dataset with 100k images will consistently underperform a well-labelled dataset with 10k images. This is why Human-in-the-Loop (HITL) labeling pipelines — where model predictions are reviewed and corrected by humans — are now the professional standard.
/data/raw/
directory. Never modify raw data — all transformations produce new artefacts. Maintain a
manifest.csv with image paths, capture dates, and source
metadata for traceability. Minimum class balance: no class should have fewer than 10% of the
dominant class's samples.
__getitem__ method so they're
generated on-the-fly (saves disk space). Critical augmentations for CV: horizontal/vertical
flip, random crop, brightness/contrast jitter, Gaussian blur, and for detection: mosaic and
mixup (used in YOLOv8 by default).yolo train data=config.yaml model=yolov8n.pt epochs=100 imgsz=640
to fine-tune on a custom dataset. PyTorch + torchvision gives more
architectural control: use resnet50 pretrained on ImageNet,
freeze backbone layers, and replace the final fully-connected layer with your class count.
Monitor with W&B or TensorBoard. Use Kaggle's free T4 GPU (30hr/week) for training.model.export(format='onnx'). ONNX models run with onnxruntime — no PyTorch dependency at inference time. Use OpenCV
for pre/post-processing: reading frames, resizing, drawing detection outputs. Wrap in
FastAPI: accept multipart image uploads, preprocess, run inference, return JSON with
bounding boxes and confidence scores. Deploy the FastAPI app in a Docker container.This structure follows the YOLO convention,
compatible with both Ultralytics and standard PyTorch training loops. The separation of /images and /labels mirrors how YOLO tools expect
data on disk.
cv-project/
├── .env # API keys, model endpoints — NEVER commit
├── .gitignore # Ignore /data, /weights, __pycache__
├── requirements.txt # Pinned: torch==2.2.0, ultralytics==8.2.0
├── config.yaml # YOLO dataset config (path, nc, names)
│
├── data/
│ ├── raw/ # Original images — READ ONLY, never modify
│ │ ├── images/
│ │ └── manifest.csv # filename, source, capture_date, split
│ ├── processed/ # After augmentation/resize
│ │ ├── train/
│ │ │ ├── images/ # .jpg / .png — 640x640 normalized
│ │ │ └── labels/ # .txt — YOLO format: class cx cy w h
│ │ ├── val/
│ │ │ ├── images/
│ │ │ └── labels/
│ │ └── test/
│ │ ├── images/
│ │ └── labels/
│ └── README.md # Dataset card: source, license, version
│
├── src/
│ ├── dataset.py # PyTorch Dataset class, augmentation pipeline
│ ├── train.py # Training loop, optimizer, scheduler
│ ├── evaluate.py # mAP, IoU, confusion matrix
│ ├── inference.py # Load model, preprocess, postprocess
│ └── utils.py # draw_boxes(), iou(), xywh2xyxy()
│
├── api/
│ ├── app.py # FastAPI app — /predict, /health endpoints
│ ├── schemas.py # Pydantic request/response models
│ └── Dockerfile # Production container
│
├── weights/ # Trained model artefacts — in .gitignore
│ ├── best.pt # Best checkpoint (lowest val loss)
│ ├── last.pt # Final epoch checkpoint
│ └── best.onnx # Exported for deployment
│
├── inference/
│ ├── test_images/ # Sample images for smoke testing
│ └── results/ # Output images with drawn detections
│
└── notebooks/
├── 01_eda.ipynb # Dataset exploration, class distribution
├── 02_training.ipynb # Training experiments
└── 03_evaluation.ipynb # mAP curves, error analysis/images/ in the path with /labels/. If your structure deviates, the dataloader silently skips
annotations. Additionally, separating raw from processed data ensures you always have an uncorrupted
source to re-augment from if you change your preprocessing strategy.torch.save(model.state_dict(), 'best.pt') and load with
model.load_state_dict(torch.load('best.pt')). This is the preferred
format for PyTorch projects — portable and architecture-agnostic.torch.save({'epoch': epoch, 'model': model.state_dict(), 'optimizer': opt.state_dict(), 'loss': loss}, 'checkpoint.pth').
Larger file but contains everything needed to continue training exactly where it stopped.
.pt
instead. If you receive a .weights file, convert it: python convert_weights.py --cfg yolov4.cfg --weights yolov4.weights --output yolov4.pt.
torch.onnx.export(model, dummy_input, 'model.onnx'). Run on any
hardware (CPU/GPU/NPU) using onnxruntime. Zero PyTorch dependency at
inference — critical for lightweight deployment. Validate with onnx.checker.check_model(model).converter = tf.lite.TFLiteConverter.from_saved_model(path); converter.optimizations = [tf.lite.Optimize.DEFAULT].
INT8 quantisation reduces model size by 75% with <1% accuracy drop.trtexec --onnx=model.onnx --saveEngine=model.engine --fp16.
Store Roboflow API
keys, W&B tokens, and cloud credentials in .env. Load with from dotenv import load_dotenv; load_dotenv(). Your .gitignore must include .env — one
accidental commit exposes your credentials to the entire internet permanently (git history is
forever).
Add data/raw/, data/processed/, weights/, *.pt, *.onnx to .gitignore. Use
DVC (Data Version Control) instead: dvc add data/processed/ pushes data to S3/GDrive while tracking
metadata in git. Never store multi-GB datasets in GitHub.
Always seed
everything: torch.manual_seed(42); np.random.seed(42); random.seed(42). For full
determinism on GPU: torch.backends.cudnn.deterministic = True. Log
the exact commit hash, dataset version, and hyperparameters to W&B so any experiment can be
perfectly reproduced 6 months later.
Use FROM python:3.11-slim as base, not FROM pytorch/pytorch (adds 3GB). Install only onnxruntime-gpu and opencv-python-headless
for a lean inference container. Copy weights in the Dockerfile: COPY weights/best.onnx /app/weights/. The resulting image should be
under 1GB.
Natural Language Processing has been completely transformed by the Transformer architecture. The paradigm shift from task-specific models to fine-tuning pre-trained Large Language Models means that most production NLP today works like this: take a model that has read the entire internet (GPT, BERT, LLaMA, Mistral), then fine-tune it on your domain-specific data using a fraction of the compute. This is called Transfer Learning for NLP.
The architectural journey of text: Raw Text → Tokenization (split into subword tokens via BPE/WordPiece) → Embeddings (each token becomes a high-dimensional vector, e.g., 768-dim for BERT-base) → Transformer Layers (attention + feed-forward) → Inference (classification head, generation head, or embedding extraction). Understanding this pipeline lets you intervene at exactly the right stage.
trafilatura for web scraping, pdfminer.six for PDFs, beautifulsoup4
for HTML. Cleaning pipeline: remove HTML tags, normalize Unicode (unicodedata.normalize('NFKC', text)), strip boilerplate,
deduplicate with MinHash LSH (use datasketch).
Near-duplicates destroy evaluation metrics — a model that memorises
training samples scores perfectly on test if the test set has duplicates.
pip install label-studio && label-studio start. Configure
a NER task with a simple XML template: highlight spans and assign entity types (PERSON, ORG,
PRODUCT). Export as CONLL-2003 format for token-classification models. Inter-annotator
agreement (Cohen's Kappa) should exceed 0.8 before training.input_ids (token
integers), attention_mask (1 for real tokens, 0 for padding),
token_type_ids (segment IDs for BERT). Save the tokenizer
alongside your model: tokenizer.save_pretrained('./checkpoints/v1/').
bitsandbytes), enabling fine-tuning of LLaMA-7B on a single 16GB
GPU. Use the peft library: model = get_peft_model(model, lora_config).
text-embedding-3-small or all-MiniLM-L6-v2 → (3) Store in a vector database
(Chroma, Pinecone, Weaviate) → (4) At query time: embed query, retrieve
top-k chunks by cosine similarity, inject into system prompt → (5) LLM
generates grounded response. LangChain and LlamaIndex provide orchestration.wandb.init(project="my-llm", config=hyperparams). Log metrics:
wandb.log({"train/loss": loss, "eval/perplexity": ppl}). W&B
automatically tracks your system metrics (GPU utilisation, memory), code version (git commit
hash), and environment. Compare runs visually to identify which hyperparameter changes
actually improved the model. Create Model Registry entries for promoted checkpoints.
nlp-project/
├── .env # HF_TOKEN, WANDB_API_KEY, OPENAI_API_KEY
├── .gitignore # /datasets, /checkpoints, *.bin, *.safetensors
├── requirements.txt # transformers, peft, bitsandbytes, datasets, wandb
│
├── configs/
│ ├── lora_config.yaml # r=16, lora_alpha=32, target_modules=[q_proj,v_proj]
│ ├── training_args.yaml # lr=2e-4, epochs=3, batch_size=4, fp16=True
│ └── data_config.yaml # max_seq_length=2048, split ratios
│
├── datasets/ # In .gitignore — track with DVC
│ ├── raw/ # Original text files, CSVs, JSONL
│ ├── labeled/ # Label Studio exports (NER, classification)
│ │ ├── train.jsonl # {"text": "...", "label": "POSITIVE"}
│ │ ├── val.jsonl
│ │ └── test.jsonl
│ └── processed/ # After tokenization + formatting
│ └── tokenized_train/ # HuggingFace datasets on-disk cache
│
├── src/
│ ├── data/
│ │ ├── loader.py # load_dataset(), preprocessing_fn()
│ │ └── collator.py # DataCollatorForSeq2Seq, padding logic
│ ├── models/
│ │ ├── base_model.py # Load pretrained + apply LoRA config
│ │ └── rag_pipeline.py # Retriever + LLM chain
│ └── training/
│ ├── trainer.py # HuggingFace Trainer wrapper
│ └── evaluate.py # ROUGE, BLEU, perplexity, exact match
│
├── tokenizer/ # Saved tokenizer files
│ ├── tokenizer.json # Vocabulary + merges rules
│ ├── tokenizer_config.json # Special tokens, model_max_length
│ └── special_tokens_map.json # [PAD], [CLS], [SEP], [MASK]
│
├── checkpoints/ # In .gitignore — track with DVC/HF Hub
│ ├── checkpoint-500/ # Mid-training checkpoint (auto-saved)
│ ├── checkpoint-1000/
│ └── final/ # Best model checkpoint
│ ├── adapter_model.safetensors # LoRA weights only (~50MB)
│ ├── adapter_config.json # LoRA configuration
│ └── trainer_state.json # Training history
│
├── api/
│ ├── app.py # FastAPI: /chat, /embed, /classify endpoints
│ ├── inference.py # Load merged model, tokenizer, generate()
│ └── Dockerfile
│
└── notebooks/
├── 01_data_exploration.ipynb
├── 02_fine_tuning.ipynb
└── 03_evaluation.ipynb.bin. Safe
to load from untrusted sources (unlike pickle-based formats). Supports lazy loading (read only
the tensors you need), memory-mapped for zero-copy loading, and verified with a checksum. Always
prefer this format: model.save_pretrained('./output', safe_serialization=True).
torch.save()
under the hood (pickle-based). Large models are split into shards: pytorch_model-00001-of-00003.bin. Functional but prefer safetensors
for new projects. Security risk if loading from untrusted sources.llama.cpp and Ollama for running LLMs on CPU without CUDA. Stores
the model in quantized format: Q4_K_M (4-bit mixed quantization) reduces a 7B model from ~14GB
to ~4GB with minimal quality loss. Essential for local deployment on laptops. Convert from
HuggingFace format: python convert_hf_to_gguf.py --model llama3.model = PeftModel.from_pretrained(base_model, './adapter/'). This
means you can fine-tune 10 different tasks and share the same base model weights — storing only
the small adapters.Most gated models
(LLaMA-3, Mistral) require a HuggingFace token. Store it as HF_TOKEN=hf_xxx in .env and load with
login(token=os.getenv('HF_TOKEN')). Never hardcode tokens in
notebooks — Jupyter notebooks save outputs to git history.
Loss decreasing doesn't mean the model is improving — it might be overfitting. Track perplexity ($e^{\text{loss}}$) on a held-out evaluation set. For chat models, also run MT-Bench or MMLU to measure general capability degradation from fine-tuning.
Text datasets
change (new annotations, cleaned versions). Track with DVC: dvc add datasets/labeled/train.jsonl pushes data to S3 and stores a
.dvc pointer in git. You can check out the exact dataset version
from any git commit.
Enable Flash
Attention 2 for 2-4x training speedup: model = AutoModelForCausalLM.from_pretrained(name, attn_implementation="flash_attention_2").
Requires Ampere or newer GPU (A100, RTX 3090+). Reduces memory from $O(n^2)$ to $O(n)$ with
respect to sequence length.
Despite the hype around deep learning, the majority of production ML systems in finance, healthcare, and e-commerce run on tabular data with tree-based models. Gradient boosted decision trees (XGBoost, LightGBM, CatBoost) consistently outperform neural networks on tabular tasks. The real complexity isn't the model — it's the feature engineering pipeline.
A production tabular ML system is a scikit-learn Pipeline with carefully ordered transformations: imputation → encoding → scaling → feature selection → model. Wrapping this in a Pipeline object is non-negotiable — it ensures preprocessing is always fitted on training data only and applied consistently at inference time, preventing the #1 source of production ML bugs: data leakage.
from ydata_profiling import ProfileReport; profile = ProfileReport(df, title="EDA"); profile.to_file("eda_report.html").
This generates a comprehensive report: distributions, missing value patterns, correlations,
duplicate detection, and interaction plots — in one command. Read the report before writing
a single preprocessing line. It tells you which features are constants (drop them), which
are highly correlated (consider dropping one), and which have MNAR (Missing Not At Random)
patterns.ColumnTransformer for heterogeneous features:
apply StandardScaler to numeric columns, OneHotEncoder to categorical columns, and SimpleImputer to handle nulls — all within a single Pipeline(steps=[...]). The final step is always the model. This
prevents any possible data leakage because the entire transformation chain is fit only on
training data.np.log1p(x)), interaction features (feature1 ×
feature2), target encoding with K-fold smoothing for high-cardinality
categoricals. Use FeatureEngine or custom transformers that
inherit from BaseEstimator, TransformerMixin to make them
Pipeline-compatible.
explainer = shap.TreeExplainer(model); shap_values = explainer.shap_values(X_test).
SHAP summary plots show global feature importance. SHAP force plots show why the model made
a specific prediction. This is often a regulatory requirement in finance
(GDPR Article 22: right to explanation) and healthcare. Tree-based SHAP is exact (not
approximate) and runs in O(n log n).tabular-ml-project/
├── .env # DB connection strings, API keys
├── .gitignore # /data/raw, /data/processed, *.pkl, *.joblib
├── pyproject.toml # PEP 517 build, locked dependencies
│
├── data/
│ ├── raw/ # Original data — READ ONLY
│ │ ├── train.csv
│ │ ├── test.csv
│ │ └── data_dictionary.md # Schema: column name, type, description, nulls
│ ├── processed/ # Cleaned + engineered features
│ │ ├── X_train.parquet # Parquet > CSV: 5-10x smaller, preserves dtypes
│ │ ├── X_val.parquet
│ │ ├── y_train.parquet
│ │ └── y_val.parquet
│ └── external/ # Lookup tables, reference data
│
├── features/
│ ├── transformers.py # Custom sklearn transformers (inherit BaseEstimator)
│ ├── datetime_features.py # DatetimeFeatureExtractor
│ ├── target_encoder.py # KFoldTargetEncoder
│ └── column_config.py # NUMERIC_COLS, CAT_COLS, TARGET, ID_COLS lists
│
├── preprocessing/
│ ├── pipeline.py # build_pipeline() → sklearn Pipeline object
│ ├── validation.py # Great Expectations: schema, distribution checks
│ └── split.py # train_test_split with stratification logic
│
├── models/
│ ├── train_xgb.py # XGBoost + Optuna hyperparameter search
│ ├── train_lgbm.py # LightGBM equivalent
│ ├── train_catboost.py # CatBoost equivalent
│ ├── stacking.py # Stacking ensemble: StackingClassifier
│ └── baseline.py # DummyClassifier, LogisticRegression baselines
│
├── evaluation/
│ ├── metrics.py # AUC, F1, calibration, lift curves
│ ├── shap_analysis.py # SHAP summary, force, dependency plots
│ └── fairness.py # Demographic parity, equal opportunity checks
│
├── artifacts/ # In .gitignore — track with MLflow/DVC
│ ├── pipeline_v1.joblib # Complete sklearn Pipeline (preprocessor + model)
│ ├── metadata_v1.json # {"version":"1.0","train_auc":0.94,"features":42}
│ └── shap_explainer_v1.pkl # Pre-computed SHAP explainer for fast inference
│
└── api/
├── app.py # FastAPI: /predict, /explain endpoints
├── schemas.py # Pydantic input validation (catches wrong dtypes)
└── Dockerfilejoblib.dump(pipeline, 'pipeline.joblib', compress=3). compress=3 gives
~50% size reduction with minimal speed penalty. Always save the full Pipeline, never just the
model.pickle.HIGHEST_PROTOCOL
for smallest file size. Limitation: pickle files are Python-version and sometimes
library-version sensitive. A .pkl saved with scikit-learn 1.3 may
not load in scikit-learn 1.1. Always record the exact library version in metadata.json.df.to_parquet('features.parquet', index=False).Generative AI learns the underlying data distribution and samples from it to create new content — images, audio, video, code, molecules. Two paradigms dominate: Diffusion Models (Stable Diffusion, DALL-E, Flux) which iteratively denoise random noise into structured output, and Generative Adversarial Networks (GANs) where a generator and discriminator play an adversarial game.
The key conceptual insight for diffusion models is the latent space: the VAE (Variational Autoencoder) compresses a 512×512×3 image (786k values) into a 64×64×4 latent tensor (16k values) — a 49x compression. All the denoising happens in this compressed space, making it computationally feasible. The U-Net (or DiT Transformer) learns to predict the noise added at each timestep. At inference, you start from pure Gaussian noise and iteratively remove predicted noise, guided by CLIP text embeddings.
kohya-ss/sd-scripts for LoRA training. Dataset
requirement: 10-30 high-quality images with descriptive captions. Use a unique trigger word:
"photo of [V] person".cog.yaml and predict.py; cog push r8.im/username/model deploys it globally. Billed per
second of GPU time. Modal provides a Python-native serverless GPU platform
— write a function, decorate with @app.function(gpu="A10G"), and
it scales to zero. ComfyUI provides a node-based workflow editor for
chaining diffusion operations visually, with API mode for programmatic access.genai-project/
├── .env # HF_TOKEN, REPLICATE_API_TOKEN
├── .gitignore # /models, /datasets, *.safetensors, *.ckpt
│
├── datasets/ # DVC-tracked, not in git
│ ├── raw_images/ # Original training images
│ ├── captions/ # BLIP-2 auto-captions or manual .txt files
│ └── preprocessed/ # Resized to 512x512 / 1024x1024 + buckets
│
├── training/
│ ├── lora_config.yaml # network_dim=32, alpha=16, scheduler=cosine
│ ├── train_dreambooth.py # DreamBooth fine-tuning script
│ ├── train_lora.py # LoRA fine-tuning (kohya-style)
│ └── caption_generator.py # BLIP-2 auto-captioning pipeline
│
├── models/ # In .gitignore
│ ├── base/ # Base model (e.g., SD1.5, SDXL, Flux.1)
│ │ └── model_index.json # Diffusers model manifest
│ ├── lora/ # Trained LoRA adapters
│ │ ├── subject_lora.safetensors
│ │ └── style_lora.safetensors
│ └── vae/ # Optional improved VAE
│
├── inference/
│ ├── generate.py # StableDiffusionPipeline, prompt engineering
│ ├── img2img.py # Image-to-image with denoising strength
│ └── controlnet.py # ControlNet: pose/depth/edge conditioning
│
├── evaluation/
│ ├── fid_score.py # Fréchet Inception Distance (lower = better)
│ ├── clip_score.py # Text-image alignment (higher = better)
│ └── sample_grid.py # Generate evaluation sample grids
│
├── api/
│ ├── predict.py # Cog prediction class (for Replicate)
│ ├── cog.yaml # Replicate container config
│ └── modal_app.py # Modal serverless deployment
│
└── notebooks/
├── 01_data_prep.ipynb
├── 02_training_lora.ipynb
└── 03_inference_experiments.ipynbmodel_index.json (manifest), unet/, vae/, text_encoder/ subdirectories each
containing diffusion_pytorch_model.safetensors and config.json. Load with StableDiffusionPipeline.from_pretrained('./model_dir/')..safetensors. Convert: python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path model.ckpt --dump_path ./output_dir/.
from_pretrained() to correctly reconstruct the full pipeline. Always
include when distributing a model.Time series data has a critical property that breaks standard ML assumptions: temporal autocorrelation. Observations are not independent — yesterday's sales predict today's sales. This invalidates random train/test splits (leaks future data into training) and requires specialized models that explicitly model temporal dependencies. Applications span demand forecasting, stock prediction, anomaly detection in telemetry, weather prediction, and energy consumption modeling.
train = df[df['date'] < '2024-01-01']; test = df[df['date'] >= '2024-01-01'].
For cross-validation, use TimeSeriesSplit(n_splits=5) — each
fold's test set is chronologically after its training set. Never shuffle time series data.
For multi-step ahead validation, use a walk-forward validation scheme.
df['sales_lag_1'] = df['sales'].shift(1) — yesterday's sales.
Rolling statistics: df['sales_roll_7'] = df['sales'].rolling(7).mean() — 7-day moving
average. Calendar features: day of week, month, quarter, is_holiday,
days_to_holiday, fiscal_week. These transform a time-dependent problem into a standard
regression problem solvable by LightGBM — often outperforming LSTM by significant margins on
structured time series.
auto_arima() from pmdarima to auto-select (p,d,q). Prophet (Meta):
decomposes into trend + seasonality + holidays. Robust to missing data. Ideal for business
forecasting with multiple seasonalities. LightGBM + lag features:
consistently wins on tabular time series. N-BEATS / N-HiTS (Nixtla): pure
deep learning, no feature engineering needed. TFT (Temporal Fusion
Transformer): attention-based, handles multivariate series with static covariates.timeseries-project/
├── data/
│ ├── raw/ # Original CSV exports, SQL dumps
│ ├── processed/
│ │ ├── cleaned.parquet # Resampled, imputed, timezone-normalized
│ │ └── features.parquet # Lag features + rolling stats + calendar
│ └── external/ # Weather, holidays, macro indicators
│
├── src/
│ ├── preprocessing/
│ │ ├── resampler.py # Irregular → regular grid (resample/interpolate)
│ │ ├── stationarity.py # ADF test, KPSS test, differencing
│ │ └── decompose.py # STL decomposition: trend + seasonal + residual
│ ├── features/
│ │ ├── lag_features.py # create_lag_features(df, lags=[1,7,28,365])
│ │ ├── rolling_features.py # Rolling mean, std, min, max
│ │ └── calendar_features.py # Datetime → day_of_week, is_holiday, etc.
│ └── models/
│ ├── prophet_model.py # Prophet wrapper + hyperparameter tuning
│ ├── lgbm_model.py # LightGBM with TimeSeriesSplit CV
│ ├── lstm_model.py # Keras LSTM with sliding window
│ └── ensemble.py # Weighted average of model forecasts
│
├── artifacts/
│ ├── lgbm_model.joblib # LightGBM model + feature names
│ ├── scaler.joblib # MinMaxScaler state for LSTM inputs
│ ├── prophet_params.json # Fitted Prophet component parameters
│ └── forecast_config.json # horizon=30, freq="D", confidence=0.95
│
└── api/
├── app.py # FastAPI: /forecast?horizon=30
└── scheduler.py # APScheduler: retrain weeklyRecommendation systems are among the highest-ROI ML applications. Netflix's recommendation system saves $1B/year in churn prevention. Amazon attributes 35% of revenue to recommendations. The core problem: given a user-item interaction matrix (sparse, often 99%+ empty), predict which items a user will engage with next. Three paradigms dominate: Collaborative Filtering (similar users like similar items), Content-Based Filtering (items similar to what you liked), and Hybrid Systems combining both with a two-tower neural network.
implicit library for implicit feedback ALS: model = implicit.als.AlternatingLeastSquares(factors=128). Scales
to millions of users/items.
TensorFlow Recommenders (TFRS).np.save('user_embeddings.npy', U) / np.load('user_embeddings.npy'). Much faster than CSV (no text
parsing). Store alongside a user_id → row_index mapping in a separate file for O(1)
lookup.IndexFlatL2 (exact, slow), IndexIVFFlat (approximate, fast), IndexHNSWFlat (graph-based, fastest). Save/load: faiss.write_index(index, 'index.bin'); faiss.read_index('index.bin').
scipy.sparse.save_npz('interactions.npz', sparse_matrix). Enables
loading just the structure without deserializing the entire matrix.Most real-world data has relational structure that standard ML ignores: molecules (atoms + bonds), social networks (users + friendships), knowledge graphs (entities + relationships), citation networks (papers + references), fraud detection (transactions + accounts). Graph Neural Networks process this structure by iteratively aggregating information from neighbouring nodes — "message passing" — learning representations that capture both node features and graph topology.
negative_sampling(edge_index) to generate
training negatives. Graph classification: predict a label for the entire
graph (e.g., classify molecules as toxic/non-toxic). Requires a readout function (global
mean/max pool) to aggregate node embeddings into a single graph-level vector.NeighborLoader: loader = NeighborLoader(data, num_neighbors=[25, 10], batch_size=1024, input_nodes=train_idx).
Scales to billion-edge graphs with constant memory per batch.gnn-project/
├── data/
│ ├── raw/
│ │ ├── nodes.csv # node_id, feature_1, ..., feature_n, label
│ │ └── edges.csv # src_id, dst_id, edge_weight, edge_type
│ └── processed/
│ └── graph.pt # PyG Data() object saved with torch.save()
├── src/
│ ├── dataset.py # Build PyG Data() from raw CSVs
│ ├── models/
│ │ ├── gcn.py # GCNConv layers
│ │ ├── gat.py # GATConv with multi-head attention
│ │ └── sage.py # SAGEConv for inductive learning
│ ├── train.py # Training loop with NeighborLoader
│ └── evaluate.py # Node accuracy, link AUC, Hits@K
├── artifacts/
│ ├── gnn_model.pt # Trained GNN model state_dict
│ └── node_embeddings.pt # Pre-computed embeddings for all nodes
└── api/
└── app.py # FastAPI: /classify_node, /predict_linkAudio ML is fundamentally a signal processing + deep learning problem. Raw audio is a 1D time-domain signal sampled at 16,000-44,100 Hz. The key insight: neural networks don't operate on raw waveforms efficiently. Instead, audio is converted to a mel spectrogram — a 2D time-frequency representation that mimics the logarithmic sensitivity of the human cochlea. This turns audio classification into image classification, enabling use of CNNs and Vision Transformers.
librosa.feature.melspectrogram(y=audio, sr=16000, n_mels=128, hop_length=160).
Result: a (128, T) matrix that CNNs can process like an image.WhisperForConditionalGeneration. Format: audio → 80-channel
log-mel spectrogram → Whisper processor → tokenized transcript. Training: minimize
cross-entropy on predicted token sequences. CER (Character Error Rate) and WER (Word Error
Rate) are the evaluation metrics.spec = librosa.feature.melspectrogram(...) → resize to (224, 224)
→ pass through EfficientNet-B0 pretrained on ImageNet →
fine-tune on audio classes. This works because ImageNet pretrained features capture texture
patterns analogous to spectral patterns. Used for: environmental sound classification
(ESC-50), music genre recognition, keyword spotting, emotion recognition. Achieves near-SOTA
with minimal audio-specific engineering.librosa.load(path, sr=16000, mono=True) resamples
to 16kHz on load. 1 hour of 16kHz mono WAV ≈ 112MB.processor.save_pretrained('./whisper_finetuned/'). Includes special
tokens: <pad>, <bos>, <eos>, language tokens (<|en|>), task tokens (<|transcribe|>).Anomaly detection (also called outlier detection or novelty detection) identifies data points that deviate significantly from the expected pattern. Unlike standard classification, anomalies are by definition rare — often constituting 0.1–1% of data. This extreme class imbalance makes standard classifiers fail. The solution: train only on normal data and flag anything that doesn't fit the learned normality model. Applications: financial fraud, network intrusion, industrial equipment failure prediction, medical outlier detection.
IsolationForest(contamination=0.01).
LocalOutlierFactor(n_neighbors=20, novelty=True).pyod.models.ecod import ECOD.Multi-modal models process and relate information from multiple data types simultaneously — images + text, audio + video, graphs + text. The breakthrough was CLIP (Contrastive Language-Image Pretraining): by training an image encoder and a text encoder jointly on 400M image-caption pairs, it learns a shared embedding space where "a photo of a cat" is near an image of a cat. This enables zero-shot image classification, visual question answering, and image-text retrieval without task-specific training.
from transformers import CLIPModel, CLIPProcessor. Fine-tune on
domain images with only a few hundred examples.from transformers import LlavaForConditionalGeneration.These practices apply to every ML domain. Skipping any of them is technical debt that will cause a production incident. Apply them from day one, not as an afterthought.
Every ML project touches external services: Hugging Face, W&B, AWS, Roboflow, OpenAI. Their API keys must never be committed to git — even to private repos, because git history is permanent and repos can be made public accidentally.
# .env — NEVER commit this file
HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
WANDB_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxxxxxxxxxxxx
AWS_ACCESS_KEY_ID=AKIAXXXXXXXXXXXXXXXX
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxx
REPLICATE_API_TOKEN=r8_xxxxxxxxxxxxxxxxxxxxxxx
ROBOFLOW_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
DATABASE_URL=postgresql://user:pass@localhost:5432/mldb
# Load in Python
# pip install python-dotenv
from dotenv import load_dotenv
import os
load_dotenv() # reads .env from current directory
hf_token = os.getenv("HF_TOKEN")
# In Docker: pass as environment variables
# docker run -e HF_TOKEN=$HF_TOKEN my-ml-image
# Or use Docker secrets for productiongit-filter-repo to purge the
commit from history, then force push. GitHub's secret scanning will flag it automatically and email
you if it detects committed credentials.# ── Secrets & credentials
.env
.env.*
*.pem
secrets/
# ── Python
__pycache__/
*.py[cod]
*.so
*.egg
*.egg-info/
dist/
build/
.venv/
venv/
env/
# ── Jupyter
.ipynb_checkpoints/
*.ipynb # Optional: only if using .py scripts, not notebooks
# ── ML data (track with DVC instead)
data/raw/
data/processed/
datasets/
*.csv # Large CSVs only — small reference CSVs are OK in git
*.parquet
*.h5
*.hdf5
# ── Model weights (track with DVC or MLflow)
*.pt
*.pth
*.pkl
*.joblib
*.safetensors
*.bin
*.onnx
*.gguf
*.ckpt
*.weights
*.engine
*.tflite
weights/
models/
checkpoints/
artifacts/
# ── Experiment outputs
runs/ # TensorBoard logs
wandb/ # W&B local runs
mlruns/ # MLflow tracking
outputs/
results/
logs/
# ── OS & Editor
.DS_Store
Thumbs.db
.idea/
.vscode/ # Optional: some teams commit .vscode/settings.json
*.swpGit tracks code. DVC tracks data and models.
Every dvc add creates a lightweight pointer file (.dvc) that git tracks, while the actual large file is pushed to remote storage
(S3, GCS, GDrive, SSH). Checking out any git commit gives you the corresponding data version
automatically.
# Install and initialise
pip install dvc[s3] # or dvc[gcs], dvc[gdrive]
dvc init # creates .dvc/ directory
# Configure remote storage (S3 example)
dvc remote add -d myremote s3://my-ml-bucket/zerotoml
dvc remote modify myremote profile default # use AWS CLI profile
# Track large files
dvc add data/processed/train.parquet # creates data/processed/train.parquet.dvc
dvc add weights/best.pt
git add data/processed/train.parquet.dvc weights/best.pt.dvc .gitignore
git commit -m "Track processed data and model weights with DVC"
# Push data to remote
dvc push # uploads to S3
# On another machine / CI pipeline
git clone https://github.com/Muhammad-waqas1/zerotoml
dvc pull # downloads tracked files from S3
# Switch to a previous data version
git checkout v1.0.0
dvc checkout # downloads the data from that commit# ── Stage 1: dependency install (cached layer)
FROM python:3.11-slim AS base
WORKDIR /app
# System deps — install before copying code (better layer caching)
RUN apt-get update && apt-get install -y --no-install-recommends \
curl libgomp1 libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
# Install Python deps first (changes less often than code)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# ── Stage 2: application
COPY src/ ./src/
COPY api/ ./api/
COPY artifacts/ ./artifacts/ # Model weights baked into image
# Security: non-root user
RUN useradd -m -u 1000 mluser && chown -R mluser /app
USER mluser
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=10s --start-period=10s --retries=3 \
CMD curl -f http://localhost:8000/health || exit 1
CMD ["uvicorn", "api.app:app", "--host", "0.0.0.0", "--port", "8000", \
"--workers", "2", "--log-level", "info"]import wandb
import mlflow
import os, subprocess
# ── Weights & Biases (cloud, great UI, hyperparameter sweeps)
wandb.init(
project="zerotoml-cv",
name=f"yolov8-exp-{run_id}",
config={
"model": "yolov8n",
"epochs": 100,
"img_size": 640,
"batch_size": 16,
"learning_rate": 0.01,
"dataset_version": "v3.2", # Always version your dataset!
"git_commit": subprocess.check_output(["git", "rev-parse", "HEAD"]).decode().strip(),
},
tags=["baseline", "augmentation-v2"],
)
# Log metrics each epoch
for epoch in range(epochs):
metrics = train_one_epoch(model, dataloader)
wandb.log({
"train/loss": metrics["loss"],
"val/mAP50": metrics["map50"],
"val/precision": metrics["precision"],
"lr": scheduler.get_last_lr()[0],
})
# Save model artifact to W&B
artifact = wandb.Artifact("yolov8-trained", type="model")
artifact.add_file("weights/best.pt")
wandb.log_artifact(artifact)
wandb.finish()
# ── MLflow (self-hosted, great for enterprise)
mlflow.set_tracking_uri("http://mlflow-server:5000")
with mlflow.start_run(run_name="yolov8-baseline"):
mlflow.log_params({"epochs": 100, "img_size": 640})
mlflow.log_metrics({"val_map50": 0.847, "val_precision": 0.891})
mlflow.log_artifact("weights/best.pt", artifact_path="model")
mlflow.set_tag("model_type", "object_detection")Every tool categorized by function, pricing model, and primary use case. Updated for 2025.
| Tool / Platform | Category | Primary Use | Pricing | Best For |
|---|---|---|---|---|
| DATA LABELING & ANNOTATION | ||||
| CVAT.ai | Labeling | CV annotation (bbox, polygon, segmentation, keypoints) | Freemium | Self-hosted teams, YOLO/COCO export |
| Roboflow | Labeling | Labeling + augmentation + dataset versioning + deploy | Freemium | End-to-end CV pipeline, quick iteration |
| Label Studio | Labeling | NLP (NER, classification), CV, audio, time-series | Free | Multi-modal labeling, self-hosted, RLHF |
| Scale AI | Labeling | Managed human labeling at scale | Paid | Production annotation, autonomous vehicles |
| Prodigy | Labeling | Active learning annotation, NLP + CV | Paid | spaCy ecosystem, small team annotation |
| Argilla | Labeling | LLM data labeling, RLHF preference collection | Free | Fine-tuning LLMs with human feedback |
| COMPUTE & GPU ACCESS | ||||
| Kaggle Notebooks | Compute | Free GPU/TPU for training | Free | 30hr/week T4 GPU, pre-installed ML stack |
| Google Colab | Compute | Notebook GPU compute | Freemium | Prototyping, free T4 with limits |
| Lambda Labs | Compute | On-demand GPU cloud (H100, A100, 3090) | Paid | LLM training, cheapest H100 hourly rate |
| Vast.ai | Compute | Peer-to-peer GPU marketplace | Paid | Cheapest GPU rates, flexible configs |
| Modal | Compute | Serverless Python GPU functions | Freemium | Zero-to-scale GPU jobs, $30/mo free credit |
| RunPod | Compute | GPU cloud with persistent storage | Paid | LLM fine-tuning, stable long-running jobs |
| EXPERIMENT TRACKING & MODEL REGISTRY | ||||
| Weights & Biases | Tracking | Experiment tracking, hyperparameter sweeps, model registry | Freemium | Best visualization, sweep automation, free tier for academics |
| MLflow | Tracking | Self-hosted experiment tracking + model serving | Free | Enterprise, on-premise, multi-framework |
| Neptune.ai | Tracking | Metadata store for ML experiments | Freemium | Large teams, compliance logging |
| DVC | Tracking | Data + model versioning for git | Free | Dataset versioning, pipeline reproducibility |
| Comet ML | Tracking | Experiment tracking + model production monitoring | Freemium | Production drift monitoring included |
| MODEL HOSTING & INFERENCE | ||||
| HF Inference Endpoints | Hosting | Deploy any HF model as dedicated endpoint | Paid | NLP/LLM APIs, auto-scaling, GPU |
| Replicate | Hosting | Deploy ML models via Docker + cog | Freemium | Diffusion models, per-second billing |
| Streamlit Community Cloud | Hosting | Deploy Streamlit ML dashboards | Free | ML demos, data apps, internal tools |
| Render | Hosting | Docker container deployment from GitHub | Freemium | FastAPI ML APIs, auto-deploy from GitHub |
| vLLM | Hosting | High-throughput LLM inference server | Free | Self-hosted LLM serving, OpenAI-compatible API |
| Ollama | Hosting | Run LLMs locally (GGUF models) | Free | Local development, privacy-sensitive deployments |
| ML FRAMEWORKS & LIBRARIES | ||||
| PyTorch | Framework | Deep learning research + production | Free | Research, flexibility, 80% of papers |
| scikit-learn | Framework | Classical ML: Pipeline, GridSearch, metrics | Free | Tabular ML, preprocessing pipelines |
| XGBoost | Framework | Gradient boosted trees, GPU support | Free | Tabular competition standard |
| LightGBM | Framework | Fast gradient boosting, leaf-wise splitting | Free | Large tabular datasets, 10x faster than XGBoost |
| fastai | Framework | High-level PyTorch API for CV/NLP/tabular | Free | Rapid prototyping, learning PyTorch patterns |
| Keras / TensorFlow | Framework | High-level DL, production TF Serving | Free | TFLite mobile export, TPU training |
| DATASETS & DATA TOOLS | ||||
| UCI ML Repository | Data | 650+ structured datasets for ML benchmarking | Free | Tabular ML benchmarking, research |
| HF Datasets | Data | 70k+ ML datasets, streaming, one-line load | Free | NLP datasets, multi-modal, benchmark |
| Kaggle Datasets | Data | 200k+ community datasets across domains | Free | Competition data, real-world messy datasets |
| Great Expectations | Data | Data quality validation, schema enforcement | Freemium | Production data pipelines, CI/CD for data |
| MLOPS & DEPLOYMENT INFRASTRUCTURE | ||||
| FastAPI | MLOps | Production REST API framework for ML services | Free | ML inference APIs, typed endpoints, auto-docs |
| BentoML | MLOps | ML model serving framework | Freemium | Multi-model APIs, adaptive batching |
| Evidently AI | MLOps | ML monitoring, drift detection, data quality | Freemium | Production model monitoring, drift alerts |
| Apache Airflow | MLOps | Pipeline orchestration and scheduling | Free | ETL pipelines, scheduled retraining |
| Prefect | MLOps | Modern data pipeline orchestration | Freemium | ML workflow automation, retry logic |
Ready to Build?
The best way to learn these architectures is to build with them. Start with the ZeroToML curriculum to build the mathematical and practical foundations, then apply the lifecycle patterns from this guide to your own projects.