Machine Learning Fundamentals

Supervised learning: The model learns from labeled examples (input → known output). Classification (predicting categories) and regression (predicting numbers) are both supervised. 80%+ of production ML is supervised.

Unsupervised learning: No labels — the model finds patterns in data on its own. Clustering (grouping similar items) and dimensionality reduction are common applications.

Reinforcement learning: An agent learns by trial and error, receiving rewards for good actions. Used in game AI, robotics, and recommendation systems.

Every ML model makes errors from two sources: bias (underfitting — model too simple to capture patterns) and variance (overfitting — model memorizes training data, fails on new data). The goal is minimizing total error. High-complexity models have low bias but high variance. Simple models have high bias and low variance. Regularization techniques (L1/L2, dropout, early stopping) help find the right balance.

Accuracy is rarely the right metric. Choose your metric based on what matters in your domain:

Never evaluate your model on the same data it was trained on — this gives you a falsely optimistic performance estimate. Use k-fold cross-validation:

from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(n_estimators=100)
scores = cross_val_score(model, X, y, cv=5, scoring='f1')

print(f"Mean F1: {scores.mean():.3f} ± {scores.std():.3f}")