Mathematics & Techniques for Data Science

data science
mathematics
ML
AI
A personal list of topics across the data science field.
Author

Oscar Cardec

Published

March 8, 2025


This is an opinionated list of mathematical topics and techniques essential across the data science field. Although, not an all-inclusive list, knowledgement and mastering of these provide a solid foundation useful for understanding of the diverse branches sustaining machine learning and artificial intelligence.

Linear Algebra

  • Vectors and Matrices
    • Vector Operations: Basic operations such as addition, subtraction, and scalar multiplication.
    • Matrix Operations: Matrix multiplication, inversion, and transposition.
    • Types of Matrices: Special matrices like diagonal, symmetric, and orthogonal matrices.
  • Systems of Linear Equations
    • Gaussian Elimination: Method for solving linear systems by reducing matrices to row echelon form.
    • LU Decomposition: Factorization of a matrix into lower and upper triangular matrices.
  • Matrix Decompositions
    • Eigenvalues and Eigenvectors: Key concepts for understanding matrix transformations.
    • Singular Value Decomposition (SVD): Decomposition of a matrix into singular vectors and singular values.
    • Principal Component Analysis (PCA): Technique for reducing the dimensionality of data.
  • Vector Spaces
    • Basis and Dimension: Fundamental properties of vector spaces.
    • Subspaces: Subsets of vector spaces that themselves are vector spaces.
    • Orthogonality and Orthogonal Projections: Concepts for projecting vectors onto subspaces.
  • Linear Transformations
    • Matrix Representation: Representation of linear transformations using matrices.
    • Change of Basis: Transforming coordinates from one basis to another.

Probability and Statistics

  • Probability Theory
    • Basic Probability Concepts: Definitions and rules of probability.
    • Conditional Probability and Bayes’ Theorem: Probability of events given other events.
    • Random Variables: Variables whose values are subject to randomness.
    • Probability Distributions: Descriptions of how probabilities are distributed over values.
    • Joint, Marginal, and Conditional Distributions: Relationships between multiple random variables.
    • Expectation, Variance, and Covariance: Measures of central tendency and variability.
  • Statistical Inference
    • Point Estimation: Estimating population parameters from sample data.
    • Confidence Intervals: Range of values within which a parameter is expected to lie.
    • Hypothesis Testing: Procedure for testing assumptions about population parameters.
    • p-values and Significance Levels: Metrics for assessing hypothesis test results.
    • Maximum Likelihood Estimation (MLE): Method for estimating parameters by maximizing likelihood.
  • Bayesian Statistics
    • Bayesian Inference: Updating probabilities based on new data.
    • Prior and Posterior Distributions: Distributions representing beliefs before and after observing data.
    • Markov Chain Monte Carlo (MCMC): Algorithms for sampling from complex distributions.
  • Regression Analysis
    • Simple and Multiple Linear Regression: Modeling relationships between variables.
    • Logistic Regression: Modeling binary outcome variables.
    • Assumptions and Diagnostics: Checking the validity of regression models.
  • Advanced Topics in Statistics
    • Time Series Analysis: Analyzing data points collected over time.
    • Survival Analysis: Analyzing time-to-event data.
    • Non-parametric Methods: Statistical methods not assuming a specific data distribution.

Numerical Methods

  • Optimization Techniques
    • Gradient Descent: Iterative method for finding local minima of functions.
    • Stochastic Gradient Descent: Variant of gradient descent using random subsets of data.
    • Conjugate Gradient Method: Optimization algorithm for large-scale linear systems.
    • Newton’s Method: Iterative method for finding successively better approximations to roots.
  • Numerical Linear Algebra
    • Matrix Factorization: Decomposing matrices into products of simpler matrices.
    • Solving Linear Systems: Methods for finding solutions to linear equations.
    • Eigenvalue Problems: Finding eigenvalues and eigenvectors of matrices.

Machine Learning

  • Supervised Learning
    • Regression (Linear, Polynomial): Predicting continuous outcomes from input features.
    • Classification (k-NN, SVM, Decision Trees, Random Forests): Categorizing data into predefined classes.
  • Unsupervised Learning
    • Clustering (k-Means, Hierarchical, DBSCAN): Grouping similar data points together.
    • Dimensionality Reduction (PCA, t-SNE, LDA): Reducing the number of variables in data.
  • Model Evaluation
    • Cross-Validation: Technique for assessing model performance on unseen data.
    • ROC Curves and AUC: Metrics for evaluating classification model performance.
    • Precision, Recall, F1-Score: Metrics for evaluating model accuracy and relevance.
  • Ensemble Methods
    • Bagging and Boosting: Techniques for improving model performance by combining multiple models.
    • Random Forests: Ensemble learning method using multiple decision trees.
    • Gradient Boosting Machines (GBM, XGBoost): Powerful ensemble methods for regression and classification.

Neural Networks and Deep Learning

  • Fundamentals of Neural Networks
    • Perceptrons and Multilayer Perceptrons (MLP): Basic building blocks of neural networks.
    • Activation Functions (ReLU, Sigmoid, Tanh): Functions introducing non-linearity into neural networks.
    • Backpropagation and Gradient Descent: Algorithms for training neural networks.
  • Deep Learning Architectures
    • Convolutional Neural Networks (CNNs): Networks for processing grid-like data such as images.
    • Recurrent Neural Networks (RNNs): Networks for processing sequential data.
    • Long Short-Term Memory (LSTM): RNN variant for capturing long-term dependencies.
    • Generative Adversarial Networks (GANs): Networks for generating new, synthetic data.
  • Deep Learning Techniques
    • Regularization (Dropout, Batch Normalization): Techniques for preventing overfitting.
    • Transfer Learning: Leveraging pre-trained models for new tasks.
    • Hyperparameter Tuning: Optimizing model parameters for better performance.
    • Autoencoders: Networks for unsupervised learning of efficient codings.
  • Advanced Topics in Deep Learning
    • Attention Mechanisms: Techniques for focusing on relevant parts of input data.
    • Transformers: Architectures for handling sequential data with attention mechanisms.
    • Reinforcement Learning: Training models to make sequences of decisions.

Dimensionality Reduction

  • Principal Component Analysis (PCA)
    • Eigenvalues and Eigenvectors: Key concepts for understanding PCA.
    • Variance Explained: Measure of how much information is retained by principal components.
  • Singular Value Decomposition (SVD)
    • Low-Rank Approximations: Simplifying data by reducing its dimensionality.
  • Manifold Learning
    • t-SNE (t-Distributed Stochastic Neighbor Embedding): Technique for visualizing high-dimensional data.
    • UMAP (Uniform Manifold Approximation and Projection): Method for dimensionality reduction and visualization.
  • Feature Selection and Extraction
    • L1 Regularization (Lasso): Technique for feature selection in regression models.
    • Recursive Feature Elimination: Method for selecting important features by recursively removing less important ones.

Additional Important Topics

  • Information Theory
    • Entropy and Information Gain: Measures of uncertainty and information content.
    • Mutual Information: Measure of the mutual dependence between variables.
  • Graph Theory
    • Graph Representation: Ways to represent graphs using matrices and lists.
    • Graph Algorithms (PageRank, Graph Neural Networks): Algorithms for processing graph-structured data.
  • Time Series Analysis
    • Autoregressive Models (AR, MA, ARIMA): Models for analyzing and forecasting time series data.
    • Seasonal Decomposition: Breaking down time series data into seasonal components.
    • Forecasting Techniques: Methods for predicting future values in time series data.
  • Natural Language Processing (NLP)
    • Text Preprocessing: Techniques for preparing text data for analysis.
    • Word Embeddings (Word2Vec, GloVe): Methods for representing words as vectors.
    • Sequence Models (RNN, LSTM, Transformer): Models for processing and understanding sequential data.