Types of Data

How data is classified determines which statistical methods are appropriate to use.

Categorical Data

Numeric Data

Quick Reference Table

Random Experiments

The foundation of probability — understanding how uncertainty arises and is formalized.

Random Variables

A mathematical bridge that maps experimental outcomes to numbers on the real line.

Discrete vs Continuous

Probability Functions

Events

Any subset of the sample space we care about assigning a probability to.

Axioms of Probability

Distributions

How probability is spread across the possible values of a random variable.

Uniform Distribution

Gaussian (Normal) Distribution N(μ, σ)

Exponential Distribution (λ)

Triangular Distribution Tri(a, b, c)

Absolute vs Relative Frequency

Descriptive Statistics

Measures that summarize the center and spread of a dataset.

Range

Mean (Arithmetic Average)

Median

Percentile — Linear Interpolation Method

Trimmed Mean

Measures of Dispersion

How spread out the data is — and how robust each measure is to outliers.

Mean Absolute Deviation

Median Absolute Deviation (MAD)

Variance & Standard Deviation

Interquartile Range (IQR)

Robustness Comparison

Box-and-Whisker Plot (Tukey Convention)

Outliers & Skew

How extreme values distort statistics and reveal the shape of a distribution.

Skewness

Effect of One Outlier on Each Statistic

Population vs Sample Statistics

The critical difference in formulas — and why Bessel's correction exists.

Formulas Side by Side

Z-scores, Φ (Phi) & Erf

Computing probabilities under any Gaussian by standardizing to N(0,1).

Z-Score

Φ (Phi) — CDF of N(0,1)

Error Function (erf)

Computing Probabilities

Key Φ Values to Memorize

Covariance & Pearson Correlation

Measuring how two variables move together — direction and strength.

Covariance

Pearson Correlation Coefficient (r)

Bias & Error Types

Understanding systematic and random sources of error in data collection.

Types of Bias

Quick Reference

Sampling Distributions & CLT

Why sample means behave predictably — even when individual data doesn't.

Central Limit Theorem (CLT)

Standard Error of the Mean

Standard Deviation vs Standard Error

Spark & PySpark

Distributed computing for datasets too large for a single machine.

Core Concepts

Lazy Evaluation

Transformations (Lazy — do not trigger execution)

Actions (Trigger full execution)

Catalyst Optimizer

NumPy vs Pandas vs Spark

Common PySpark Patterns

# Select specific columns
df.select("PassengerId", "Survived", "Pclass").show(5)

# Filter rows (two equivalent ways)
df.filter((col("Survived")==1) & (col("Pclass")==1)).show()
df.filter("Survived == 1").filter("Pclass == 1").show()

# Add a new column
df.withColumn("FamilySize", col("SibSp") + col("Parch")).show()

# Group and aggregate
df.groupBy("Pclass").count().orderBy("count", ascending=False).show()

# Read a CSV file
df = spark.read.csv("/path/file.csv", header=True, inferSchema=True)

Object Classes

A regular object class in Java has exactly 4 types of methods.

Constructors

public Person(String name, int age){ this.name = name; this.age = age; }

Accessors (Getters)

public String getName(){ return name; }
public int getAge(){ return age; }

Mutators (Setters)

public void setName(String name){ this.name = name; }
public void setAge(int age){ this.age = age; }

toString

public String toString(){
  return "Person: " + name + ", Age: " + age;
}

Quick Reference

The this Keyword & Shadowing

Shadowing is one of the most common bugs in object classes.

// BUG — shadowing! name = name assigns param to itself
public void setName(String name){ name = name; }

// FIX — use this.name
public void setName(String name){ this.name = name; }

Method Overloading & Return Types

Same name, different parameters — Java picks the right version automatically.

public void print(int x){ ... }       // version 1
public void print(String s){ ... }    // version 2 — different param type
public void print(int x, int y){ ... } // version 3 — different param count

public int add(int a, int b){ return a + b; }   // returns int
public String greet(){ return "Hello"; }         // returns String
public void print(){ System.out.println("Hi"); } // void, no return value

The static Keyword

Static members belong to the class, not any particular instance.

public class Counter {
  public static int count = 0;  // one copy, shared
  public Counter(){ count++; }  // every new object increments the same count
}

public static int square(int n){ return n * n; }
// called as: ClassName.square(5);

Aggregation (Has-A Relationship)

One class uses another class as a field — modeling real-world ownership.

public class University {
  private String name;
  private Person[] faculty;  // aggregation: University HAS-A Person[]
  private int count;
  ...
}

public void addPerson(String name, int age){
  faculty[count] = new Person(name, age);
  count++;
}

Arrays

Fixed-size, ordered collection of elements of the same type.

int[] scores = new int[5];       // size 5, all zeros
String[] names = new String[3];  // size 3, all null

int[] data = new int[10];  // physical = 10
int count = 0;             // logical starts at 0
data[count] = 42; count++; // logical becomes 1

// Print all elements (full array):
for(int i = 0; i < arr.length; i++){
  System.out.println(arr[i]);
}
// Enhanced for-loop:
for(int val : arr){ System.out.println(val); }

ArrayList

Dynamic array that grows and shrinks automatically. From java.util.ArrayList.

import java.util.ArrayList;
ArrayList<String> list = new ArrayList<String>();

Key Methods

Wrapper Classes

Object versions of Java primitives. Required for collections like ArrayList.

Useful Methods

Integer.parseInt("42")     // String → int
Integer.toString(42)       // int → String
Double.parseDouble("3.14") // String → double
Character.isLetter('a')    // true
Character.toUpperCase('a') // 'A'

ArrayList<Integer> nums = new ArrayList<>();
nums.add(5);      // autoboxing: int 5 → Integer(5)
int x = nums.get(0); // unboxing: Integer(5) → int 5

String Methods

Strings are objects in Java — immutable and full of useful methods.

Formatting Output

Three ways to control decimal precision and field width.

printf / String.format Format Specifiers

import java.text.DecimalFormat;
DecimalFormat df = new DecimalFormat("0.00");
System.out.println(df.format(3.14159)); // → 3.14

DecimalFormat df2 = new DecimalFormat("#,##0.00");
System.out.println(df2.format(1234567.8)); // → 1,234,567.80

String s = String.format("%.2f", 3.14159); // s = "3.14"
String t = String.format("%-10s|", "Hi");  // t = "Hi        |"

Primitives vs Objects & Pass by Value/Reference

Understanding how Java passes data to methods.

The 8 Primitive Types

Pass by Value (Primitives)

void addTen(int x){ x += 10; } // only changes the copy
int n = 5;
addTen(n);
System.out.println(n); // still 5!

Pass by Reference (Objects)

void rename(Person p){ p.setName("Bob"); }
Person person = new Person("Alice", 20);
rename(person);
System.out.println(person.getName()); // "Bob" — object was changed!

Random Class & Nested Loops

Generating random numbers and using nested iteration.

import java.util.Random;
Random rng = new Random();

rng.nextInt(n)         // random int: 0 to n-1 (inclusive)
rng.nextInt(10)        // 0–9
rng.nextInt(10) + 1    // 1–10 (shift range up)
rng.nextInt(6) + 1     // 1–6 (simulate die roll)

// General range [min, max] inclusive:
rng.nextInt(max - min + 1) + min

for(int i = 1; i <= 3; i++){
  for(int j = 1; j <= 3; j++){
    System.out.print(i * j + "\t");
  }
  System.out.println();
}
// Output: multiplication table 1..3

JUnit Testing

Testing individual methods in isolation.

import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

public class PersonTest {
  @Test
  public void testGetName(){
    Person p = new Person("Alice", 20);
    assertEquals("Alice", p.getName());
  }
}

Common Assert Methods