Big-O, Big-Θ, Big-Ω — Measuring algorithm efficiency
Foundation Module - Understand HOW to measure algorithm efficiency before learning algorithms
An algorithm is a step-by-step procedure to solve a problem.
Problem: Find the largest number in a list
Algorithm:
1. Assume first element is largest
2. Compare with each remaining element
3. If current element is bigger, update largest
4. After checking all, return largest
This is O(n) — we look at each element once
Two key questions about any algorithm:
| Aspect | Time Complexity | Space Complexity |
|---|---|---|
| Measures | Number of operations | Memory used |
| Depends on | Input size (n) | Input size (n) |
| Trade-off | Often can trade space for time | Caching, memoization |
# Example: Two approaches to find duplicates
# Approach 1: Brute Force — O(n²) time, O(1) space
def has_duplicate_v1(arr):
for i in range(len(arr)):
for j in range(i + 1, len(arr)):
if arr[i] == arr[j]:
return True
return False
# Approach 2: Hash Set — O(n) time, O(n) space
def has_duplicate_v2(arr):
seen = set()
for num in arr:
if num in seen:
return True
seen.add(num)
return False
# V2 is FASTER but uses MORE MEMORY — classic trade-off!
We describe algorithm performance using three notations:
| Notation | Meaning | Analogy |
|---|---|---|
| Big-O (O) | Upper bound (worst case) | "At MOST this slow" |
| Big-Ω (Ω) | Lower bound (best case) | "At LEAST this fast" |
| Big-Θ (Θ) | Tight bound (exact) | "EXACTLY this fast" |
Example: Linear Search
Best case: Ω(1) — Found at index 0
Worst case: O(n) — Found at last index (or not found)
Average: Θ(n/2) = Θ(n) — Found somewhere in the middle
When we say "Linear Search is O(n)" we mean:
→ In the WORST case, it checks all n elements
Interview Tip: When someone asks "What is the complexity?", they almost always mean Big-O (worst case).
FAST ────────────────────────────────────── SLOW
O(1) O(log n) O(n) O(n log n) O(n²) O(2^n) O(n!)
│ │ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼ ▼
Const Logarith Linear Linearith Quadra Expon Factor
| Complexity | Name | n=10 | n=100 | n=1000 | Example |
|---|---|---|---|---|---|
| O(1) | Constant | 1 | 1 | 1 | Array index access |
| O(log n) | Logarithmic | 3 | 7 | 10 | Binary Search |
| O(n) | Linear | 10 | 100 | 1000 | Linear Search |
| O(n log n) | Linearithmic | 33 | 664 | 9966 | Merge Sort |
| O(n²) | Quadratic | 100 | 10,000 | 1,000,000 | Bubble Sort |
| O(2^n) | Exponential | 1,024 | 1.26×10³⁰ | ∞ | Naive Fibonacci |
| O(n!) | Factorial | 3.6M | ∞ | ∞ | Brute-force permutations |
# This is O(n), NOT O(3n)
for i in range(n): # n operations
print(i)
for i in range(n): # + n operations
print(i)
for i in range(n): # + n operations
print(i)
# Total: 3n → O(n)
# This is O(n²), NOT O(n² + n)
for i in range(n): # O(n²)
for j in range(n):
print(i, j)
for k in range(n): # + O(n)
print(k)
# Total: n² + n → O(n²) (n² dominates)
for i in range(n): # n ×
for j in range(n): # n ×
for k in range(n): # n
print(i, j, k)
# Total: n × n × n = O(n³)
def example(n):
# Block 1: O(n)
for i in range(n):
print(i)
# Block 2: O(n²)
for i in range(n):
for j in range(n):
print(i, j)
# Total: O(n) + O(n²) = O(n²)
# Each step halves the input → O(log n)
i = n
while i > 1:
i = i // 2 # Halving each time!
print(i)
# For n = 1000: only ~10 steps (log₂ 1000 ≈ 10)
# O(1) space — uses fixed variables
def sum_array(arr):
total = 0
for num in arr:
total += num
return total
# O(n) space — creates a new array of size n
def double_array(arr):
result = []
for num in arr:
result.append(num * 2)
return result
# O(n) space — recursion uses call stack
def factorial(n):
if n <= 1:
return 1
return n * factorial(n - 1) # n stack frames!
Key insight: Recursion uses O(depth) space on the call stack!
Some operations are slow occasionally but fast on average.
Example: Dynamic Array (Python list)
| Operation | Amortized | Worst Case |
|---|---|---|
append() | O(1) | O(n) when resizing |
pop() | O(1) | O(1) |
insert(0, x) | O(n) | O(n) |
When the array is full, Python doubles its size (O(n) copy), but this happens rarely. Over many appends, it averages out to O(1) per append.
| Concept | Key Point |
|---|---|
| Big-O | Worst case upper bound (most commonly used) |
| Big-Ω | Best case lower bound |
| Big-Θ | Tight/exact bound |
| Drop constants | O(3n) = O(n) |
| Drop lower terms | O(n² + n) = O(n²) |
| Nested loops | Multiply: O(n) × O(n) = O(n²) |
| Halving | O(log n) |
| Recursion space | O(depth of recursion) |
One-liner for each concept:
| Concept | Key Takeaway |
|---|---|
| Big-O | Upper bound. Worst case scenario. "At most this slow." |
| Big-Ω | Lower bound. Best case scenario. "At least this fast." |
| Big-Θ | Tight bound. Exact growth rate. |
| O(1) | Constant. Doesn't depend on input size. Array index access. |
| O(log n) | Logarithmic. Halving each step. Binary Search. |
| O(n) | Linear. Visit each element once. Linear Search. |
| O(n log n) | Linearithmic. Best comparison-based sorting (Merge/Quick). |
| O(n²) | Quadratic. Nested loops over same input. Bubble Sort. |
Essential Code Snippets:
# O(1) — Constant
x = arr[5]
# O(log n) — Halving
while n > 1: n //= 2
# O(n) — Single loop
for x in arr: print(x)
# O(n²) — Nested loop
for i in arr:
for j in arr: print(i, j)
The Golden Rules:
Video Courses:
Articles & Visualizations:
Practice Problems:
Test your understanding with step-by-step solutions
10 questions · 90s per question
Each question has a 90-second time limit. Unanswered questions will be auto-submitted when time runs out.