Databases

MongoDB, PostgreSQL, Redis, Prisma, indexing and caching

31Questions

๐Ÿง  Simple Definition

A Primary Key is a column (or set of columns) that uniquely identifies every row in a table. It cannot be null and must be unique across all rows. A Foreign Key is a column in one table that references a Primary Key in another table, creating a relationship and enforcing referential integrity.


๐Ÿงช SQL Example

-- Primary key: uniquely identifies each user
CREATE TABLE users (
  id       INT          PRIMARY KEY,  -- โ† Primary Key (auto-unique, not null)
  email    VARCHAR(255) UNIQUE NOT NULL,
  name     VARCHAR(100) NOT NULL
);

-- Foreign key: orders.user_id references users.id
CREATE TABLE orders (
  id       INT  PRIMARY KEY,
  user_id  INT  NOT NULL,
  amount   DECIMAL(10, 2),

  FOREIGN KEY (user_id) REFERENCES users(id)  -- โ† Foreign Key
  ON DELETE CASCADE  -- if user deleted, delete their orders too
);

๐Ÿงช Prisma Schema Example

model User {
  id     Int     @id @default(autoincrement())  // โ† Primary Key
  email  String  @unique
  orders Order[]
}

model Order {
  id     Int   @id @default(autoincrement())
  amount Float
  userId Int
  user   User  @relation(fields: [userId], references: [id])  // โ† Foreign Key
}

๐Ÿ“Š Key Properties

PropertyPrimary KeyForeign Key
Uniquenessโœ… Must be uniqueโŒ Can repeat (many orders per user)
NULL allowedโŒ Never NULLโš ๏ธ Depends (can be nullable for optional relations)
Indexโœ… Automatically indexedโœ… Should be indexed (for JOIN performance)
One per tableโœ… One PK per tableโŒ Can have multiple FKs
PurposeRow identificationRelationship + referential integrity

๐Ÿ”‘ Foreign Key Actions

ActionON DELETEON UPDATE
CASCADEDelete child rows when parent deletedUpdate child FK when parent PK updated
SET NULLSet FK to NULL when parent deletedSet FK to NULL when parent PK updated
RESTRICTPrevent parent deletion if children existPrevent parent update if children exist
NO ACTIONDefault โ€” like RESTRICT (checked at end of TX)Default

๐ŸŒ Composite Primary Key

-- Junction table for many-to-many relationship
CREATE TABLE user_roles (
  user_id INT NOT NULL,
  role_id INT NOT NULL,
  PRIMARY KEY (user_id, role_id),  -- โ† Composite PK
  FOREIGN KEY (user_id) REFERENCES users(id),
  FOREIGN KEY (role_id) REFERENCES roles(id)
);

๐Ÿ’ก Possible Follow-up Questions

  • What is a composite primary key and when would you use one?
  • What is referential integrity and how do foreign keys enforce it?
  • What is the difference between CASCADE DELETE and RESTRICT?
  • What is a surrogate key vs a natural key?
  • Why should you index foreign key columns?

โšก One-line Interview Answer

A primary key uniquely identifies each row in a table (enforces uniqueness and non-null constraint, automatically indexed). A foreign key in one table references a primary key in another, creating a relationship and enforcing referential integrity โ€” controlling what happens to child rows when the parent is deleted or updated (CASCADE, SET NULL, RESTRICT).

๐Ÿง  Simple Definition

WHERE filters rows before any grouping or aggregation happens. HAVING filters groups after GROUP BY aggregation. You cannot use aggregate functions (COUNT, SUM, AVG) in a WHERE clause โ€” that's what HAVING is for.


โšก Super Simple Line

WHERE = filter individual rows (before grouping).
HAVING = filter aggregate results (after grouping).
WHERE can't use SUM/COUNT. HAVING can.


๐Ÿงช Example: Both in Same Query

SELECT
  department,
  COUNT(*) AS employee_count,
  AVG(salary) AS avg_salary
FROM employees
WHERE status = 'active'           -- โ† WHERE: filter ROWS first (active only)
GROUP BY department
HAVING COUNT(*) > 5               -- โ† HAVING: filter GROUPS (only depts with 6+ people)
   AND AVG(salary) > 60000;       -- โ† HAVING: aggregate condition on the group

Execution order:

  1. FROM employees (get all rows)
  2. WHERE status = 'active' (filter to active rows)
  3. GROUP BY department (form groups)
  4. HAVING COUNT(*) > 5 (filter groups)
  5. SELECT (compute final output)

โŒ Common Mistake: Using Aggregate in WHERE

-- โŒ WRONG โ€” cannot use aggregate in WHERE
SELECT department, COUNT(*)
FROM employees
WHERE COUNT(*) > 5   -- ERROR: aggregate functions not allowed in WHERE
GROUP BY department;

-- โœ… CORRECT โ€” use HAVING for aggregate conditions
SELECT department, COUNT(*)
FROM employees
GROUP BY department
HAVING COUNT(*) > 5;

๐Ÿ“Š WHERE vs HAVING

FeatureWHEREHAVING
When it filtersBefore GROUP BYAfter GROUP BY
FiltersIndividual rowsGroups (aggregated results)
Aggregate functionsโŒ Not allowedโœ… Allowed (COUNT, SUM, AVG)
Can reference aliasesโŒ No (evaluated before SELECT)โŒ No (depends on DB)
Performanceโœ… Faster (reduces rows early)Slower (aggregates all first)
Works without GROUP BYโœ… YesRarely

๐ŸŒ Real-World Examples

-- Find categories with more than 10 active products
SELECT category_id, COUNT(*) AS product_count
FROM products
WHERE status = 'active'          -- WHERE: filter to active products only
GROUP BY category_id
HAVING COUNT(*) > 10;            -- HAVING: only categories with 10+ active products

-- Find customers with total orders over $1000
SELECT customer_id, SUM(amount) AS total
FROM orders
WHERE created_at > '2024-01-01'  -- WHERE: recent orders only
GROUP BY customer_id
HAVING SUM(amount) > 1000;       -- HAVING: only high-value customers

๐Ÿ’ก Possible Follow-up Questions

  • What is the SQL query execution order (FROM, WHERE, GROUP BY, HAVING, SELECT, ORDER BY)?
  • Can you use a WHERE clause without a GROUP BY?
  • Why is WHERE more efficient than HAVING for row filtering?
  • Can you use column aliases in HAVING? (PostgreSQL: yes, MySQL: sometimes)
  • What is the difference between HAVING COUNT(*) and WHERE COUNT(*)?

๐Ÿง  Simple Definition

SQL JOINs combine rows from two or more tables based on a matching column (usually a foreign key relationship). The type of JOIN determines which rows are included when there's no match between tables.


โšก Super Simple Line

INNER = only matching rows from both tables.
LEFT = all left rows + matched right rows (NULL if no right match).
RIGHT = all right rows + matched left rows (NULL if no left match).
FULL OUTER = all rows from both sides (NULLs where no match).
CROSS = every combination of every row (cartesian product).


๐Ÿงช Sample Data Setup

-- Customers table
| id | name    |
|----|---------|
| 1  | Alice   |
| 2  | Bob     |
| 3  | Charlie |

-- Orders table
| id | customer_id | amount |
|----|-------------|--------|
| 1  | 1           | 100    |
| 2  | 1           | 250    |
| 3  | 2           | 75     |
-- Note: Charlie (id=3) has no orders

๐Ÿงช JOIN Examples

-- 1. INNER JOIN: only customers who have orders
SELECT c.name, o.amount
FROM customers c
INNER JOIN orders o ON c.id = o.customer_id;
-- Result: Alice, Bob only (Charlie excluded)

-- 2. LEFT JOIN: ALL customers, even those without orders
SELECT c.name, o.amount
FROM customers c
LEFT JOIN orders o ON c.id = o.customer_id;
-- Result: Alice, Bob, Charlie (Charlie has NULL amount)

-- 3. RIGHT JOIN: ALL orders, even if no matching customer
SELECT c.name, o.amount
FROM customers c
RIGHT JOIN orders o ON c.id = o.customer_id;

-- 4. FULL OUTER JOIN: everything from both sides
SELECT c.name, o.amount
FROM customers c
FULL OUTER JOIN orders o ON c.id = o.customer_id;

-- 5. SELF JOIN: join a table with itself
SELECT a.name AS employee, b.name AS manager
FROM employees a
JOIN employees b ON a.manager_id = b.id;

๐Ÿ“Š Visual Comparison

JOIN TypeLeft RowsRight RowsUnmatched
INNER JOINMatching onlyMatching onlyDiscarded both sides
LEFT JOINAll rowsMatching onlyNULL on right side
RIGHT JOINMatching onlyAll rowsNULL on left side
FULL OUTER JOINAll rowsAll rowsNULL on both sides
CROSS JOINAll rowsAll rowsCartesian product

๐ŸŒ Real-World Use Cases

  • INNER JOIN: Get all orders with their customer info (both must exist)
  • LEFT JOIN: Get all users, show their last login if it exists (users without logins still appear)
  • FULL OUTER JOIN: Find unmatched rows on both sides (data integrity checks)
  • SELF JOIN: Employee hierarchy tree (employee โ†’ manager)

๐Ÿ’ก Possible Follow-up Questions

  • What is the difference between INNER JOIN and WHERE with multiple tables?
  • When would you use a CROSS JOIN?
  • What is a self-join and give a use case?
  • How do NULLs affect JOIN results?
  • How does JOIN performance depend on indexes?

โšก One-line Interview Answer

INNER JOIN returns only rows matching in both tables; LEFT JOIN returns all left-table rows with NULLs for unmatched right-table columns; FULL OUTER JOIN returns all rows from both tables with NULLs where matches don't exist on either side.

๐Ÿง  Simple Definition

A subquery (also called an inner query or nested query) is a SQL query embedded inside another SQL query. The inner query executes first and its result is used by the outer query. Subqueries can appear in SELECT, WHERE, FROM, and HAVING clauses.


๐Ÿงช Types of Subqueries

1. Scalar Subquery โ€” Returns a single value

-- Find employees earning above average salary
SELECT name, salary
FROM employees
WHERE salary > (
  SELECT AVG(salary) FROM employees  -- โ† scalar: returns one number
);

2. Row Subquery โ€” Returns a single row

SELECT * FROM products
WHERE (category_id, price) = (
  SELECT category_id, MIN(price) FROM products GROUP BY category_id LIMIT 1
);

3. Table Subquery (Derived Table) โ€” Returns multiple rows/columns

-- Subquery in FROM clause (like a temporary table)
SELECT dept_avg.department, dept_avg.avg_salary
FROM (
  SELECT department, AVG(salary) AS avg_salary
  FROM employees
  GROUP BY department
) AS dept_avg  -- โ† named derived table
WHERE dept_avg.avg_salary > 60000;

4. Correlated Subquery โ€” References outer query

-- Find employees earning more than their department's average
SELECT e.name, e.salary, e.department
FROM employees e
WHERE e.salary > (
  SELECT AVG(e2.salary)
  FROM employees e2
  WHERE e2.department = e.department  -- โ† references outer query's e.department
);
-- Note: Correlated subqueries run ONCE PER ROW โ€” can be slow on large tables

5. EXISTS Subquery โ€” Check existence

-- Find customers who have placed at least one order
SELECT c.name
FROM customers c
WHERE EXISTS (
  SELECT 1 FROM orders o WHERE o.customer_id = c.id
);
-- EXISTS stops at first match โ€” more efficient than IN for large sets

๐Ÿ“Š Subquery vs JOIN vs CTE

FeatureSubqueryJOINCTE
ReadabilityMediumHighโœ… Highest
Reusability in same queryโŒ Repeat each timeโŒโœ… Reference multiple times
PerformanceDepends on optimizerUsually bestSimilar to subquery
Recursive queriesโŒโŒโœ… RECURSIVE CTEs

โš ๏ธ Performance Note

  • Non-correlated subqueries: run once, result cached โ†’ generally OK
  • Correlated subqueries: run once per outer row โ†’ can be very slow on large tables
  • Modern query optimizers (PostgreSQL) often rewrite subqueries as JOINs automatically
  • Use EXPLAIN ANALYZE to check if subquery is causing a performance problem

๐Ÿ’ก Possible Follow-up Questions

  • What is a correlated subquery and why can it be slow?
  • What is the difference between IN and EXISTS?
  • When would you use a subquery vs a CTE?
  • Can a subquery be used in an UPDATE or DELETE statement?
  • How does the database optimizer handle subqueries?

โšก One-line Interview Answer

A subquery is a query nested inside another query where the inner query executes first and its result is used by the outer query. Types include scalar (single value in WHERE), derived table (in FROM clause), and correlated (references outer query's row โ€” runs once per outer row, so can be slow on large datasets and often better replaced with a JOIN or CTE).

๐Ÿง  Simple Definition

A CTE (Common Table Expression) is a named temporary result set defined at the start of a query using the WITH keyword. It exists only for the duration of the query and can be referenced like a table. CTEs make complex queries more readable, reusable, and support recursive queries for hierarchical data.


โšก Super Simple Line

CTE = a named temporary "virtual table" you define once at the top and reuse multiple times in the same query. Like a variable for a SQL result set.


๐Ÿงช Basic CTE Syntax

WITH cte_name AS (
  SELECT department, AVG(salary) AS avg_salary
  FROM employees
  GROUP BY department
)
SELECT e.name, e.salary, c.avg_salary
FROM employees e
JOIN cte_name c ON e.department = c.department
WHERE e.salary > c.avg_salary;  -- employees earning above their dept average

๐Ÿงช Multiple CTEs (Chained)

WITH
  active_users AS (
    SELECT * FROM users WHERE active = true
  ),
  high_spenders AS (
    SELECT user_id, SUM(amount) AS total
    FROM orders
    GROUP BY user_id
    HAVING SUM(amount) > 1000
  )
SELECT u.name, h.total
FROM active_users u
JOIN high_spenders h ON u.id = h.user_id;
-- Active users who spent more than $1000

๐Ÿงช Recursive CTE (Hierarchy / Tree)

-- Find all employees in a management chain
WITH RECURSIVE org_chart AS (
  -- Anchor: start with CEO (no manager)
  SELECT id, name, manager_id, 0 AS depth
  FROM employees
  WHERE manager_id IS NULL

  UNION ALL

  -- Recursive: find each person's reports
  SELECT e.id, e.name, e.manager_id, o.depth + 1
  FROM employees e
  JOIN org_chart o ON e.manager_id = o.id
)
SELECT name, depth FROM org_chart ORDER BY depth;
-- Outputs every employee with their level in the hierarchy

๐Ÿ“Š CTE vs Subquery

FeatureCTESubquery
Readabilityโœ… Named, defined once at topโŒ Nested, harder to read
Reusability in same queryโœ… Can reference same CTE multiple timesโŒ Must repeat subquery
Recursive queriesโœ… Supported (RECURSIVE keyword)โŒ Not possible
PerformanceSimilar in modern PostgreSQLSimilar in modern PostgreSQL
Debuggingโœ… Easier (run CTE independently)โŒ Harder to isolate

๐ŸŒ When to Use CTE Over Subquery

  • When you'd need the same subquery twice or more in a query
  • For recursive data (org charts, category trees, comment threads)
  • When you want to break complex logic into named, readable steps
  • When readability and maintainability matter

๐Ÿ’ก Possible Follow-up Questions

  • What is a recursive CTE and give a real-world use case?
  • Does PostgreSQL optimize CTEs the same way as subqueries?
  • What is the difference between a CTE and a temporary table?
  • Can a CTE be used in an INSERT, UPDATE, or DELETE statement?
  • What is a "materialized" CTE in PostgreSQL?

โšก One-line Interview Answer

A CTE is a named temporary result set defined with the WITH keyword that exists only for the duration of the query. I prefer CTEs over subqueries when the same result is needed multiple times, when writing recursive hierarchical queries, or when the logic is complex enough that naming each step dramatically improves readability.

๐Ÿง  Simple Definition

Window functions perform calculations across a set of rows related to the current row, but unlike GROUP BY, they do NOT collapse rows into a single output row. Each row retains its individual identity while also having access to aggregated information from its "window" of related rows.


โšก Super Simple Line

Window functions = run aggregate calculations OVER a window of rows without collapsing the result set.
GROUP BY collapses rows. Window functions preserve all rows.


๐Ÿ”‘ Syntax

FUNCTION() OVER (
  PARTITION BY column   -- divide rows into groups
  ORDER BY column       -- order within each partition
  ROWS BETWEEN ...      -- optional frame clause
)

๐Ÿงช RANK() Example

SELECT
  name,
  department,
  salary,
  RANK() OVER (
    PARTITION BY department
    ORDER BY salary DESC
  ) AS salary_rank
FROM employees;

-- Output:
-- | name    | department | salary | salary_rank |
-- |---------|------------|--------|-------------|
-- | Alice   | Engineering| 95000  | 1           |
-- | Bob     | Engineering| 90000  | 2           |
-- | Charlie | Engineering| 90000  | 2           |  โ† same rank (tie)
-- | Dave    | Engineering| 80000  | 4           |  โ† skips 3 (gap after tie)
-- | Eve     | Sales      | 70000  | 1           |  โ† restarts for new partition

๐Ÿ“Š Common Window Functions

FunctionWhat It DoesTie Behavior
ROW_NUMBER()Unique sequential number (1,2,3...)Arbitrary order for ties
RANK()Rank with gaps after ties (1,2,2,4)Same rank, next skipped
DENSE_RANK()Rank without gaps (1,2,2,3)Same rank, no gaps
LAG(col, n)Value from n rows before currentN/A
LEAD(col, n)Value from n rows after currentN/A
SUM() OVER()Running totalN/A
AVG() OVER()Moving averageN/A
FIRST_VALUE()First value in windowN/A

๐Ÿงช LAG/LEAD Example (Month-over-Month Growth)

SELECT
  month,
  revenue,
  LAG(revenue, 1) OVER (ORDER BY month) AS prev_month_revenue,
  revenue - LAG(revenue, 1) OVER (ORDER BY month) AS growth
FROM monthly_sales;
-- Calculates revenue growth vs previous month for each row

๐Ÿงช Running Total Example

SELECT
  order_date,
  amount,
  SUM(amount) OVER (ORDER BY order_date) AS running_total
FROM orders;
-- Each row shows the cumulative total up to that date

๐ŸŒ Real-World Use Cases

  • Leaderboards (rank users by score per category)
  • Running totals (cumulative revenue per day)
  • Period-over-period comparisons (LAG for month-over-month growth)
  • Top N per group (ROW_NUMBER for top 3 products per category)

๐Ÿ’ก Possible Follow-up Questions

  • What is the PARTITION BY clause and how does it work?
  • What's the difference between RANK and DENSE_RANK?
  • How would you find the top 3 earners per department?
  • What is the ROWS BETWEEN frame clause?
  • How do window functions differ from GROUP BY + subquery?

โšก One-line Interview Answer

Window functions perform aggregate calculations across a related set of rows (the "window") without collapsing them into a single row, using OVER (PARTITION BY ... ORDER BY ...) syntax. RANK() assigns ranks within each partition, giving ties the same rank with gaps, while DENSE_RANK() gives no gaps and ROW_NUMBER() gives unique sequential numbers regardless of ties.

๐Ÿง  Simple Definition

DELETE, TRUNCATE, and DROP all remove data, but at different levels of scope and with different implications for transactions, triggers, and recovery.


โšก Super Simple Line

DELETE = remove specific rows (with WHERE), logged, rollbackable.
TRUNCATE = remove ALL rows instantly, minimal logging, usually not rollbackable.
DROP = delete the entire table (structure + data) permanently.


๐Ÿงช Examples

-- DELETE: remove specific rows with optional WHERE
DELETE FROM orders WHERE status = 'cancelled';  -- removes matching rows only
DELETE FROM orders;                              -- removes ALL rows (very slow for large tables)

-- TRUNCATE: removes ALL rows instantly
TRUNCATE TABLE orders;          -- fast, no WHERE support
TRUNCATE TABLE orders RESTART IDENTITY;  -- also resets auto-increment counter

-- DROP: removes the entire table
DROP TABLE orders;              -- table is gone โ€” cannot SELECT from it anymore
DROP TABLE IF EXISTS orders;    -- won't error if table doesn't exist

๐Ÿ“Š Comparison Table

FeatureDELETETRUNCATEDROP
RemovesSpecific rowsAll rowsEntire table + structure
WHERE clauseโœ… YesโŒ NoโŒ No
SpeedSlow (row by row)โœ… Very fastโœ… Fast
Transaction safeโœ… Fullyโœ… In PostgreSQLโŒ Usually not
Triggers fireโœ… Yes (per row)โŒ NoโŒ No
Resets auto-incrementโŒ Noโœ… With RESTART IDENTITYโœ… Table gone
Table structure keptโœ… Yesโœ… YesโŒ No
WAL loggingFully loggedMinimal loggingMinimal

๐ŸŒ When to Use Each

  • DELETE: Remove specific records (expired sessions, cancelled orders)
  • TRUNCATE: Clear a staging/temp table before reloading data (ETL pipelines)
  • DROP: Remove a table during schema migration (deprecated tables)

๐Ÿšจ PostgreSQL Note on TRUNCATE

Unlike MySQL, PostgreSQL TRUNCATE IS transactional โ€” you CAN rollback it inside a transaction. This makes it safer for use in migrations.

-- In PostgreSQL (unlike MySQL): TRUNCATE can be rolled back!
BEGIN;
TRUNCATE TABLE staging_data;
-- If something goes wrong:
ROLLBACK;  -- โœ… data restored in PostgreSQL

๐Ÿ’ก Possible Follow-up Questions

  • Why is TRUNCATE faster than DELETE?
  • Does TRUNCATE fire triggers?
  • How do you recover from an accidental DROP TABLE?
  • What is the difference between TRUNCATE and DELETE in MySQL vs PostgreSQL?
  • What DDL vs DML categories do these belong to?

โšก One-line Interview Answer

DELETE removes specific rows using a WHERE condition, is fully logged and rollbackable, and fires row-level triggers. TRUNCATE removes all rows instantly with minimal logging (no WHERE clause, no trigger firing), and resets the identity counter. DROP removes the entire table structure and data permanently โ€” they differ in scope, speed, rollback support, and trigger behavior.

๐Ÿง  Simple Definition

A stored procedure is a named, precompiled block of SQL statements stored directly in the database that can be called by name. It can accept parameters, contain conditional logic, loops, and multiple SQL statements, and return results. Stored procedures run server-side, reducing network roundtrips.


๐Ÿงช Creating and Calling a Stored Procedure (PostgreSQL)

-- Create a stored procedure
CREATE OR REPLACE PROCEDURE transfer_funds(
  sender_id   INT,
  receiver_id INT,
  amount      DECIMAL
)
LANGUAGE plpgsql
AS $$
BEGIN
  -- Check sufficient balance
  IF (SELECT balance FROM accounts WHERE id = sender_id) < amount THEN
    RAISE EXCEPTION 'Insufficient funds';
  END IF;

  -- Perform transfer atomically
  UPDATE accounts SET balance = balance - amount WHERE id = sender_id;
  UPDATE accounts SET balance = balance + amount WHERE id = receiver_id;

  -- Log the transaction
  INSERT INTO transaction_log(from_id, to_id, amount, created_at)
  VALUES (sender_id, receiver_id, amount, NOW());

  COMMIT;
END;
$$;

-- Call the procedure
CALL transfer_funds(1, 2, 500.00);

๐Ÿ“Š Stored Procedure vs Function vs Application Code

FeatureStored ProcedureDB FunctionApp Code
Returns valueVia OUT paramsโœ… Returns valueโœ… Returns value
Transaction controlโœ… Can COMMIT/ROLLBACKโŒ Cannotโœ… Full control
Network roundtripsโœ… Minimal (runs on DB)โœ… MinimalโŒ Multiple roundtrips
MaintainabilityโŒ Hard to version/testโŒ Same issueโœ… Version controlled code
Performanceโœ… Precompiled planโœ… PrecompiledDepends
Called withCALLSELECTApplication

โœ… When to Use Stored Procedures

  • Complex multi-step database operations that should be atomic
  • Operations called frequently from many different clients/languages
  • Security: expose only procedure interface, not underlying table structure
  • Legacy systems or DBA-managed database logic

โŒ When to Avoid Stored Procedures

  • Modern applications with ORM-managed migrations (version control is hard)
  • Logic that needs testing with unit tests (hard to test SP in isolation)
  • When business logic should stay in the application layer
  • Microservices (each service should own its own data layer logic)

๐ŸŒ Real-World Use Cases

  • Banking: Fund transfers (multi-step atomicity)
  • Reporting: Complex aggregations run by many report viewers
  • Data ETL: Transform and load pipelines inside the DB
  • Audit logging: Automatically record changes on sensitive tables

๐Ÿ’ก Possible Follow-up Questions

  • What is the difference between a stored procedure and a function?
  • What is a trigger and how does it differ from a stored procedure?
  • What are the downsides of storing business logic in stored procedures?
  • How do you debug a stored procedure?
  • How does Prisma handle stored procedures?

โšก One-line Interview Answer

A stored procedure is a named, precompiled block of SQL logic stored in the database that accepts parameters and can contain conditional logic, transactions, and multiple SQL operations. It runs server-side reducing network roundtrips, and is useful for complex atomic operations shared across multiple clients โ€” though I prefer to keep business logic in the application layer for testability and maintainability, using stored procedures mainly for DBA-managed reporting or security-isolated database operations.

๐Ÿง  Simple Definition

Normalization is the process of organizing database tables to eliminate data redundancy and insertion/update/deletion anomalies by splitting data into related tables with proper relationships. Denormalization is the intentional reversal โ€” adding redundancy back to speed up read queries by reducing costly JOINs.


โšก Super Simple Line

Normalization = split tables โ†’ reduce redundancy โ†’ ensure consistency (optimizes writes).
Denormalization = merge/duplicate โ†’ reduce JOINs โ†’ faster reads (optimizes reads).


๐Ÿ“š Normal Forms (1NF, 2NF, 3NF)

First Normal Form (1NF)

All column values must be atomic (no arrays or comma-separated lists). Each row must be uniquely identifiable.

-- โŒ Violates 1NF (phone_numbers is not atomic)
| id | name  | phone_numbers          |
|----|-------|------------------------|
| 1  | Alice | "555-0001, 555-0002"   |

-- โœ… 1NF compliant
| id | name  | phone_number |
|----|-------|--------------|
| 1  | Alice | 555-0001     |
| 1  | Alice | 555-0002     |

Second Normal Form (2NF)

Must be in 1NF. All non-key columns must be fully dependent on the ENTIRE primary key (no partial dependencies on composite keys).

Third Normal Form (3NF)

Must be in 2NF. No non-key column can depend on another non-key column (no transitive dependencies).

-- โŒ Violates 3NF (zip determines city โ€” transitive dependency)
| id | name  | zip   | city       |
|----|-------|-------|------------|
| 1  | Alice | 10001 | New York   |

-- โœ… 3NF โ€” separate zip-to-city relationship
| id | name  | zip   |     | zip   | city     |
|----|-------|-------|     |-------|----------|
| 1  | Alice | 10001 |     | 10001 | New York |

๐Ÿ”ฅ When to Denormalize Intentionally

1. Read-Heavy Reporting Dashboards

Joining 8+ tables per analytics query is expensive. Store pre-computed aggregates.

-- Denormalized: store total_orders directly on customer (avoid counting every query)
| customer_id | name  | total_orders | lifetime_value |
|-------------|-------|--------------|----------------|
| 1           | Alice | 47           | 12500          |

2. Caching Computed Values

-- Instead of: SELECT COUNT(*) FROM products WHERE category_id = X
-- Store it: UPDATE categories SET product_count = product_count + 1
-- Update on write, read directly โ†’ no join

๐Ÿ“Š Normalization vs Denormalization

AspectNormalizedDenormalized
Write performanceโœ… Fast (update once)โŒ Slower (update multiple places)
Read performanceโŒ Slower (many JOINs)โœ… Fast (data co-located)
Data consistencyโœ… Easy (single source of truth)โŒ Harder (must keep copies in sync)
Storageโœ… MinimalโŒ More redundant data
Best forOLTP (transactional apps)OLAP (analytics, dashboards)

๐Ÿ’ก Possible Follow-up Questions

  • What is Boyce-Codd Normal Form (BCNF)?
  • When should you denormalize in practice?
  • What is an OLTP vs OLAP database design?
  • What are insertion anomalies, update anomalies, and deletion anomalies?
  • How does MongoDB's document model relate to denormalization?

โšก One-line Interview Answer

Normalization organizes tables into normal forms (1NF, 2NF, 3NF) to eliminate data redundancy and ensure update consistency at the cost of more JOINs, while denormalization intentionally duplicates data to accelerate read-heavy queries in analytics systems โ€” the key tradeoff is write consistency vs read performance.

๐Ÿง  Simple Definition

SQL (Relational) databases like PostgreSQL store data in structured tables with fixed schemas, enforce ACID compliance, and use powerful JOINs for relationships. NoSQL databases like MongoDB store flexible semi-structured data (documents, key-values, graphs) optimized for horizontal scaling and dynamic schemas.


โšก Super Simple Line

SQL = structured, consistent, relational, ACID (use for financial/transactional data).
NoSQL = flexible, scalable, document-based (use for social, real-time, or schema-changing data).


๐Ÿ“Š Comparison Table

FeatureSQL (PostgreSQL)NoSQL (MongoDB)
Data ModelTables with rows and columnsJSON-like BSON documents
SchemaStrict, predefined, enforced at DB levelDynamic, flexible, schema-on-read
RelationshipsRich JOINs + foreign key constraintsEmbedding or manual references
ACIDโœ… Full transactions out-of-the-boxโœ… Single-document atomic; multi-doc needs sessions
ScalingVertical (more RAM/CPU) + read replicasโœ… Horizontal (sharding across many nodes)
Query LanguageSQL (standardized, powerful)MongoDB Query Language (JSON-based)
IndexingB-Tree, partial, expression indexesB-Tree, text, geospatial, TTL indexes
Best ForBanking, ERP, analyticsCatalogs, social feeds, real-time apps

โœ… Choose PostgreSQL When:

  • Data integrity is critical: Banking, payments, healthcare (ACID guarantees)
  • Complex relational queries: Multi-table JOINs, window functions, complex aggregations
  • Schema is stable and well-defined: You know the data shape in advance
  • Strong consistency required: Financial ledgers, inventory management
  • Regulatory compliance: Auditing, transactions that cannot be lost

โœ… Choose MongoDB When:

  • Schema evolves frequently: SaaS apps where each customer configures their own fields
  • Document-centric data: Product catalogs with varying attributes per category
  • Massive write scale: IoT sensor data, event logging, social media posts
  • Horizontal sharding needed: Data volume exceeds what one SQL server can handle
  • Geospatial or real-time data: Location tracking, user feeds

๐ŸŒ Real-World Hybrid Example

E-commerce Platform:
โ”œโ”€โ”€ PostgreSQL: orders, payments, inventory (ACID critical)
โ”œโ”€โ”€ MongoDB: product catalog (varying attributes per category)
โ””โ”€โ”€ Redis: cart sessions, flash sale inventory counters

๐Ÿ’ก Possible Follow-up Questions

  • What is the CAP theorem and how does it apply to SQL vs NoSQL?
  • When would you use a graph database vs SQL vs MongoDB?
  • What is BASE consistency and how does it differ from ACID?
  • Can PostgreSQL also store JSON documents? (Yes โ€” JSONB type)
  • How does MongoDB handle transactions across multiple collections?

โšก One-line Interview Answer

I choose PostgreSQL when data integrity, complex relational queries, and strong ACID guarantees are critical โ€” like financial transactions or reporting. I choose MongoDB when the schema evolves frequently, data is document-centric with varying structures, or horizontal sharding is needed for massive write scale โ€” and in practice I often use both together, with PostgreSQL handling transactional data and MongoDB handling catalog or event data.

๐Ÿง  Simple Definition

An ORM (Object-Relational Mapper) like Prisma or TypeORM abstracts SQL queries into JavaScript/TypeScript method calls and provides type safety, migrations, and automatic query generation. Raw SQL gives you direct control over every query with no abstraction layer.


โšก Super Simple Line

ORM = type-safe, faster to write, less control. Raw SQL = full power, more verbose, requires manual parameterization.


๐Ÿ“Š ORM vs Raw SQL Comparison

AspectORM (Prisma)Raw SQL
Type Safetyโœ… Full TypeScript types auto-generated from schemaโŒ Manual โ€” no types unless using a query builder
SQL Injection Preventionโœ… Automatic parameterizationโš ๏ธ Manual โ€” must parameterize yourself
Development Speedโœ… Faster CRUD โ€” less boilerplateโŒ More verbose for simple queries
Migrationsโœ… Schema-as-code, auto-generated migrationsโŒ Manual migration SQL files
Complex QueriesโŒ Can be awkward for complex JOINs, CTEs, window functionsโœ… Full SQL power
Query EfficiencyโŒ Can generate inefficient SQL (N+1, over-fetching)โœ… Precise control over query
DB-Specific FeaturesโŒ Hard to access (LATERAL joins, materialized views)โœ… Direct access
DebuggingโŒ Must inspect generated SQLโœ… Write exactly what runs

๐Ÿงช Prisma (ORM) Example

// โœ… Type-safe, auto-parameterized, readable
const user = await prisma.user.findUnique({
  where: { id: userId },
  include: {
    orders: {
      where: { status: "completed" },
      orderBy: { createdAt: "desc" },
      take: 5
    }
  }
});
// user.orders is fully typed as Order[]

๐Ÿงช Raw SQL Example (Same Query)

// More control, but verbose and requires manual handling
const result = await pool.query(`
  SELECT u.*, json_agg(o.*) AS orders
  FROM users u
  LEFT JOIN orders o ON o.user_id = u.id
  WHERE u.id = $1
    AND o.status = 'completed'
  GROUP BY u.id
  ORDER BY o.created_at DESC
  LIMIT 5
`, [userId]);

๐Ÿšจ ORM Pitfalls to Watch For

N+1 Problem with ORM

// โŒ N+1: fetches users, then 1 query per user for orders
const users = await prisma.user.findMany();
for (const user of users) {
  const orders = await prisma.order.findMany({ where: { userId: user.id } });
  // 1 + N queries! โŒ
}

// โœ… Fix: use include for eager loading (1 query with JOIN)
const users = await prisma.user.findMany({
  include: { orders: true }  // single optimized query
});

๐ŸŒ When to Use Each

  • โœ… ORM for: CRUD operations, rapid development, type safety requirements, standard queries
  • โœ… Raw SQL for: complex analytics, window functions, CTEs, reporting, performance-critical paths, DB-specific optimizations
  • โœ… Best of both: Use ORM for 80% of queries + raw SQL escape hatches via prisma.$queryRaw for complex ones

๐Ÿ’ก Possible Follow-up Questions

  • How do you use raw SQL inside Prisma (prisma.$queryRaw)?
  • What is the N+1 problem and how does an ORM cause it?
  • How does Prisma generate TypeScript types from the schema?
  • What is a query builder (like Knex.js) and how does it differ from an ORM?
  • How do ORMs handle database migrations?

โšก One-line Interview Answer

ORMs like Prisma offer type safety, automatic SQL injection prevention, schema-as-code migrations, and faster CRUD development at the cost of potential query inefficiency, N+1 problems, and limited access to advanced SQL features. Raw SQL offers full control and optimization at the cost of verbosity and manual parameterization โ€” I typically use an ORM for standard operations and drop to raw SQL for complex analytics or performance-critical queries.

๐Ÿง  Simple Definition

The N+1 query problem occurs when code fetches a list of N records with one query, then executes one additional query for each record to load related data โ€” resulting in N+1 total queries instead of 1 or 2 optimized queries. This causes unnecessary database load and high latency.


โšก Super Simple Line

N+1 = 1 query to get 100 orders + 100 separate queries to get the customer for each order = 101 queries instead of 1 JOIN query.


๐Ÿ”ฅ Classic N+1 Example

// โŒ N+1 Problem
const orders = await prisma.order.findMany();  // 1 query: gets 100 orders

for (const order of orders) {
  // โŒ 1 extra query per order = 100 more queries!
  const customer = await prisma.customer.findUnique({
    where: { id: order.customerId }
  });
  console.log(`${customer.name}: ${order.total}`);
}
// Total: 1 + 100 = 101 queries ๐Ÿ’€

โœ… Fix 1: Eager Loading (ORM)

// โœ… Single optimized query with JOIN
const orders = await prisma.order.findMany({
  include: { customer: true }  // JOIN in one query
});
// Total: 1 query โœ…

โœ… Fix 2: SQL JOIN (Raw)

-- โœ… One query with JOIN
SELECT o.*, c.name AS customer_name
FROM orders o
JOIN customers c ON c.id = o.customer_id;
-- Total: 1 query โœ…

โœ… Fix 3: Batching / DataLoader Pattern

// โœ… Collect all IDs, fetch in one batch query
const orders = await prisma.order.findMany();
const customerIds = [...new Set(orders.map(o => o.customerId))];

// One batch query instead of N
const customers = await prisma.customer.findMany({
  where: { id: { in: customerIds } }
});

// Map customers to orders in memory
const customerMap = new Map(customers.map(c => [c.id, c]));
const ordersWithCustomers = orders.map(o => ({
  ...o,
  customer: customerMap.get(o.customerId)
}));
// Total: 2 queries โœ…

๐Ÿ” How to Detect N+1

  • Enable Prisma query logging: log: ["query"] in PrismaClient
  • Look for the same query repeated N times in logs
  • Use database monitoring tools (e.g., PostgreSQL's pg_stat_statements)
  • Check response times โ€” N+1 causes latency to scale linearly with data size
const prisma = new PrismaClient({ log: ["query"] });
// Logs every SQL query โ€” N+1 will be obvious (same query repeated)

๐Ÿ“Š Impact

ScenarioWithout FixWith Fix
100 orders, load customers101 queries1-2 queries
1000 orders, load customers1001 queries1-2 queries
Response time~2000ms (network RTT per query)~20ms

๐Ÿ’ก Possible Follow-up Questions

  • How do you detect N+1 in a running application?
  • What is DataLoader and how does it batch requests?
  • How does Prisma's include/select help avoid N+1?
  • What is the difference between eager loading and lazy loading?
  • How does N+1 manifest differently in REST vs GraphQL APIs?

๐Ÿง  Simple Definition

Soft delete marks a record as deleted (e.g., sets deletedAt timestamp or isDeleted = true) without physically removing it from the database. Hard delete permanently removes the row. The choice depends on whether you need auditability, data recovery, or must preserve referential integrity.


โšก Super Simple Line

Soft delete = mark as deleted, keep data โ†’ recoverable, auditable.
Hard delete = remove permanently โ†’ simple, storage-efficient.


๐Ÿงช Soft Delete Implementation

-- Schema with soft delete columns
ALTER TABLE users ADD COLUMN deleted_at TIMESTAMP;
ALTER TABLE users ADD COLUMN deleted_by INTEGER;

-- Soft delete a user
UPDATE users
SET deleted_at = NOW(), deleted_by = admin_id
WHERE id = 42;

-- Query active users (must always filter!)
SELECT * FROM users WHERE deleted_at IS NULL;

-- Query deleted users (for admin / audit)
SELECT * FROM users WHERE deleted_at IS NOT NULL;

-- Restore a user
UPDATE users SET deleted_at = NULL WHERE id = 42;

๐Ÿงช Soft Delete in Prisma

// Schema
model User {
  id        Int       @id @default(autoincrement())
  name      String
  deletedAt DateTime?  // null = active, set = deleted
}

// Soft delete
await prisma.user.update({
  where: { id: userId },
  data: { deletedAt: new Date() }
});

// Always filter in queries (use middleware to automate)
const activeUsers = await prisma.user.findMany({
  where: { deletedAt: null }
});

๐Ÿ“Š When to Use Soft vs Hard Delete

Use CaseSoft DeleteHard Delete
Need data recoveryโœ… User deleted account by mistakeโŒ Gone forever
Audit trail requiredโœ… Who deleted what and whenโŒ No record
Legal data retentionโœ… Must keep for 7 years (GDPR exception)โŒ Not compliant
Referential integrityโœ… Orders reference deleted user โ†’ still validโŒ Foreign key violation or cascade delete
Truly unnecessary dataโŒ Storage wasteโœ… Clean removal
GDPR right to erasureโŒ Data still existsโœ… Actually deleted

โš ๏ธ Soft Delete Pitfalls

  • Must filter everywhere: Every query needs WHERE deleted_at IS NULL โ€” easy to forget
  • Unique constraints break: If user with email "a@b.com" is soft-deleted, you can't create a new user with the same email unless you handle it
  • Storage grows: Data never actually removed
  • Fix: Use Prisma middleware or a global query extension to auto-filter soft-deleted records
// Prisma middleware to automatically filter soft-deleted records
prisma.$use(async (params, next) => {
  if (params.model === 'User') {
    if (params.action === 'findMany' || params.action === 'findFirst') {
      params.args.where = { ...params.args.where, deletedAt: null };
    }
  }
  return next(params);
});

๐Ÿ’ก Possible Follow-up Questions

  • How do you handle unique constraints with soft deletes?
  • How does GDPR "right to erasure" interact with soft deletes?
  • How do you automate soft-delete filtering in Prisma?
  • What is the difference between soft delete and archiving?
  • How do you handle foreign key relationships with soft-deleted records?

โšก One-line Interview Answer

I use soft delete (setting deletedAt timestamp) when data recovery, audit trails, legal retention, or referential integrity are required, and hard delete when data genuinely should disappear and storage or GDPR compliance matters. The main tradeoff is that soft delete requires filtering deleted records in every query, which I automate using ORM middleware to prevent accidentally exposing deleted data.

๐Ÿง  Simple Definition

In MongoDB, embedding means storing related data inside a single document. Referencing means storing just an ID and looking up the related document separately. This is the core data modeling decision in MongoDB, and it fundamentally affects performance, query complexity, and scalability.


โšก Super Simple Line

Embed = put related data inside the document (like a nested JSON object).
Reference = store only the ID, and join later (like a foreign key in SQL).


๐Ÿงช Embedding Example

// User with embedded addresses
{
  _id: ObjectId("abc"),
  name: "Alice",
  addresses: [
    { type: "home", city: "New York" },
    { type: "work", city: "Boston" }
  ]
}
// โœ… One query retrieves everything โ€” no join needed

๐Ÿงช Referencing Example

// User document
{ _id: ObjectId("u1"), name: "Alice" }

// Order documents referencing the user
{ _id: ObjectId("o1"), userId: ObjectId("u1"), total: 100 }
{ _id: ObjectId("o2"), userId: ObjectId("u1"), total: 250 }

// To get user + orders, need two queries or $lookup

๐Ÿ“Š When to Embed vs Reference

SituationStrategyWhy
Data always fetched togetherโœ… EmbedSingle read, no join
One-to-few (e.g., addresses per user)โœ… EmbedBounded size, always related
Child has no independent existenceโœ… EmbedLogically inseparable
Array could grow unbounded (>100)โœ… ReferenceMongoDB 16MB document limit
Data shared across multiple documentsโœ… ReferenceAvoids duplication inconsistency
Many-to-many relationshipโœ… ReferenceCannot embed in both directions
Child accessed independently oftenโœ… ReferenceSeparate queries are more efficient

๐Ÿšจ The 16MB Document Limit

MongoDB has a hard limit of 16MB per document. If you embed an array that grows unboundedly (e.g., all comments on a popular post), you will hit this limit. Always reference when arrays could grow large.


๐ŸŒ Real-World Examples

โœ… Embed: Blog Post + Comments (if limited)

{
  _id: ObjectId("p1"),
  title: "MongoDB Tips",
  comments: [   // โœ… OK if comments stay small
    { author: "Bob", text: "Great post!" }
  ]
}

โœ… Reference: Blog Post + Comments (if unlimited)

// Post
{ _id: ObjectId("p1"), title: "MongoDB Tips" }

// Comments referencing post
{ _id: ObjectId("c1"), postId: ObjectId("p1"), text: "Great!" }
{ _id: ObjectId("c2"), postId: ObjectId("p1"), text: "Helpful!" }

๐Ÿง  Rule of Thumb

  • One-to-few โ†’ Embed
  • One-to-many โ†’ Reference (with array of IDs on parent)
  • One-to-squillions โ†’ Reference (store parent ID on child side)

๐Ÿ’ก Possible Follow-up Questions

  • What happens if an embedded array grows too large?
  • How does $lookup work in MongoDB for references?
  • How do you handle many-to-many relationships in MongoDB?
  • What is the 16MB document size limit and how do you work around it?
  • How does this compare to how SQL handles relationships?

โšก One-line Interview Answer

Embed when data is always accessed together and the child array is bounded; reference when data is accessed independently, shared across documents, or the array could grow unbounded beyond MongoDB's 16MB document size limit.

๐Ÿง  Simple Definition

The MongoDB Aggregation Pipeline is a data processing framework where documents pass through a series of stages, each transforming the data until the final result is produced. Think of it like an assembly line: data goes in, gets filtered, grouped, reshaped, sorted, and comes out transformed.


โšก Super Simple Line

Aggregation pipeline = a chain of stages that transform documents one step at a time, similar to SQL's SELECT + WHERE + GROUP BY + ORDER BY combined.


๐Ÿ”‘ Core Pipeline Stages

StageSQL EquivalentWhat It Does
$matchWHEREFilter documents by condition
$groupGROUP BYAggregate values (sum, avg, count)
$projectSELECTInclude, exclude, or rename fields
$sortORDER BYSort documents
$limitLIMITLimit number of results
$skipOFFSETSkip N documents (pagination)
$lookupLEFT JOINJoin data from another collection
$unwindN/AFlatten array fields into separate documents
$addFieldscomputed columnsAdd computed fields to documents
$facetN/ARun multiple pipelines in parallel

๐Ÿงช Real Example: Total Sales Per User

db.orders.aggregate([
  // Stage 1: filter only completed orders
  { $match: { status: "completed" } },

  // Stage 2: group by userId and sum their totals
  { $group: {
    _id: "$userId",
    totalSpent: { $sum: "$amount" },
    orderCount: { $sum: 1 }
  }},

  // Stage 3: sort by highest spenders
  { $sort: { totalSpent: -1 } },

  // Stage 4: top 10 only
  { $limit: 10 }
]);

๐Ÿงช $lookup Example (Join)

db.orders.aggregate([
  { $lookup: {
    from: "users",           // collection to join
    localField: "userId",    // field in orders
    foreignField: "_id",     // field in users
    as: "userDetails"        // output array field
  }}
]);
// Result: each order has a "userDetails" array with user data embedded

๐Ÿงช $unwind Example

// If a document has: { tags: ["mongodb", "nosql", "database"] }
{ $unwind: "$tags" }
// Creates 3 separate documents, one per tag
// Useful for counting tag frequency

โš™๏ธ Performance Tips

  • Always put $match first to filter early and reduce documents processed
  • Put $project early to reduce document size before heavy operations
  • Use indexes on fields used in $match and $sort
  • Use allowDiskUse: true for large aggregations that exceed 100MB RAM limit

๐Ÿ’ก Possible Follow-up Questions

  • How does $facet allow multiple aggregation pipelines?
  • How do you optimize a slow aggregation pipeline?
  • How does $lookup compare to SQL JOIN in terms of performance?
  • What is $unwind and when would you use it?
  • How do you paginate results using aggregation?

โšก One-line Interview Answer

MongoDB's aggregation pipeline processes documents through sequential stages like $match (filter), $group (aggregate), $project (reshape), $lookup (join), and $sort, where each stage transforms the output of the previous one, similar to SQL's query clauses chained together.

๐Ÿง  Simple Definition

A database index is a data structure (typically a B-Tree) that the database maintains separately from the main data, mapping field values to document locations to enable fast lookups without scanning every document. Without an index, MongoDB does a collection scan (reads every document). With an index, it jumps directly to matching documents.


โšก Super Simple Line

Index = a sorted shortcut to find documents fast, like a book's index pointing you to the right page.
Without index โ†’ scan all documents (slow).
With index โ†’ jump straight to matches (fast).


๐Ÿ“š Types of Indexes in MongoDB

Index TypeBest ForExample
Single FieldSimple field lookupsdb.users.createIndex({ email: 1 })
CompoundMulti-field queries + sortsdb.users.createIndex({ lastName: 1, age: -1 })
MultikeyArray field indexingdb.posts.createIndex({ tags: 1 })
TextFull-text keyword searchdb.articles.createIndex({ content: "text" })
Geospatial 2dsphereLocation-based queriesdb.places.createIndex({ location: "2dsphere" })
HashedSharding shard keydb.users.createIndex({ _id: "hashed" })
TTLAuto-expiring documentsdb.sessions.createIndex({ createdAt: 1 }, { expireAfterSeconds: 3600 })

๐Ÿงช Compound Index Code

// Compound index - field order matters!
db.users.createIndex({ lastName: 1, age: -1 });

// This index CAN serve:
db.users.find({ lastName: "Smith" });                     // โœ… prefix match
db.users.find({ lastName: "Smith", age: { $gt: 25 } });  // โœ… both fields
db.users.find({ lastName: "Smith" }).sort({ age: -1 });  // โœ… sort covered

// This index CANNOT serve:
db.users.find({ age: 25 });  // โŒ age alone (not a prefix)

๐Ÿงช Text Index Code

// Create text index on multiple fields
db.articles.createIndex({ title: "text", content: "text" });

// Search using $text operator
db.articles.find({
  $text: { $search: "database index performance" }
});

// โš ๏ธ Only ONE text index allowed per collection

๐Ÿ† The ESR Rule (Interview Gold)

When building compound indexes, order fields in this sequence for maximum performance:

  1. Equality first โ€” fields with exact match conditions (e.g., status: "active")
  2. Sort next โ€” fields in the sort clause (e.g., createdAt: -1)
  3. Range last โ€” fields with range conditions (e.g., age: { $gt: 25 })
// Query: find active users older than 25, sorted by name
db.users.find({ status: "active", age: { $gt: 25 } }).sort({ name: 1 });

// โœ… ESR-correct index:
db.users.createIndex({ status: 1, name: 1, age: 1 });
// E=status, S=name, R=age

โš ๏ธ When to AVOID Indexes

  • Collections with very few documents (full scan is fine)
  • Fields with very low cardinality (e.g., a boolean field โ€” only 2 values)
  • Write-heavy collections (every write must update all indexes)
  • Too many indexes slow down writes significantly

๐Ÿ’ก Possible Follow-up Questions

  • What is a covered query and how do indexes enable it?
  • How do you check if an index is being used with explain()?
  • What is index selectivity and why does it matter?
  • What is the ESR rule for compound indexes?
  • How does a TTL index work and when would you use it?
  • What is the difference between a multikey index and a regular index?

โšก One-line Interview Answer

MongoDB indexes are B-Tree data structures that enable fast document lookups by mapping field values to locations, avoiding full collection scans. Compound indexes support multi-field queries and sorts following the ESR rule, while text indexes tokenize string fields for full-text keyword search, and TTL indexes automatically expire documents after a set duration.

๐Ÿง  Simple Definition

EXPLAIN ANALYZE is a PostgreSQL command that actually executes a query and then shows you a detailed query execution plan including actual runtime statistics (rows processed, time spent, memory used). It's the primary tool for diagnosing slow queries.


โšก Super Simple Line

EXPLAIN = shows the plan (doesn't run the query).
EXPLAIN ANALYZE = runs the query AND shows plan + actual performance stats.
Use it to find why a query is slow and where to add indexes.


๐Ÿงช Basic Usage

EXPLAIN ANALYZE
SELECT * FROM orders
WHERE customer_id = 42
ORDER BY created_at DESC
LIMIT 10;

๐Ÿ“‹ Sample Output Explained

Limit (cost=0.42..20.56 rows=10 width=100) (actual time=0.052..0.185 rows=10 loops=1)
  -> Index Scan using orders_customer_id_idx on orders
      (cost=0.42..1854.20 rows=900 width=100)
      (actual time=0.045..0.172 rows=10 loops=1)
      Index Cond: (customer_id = 42)
Planning Time: 0.5 ms
Execution Time: 0.2 ms

๐Ÿ” What to Look For

Warning SignWhat It MeansFix
Seq Scan on large tableFull table scan โ€” no index usedAdd index on filter/join column
Rows estimate โ‰  actual rowsStale query planner statisticsRun ANALYZE or VACUUM ANALYZE
Nested Loop on large setsO(nยฒ) join โ€” bad for big tablesConsider Hash Join or index
High actual timeSlow node โ€” bottleneck found hereOptimize that specific operation
Sort without indexIn-memory sort of many rowsAdd index on ORDER BY columns
High rows ร— loopsNested loop multiplying workRewrite join or add index

๐Ÿงช Identifying a Missing Index

-- Bad output (no index):
Seq Scan on orders (cost=0.00..35000.00 rows=1000000 width=50)
  Filter: (customer_id = 42)
  Rows Removed by Filter: 999990  โ† 999,990 rows scanned to find 10!

-- Fix: add an index
CREATE INDEX orders_customer_id_idx ON orders(customer_id);

-- Good output (with index):
Index Scan using orders_customer_id_idx on orders
  Index Cond: (customer_id = 42)  โ† only 10 rows touched โœ…

โš™๏ธ EXPLAIN Options

EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) SELECT ...;
-- ANALYZE: run the query + show actual times
-- BUFFERS: show cache hit/miss info (shared_blks_hit vs shared_blks_read)
-- FORMAT JSON: machine-readable output for tools like explain.depesz.com

๐Ÿ’ก Possible Follow-up Questions

  • What is the difference between EXPLAIN and EXPLAIN ANALYZE?
  • What does a high row estimate mismatch mean and how do you fix it?
  • What is VACUUM ANALYZE and why is it important?
  • How do you read the cost numbers in an EXPLAIN output?
  • What is a Bitmap Index Scan vs an Index Scan?
  • What tools help visualize EXPLAIN ANALYZE output?

โšก One-line Interview Answer

EXPLAIN ANALYZE executes a query and returns the full execution plan with actual runtime stats. I look for Seq Scans on large tables indicating missing indexes, high rows-removed-by-filter counts, mismatches between estimated and actual rows suggesting stale statistics requiring VACUUM ANALYZE, and slow sort operations that could benefit from an index on the ORDER BY column.

๐Ÿง  Simple Definition

A database transaction is a sequence of operations that are executed as a single logical unit of work. Transactions must satisfy ACID properties to guarantee data integrity. PostgreSQL implements transactions natively and uses MVCC for isolation and WAL (Write-Ahead Log) for durability.


โšก Super Simple Line

Transaction = group of operations that all succeed together or all fail together. ACID = the 4 guarantees every reliable database transaction must have.


๐Ÿ”‘ ACID Properties Explained

A โ€” Atomicity

All operations in a transaction succeed, or ALL are rolled back. There is no partial execution.

BEGIN;
  UPDATE accounts SET balance = balance - 500 WHERE id = 1;  -- debit
  UPDATE accounts SET balance = balance + 500 WHERE id = 2;  -- credit
COMMIT;  -- both succeed together
-- If second UPDATE fails โ†’ ROLLBACK automatically reverts first UPDATE

C โ€” Consistency

The database always moves from one valid state to another. Constraints (NOT NULL, foreign keys, CHECK) are enforced. Invalid state is never written.

I โ€” Isolation

Concurrent transactions don't see each other's intermediate state. PostgreSQL uses MVCC โ€” each transaction reads a consistent snapshot.

D โ€” Durability

Committed transactions survive crashes. PostgreSQL uses WAL (Write-Ahead Log) โ€” changes are written to the log before the actual data files, so a crash can always be recovered.


โš™๏ธ PostgreSQL Under the Hood

MVCC (Multi-Version Concurrency Control)

  • When a row is updated, PostgreSQL writes a NEW version of the row (doesn't overwrite)
  • Old version remains visible to active transactions that started before the update
  • Readers never block writers, writers never block readers
  • Dead row versions are cleaned up by VACUUM

WAL (Write-Ahead Log)

  • Before changing actual data files, PostgreSQL writes the change to the WAL log first
  • On crash: PostgreSQL replays WAL to recover committed but unwritten changes
  • WAL also enables streaming replication (replica servers replay the same WAL)

๐Ÿ“Š Isolation Levels in PostgreSQL

Isolation LevelDirty ReadNon-Repeatable ReadPhantom Read
Read UncommittedPossiblePossiblePossible
Read Committed (default)โŒ PreventedPossiblePossible
Repeatable ReadโŒ PreventedโŒ PreventedโŒ Prevented
SerializableโŒ PreventedโŒ PreventedโŒ Prevented

๐Ÿ’ก Possible Follow-up Questions

  • What is the difference between COMMIT and ROLLBACK?
  • What is a savepoint in a transaction?
  • What are the different isolation levels and their tradeoffs?
  • What is a dirty read, phantom read, and non-repeatable read?
  • How does VACUUM work in PostgreSQL and why is it needed?
  • How does WAL enable replication?

โšก One-line Interview Answer

A database transaction is a logical unit of work that must satisfy ACID properties: Atomicity (all-or-nothing), Consistency (valid state transitions), Isolation (concurrent transactions don't interfere), and Durability (committed data survives crashes). PostgreSQL implements isolation using MVCC where each transaction reads a consistent snapshot without blocking others, and durability using Write-Ahead Logging which ensures changes can be recovered after a crash.

๐Ÿง  Simple Definition

MongoDB supports multi-document ACID transactions since version 4.0 (replica sets) and 4.2 (sharded clusters). By default, individual document writes in MongoDB are atomic. For operations spanning multiple documents or collections, you use session-based transactions to guarantee all-or-nothing execution.


โšก ACID Properties Explained Simply

PropertyMeaningMongoDB Implementation
AtomicityAll operations succeed or all roll backcommitTransaction() / abortTransaction()
ConsistencyData always moves between valid statesSchema validation + constraints
IsolationConcurrent transactions don't interfereMVCC snapshot isolation in WiredTiger
DurabilityCommitted data survives crashesWrite-ahead journal + replica acknowledgment

๐Ÿงช Transaction Code Example (Node.js)

const { MongoClient } = require("mongodb");
const client = new MongoClient("mongodb://localhost:27017");

async function transferFunds(fromId, toId, amount) {
  const session = client.startSession();

  try {
    session.startTransaction({
      readConcern: { level: "snapshot" },
      writeConcern: { w: "majority" }
    });

    const accounts = client.db("bank").collection("accounts");

    // Debit sender
    await accounts.updateOne(
      { _id: fromId, balance: { $gte: amount } },  // ensure sufficient funds
      { $inc: { balance: -amount } },
      { session }
    );

    // Credit receiver
    await accounts.updateOne(
      { _id: toId },
      { $inc: { balance: amount } },
      { session }
    );

    await session.commitTransaction();
    console.log("โœ… Transfer successful");

  } catch (error) {
    await session.abortTransaction();
    console.error("โŒ Transaction rolled back:", error.message);
    throw error;
  } finally {
    session.endSession();
  }
}

โš ๏ธ When Should You Use Transactions?

  • โœ… Transferring funds between accounts (multi-document atomicity required)
  • โœ… Creating an order + updating inventory + recording payment (3 collections)
  • โœ… Any operation where partial failure would leave data inconsistent
  • โŒ Simple CRUD on a single document (MongoDB is already atomic at document level)
  • โŒ As a substitute for good schema design (embed related data to avoid transactions)

๐Ÿšจ Transaction Performance Considerations

Transactions in MongoDB have higher overhead than single-document operations. Best practice: 80-90% of operations should be single-document. Use transactions only when truly needed for cross-document consistency.

  • Transactions hold locks โ€” keep them short
  • Default timeout: 60 seconds (configurable)
  • Transactions require a replica set (even in development)

๐Ÿ’ก Possible Follow-up Questions

  • How do MongoDB transactions compare to PostgreSQL transactions?
  • What happens if a transaction times out?
  • Can transactions span multiple replica set members?
  • What are read concerns and write concerns in MongoDB?
  • Why is single-document atomicity often sufficient in MongoDB?
  • How does the session object relate to transactions?

โšก One-line Interview Answer

MongoDB guarantees ACID at the single-document level by default, and supports multi-document ACID transactions via session-based transactions that span collections and replica set members, using MVCC for isolation and write-ahead journaling for durability โ€” though transactions should be used sparingly since good schema design with document embedding often eliminates the need for them.

๐Ÿง  Simple Definition

WiredTiger is MongoDB's default storage engine since version 3.2. It replaced the old MMAPv1 engine. MVCC (Multi-Version Concurrency Control) is WiredTiger's concurrency mechanism that allows multiple readers and writers to access data simultaneously without blocking each other by maintaining multiple versions of each document.


โšก Super Simple Line

WiredTiger = MongoDB's storage engine (handles disk I/O, compression, locking).
MVCC = readers see a consistent snapshot of data without blocking writers, and writers don't block readers.


๐Ÿง  How MVCC Works (Step by Step)

  1. Transaction A starts a read โ†’ gets a read timestamp
  2. Transaction B starts writing โ†’ creates a new version of the document
  3. Transaction A still reads the old version (its snapshot)
  4. After Transaction B commits โ†’ Transaction A's next read would see the new version
  5. Old versions are cleaned up when no longer needed (called checkpoint)

๐Ÿ“Š WiredTiger vs Old MMAPv1

FeatureWiredTigerMMAPv1 (old)
Locking levelDocument-level (fine-grained)Collection-level (coarse)
ConcurrencyHigh (MVCC โ€” readers don't block writers)Low (writers block all readers)
CompressionSnappy (default), zlib, zstdNone
MemoryUses its own cache (configurable)Used OS mmap
Write performanceBetter (journal batching)Slower

โš™๏ธ Key WiredTiger Features

  • Document-level concurrency: Multiple documents in the same collection can be modified simultaneously
  • Compression: Data is compressed by default using Snappy, reducing disk usage ~50-70%
  • Cache: WiredTiger maintains its own in-memory cache (default: 50% of RAM minus 1GB)
  • Checkpointing: Periodically writes a consistent snapshot to disk for crash recovery
  • Journaling: Write-ahead log ensures durability even before checkpoint

๐Ÿงช Checking Your Storage Engine

db.serverStatus().storageEngine
// Returns: { name: 'wiredTiger', supportsCommittedReads: true, ... }

๐Ÿ’ก Possible Follow-up Questions

  • How does MVCC prevent dirty reads?
  • What is a checkpoint in WiredTiger?
  • How does WiredTiger compression affect performance?
  • What is the WiredTiger cache and how do you tune it?
  • How does MVCC differ from pessimistic locking?
  • What is journaling and how does it relate to durability?

โšก One-line Interview Answer

WiredTiger is MongoDB's default storage engine that provides document-level locking, data compression, and MVCC (Multi-Version Concurrency Control), which allows readers to see a consistent snapshot of data at their read timestamp without blocking writers, enabling high concurrency compared to the old collection-level locking in MMAPv1.

๐Ÿง  Simple Definition

A unique constraint is a database-level rule that prevents two rows from having the same value(s) in a specific column or combination of columns. It's enforced by the database engine as an atomic operation, making it the only reliable way to prevent duplicates under concurrent load.


โšก Super Simple Line

Unique constraint = the database guarantees no duplicates, even if 1,000 requests try to create the same record simultaneously.


๐Ÿ”ฅ Why Application-Level Checks Are Not Enough

// โŒ DANGEROUS โ€” Race condition!
async function createUser(email: string) {
  // Two simultaneous requests can BOTH pass this check
  const existing = await prisma.user.findUnique({ where: { email } });
  if (existing) throw new Error("Email taken");

  // Both proceed to insert โ†’ DUPLICATE USER โŒ
  return prisma.user.create({ data: { email } });
}
// Time: Request A checks โ†’ no user found
// Time: Request B checks โ†’ no user found (A hasn't inserted yet)
// Time: Request A inserts โ†’ success
// Time: Request B inserts โ†’ success but creates duplicate! โŒ

โœ… The Fix: Database-Level Unique Constraint

-- PostgreSQL
ALTER TABLE users ADD CONSTRAINT users_email_unique UNIQUE (email);

-- Composite unique constraint (e.g., one OAuth account per provider per user)
ALTER TABLE oauth_accounts
ADD CONSTRAINT unique_user_provider UNIQUE (user_id, provider_id);
// Prisma schema
model User {
  id    Int    @id @default(autoincrement())
  email String @unique  // โ† database-enforced unique constraint
}

model OAuthAccount {
  id         Int    @id
  userId     Int
  providerId String
  @@unique([userId, providerId])  // โ† composite unique
}

๐Ÿงช Handling Unique Constraint Violations

import { Prisma } from "@prisma/client";

async function createUser(email: string) {
  try {
    return await prisma.user.create({ data: { email } });
  } catch (error) {
    if (error instanceof Prisma.PrismaClientKnownRequestError) {
      if (error.code === "P2002") {  // Unique constraint violation
        throw new Error("409: Email already registered");
      }
    }
    throw error;
  }
}

๐ŸŒ Upsert โ€” The Atomic Alternative

// Instead of check-then-insert, use upsert (atomic)
const user = await prisma.user.upsert({
  where: { email },
  update: {},          // nothing to update if exists
  create: { email }    // create if not exists
});
// โœ… Database guarantees atomicity โ€” no race condition

๐Ÿ“Š Common Unique Constraint Use Cases

TableUnique Column(s)Reason
usersemailOne account per email
usersusernameUnique usernames
oauth_accounts(user_id, provider)One Google/GitHub account per user
product_variants(product_id, sku)No duplicate SKUs per product
friendships(user_id, friend_id)No duplicate friend connections

๐Ÿ’ก Possible Follow-up Questions

  • What is the difference between a unique constraint and a unique index?
  • How do you handle unique constraint violations gracefully in an API?
  • What is an upsert and how does it prevent race conditions?
  • How do unique constraints interact with soft deletes?
  • What error code does Prisma return for unique constraint violations?

โšก One-line Interview Answer

Unique constraints enforce uniqueness at the database level as an atomic operation, which is critical because application-level "check then insert" patterns are vulnerable to race conditions under concurrent load โ€” two simultaneous requests can both pass the check and both insert duplicates. The correct approach is to rely on database constraints and catch the P2002 (unique constraint violated) error to return a clean 409 Conflict response.

๐Ÿง  Simple Definition

Pessimistic locking assumes conflicts are likely and locks a resource immediately when reading it, preventing others from modifying it. Optimistic locking assumes conflicts are rare and doesn't lock anything, but detects conflicts at update time using a version field.


โšก Super Simple Line

Pessimistic = "I expect conflict, so I lock it now."
Optimistic = "I expect no conflict, so I just check at update time."


๐Ÿงช Pessimistic Locking (SQL: SELECT FOR UPDATE)

BEGIN;
  -- Lock the row โ€” no one else can update this until we commit
  SELECT * FROM inventory
  WHERE product_id = 42
  FOR UPDATE;  -- โ† pessimistic lock

  -- Safely update knowing no one changed it
  UPDATE inventory
  SET quantity = quantity - 1
  WHERE product_id = 42;

COMMIT;
-- Lock is released on commit

โœ… Good for:

  • High-conflict scenarios (many users competing for same resource)
  • Critical operations like inventory management, seat booking
  • When conflict resolution after the fact is expensive

โŒ Downside:

  • Reduces concurrency (other transactions wait)
  • Risk of deadlocks if multiple rows locked in different order

๐Ÿงช Optimistic Locking (Version Field)

-- Table has a 'version' column
-- Step 1: Read the row and note the version
SELECT id, balance, version FROM accounts WHERE id = 1;
-- Returns: { id: 1, balance: 500, version: 3 }

-- Step 2: Try to update โ€” include version in WHERE clause
UPDATE accounts
SET balance = 450, version = version + 1
WHERE id = 1 AND version = 3;  -- โ† check no one changed it

-- If 0 rows affected โ†’ someone else updated it first โ†’ retry!

โœ… Good for:

  • Low-conflict scenarios (most writes don't collide)
  • Better concurrency and throughput
  • Distributed systems and microservices (no shared lock state)

โŒ Downside:

  • Requires retry logic in application code
  • High conflict = many retries = poor performance

๐Ÿ“Š Comparison Table

FeaturePessimistic LockingOptimistic Locking
Locks data?โœ… Yes (immediate)โŒ No lock acquired
Conflict assumptionConflicts are frequentConflicts are rare
ConcurrencyLower (others wait)Higher (no blocking)
Deadlock riskโœ… YesโŒ No
Conflict handlingPrevented upfrontDetected at update time, retry
Best use caseBank transfers, inventoryUser profile updates, blog posts

๐Ÿ’ก Possible Follow-up Questions

  • How do you implement optimistic locking in Prisma or TypeORM?
  • What happens if two optimistic updates conflict โ€” how do you retry?
  • How does pessimistic locking lead to deadlocks?
  • What is SELECT FOR UPDATE SKIP LOCKED and when is it useful?
  • How does optimistic locking relate to MVCC?

โšก One-line Interview Answer

Pessimistic locking prevents conflicts by locking the row immediately on read (SELECT FOR UPDATE), suitable for high-conflict scenarios like banking. Optimistic locking avoids locks entirely by using a version field and checking at update time whether the data changed, suitable for low-conflict scenarios with better concurrency but requiring retry logic on conflict.

๐Ÿง  Simple Definition

A deadlock is a state where two or more database transactions are each waiting for a lock held by the other, forming a circular dependency that prevents any of them from proceeding. Databases automatically detect and resolve deadlocks by aborting one of the transactions and returning an error.


โšก Super Simple Line

Deadlock = Transaction A holds lock on Row X and wants Row Y. Transaction B holds lock on Row Y and wants Row X. Both wait forever โ†’ deadlock โ†’ database kills one.


๐Ÿ”ฅ Deadlock Visualization

Transaction A:
  1. LOCK Row X โœ… (acquired)
  2. Try to LOCK Row Y... โณ (waiting for B)

Transaction B:
  1. LOCK Row Y โœ… (acquired)
  2. Try to LOCK Row X... โณ (waiting for A)

โ†’ Circular wait โ†’ Deadlock detected โ†’ DB aborts one transaction

๐Ÿงช SQL Deadlock Example

-- Transaction A (session 1)
BEGIN;
UPDATE accounts SET balance = balance - 100 WHERE id = 1;  -- locks row 1
-- (pause here)
UPDATE accounts SET balance = balance + 100 WHERE id = 2;  -- tries to lock row 2 โ†’ BLOCKS

-- Transaction B (session 2)
BEGIN;
UPDATE accounts SET balance = balance - 50 WHERE id = 2;   -- locks row 2
-- (pause here)
UPDATE accounts SET balance = balance + 50 WHERE id = 1;   -- tries to lock row 1 โ†’ BLOCKS

-- Result: PostgreSQL detects circular dependency, aborts one transaction:
-- ERROR: deadlock detected
-- DETAIL: Process X waits for Lock on tuple (...)

๐Ÿ›ก๏ธ How to Prevent Deadlocks

1. Consistent Lock Ordering (Most Important)

// โŒ Inconsistent order โ€” deadlock risk
async function transfer(fromId: number, toId: number, amount: number) {
  await lockAccount(fromId);  // TX A locks id=1, TX B locks id=2
  await lockAccount(toId);    // both now blocked โ† deadlock
}

// โœ… Always lock in ascending ID order โ€” prevents circular dependency
async function transfer(fromId: number, toId: number, amount: number) {
  const [first, second] = fromId < toId ? [fromId, toId] : [toId, fromId];
  await lockAccount(first);   // Both TXs lock the smaller ID first
  await lockAccount(second);  // No circular dependency โœ…
}

2. Keep Transactions Short

// โŒ Bad โ€” long transaction holds locks
async function processPayment() {
  await db.beginTransaction();
  await lockAccount(userId);
  await callExternalPaymentAPI();  // Network call while holding lock! โŒ
  await db.commit();
}

// โœ… Good โ€” do external work OUTSIDE the transaction
async function processPayment() {
  const paymentResult = await callExternalPaymentAPI();  // before lock
  await db.beginTransaction();
  await recordPayment(paymentResult);  // quick, no external calls
  await db.commit();
}

3. Lock Timeouts + Retry Logic

-- Set a lock timeout (PostgreSQL)
SET lock_timeout = '5s';  -- fail if can't acquire lock in 5 seconds
// Application-level retry on deadlock
async function withRetry(fn: () => Promise, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (isDeadlockError(error) && attempt < maxRetries - 1) {
        await sleep(Math.random() * 100 * (attempt + 1));  // exponential backoff
        continue;
      }
      throw error;
    }
  }
}

4. Index Foreign Keys (PostgreSQL)

PostgreSQL locks parent rows when updating foreign keys on child rows. Adding indexes on foreign key columns reduces lock scope and deadlock probability.


๐Ÿ“Š Prevention Techniques

TechniqueHow It Helps
Consistent lock orderingEliminates circular dependency possibility
Short transactionsReduces time locks are held
Lock timeout + retryAutomatically recovers from deadlocks
Index foreign keysReduces lock scope on parent rows
Avoid user interaction in TXDon't wait for user input while holding locks

๐Ÿ’ก Possible Follow-up Questions

  • How does a database detect a deadlock cycle?
  • What is the difference between a deadlock and a livelock?
  • How does MongoDB handle deadlocks with document-level locking?
  • What is SELECT FOR UPDATE SKIP LOCKED and how does it help queues avoid deadlocks?
  • How do you identify deadlocks in PostgreSQL logs?

โšก One-line Interview Answer

A deadlock occurs when concurrent transactions form a circular lock dependency, each waiting for the other's lock. Databases auto-detect and abort one transaction to resolve it. Prevention involves always acquiring locks in a consistent order (e.g., sorted by ID), keeping transactions short to minimize lock hold time, setting lock timeouts with retry logic, and indexing foreign key columns to reduce lock scope.

๐Ÿง  Simple Definition

Redis (Remote Dictionary Server) is an open-source, in-memory data store that functions as a database, cache, and message broker. It stores data entirely in RAM for sub-millisecond latency and supports multiple native data structures, each optimized for specific use cases.


โšก Super Simple Line

Redis = blazing-fast in-memory key-value store with multiple data structures, used for caching, session storage, queues, leaderboards, pub/sub messaging, and rate limiting.


๐Ÿ“š Redis Data Structures

1. String โ€” Most basic type

SET user:1:name "Alice"
GET user:1:name         # โ†’ "Alice"
INCR page_views         # Atomic counter (no race condition)
EXPIRE session:abc 3600 # Auto-expire in 1 hour

๐Ÿ“Œ Use for: caching API responses, session tokens, counters, feature flags

2. Hash โ€” Object storage

HSET user:1 name "Alice" age 30 email "alice@example.com"
HGET user:1 name    # โ†’ "Alice"
HGETALL user:1      # โ†’ all fields

๐Ÿ“Œ Use for: storing structured objects (user profiles, product details)

3. List โ€” Ordered collection / Queue

LPUSH job_queue "job:1"   # Push to left
RPUSH job_queue "job:2"   # Push to right
RPOP job_queue             # Pop from right (FIFO queue)
LRANGE job_queue 0 -1     # Get all items

๐Ÿ“Œ Use for: task queues, activity feeds, recent items history

4. Set โ€” Unique unordered collection

SADD active_users "user:1" "user:2" "user:3"
SMEMBERS active_users        # All unique members
SISMEMBER active_users "user:1"  # Check membership O(1)
SINTER premium_users active_users  # Intersection

๐Ÿ“Œ Use for: unique visitor tracking, tags, mutual friends, permissions sets

5. Sorted Set โ€” Scored ranking

ZADD leaderboard 5000 "alice" 8000 "bob" 3000 "charlie"
ZRANGE leaderboard 0 -1 WITHSCORES  # Ascending order
ZREVRANGE leaderboard 0 2           # Top 3 (descending)
ZINCRBY leaderboard 500 "alice"     # Increment score

๐Ÿ“Œ Use for: leaderboards, rate limiting, priority queues, time-series events

6. Stream โ€” Event log

XADD events * action "purchase" userId "1" amount "100"
XREAD COUNT 10 STREAMS events 0   # Read 10 events

๐Ÿ“Œ Use for: event sourcing, audit logs, message streaming


๐Ÿ“Š Quick Reference Table

StructureCommandsBest Use Case
StringGET, SET, INCR, EXPIRECache, counters, sessions
HashHSET, HGET, HGETALLObject fields (user profile)
ListLPUSH, RPOP, LRANGEQueues, feeds
SetSADD, SMEMBERS, SINTERUnique collections, tags
Sorted SetZADD, ZRANGE, ZINCRBYLeaderboards, rate limits
StreamXADD, XREADEvent streaming, audit log

๐Ÿ’ก Possible Follow-up Questions

  • How does Redis handle persistence if it's in-memory? (RDB vs AOF)
  • What is Redis pub/sub and how does it work?
  • How do you implement rate limiting with Redis?
  • What is Redis Cluster and how does sharding work?
  • What is the difference between Redis EXPIRE and TTL?
  • How would you implement a distributed lock with Redis?

โšก One-line Interview Answer

Redis is an in-memory data store that provides sub-millisecond latency using multiple data structures: Strings for caching and counters, Hashes for object storage, Lists for queues, Sets for unique collections, Sorted Sets for leaderboards and rate limiting, and Streams for event logging โ€” making it suitable for caching, session management, real-time analytics, and message brokering.

๐Ÿง  Simple Definition

Redis stores data in RAM for speed, but provides two persistence mechanisms to prevent data loss on restart: RDB (Redis Database) creates periodic point-in-time binary snapshots of the entire dataset, while AOF (Append Only File) logs every write command sequentially to a file as it happens.


โšก Super Simple Line

RDB = take a photo of data every N minutes (fast restart, some data loss risk).
AOF = record every write in a log (slower restart, maximum durability).
Production = use both together.


๐Ÿ”‘ RDB โ€” Snapshots

# redis.conf configuration
save 900 1    # Save if at least 1 key changed in 900 seconds
save 300 10   # Save if at least 10 keys changed in 300 seconds
save 60 10000 # Save if at least 10,000 keys changed in 60 seconds

# Manual snapshot
redis-cli BGSAVE  # Fork background process to create snapshot

# Snapshot file location
dbfilename dump.rdb
dir /var/lib/redis

How RDB works:

  1. Redis forks a child process
  2. Child writes entire memory dataset to a temp .rdb file
  3. On success, temp file replaces the old dump.rdb
  4. Main process continues serving requests (no blocking)

๐Ÿ”‘ AOF โ€” Append Only File

# redis.conf configuration
appendonly yes
appendfilename "appendonly.aof"

# Sync frequency options:
appendfsync always    # Sync every write โ€” safest, slowest
appendfsync everysec  # Sync every 1 second โ€” good balance (default)
appendfsync no        # Let OS decide โ€” fastest, least durable

AOF Rewrite (Compaction):

# AOF file can grow large โ€” rewrite compacts it
# (e.g., 1000 INCR operations โ†’ single SET with final value)
redis-cli BGREWRITEAOF  # background rewrite/compact the AOF file

๐Ÿ“Š RDB vs AOF Comparison

FeatureRDB (Snapshots)AOF (Append Only)
DurabilityLower โ€” can lose changes since last snapshotโœ… Higher โ€” max 1 second loss with everysec
Recovery Speedโœ… Very fast (direct binary load)Slower (replay all commands)
File Sizeโœ… Compact binary fileLarger (all commands + needs rewrite)
Write Performanceโœ… No overhead during snapshotsSlight overhead (fsync per second)
Point-in-time Recoveryโœ… Can keep multiple snapshotsโŒ Only last state
Human-readableโŒ Binary formatโœ… Text log (easier to inspect)

๐Ÿญ Production Best Practice

# Enable BOTH for maximum durability + fast recovery
appendonly yes          # AOF for durability
appendfsync everysec    # 1 second max data loss
save 3600 1             # RDB backup every hour

# Recovery on startup:
# Redis uses AOF if both exist (more complete)
# Falls back to RDB if no AOF found

Why both?

  • AOF ensures maximum durability (1 second data loss max)
  • RDB provides fast restart (load binary snapshot, then replay only recent AOF)
  • RDB snapshots serve as backups and point-in-time recovery

๐Ÿ’ก Possible Follow-up Questions

  • What happens to AOF data if Redis crashes mid-write?
  • What is AOF rewrite and why is it needed?
  • What is the BGSAVE command and how does it work?
  • How does Redis persistence affect performance?
  • What is Redis persistence vs Redis Sentinel vs Redis Cluster?

โšก One-line Interview Answer

Redis persists data through two mechanisms: RDB creates periodic binary snapshots of the full dataset (fast restart, some data loss risk), and AOF logs every write command sequentially (maximum durability with up to 1 second loss using everysec sync). In production, both are enabled together โ€” AOF ensures minimal data loss while RDB enables faster restart and serves as a point-in-time backup.

๐Ÿง  Simple Definition

Cache invalidation is the process of removing or updating cached data when the underlying source data changes. It's one of the hardest problems in computer science because you must balance keeping cache fresh (accuracy) vs keeping it fast (performance).


โšก Super Simple Line

Cache invalidation = deciding WHEN and HOW to remove stale cached data and replace it with fresh data from the source.


๐Ÿ“š Cache Invalidation Strategies

1. Cache-Aside (Lazy Loading) โ€” Most Common

async function getUser(userId) {
  // 1. Check cache first
  const cached = await redis.get(`user:${userId}`);
  if (cached) return JSON.parse(cached);  // โœ… cache hit

  // 2. Cache miss โ€” fetch from DB
  const user = await db.users.findById(userId);

  // 3. Store in cache for next time
  await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));

  return user;
}
// On update: explicitly delete the cache key
async function updateUser(userId, data) {
  await db.users.update(userId, data);
  await redis.del(`user:${userId}`);  // โ† invalidate
}

โœ… Pros: Simple, only caches what's actually needed
โŒ Cons: First request after expiry is slow (cache miss)

2. Write-Through โ€” Always Current

async function updateUser(userId, data) {
  // Write to DB AND cache simultaneously
  await Promise.all([
    db.users.update(userId, data),
    redis.setex(`user:${userId}`, 3600, JSON.stringify(data))
  ]);
}
// Cache is always fresh โ€” no stale reads

โœ… Pros: Cache always fresh, no cold start
โŒ Cons: Slower writes (two writes per update), cache may hold unused data

3. Write-Behind (Write-Back) โ€” Async

// Write to cache immediately, sync DB asynchronously
async function updateUser(userId, data) {
  await redis.setex(`user:${userId}`, 3600, JSON.stringify(data));
  queue.add('sync-db', { userId, data });  // background job
}

โœ… Pros: Fastest write performance
โŒ Cons: Risk of data loss if cache fails before DB sync

4. TTL (Time-to-Live) โ€” Simplest

SET product:123 "{...}" EX 300  # Auto-expire in 5 minutes

โœ… Pros: Zero application logic needed
โŒ Cons: Data can be stale up to TTL duration; no instant invalidation

5. Event-Driven Invalidation โ€” Most Precise

// On any write event, publish invalidation message
eventBus.on('user.updated', async ({ userId }) => {
  await redis.del(`user:${userId}`);
  await redis.del(`user:${userId}:profile`);
  await redis.del(`user:${userId}:permissions`);
});

โœ… Pros: Precise, immediate invalidation
โŒ Cons: Complex event system, risk of missed events


๐Ÿ“Š Strategy Comparison

StrategyFreshnessWrite SpeedComplexity
Cache-AsideGood (on miss)โœ… FastLow
Write-Throughโœ… BestSlowerMedium
Write-BehindGoodโœ…โœ… FastestHigh + risk
TTLAcceptableโœ… FastLowest
Event-Drivenโœ… Bestโœ… FastHighest

๐Ÿ’ก Possible Follow-up Questions

  • What is a cache stampede and how do you prevent it?
  • How do you handle cache invalidation in a distributed system?
  • What is the difference between cache-aside and read-through caching?
  • How do you decide what TTL to use?
  • What happens if the cache goes down โ€” how do you protect the database?

โšก One-line Interview Answer

Cache invalidation determines when stale cached data should be removed or refreshed. The main strategies are cache-aside (fetch-on-miss, delete-on-write), write-through (update cache and DB together), write-behind (update cache first and sync DB asynchronously), TTL expiry (auto-expire after a time limit), and event-driven invalidation (explicitly delete specific cache keys when source data changes).

๐Ÿง  Simple Definition

A cache stampede (also called a dogpile effect or thundering herd) occurs when a popular cached value expires simultaneously for many concurrent requests. All of them detect the cache miss at the same moment and flood the database with the same query, potentially overwhelming it.


๐Ÿ”ฅ Why It Happens

Time: 12:00:00 โ€” Cache key "homepage_data" set with TTL=60s
Time: 12:01:00 โ€” TTL expires

At 12:01:00, 500 concurrent requests all:
  1. Check cache โ†’ MISS
  2. ALL query the database โ†’ 500 simultaneous DB queries โŒ
  3. ALL write the result to cache โ†’ 500 redundant writes

๐Ÿ›ก๏ธ Prevention Strategies

1. Mutex / Lock (First request fetches, others wait)

async function getDataWithLock(key) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  // Try to acquire a lock
  const lockAcquired = await redis.set(
    `lock:${key}`, "1",
    "NX", "EX", 5  // NX = only set if not exists, expire in 5s
  );

  if (lockAcquired) {
    // This request got the lock โ€” fetch and populate cache
    const data = await fetchFromDB(key);
    await redis.setex(key, 60, JSON.stringify(data));
    await redis.del(`lock:${key}`);
    return data;
  } else {
    // Another request has the lock โ€” wait and retry
    await sleep(100);
    return getDataWithLock(key);  // retry
  }
}

2. Probabilistic Early Expiration (XFetch Algorithm)

// Re-cache BEFORE expiry based on probability
// Probability increases as TTL gets closer to 0
function shouldEarlyRecompute(ttlRemaining, beta = 1.0) {
  const now = Date.now() / 1000;
  return now - beta * Math.log(Math.random()) > (now + ttlRemaining);
}

async function getData(key) {
  const [cached, ttl] = await Promise.all([redis.get(key), redis.ttl(key)]);

  if (cached && !shouldEarlyRecompute(ttl)) {
    return JSON.parse(cached);
  }

  // Either miss or probabilistic early refresh
  const data = await fetchFromDB(key);
  await redis.setex(key, 60, JSON.stringify(data));
  return data;
}

3. Background Refresh (Stale-While-Revalidate)

async function getData(key) {
  const cached = await redis.get(key);

  // Return stale data immediately
  if (cached) {
    const { data, expiresAt } = JSON.parse(cached);

    // If close to expiry, refresh in background
    if (Date.now() > expiresAt - 5000) {
      fetchFromDB(key).then(fresh => {
        redis.setex(key, 60, JSON.stringify({
          data: fresh,
          expiresAt: Date.now() + 60000
        }));
      });
    }

    return data;  // โ† user gets stale data instantly, no wait
  }

  // True cache miss โ€” fetch and block
  const data = await fetchFromDB(key);
  await redis.setex(key, 60, JSON.stringify({ data, expiresAt: Date.now() + 60000 }));
  return data;
}

4. Jitter (Randomized TTL)

// Instead of all keys expiring at exactly the same time
const TTL = 60 + Math.floor(Math.random() * 30);  // 60-90 seconds
await redis.setex(key, TTL, data);
// Different keys expire at different times โ†’ no simultaneous stampede

๐Ÿ“Š Strategy Comparison

StrategyComplexityUser ExperienceDB Protection
Mutex LockMediumWait (other users pause)โœ… Strong
Probabilistic Early ExpireMediumSmooth (no wait)โœ… Good
Background RefreshHighโœ… Best (instant stale)โœ… Strong
TTL JitterLowNormalModerate

๐Ÿ’ก Possible Follow-up Questions

  • What is the thundering herd problem?
  • How does a distributed lock work in Redis?
  • What is stale-while-revalidate and when is it safe to use?
  • How would you prevent stampede for a real-time leaderboard?
  • What is the XFetch algorithm?

โšก One-line Interview Answer

A cache stampede happens when many concurrent requests simultaneously miss the cache after a key expires and overwhelm the database with duplicate queries. Prevention strategies include mutex locking so only one request fetches and others wait, probabilistic early expiration to refresh before expiry, background refresh to serve stale data instantly while refreshing asynchronously, and randomized TTL jitter to prevent synchronized expiration.

๐Ÿง  Simple Definition

Queues build up when a component in a system receives work faster than it can process it. Identifying queue buildup points (bottlenecks) is essential for diagnosing performance problems and designing scalable systems.


โšก Super Simple Line

Queue builds up wherever incoming rate > processing rate. Find the bottleneck, fix the bottleneck, monitor continuously.


๐Ÿ“ Where Queues Can Build Up

1. Network Layer

  • High latency or limited bandwidth causes packets to queue up in buffers
  • TCP connection pool exhaustion causes connection queue to grow
  • Fix: CDN, connection pooling (PgBouncer), compression, TCP tuning

2. Database

  • Slow queries hold locks โ†’ other queries queue waiting for the lock
  • Too many concurrent connections exceed DB connection limit
  • Unindexed queries cause sequential scans that are slow
  • Fix: Add indexes, use connection pooler, optimize queries, read replicas
-- See active queries and wait times in PostgreSQL
SELECT pid, query, state, wait_event, now() - query_start AS duration
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY duration DESC;

3. Application Server

  • CPU-bound processing uses all available threads โ†’ new requests queue in web server
  • Memory pressure causes GC pauses โ†’ spikes in response time
  • Node.js: blocking the event loop with synchronous CPU work
  • Fix: Horizontal scaling, worker threads, load balancing, async processing

4. Message Queue (Kafka, Redis Queue, BullMQ)

  • Consumer is slower than producer โ†’ queue depth grows indefinitely
  • Fix: Add more consumers (horizontal scaling), optimize processing, prioritize critical jobs

5. External APIs / Third-Party Services

  • Slow upstream APIs cause application to wait โ†’ thread/connection pool exhausted
  • Rate limiting from third-party โ†’ requests queue internally
  • Fix: Timeouts, circuit breakers, async/background processing, caching API responses

6. Application Code

  • N+1 queries, missing indexes, synchronous file I/O
  • CPU-intensive computations blocking the main thread
  • Fix: Code profiling, algorithmic improvements, offload to worker threads

๐Ÿ” How to Find Queue Buildup

LayerMonitoring ToolSignal
Databasepg_stat_activity, slow query logsLong-running queries, many waiting
ApplicationAPM (Datadog, New Relic), profilerHigh CPU, memory, slow response p99
Message QueueBullMQ dashboard, Kafka Consumer LagGrowing queue depth, consumer lag
Networknetstat, load balancer metricsConnection queue length, timeouts

๐ŸŒ Real-World Example: Diagnosing a Slow API

Symptom: API response times spiking to 5+ seconds

Investigation:
1. Check application logs โ†’ requests completing, no errors
2. Check DB slow query logs โ†’ found: 200ms query running 50x/second (N+1!)
3. Add index + fix N+1 โ†’ response drops to 50ms
4. Load test โ†’ now DB connection pool exhausts at high traffic
5. Add PgBouncer connection pooler โ†’ stable at 5,000 req/min

๐Ÿ’ก Possible Follow-up Questions

  • What is a circuit breaker and how does it prevent queue buildup from cascading?
  • How do you monitor database connection pool usage?
  • What is consumer lag in Kafka and why does it matter?
  • How does the event loop queue in Node.js work?
  • What is backpressure and how do you handle it?

โšก One-line Interview Answer

Queues build up wherever processing rate falls below arrival rate โ€” common hotspots include the database (slow queries, lock contention, exhausted connection pool), the application server (CPU saturation, event loop blocking), message queues (slow consumers), external APIs (rate limiting or slow upstream), and the network (bandwidth or connection limits). I diagnose bottlenecks using observability tools (APM, slow query logs, queue depth metrics) and fix them with indexes, caching, horizontal scaling, connection pooling, or async background processing.

๐Ÿง  Simple Definition

Prisma Migrate is Prisma's schema-driven migration system. You define your database schema in schema.prisma, and Prisma generates the corresponding SQL migration files. In production, migrations are applied in strict order and tracked in the _prisma_migrations table.


๐Ÿ”„ Migration Workflow

Development Flow

# 1. Edit schema.prisma (add a new field)
# model User {
#   id    Int    @id
#   name  String
#   email String @unique  โ† NEW FIELD
# }

# 2. Create migration (generates SQL file + applies to dev DB)
npx prisma migrate dev --name add_email_to_user

# Result: creates migration file:
# prisma/migrations/20240615_add_email_to_user/migration.sql
# Content: ALTER TABLE "User" ADD COLUMN "email" TEXT NOT NULL;

# 3. Prisma Client regenerated with new types automatically
npx prisma generate

Production Flow

# Apply pending migrations without prompting (safe for CI/CD)
npx prisma migrate deploy
# Reads _prisma_migrations table โ†’ applies only unapplied migrations in order

๐Ÿšจ What Happens When a Migration Fails in Production?

_prisma_migrations table status:
| id  | migration_name              | finished_at | applied_steps | logs          |
|-----|-----------------------------|-------------|---------------|---------------|
| 1   | 20240614_initial_schema     | 2024-06-14  | 1             | null          |
| 2   | 20240615_add_email_to_user  | null        | 0             | ERROR: ...    |  โ† FAILED

Recovery Steps

  1. Identify the failure: prisma migrate status shows which migration failed
  2. Fix the root cause: Either fix the migration SQL or fix the data issue
  3. Option A โ€” Resolve manually: Apply the fix SQL manually, then mark as resolved:
    prisma migrate resolve --applied "20240615_add_email_to_user"
    
  4. Option B โ€” Create a new fix migration: If the original migration is too broken to salvage, create a new migration that corrects the state

๐Ÿ›ก๏ธ Production Migration Best Practices

  • โœ… Always backup before migrating production
  • โœ… Use backward-compatible changes: Add nullable columns, never drop/rename in one step
  • โœ… Run migrations BEFORE deploying new code (expand-contract pattern)
  • โœ… Test migrations on a staging environment first
  • โœ… Keep migrations small and focused โ€” one change per migration
  • โŒ Never edit migration files after they've been applied to production

๐Ÿงช Expand-Contract Pattern (Zero Downtime)

# Step 1: EXPAND โ€” add new column (nullable, backward compatible)
ALTER TABLE users ADD COLUMN username TEXT;  # nullable, old code works fine

# Step 2: Deploy new code that writes to BOTH old and new columns

# Step 3: Migrate existing data in batches
UPDATE users SET username = email WHERE username IS NULL;

# Step 4: CONTRACT โ€” make required, drop old if needed
ALTER TABLE users ALTER COLUMN username SET NOT NULL;

๐Ÿ’ก Possible Follow-up Questions

  • What is the _prisma_migrations table and how does it work?
  • How do you run zero-downtime migrations?
  • What is the difference between prisma migrate dev and prisma migrate deploy?
  • How do you handle a migration that locks a production table?
  • What is prisma db push and when would you use it instead of migrate?

โšก One-line Interview Answer

Prisma Migrate generates SQL migration files from schema.prisma changes using prisma migrate dev in development, and applies them in strict order in production via prisma migrate deploy, tracking each in the _prisma_migrations table. When a production migration fails, I identify the failed entry, fix the underlying issue, and use prisma migrate resolve or create a corrective migration โ€” always backing up first and using backward-compatible expand-contract patterns for zero downtime.

๐Ÿง  Simple Definition

Zero-downtime database migrations update the database schema without taking the application offline or causing errors in currently-running application instances. This requires all schema changes to be backward-compatible โ€” meaning both the old and new versions of your application code can query the same database simultaneously during a rolling deployment.


โšก Super Simple Line

Zero-downtime migration = change DB schema without breaking the old app version that's still running while the new version deploys.
Key rule: Never rename or drop columns in a single step.


๐Ÿ”„ The Expand-Contract Pattern (3 Phases)

Phase 1: EXPAND โ€” Add, Don't Remove

-- Add new column as nullable (old app doesn't know about it yet โ€” that's OK)
ALTER TABLE users ADD COLUMN username TEXT;
-- โœ… Old app still works: it ignores the new column
-- โœ… New app can start writing to username

Phase 2: MIGRATE โ€” Dual-Write + Backfill

// New app code writes to BOTH old and new columns
async function updateUser(id: number, data: UserData) {
  await prisma.user.update({
    where: { id },
    data: {
      email: data.email,      // old column โ€” still written for old app instances
      username: data.username  // new column โ€” written by new app instances
    }
  });
}

// Background job to backfill existing rows
async function backfillUsernames() {
  let cursor = 0;
  while (true) {
    const batch = await db.query(
      "SELECT id, email FROM users WHERE username IS NULL LIMIT 1000"
    );
    if (batch.length === 0) break;

    await db.query(
      "UPDATE users SET username = email WHERE id = ANY($1)",
      [batch.map(r => r.id)]
    );
    await sleep(100);  // throttle to avoid DB overload
  }
}

Phase 3: CONTRACT โ€” Make Required, Drop Old

-- Only after ALL app instances run new code + backfill is complete
ALTER TABLE users ALTER COLUMN username SET NOT NULL;

-- Later (separate deployment), drop the old column
ALTER TABLE users DROP COLUMN email;  -- only when no code references it anymore

๐Ÿšจ Dangerous Migrations (Avoid These Patterns)

OperationRiskSafe Alternative
Rename columnOld app can't find old column name โ†’ crashAdd new column + migrate + drop old
Drop columnOld app still reads it โ†’ errorRemove from code first, then drop column
Add NOT NULL without defaultOld app inserts without new column โ†’ constraint errorAdd nullable first, backfill, then add NOT NULL
Change column typeOld data may not convert โ†’ runtime errorAdd new column with new type, migrate, drop old
Add index without CONCURRENTLYTable locked during index build โ†’ downtimeCREATE INDEX CONCURRENTLY

๐Ÿ”ง PostgreSQL: Create Index Without Locking

-- โŒ Locks the table during build (downtime)
CREATE INDEX idx_users_email ON users(email);

-- โœ… No table lock โ€” safe in production
CREATE INDEX CONCURRENTLY idx_users_email ON users(email);
-- Slightly slower to build but doesn't block reads or writes

๐Ÿ› ๏ธ Deployment Pipeline

1. Backup production database
2. Run migrations (backward-compatible only โ€” expand phase)
3. Deploy new app version (rolling update โ€” old + new running together)
4. Verify application health
5. Run backfill job (background, throttled)
6. Run contract migration (remove old columns) in a LATER deployment

๐Ÿ’ก Possible Follow-up Questions

  • What is the expand-contract pattern in detail?
  • How do you create an index on a production table without downtime?
  • How do you handle a migration that must change a column type?
  • How does blue-green deployment interact with database migrations?
  • What is CREATE INDEX CONCURRENTLY and when must you use it?

โšก One-line Interview Answer

Zero-downtime migrations require backward-compatible schema changes using the expand-contract pattern: in the expand phase, add new columns as nullable while keeping old ones; deploy new code that writes to both; backfill existing rows; and only in a later contract phase make columns required and drop old ones. Critical techniques include CREATE INDEX CONCURRENTLY to avoid table locks, never renaming or dropping columns in a single deployment, and always running migrations before deploying new code in a CI/CD pipeline.

๐Ÿง  Simple Definition

Database scaling increases capacity and performance under growing load. Read replication copies data to multiple read-only replicas to distribute read queries. Sharding horizontally partitions data across multiple database servers (shards) so each shard holds only a subset of the data, scaling both reads and writes.


โšก Super Simple Line

Replication = copy the entire DB to scale reads.
Sharding = split the data across servers to scale reads + writes + storage.
Vertical scaling = bigger machine. Horizontal = more machines.


๐Ÿ“ˆ Vertical Scaling (Scale Up)

  • Give the existing server more CPU, RAM, faster SSD
  • โœ… Simple โ€” no code changes
  • โŒ Has a hard limit (you can't buy infinite RAM)
  • โŒ Single point of failure

๐Ÿ“ˆ Read Replication (Scale Reads)

            โ”Œโ”€โ”€โ”€ Primary (Read/Write) โ”€โ”€โ”€โ”
            โ”‚                             โ”‚
        Read Replica 1              Read Replica 2
        (Read Only)                 (Read Only)

Writes โ†’ Primary โ†’ async replication โ†’ Replicas
Reads โ†’ Any Replica (load balanced)
// Route reads to replica, writes to primary
const primaryDb = new PrismaClient({ datasourceUrl: PRIMARY_URL });
const replicaDb = new PrismaClient({ datasourceUrl: REPLICA_URL });

// Write โ†’ primary
await primaryDb.user.create({ data: { email } });

// Read โ†’ replica (can serve stale by replication lag)
const users = await replicaDb.user.findMany();

โœ… Pros: Low complexity, built into PostgreSQL/MySQL/MongoDB, good for read-heavy apps
โŒ Cons: All writes still go to primary (write bottleneck remains), replication lag (eventual consistency)


๐Ÿ“ˆ Sharding (Horizontal Partitioning)

Shard Key: user_id % 3

user_id 1,4,7,10  โ†’ Shard 0
user_id 2,5,8,11  โ†’ Shard 1
user_id 3,6,9,12  โ†’ Shard 2

Each shard is an independent DB with ~33% of the data
Both reads and writes distributed across shards

Common Sharding Strategies

StrategyHowProsCons
Range-baseduser_id 1-1000 โ†’ Shard 1, 1001-2000 โ†’ Shard 2Easy range queriesHot spots if not balanced
Hash-basedhash(user_id) % N โ†’ Shard NEven distributionNo range queries
Directory-basedLookup table maps key โ†’ shardFlexibleLookup overhead
GeographicEU users โ†’ EU shardCompliance (GDPR)Imbalanced if users concentrated

๐Ÿ“Š Replication vs Sharding

FeatureRead ReplicationSharding
Scales What?Read throughputReads + Writes + Storage
Data DistributionFull copy on every replicaSubset per shard
Write bottleneckโŒ Still single primaryโœ… Distributed across shards
ComplexityLow (native DB feature)High (custom routing, cross-shard queries)
Cross-shard JOINsN/AโŒ Very complex / expensive
ImplementationBuilt-in (pglogical, streaming)Application logic or Vitess/Citus

โš ๏ธ Sharding Pitfalls

  • Cross-shard JOINs are extremely expensive (often impossible)
  • Resharding when you outgrow your shard count is painful
  • Hot shards (popular shard key values) can become bottlenecks
  • Choose shard key very carefully โ€” it's very hard to change later

๐Ÿ’ก Possible Follow-up Questions

  • What is replication lag and how does it cause consistency issues?
  • How do you choose a sharding key?
  • What is Citus and how does it extend PostgreSQL with sharding?
  • What is MongoDB Atlas sharding and how does it work?
  • What is a hot shard and how do you avoid it?

โšก One-line Interview Answer

Read replication distributes read queries across identical copies of the full dataset (good for read-heavy workloads, low complexity), while sharding horizontally partitions data across multiple servers based on a shard key (scales both reads and writes and storage, but introduces cross-shard query complexity and requires careful shard key selection). In practice, I'd use read replicas first, only moving to sharding when write throughput or storage size becomes the actual bottleneck.