PostgreSQL vs SQL Server
What changes for a developer who wrote T-SQL for years and has moved to PostgreSQL: dialect and types, row versions and VACUUM, indexes, JSONB and query diagnostics. Versions as of October 2026: PostgreSQL 18 and SQL Server 2025; PostgreSQL 19 is about to ship.
Before this topic: Indexes and query plans, Transactions and locking.
How does PostgreSQL differ from SQL Server in practice?
Section titled “How does PostgreSQL differ from SQL Server in practice?”What it is
PostgreSQL and SQL Server both speak standard SQL, but each has its own dialect. The core — SELECT, JOIN, GROUP BY — is the same; names, types and utility constructs differ.
How to use
| T-SQL | PostgreSQL |
|---|---|
SELECT TOP 10 ... |
... LIMIT 10 |
int IDENTITY(1,1) |
int GENERATED ALWAYS AS IDENTITY |
nvarchar(100), bit |
text or varchar(100), boolean |
GETDATE(), ISNULL(a, b) |
now(), coalesce(a, b) |
OUTPUT inserted.Id |
RETURNING id |
[Order] |
"order" |
Common mistake
Creating a table "Users" in quotes and then writing SELECT * FROM Users. Without quotes the name becomes users, and PostgreSQL reports that the table does not exist. This happens with EF Core and Prisma tables: by default they create capitalized names in quotes.
Takeaway: the SQL core is shared, but names, types and utility constructs have to be relearned.
How it works
- Names. An unquoted identifier folds to lower case:
Usersandusersare the same name,"Users"is a different one. That is why PostgreSQL code usessnake_case. - Schemas. The hierarchy is database → schema → table, the default schema is
public, and lookup order is set bysearch_path. Querying another database on the same server is only possible through thepostgres_fdwordblinkextensions. - Auto-increment.
GENERATED ... AS IDENTITY(since PostgreSQL 10) is the standard replacement for the olderserial. Both use a sequence under the hood.
When to choose what
| SQL Server | PostgreSQL | Difference |
|---|---|---|
nvarchar(n) |
text, varchar(n) |
the database is usually UTF-8, no N prefix needed |
bit |
boolean |
WHERE is_active without = 1 |
datetimeoffset |
timestamptz |
stores the instant in UTC, drops the offset |
uniqueidentifier |
uuid |
gen_random_uuid(), since PostgreSQL 18 also uuidv7() |
| none | int[], text[] |
arrays instead of TVPs and STRING_SPLIT |
Pitfalls
- String case.
'Abc' = 'abc'isfalse. Case-insensitive search usesILIKE,lower()with an expression index, or a nondeterministic collation. - Identity after explicit ids. Inserting explicit ids during a data migration does not advance the sequence, and the next normal insert fails with a duplicate key. Fix it with
setval. SQL Server in the same situation moves the current identity value itself. - Upsert.
INSERT ... ON CONFLICTis atomic under concurrent inserts.MERGEexists since PostgreSQL 15, but it does not close the race between two inserts.
Rule of thumb
Names in snake_case without quotes, strings as text, time as timestamptz, keys as identity, upserts with ON CONFLICT.
Takeaway: what breaks in a move is not the syntax but the case of names and strings, time zones and sequences.
Under the hood
- Upsert.
ON CONFLICTrelies on a unique index and guarantees one of two outcomes — insert or update — even under high concurrency. In SQL Server,MERGEwithoutHOLDLOCKhits unique key violations under concurrent inserts, and withHOLDLOCKit holds range locks. - Procedures and functions. A result set is returned by a
RETURNS TABLEfunction called inFROM. A procedure (CALL, since PostgreSQL 11) canCOMMITinside, but it does not return aSELECTresult like a SQL Server procedure. - Time.
now()is the start time of the transaction, not of the statement;clock_timestamp()gives the current time. NULLin a unique index. PostgreSQL allows multipleNULLs by default (forbid withNULLS NOT DISTINCT, since PostgreSQL 15); SQL Server allows only one.
Cost
Procedures with several result sets get rewritten as functions or moved into the application. Code that relies on SQL Server’s case-insensitive collation moves to lower(), citext or an ICU collation.
Trade-offs
textvsvarchar(n): stored and processed the same way; a length limit is a business rule, not an optimization.- A random
gen_random_uuid()scatters inserts across the B-tree, whileuuidv7()is time-ordered and inserts almost like a sequence. - Npgsql EF Core and Prisma create quoted names by default (
"Orders"). A consistentsnake_case(EFCore.NamingConventions,@@mapin Prisma) frees hand-written SQL from quotes.
How to check in production
After a data migration, compare last_value from pg_sequences with max(id) of each table. Actual column names and types are shown by \d table in psql and by information_schema.columns.
Takeaway: the syntax translates mechanically, but the semantics — case, time, NULL, sequences and procedures — must be checked deliberately.
CREATE TABLE app.users ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, email text NOT NULL UNIQUE, -- case-sensitive: Ann@ and ann@ are different is_active boolean NOT NULL DEFAULT true, created_at timestamptz NOT NULL DEFAULT now(), tags text[] NOT NULL DEFAULT '{}');
-- Upsert: atomically insert or update and return the idINSERT INTO app.users (email) VALUES ('ann@example.com')ON CONFLICT (email) DO UPDATE SET is_active = trueRETURNING id;
-- LIMIT/OFFSET instead of TOP and OFFSET ... FETCHSELECT id, email FROM app.usersWHERE is_active AND 'admin' = ANY (tags)ORDER BY idLIMIT 20 OFFSET 40;
SELECT timestamptz '2026-10-07 12:00+05' AT TIME ZONE 'UTC'; -- 2026-10-07 07:00:00CREATE TABLE app.Users ( Id bigint IDENTITY(1,1) PRIMARY KEY, Email nvarchar(320) NOT NULL UNIQUE, -- with a CI collation Ann@ and ann@ are duplicates IsActive bit NOT NULL DEFAULT 1, CreatedAt datetimeoffset NOT NULL DEFAULT SYSDATETIMEOFFSET());
-- Upsert: without HOLDLOCK concurrent inserts hit unique key violationsMERGE app.Users WITH (HOLDLOCK) AS tUSING (VALUES (N'ann@example.com')) AS s (Email) ON t.Email = s.EmailWHEN MATCHED THEN UPDATE SET IsActive = 1WHEN NOT MATCHED THEN INSERT (Email) VALUES (s.Email)OUTPUT inserted.Id;
SELECT Id, Email FROM app.UsersWHERE IsActive = 1ORDER BY IdOFFSET 40 ROWS FETCH NEXT 20 ROWS ONLY;- Why doesn’t
SELECT * FROM Usersfind a table created by EF Core? — It was created as"Users"in quotes, and an unquoted name folds tousers. - How does
GENERATED ALWAYSdiffer fromBY DEFAULT? —ALWAYSrejects an explicit value withoutOVERRIDING SYSTEM VALUE;BY DEFAULTaccepts it. - How do you make email unique case-insensitively? — A unique index on
lower(email), thecitexttype, or a nondeterministic ICU collation. - What does
now()return twice in one transaction? — The same transaction start time; the current time isclock_timestamp(). - Can you query a table in a neighboring database? — Only through
postgres_fdwordblink; within one database, data is split by schemas.
How does MVCC work in PostgreSQL and why do you need VACUUM?
Section titled “How does MVCC work in PostgreSQL and why do you need VACUUM?”What it is
MVCC (multi-version concurrency control) is a concurrency approach in which the database keeps several versions of a row. A reader sees the version that was current as of its snapshot and does not wait for a writing transaction.
How it works
UPDATEdoes not change the row in place: it creates a new version and marks the old one as deleted.DELETEonly marks the version as deleted.- Once no one can see an old version anymore, it becomes dead.
- VACUUM makes the space of dead versions available for new rows. Usually the background autovacuum runs it.
Common mistake
Turning autovacuum off “so it doesn’t get in the way”. Tables start to grow, queries slow down, and eventually the database stops accepting writes to protect itself from transaction counter overflow.
Takeaway: in PostgreSQL every change leaves garbage behind, and VACUUM is routine cleanup, not optional maintenance.
How it works
- Every row version has hidden columns
xmin(who created it) andxmax(who deleted or replaced it). - A snapshot records which transactions had finished when it was taken. A version is visible if its
xmintransaction committed before the snapshot, andxmaxis empty or its transaction had not committed by then. - In Read Committed a snapshot is taken per statement, so PostgreSQL behaves like SQL Server with RCSI, not like lock-based Read Committed.
Comparison with SQL Server
| SQL Server (RCSI, SNAPSHOT) | PostgreSQL | |
|---|---|---|
| Where old versions live | version store in tempdb, with ADR in the database itself |
in the same table |
| Who cleans up | background cleanup | VACUUM and autovacuum |
| When it’s on | per database setting | always |
Pitfalls
- Bloat. If autovacuum can’t keep up, dead versions pile up and queries read extra pages. Plain VACUUM does not return space to the OS;
VACUUM FULLdoes, but it locks the whole table (ACCESS EXCLUSIVE). - Thresholds. Autovacuum kicks in when dead rows exceed 50 plus 20% of the table; since PostgreSQL 18 the threshold is capped at 100 million rows. For large hot tables, lower
autovacuum_vacuum_scale_factor. - Long transactions. While an old transaction is open, VACUUM won’t remove versions it might still see.
Rule of thumb
Don’t turn autovacuum off, tune it; keep transactions short.
Takeaway: MVCC in PostgreSQL pays for non-blocking reads with garbage in the table, and VACUUM is part of that price.
Under the hood
- Rollback. There is no undo log: an aborted transaction is marked in
pg_xact, and its versions simply become invisible. In SQL Server without ADR, rollback replays the log backwards and takes time proportional to the work done. - HOT (heap-only tuple). If no indexed column changed and the page has room, the new version goes on the same page with no new index entries. The chain of old versions is pruned by ordinary page access, without VACUUM. A
fillfactorbelow 100 leaves room for HOT. - Freezing. A transaction ID is 32 bits and compared modulo 2³², so every ~2 billion transactions old rows must be frozen. Past
autovacuum_freeze_max_age(200 million) a forced VACUUM starts. Warnings begin 40 million transactions before the limit, and at 3 million the database stops assigning new transaction IDs.
Cost
Every non-HOT update means new entries in all indexes and in WAL. Frequent UPDATEs of one column in a wide table cost more than SQL Server’s in-place update.
Trade-offs
- Aggressive autovacuum (lower
scale_factor, highervacuum_cost_limit) spends I/O but keeps bloat in check. - An index on a frequently updated column disables HOT for its updates — the index has to be worth it.
How to check in production
pg_stat_user_tables:n_dead_tup,last_autovacuum, the share ofn_tup_hot_updinn_tup_upd.age(datfrozenxid)frompg_database— the headroom before wraparound.backend_xmininpg_stat_activityandpg_replication_slots— who is holding the horizon.
Takeaway: versions in the table itself make rollback cheap and updates expensive; VACUUM, HOT and short transactions keep that cost under control.
-- Hidden version columnsSELECT xmin, xmax, ctid, id, status FROM orders WHERE id = 1;-- 750 | 0 | (0,1) | 1 | new
UPDATE orders SET status = 'paid' WHERE id = 1;
SELECT xmin, xmax, ctid, id, status FROM orders WHERE id = 1;-- 751 | 0 | (0,2) | 1 | paid -- new version with a new ctid, (0,1) is now dead
-- How much garbage and when autovacuum last ranSELECT relname, n_live_tup, n_dead_tup, n_tup_hot_upd, last_autovacuumFROM pg_stat_user_tablesORDER BY n_dead_tup DESCLIMIT 10;
-- Large hot table: vacuum at 2% dead rows instead of 20%ALTER TABLE orders SET (autovacuum_vacuum_scale_factor = 0.02);- Why does a table grow when the row count doesn’t? — Dead versions from
UPDATEandDELETE: autovacuum can’t keep up or a long transaction is holding it back. - How does
VACUUMdiffer fromVACUUM FULL? — The first frees space for reuse without blocking writes; the second rewrites the table underACCESS EXCLUSIVE. The online alternative is thepg_repackextension. - What is transaction ID wraparound? — Overflow of the 32-bit transaction counter; VACUUM freezes old rows, otherwise the database stops accepting writes.
- Why is rollback instant in PostgreSQL? — There is no undo: the transaction is marked aborted, and its versions become invisible.
- How does Npgsql EF Core implement optimistic concurrency? — With the
xminsystem column as a concurrency token:[Timestamp] public uint Version.
What index types does PostgreSQL have and when do you use each?
Section titled “What index types does PostgreSQL have and when do you use each?”What it is
An index is a separate structure the database uses to find rows without reading the whole table. PostgreSQL has several index types, and each speeds up its own operations.
How to use
| Type | For |
|---|---|
| B-tree (default) | =, <, >, BETWEEN, ORDER BY |
| GIN | arrays, JSONB, full-text search |
| GiST | geodata, ranges, nearest-neighbor search |
| BRIN | huge tables where the value grows with insert order |
| Hash | = only; a B-tree is almost always enough |
The type is set with USING: CREATE INDEX ON orders USING gin (tags);.
Common mistake
Expecting a B-tree to speed up LIKE '%abc%' or a search inside JSONB. Substrings need a GIN index from the pg_trgm extension, JSONB needs a GIN index on the column.
Takeaway: B-tree is the default; the other types serve special operators: “contains”, “overlaps”, “nearest”.
How it works
- A table is an unordered heap, and the primary key is a regular unique B-tree. There is no equivalent of a clustered index.
- A partial index with
WHERE status = 'new'is the equivalent of a SQL Server filtered index: small, and used by queries with the same condition. - An expression index
ON users (lower(email))is built without a computed column, but the query must contain the same expression. INCLUDE(since PostgreSQL 11) adds non-key columns for index-only scans, as in SQL Server.
When to choose what
| Condition in the query | Index |
|---|---|
id = 42, ORDER BY created_at |
B-tree |
tags @> ARRAY['vip'], attrs @> '...' |
GIN |
name ILIKE '%ann%' |
GIN with gin_trgm_ops (pg_trgm) |
| overlapping periods, nearest points | GiST |
| recent events in a log of hundreds of GB | BRIN on the timestamp |
Pitfalls
CREATE INDEXblocksINSERT,UPDATEandDELETEfor the whole build. In production writeCREATE INDEX CONCURRENTLY.LIKE 'abc%'uses a B-tree only with theCcollation or thetext_pattern_opsoperator class.- A unique index allows multiple
NULLs; forbid them withNULLS NOT DISTINCT(PostgreSQL 15+).
Rule of thumb
Start with a B-tree; reach for another type when the query has an operator a B-tree can’t handle.
Takeaway: you choose an index by its operator, and partial and expression indexes replace SQL Server’s filtered indexes and computed columns.
Under the hood
- An index stores the
ctid, the address of a row version in the heap. A non-HOTUPDATEgives the row a newctidand new entries in every index. In SQL Server a nonclustered index stores the clustered key, and updating a non-indexed column doesn’t touch it. - An index-only scan skips the heap only for pages the visibility map marks as “all visible”. Without a recent VACUUM it degrades into a regular index scan.
CLUSTERreorders the table by an index once, underACCESS EXCLUSIVE; new rows don’t maintain that order.- Since PostgreSQL 18, B-tree supports skip scan and can use a composite index without a condition on its first column.
Cost
- GIN is expensive to write: every array element or JSONB key is a separate index entry.
fastupdatespeeds up inserts with a pending list, but searching that list is slower. CONCURRENTLYscans the table twice, waits for running transactions and can’t run inside a transaction. Migration tools that wrap everything in a transaction run it separately — in Alembic viaautocommit_block().- A failed
CONCURRENTLYbuild leaves anINVALIDindex: queries don’t use it, but writes still maintain it.
Trade-offs
- BRIN is orders of magnitude smaller than a B-tree, but only while values correlate with the physical order of rows.
- Every extra index on a frequently updated column disables HOT for its updates.
How to check in production
idx_scan = 0inpg_stat_user_indexes— candidates for removal.pg_index.indisvalid = false— leftovers of failedCONCURRENTLYbuilds.
Takeaway: a PostgreSQL index points to a row version, so indexes cost more on writes than in SQL Server; pick the type by operator and create it CONCURRENTLY.
-- Partial index: the equivalent of a SQL Server filtered indexCREATE INDEX CONCURRENTLY ix_orders_new ON orders (created_at) WHERE status = 'new';
-- Expression index instead of a computed columnCREATE UNIQUE INDEX CONCURRENTLY ux_users_email_lower ON users (lower(email));SELECT id FROM users WHERE lower(email) = lower('Ann@Example.com'); -- uses the index
-- Covering index for an index-only scanCREATE INDEX CONCURRENTLY ix_orders_customer ON orders (customer_id) INCLUDE (total);
-- GIN for arrays, BRIN for a log that grows over timeCREATE INDEX CONCURRENTLY ix_orders_tags ON orders USING gin (tags);CREATE INDEX CONCURRENTLY ix_events_at ON events USING brin (occurred_at);
-- A CONCURRENTLY build failed: the index was left invalidSELECT indexrelid::regclass FROM pg_index WHERE NOT indisvalid;- Why doesn’t PostgreSQL have a clustered index? — A table is a heap, and under MVCC row versions live at different addresses;
CLUSTERorders it only once. - Why doesn’t a query with
lower(email)use an index onemail? — An expression index only matches a query with the same expression, and vice versa. - What do you do with an index after a failed
CREATE INDEX CONCURRENTLY? —DROP INDEX CONCURRENTLYand create it again, orREINDEX INDEX CONCURRENTLY. - When is BRIN useless? — When values don’t correlate with the physical row order: every block range covers almost all values.
- Why Hash if there is B-tree? — Hardly any reason: equality only; it is sometimes more compact for long keys.
JSONB: when should you store JSON in a relational database and how do you index it?
Section titled “JSONB: when should you store JSON in a relational database and how do you index it?”What it is
jsonb is a column type that holds a JSON document. PostgreSQL stores it parsed and can search inside it.
Why
For data whose shape varies from row to row: product characteristics, user settings, a raw response from an external API.
How to use
attrs -> 'size'— a nested value asjsonb.attrs ->> 'brand'— a value astext; get a number by casting:(attrs ->> 'weight')::numeric.attrs @> '{"color": "red"}'— the document contains the given pair. A GIN index speeds up this operator.
Common mistake
Putting everything into JSONB, foreign keys and filter fields included. The database stops checking types and relationships, and queries lose indexes and statistics.
Takeaway: JSONB is for the flexible part of the data; whatever you search or join on goes into regular columns.
How it works
json |
jsonb |
|
|---|---|---|
| Storage | original text | parsed binary tree |
| Whitespace, key order, duplicate keys | preserved | lost, the last key wins |
@>, ? operators and GIN |
no | yes |
When to choose what
- “Document contains” search on any keys — GIN on the column:
USING gin (attrs jsonb_path_ops). It is smaller than the defaultjsonb_opsbut doesn’t support key-existence checks (?,?|,?&). - Filtering and sorting on one known field — a B-tree on the expression
((attrs ->> 'brand')). - A field used in relationships and constraints — a separate column.
Pitfalls
->returnsjsonb,->>returnstext.attrs -> 'n' = 5is a type error; write(attrs ->> 'n')::int = 5orattrs @> '{"n": 5}'.- GIN only understands its own operators:
attrs ->> 'color' = 'red'doesn’t use it, whileattrs @> '{"color": "red"}'does. - Keys are case-sensitive:
Colorandcolorare different keys.
Rule of thumb
Columns first, JSONB for the truly variable part; match the index to the operator in WHERE.
Takeaway: jsonb by default, GIN for content search, an expression B-tree for one hot field.
Under the hood
- If a row is wider than ~2 KB, large values are compressed and moved out to TOAST. Reading one key means decompressing the whole document.
- Changing one key (
jsonb_set,attrs['color'] = ...) rewrites the whole value: a new row version and WAL proportional to the document size. The documentation recommends keeping documents small: any update locks the whole row. - There are no statistics on keys inside a document, so the planner uses default selectivity for JSONB conditions. An expression index or
CREATE STATISTICSon an expression (PostgreSQL 14+) gives it real data. - The SQL/JSON standard: constructors and
IS JSONsince PostgreSQL 16;JSON_VALUE,JSON_QUERY,JSON_EXISTSandJSON_TABLEsince PostgreSQL 17. SQL Server also hasJSON_VALUEandJSON_QUERY, andOPENJSONplays the role ofJSON_TABLEthere.
SQL Server for comparison
- Before 2025: JSON in
nvarchar(max), theJSON_VALUEandOPENJSONfunctions, and indexing via a computed column overJSON_VALUE. - SQL Server 2025: the binary
jsontype, themodifymethod for in-place changes,JSON_CONTAINSandCREATE JSON INDEX, which requires a clustered primary key.
Trade-offs
JSONB vs EAV: simpler queries and fewer joins, but no foreign keys, types or NOT NULL on keys. They are replaced by a CHECK with jsonb_path_exists or by validation in the application.
How to check in production
pg_column_size(attrs) shows the actual document size; EXPLAIN shows whether GIN was chosen (Bitmap Index Scan) and how far off the row estimate was.
Takeaway: JSONB buys flexibility at the price of statistics, expensive partial updates and schema control; keep documents small and index for specific operators.
CREATE TABLE products ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, name text NOT NULL, attrs jsonb NOT NULL DEFAULT '{}');
-- GIN for content search: @>, @?, @@CREATE INDEX ix_products_attrs ON products USING gin (attrs jsonb_path_ops);-- B-tree on one hot fieldCREATE INDEX ix_products_brand ON products ((attrs ->> 'brand'));
SELECT id, name, attrs -> 'size' AS size -- jsonbFROM productsWHERE attrs @> '{"color": "red"}' -- uses GIN AND (attrs ->> 'weight')::numeric < 2.5; -- ->> returns text
SELECT id FROM products WHERE attrs ->> 'brand' = 'acme'; -- expression B-tree
-- SQL/JSON (PostgreSQL 17+): standard syntax, as in SQL ServerSELECT JSON_VALUE(attrs, '$.weight' RETURNING numeric) FROM products;-- Before SQL Server 2025: JSON in nvarchar, index on a computed columnALTER TABLE dbo.Products ADD Brand AS CAST(JSON_VALUE(Attrs, '$.brand') AS nvarchar(100));CREATE INDEX IX_Products_Brand ON dbo.Products (Brand);
-- SQL Server 2025: binary json type and a JSON indexCREATE TABLE dbo.Items ( Id bigint IDENTITY PRIMARY KEY, -- a JSON index requires a clustered PK Attrs json NOT NULL);CREATE JSON INDEX IX_Items_Attrs ON dbo.Items (Attrs);
SELECT Id FROM dbo.Items WHERE JSON_VALUE(Attrs, '$.color') = 'red';- How does
jsondiffer fromjsonb? —jsonstores the text as is;jsonbstores a parsed tree without duplicate keys and with index support. - Why isn’t GIN used for
attrs ->> 'brand' = 'acme'? — GIN supports@>,?,@?,@@;->>needs an expression B-tree. - When is JSONB worse than regular columns? — When you filter and join on the fields, need constraints and accurate statistics.
- How do you update a single key? —
jsonb_setor subscriptingattrs['color'] = '"blue"'; on disk the whole value is rewritten anyway.
How do you diagnose PostgreSQL performance?
Section titled “How do you diagnose PostgreSQL performance?”What it is
EXPLAIN shows the plan the planner chose, and EXPLAIN ANALYZE also runs the query and adds actual numbers. In SQL Server these are the estimated and actual execution plans.
How to use
- Find the slow query: in the log or in
pg_stat_statements. - Run
EXPLAIN (ANALYZE, BUFFERS)on it. Since PostgreSQL 18,BUFFERSis on automatically withANALYZE. - Look for the node with the highest
actual time, aSeq Scanon a big table, and a large gap between estimated and actualrows. - After the fix, compare the plan and timing again.
Common mistake
Running EXPLAIN ANALYZE on an UPDATE or DELETE in production. The query really executes, so wrap it in BEGIN and ROLLBACK.
Takeaway: EXPLAIN ANALYZE is the actual plan with real timings and row counts, the main tool for analyzing a single query.
How it works
pg_stat_statementsaccumulates statistics per normalized query: calls, total and mean time, rows, block reads. It is enabled throughshared_preload_librariesandCREATE EXTENSION. Unlike Query Store, it stores no plans.EXPLAIN (ANALYZE, BUFFERS)shows the estimate and the actuals at every node:rows,loops, time,shared hit(from cache) andread(from disk). Time and rows are per loop — multiply byloops.- Statistics are most common values and histograms per column in
pg_stats. They are collected byANALYZE, usually run by autovacuum.
When to choose what
| Symptom in the plan | What to do |
|---|---|
the rows estimate is off by 10x or more |
ANALYZE, CREATE STATISTICS for correlated columns |
Sort Method: external merge Disk |
raise work_mem for the query or role |
Seq Scan with a large Rows Removed by Filter |
an index for the condition |
many reads for few rows |
bloat or a poor index |
Pitfalls
work_mem(4 MB by default) is a limit per sort or hash, not per query: a complex query in 100 connections can use it hundreds of times over.EXPLAIN ANALYZEtimes every node, which noticeably inflates the total on fast queries.
Rule of thumb
Start with pg_stat_statements sorted by total_exec_time, then EXPLAIN (ANALYZE, BUFFERS) for the top of the list.
Takeaway: pg_stat_statements tells you what to fix, EXPLAIN ANALYZE tells you why it’s slow.
Under the hood
- A process per connection. Each connection is a separate OS process with its own memory, and
max_connectionsdefaults to 100. Hundreds of connections from pods eat memory before any query runs. - Pools. Npgsql keeps a client-side pool (up to 100 connections per connection string by default). PgBouncer in
transactionmode hands out a server connection for the duration of a transaction, but breaksSET,LISTENand session-level advisory locks. It supports protocol-level prepared statements since version 1.21 (max_prepared_statements), and Npgsql behind it usesNo Reset On Close=true. - The xmin horizon. The oldest transaction, a replication slot or a replica with
hot_standby_feedbackkeeps VACUUM from removing newer versions. An hour inidle in transactionis an hour of bloat across the whole database. - Plans. There is no shared plan cache as in SQL Server: a plan lives in the session with a prepared statement. After five executions PostgreSQL may switch to a generic plan that ignores parameter values; the fix is
plan_cache_mode = force_custom_plan.
Trade-offs
auto_explainwithlog_analyzelogs actual plans of slow queries but instruments every query;log_timing = offlowers the cost.statement_timeout,idle_in_transaction_session_timeoutandtransaction_timeout(PostgreSQL 17+) protect the database; set them on the application role, not server-wide.
How to check in production
pg_stat_activity:state,xact_start,backend_xmin,wait_event;pg_blocking_pids(pid)shows who blocks whom.temp_blks_writteninpg_stat_statements— queries short onwork_mem.
Takeaway: in PostgreSQL performance is plans plus connections and the VACUUM horizon: a pool, short transactions and pg_stat_statements matter more than tuning individual queries.
-- The most expensive queries by total timeSELECT calls, round(total_exec_time) AS total_ms, round(mean_exec_time::numeric, 1) AS mean_ms, rows, queryFROM pg_stat_statementsORDER BY total_exec_time DESCLIMIT 10;
-- Actual plan; DML goes in a transaction that is rolled backBEGIN;EXPLAIN (ANALYZE, BUFFERS)UPDATE orders SET status = 'paid' WHERE customer_id = 42;ROLLBACK;
-- Who holds the VACUUM horizon and whom they blockSELECT pid, state, now() - xact_start AS xact_age, backend_xmin, pg_blocking_pids(pid) AS blocked_by, left(query, 60) AS queryFROM pg_stat_activityWHERE xact_start IS NOT NULLORDER BY xact_start;EXPLAIN (ANALYZE, COSTS OFF)SELECT g FROM generate_series(1, 300000) AS g ORDER BY g DESC;
-- Sort (actual time=72.339..85.677 rows=300000.00 loops=1)-- Sort Key: g DESC-- Sort Method: external merge Disk: 3528kB -- work_mem was not enough-- Buffers: shared hit=3, temp read=954 written=955 -- BUFFERS without asking-- -> Function Scan on generate_series g (actual time=18.384..34.475 rows=300000.00 loops=1)-- Execution Time: 104.038 msAfter SET work_mem = '64MB' the sort runs in memory: Sort Method: quicksort.
// NuGet: Npgsqlusing Npgsql;
// Direct connection: Npgsql pool and auto-preparation of frequent queriesawait using var dataSource = NpgsqlDataSource.Create( "Host=db;Database=shop;Username=app;Password=secret;" + "Maximum Pool Size=50;Max Auto Prepare=20;Command Timeout=30");// Behind PgBouncer in transaction mode add: No Reset On Close=true
await using var cmd = dataSource.CreateCommand( "SELECT count(*) FROM orders WHERE status = $1");cmd.Parameters.Add(new NpgsqlParameter { Value = "paid" }); // positional $1Console.WriteLine(await cmd.ExecuteScalarAsync());- How does
EXPLAINdiffer fromEXPLAIN ANALYZE? — The first shows the estimate without running the query; the second runs it and adds actuals. - Why do you multiply
actual rowson the inner side of a Nested Loop? — Values are per loop, and the number of loops isloops. - How does
pg_stat_statementsdiffer from Query Store? — It aggregates per normalized query but stores no plans and can’t force them. - Why PgBouncer if Npgsql already has a pool? — The Npgsql pool lives in one application process, while PgBouncer funnels connections from all pods into a small number of server processes.
- What do you do with an
idle in transactionsession? — Find it inpg_stat_activity, end it withpg_terminate_backend, setidle_in_transaction_session_timeout, and fix the code that doesn’t close the transaction.