Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test-pets-rollback.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:

services:
pg_db:
image: ghcr.io/constructive-io/docker/postgres-plus:17
image: ghcr.io/constructive-io/docker/postgres-plus:18
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-pgpm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ concurrency:
cancel-in-progress: true

env:
PGPM_VERSION: '2.7.9'
PGPM_VERSION: '5.30.6'

jobs:
test-pgpm:
Expand All @@ -30,7 +30,7 @@ jobs:

services:
pg_db:
image: ghcr.io/constructive-io/docker/postgres-plus:17
image: ghcr.io/constructive-io/docker/postgres-plus:18
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
Expand All @@ -49,7 +49,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: '22'

- name: Cache pgpm CLI
uses: actions/cache@v4
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:

services:
pg_db:
image: ghcr.io/constructive-io/docker/postgres-plus:17
image: ghcr.io/constructive-io/docker/postgres-plus:18
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
Expand Down
80 changes: 66 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ The Python counterpart to [`pgsql-test`](https://www.npmjs.com/package/pgsql-tes

* **Instant test DBs** — each one seeded, isolated, and UUID-named
* **Per-test rollback** — every test runs in its own transaction with savepoint-based rollback via `before_each()`/`after_each()`
* **RLS-friendly** — test with role-based auth via `set_context()`
* **RLS-friendly** — `db` is a real non-superuser connection (`app_user`); switch roles and JWT claims via `set_context()`
* **pgpm integration** — run database migrations using [pgpm](https://pgpm.io) (PostgreSQL Package Manager)
* **Flexible seeding** — run `.sql` files, programmatic seeds, pgpm modules, or combine multiple strategies
* **Auto teardown** — no residue, no reboots, just clean exits
Expand Down Expand Up @@ -104,6 +104,10 @@ def test_my_function(db):
assert result['result'] == expected_value
```

> `db` connects as the non-superuser `app_user` (see [`pg` vs `db`](#pg-vs-db)). Your migrations
> must `GRANT USAGE`/`EXECUTE` to `anonymous`/`authenticated`/`administrator` for `db` to reach
> them — or use `conn.pg` when you only want to assert that the deploy happened.

### pgpm with Dependencies

If your module depends on other pgpm packages (like `@pgpm/faker`), install them first:
Expand Down Expand Up @@ -220,23 +224,69 @@ Without per-test rollback, tests can interfere with each other:

With `before_each()`/`after_each()`, each test is completely isolated, making your test suite reliable and deterministic.

## `pg` vs `db`

`get_connections()` returns two **different** clients, mirroring the TypeScript `pgsql-test`:

| Client | Connects as | Use it for |
|--------|-------------|------------|
| `pg` | the superuser from `PGUSER` | DDL, seeding, grants, asserting ground truth (bypasses RLS) |
| `db` | `app_user` (non-superuser), default role `anonymous` | the code under test — grants and RLS policies are enforced |

On every `get_connections()` call, pgsql-test creates the `app_user` LOGIN role (if missing),
grants it membership in the `anonymous`, `authenticated` and `administrator` roles, grants it
`CONNECT` on the test database, and opens `db` with those credentials. Every query on `db` runs
after `SET LOCAL ROLE <role>` (default `anonymous`), so tables created by `pg` are invisible to
`db` until you `GRANT` access — exactly as they would be in production.

Override the credentials/roles via `connection_options`:

```python
conn = get_connections(connection_options={
'connection': {'user': 'app_user', 'password': 'app_password', 'role': 'anonymous'},
'roles': {'anonymous': 'anonymous', 'authenticated': 'authenticated', 'administrator': 'administrator'},
})
```

## RLS Testing

Test Row Level Security policies by switching contexts:
Seed with `pg`, then exercise policies with `db`:

```python
def test_rls_policy(db):
@pytest.fixture(scope='module')
def conn():
c = get_connections()
c.pg.query("""
CREATE TABLE documents (id SERIAL PRIMARY KEY, owner_id TEXT, title TEXT);
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
GRANT SELECT ON documents TO authenticated;
CREATE POLICY owner_only ON documents FOR SELECT TO authenticated
USING (owner_id = current_setting('jwt.claims.user_id', true));
""")
c.pg.commit()
yield c
c.teardown()

def test_rls_policy(conn):
db = conn.db
db.before_each()

# Set the user context
db.set_context({'app.user_id': '123'})

# Now queries will be filtered by RLS policies
result = db.many('SELECT * FROM user_data')


# anonymous (the default role) has no grant at all
with pytest.raises(psycopg2.errors.InsufficientPrivilege):
db.query('SELECT * FROM documents')

# authenticated sees only its own rows
db.set_context({'role': 'authenticated', 'jwt.claims.user_id': '123'})
rows = db.many_or_none('SELECT * FROM documents')

db.clear_context() # back to anonymous, claims nulled
db.after_each()
```

`set_context()` turns `role` into `SET LOCAL ROLE` and every other key into
`set_config(key, value, true)`. Both are transaction-local and are re-applied before every
query, so they persist for the whole `before_each()`/`after_each()` window.

## Seeding Strategies

### pgpm Modules
Expand Down Expand Up @@ -300,8 +350,8 @@ conn = get_connections(
Creates a new isolated test database and returns connection objects.

Returns a `ConnectionResult` with:
- `pg`: PgTestClient connected as superuser
- `db`: PgTestClient for testing (same as pg for now)
- `pg`: PgTestClient connected as the superuser (bypasses RLS; use for setup/assertions)
- `db`: PgTestClient connected as `app_user` with default role `anonymous` (RLS enforced)
- `admin`: DbAdmin for database management
- `manager`: PgTestConnector managing connections
- `teardown()`: Function to clean up
Expand All @@ -316,7 +366,9 @@ Returns a `ConnectionResult` with:
- `execute(sql, params?)`: Execute and return affected row count
- `before_each()`: Start test isolation (transaction + savepoint)
- `after_each()`: End test isolation (rollback)
- `set_context(dict)`: Set session variables for RLS testing
- `set_context(dict)`: Set `role` (`SET LOCAL ROLE`) and GUCs (`set_config(..., true)`) for RLS testing
- `get_context()`: Return the current context dict
- `clear_context()`: Null every GUC and restore the default role

## GitHub Actions Example

Expand Down Expand Up @@ -356,7 +408,7 @@ jobs:

- uses: actions/setup-node@v4
with:
node-version: '20'
node-version: '22'

- name: Install pgpm
run: npm install -g pgpm
Expand Down
2 changes: 1 addition & 1 deletion WORKSPACE_SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -547,7 +547,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
node-version: "22"

- name: Install pgpm
run: npm install -g pgpm
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "pgsql-test"
version = "0.2.0"
version = "0.3.0"
description = "PostgreSQL testing framework for Python - instant, isolated databases with automatic transaction rollback"
authors = ["Constructive <developers@constructive.io>"]
license = "MIT"
Expand Down
2 changes: 1 addition & 1 deletion src/pgsql_test/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@
"seed",
]

__version__ = "0.2.0"
__version__ = "0.3.0"
60 changes: 60 additions & 0 deletions src/pgsql_test/admin.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,66 @@ def create_role(
)
logger.info(f"Created role: {role_name}")

def role_exists(self, role_name: str) -> bool:
"""Check whether a role exists."""
conn = self._get_admin_connection()
with conn.cursor() as cur:
cur.execute("SELECT 1 FROM pg_roles WHERE rolname = %s", (role_name,))
return cur.fetchone() is not None

def grant_role(self, role_name: str, member: str) -> None:
"""Grant membership in `role_name` to `member` (idempotent)."""
conn = self._get_admin_connection()
with conn.cursor() as cur:
cur.execute(
"""
SELECT 1
FROM pg_auth_members am
JOIN pg_roles r ON am.roleid = r.oid
JOIN pg_roles m ON am.member = m.oid
WHERE r.rolname = %s AND m.rolname = %s
""",
(role_name, member),
)
if cur.fetchone() is not None:
return
cur.execute(
sql.SQL("GRANT {} TO {}").format(
sql.Identifier(role_name),
sql.Identifier(member),
)
)
logger.debug(f"Granted {role_name} to {member}")

def create_user_role(self, user: str, password: str, roles: list[str]) -> None:
"""
Create the app-level LOGIN user used by the `db` client and grant it
membership in the given NOLOGIN roles (created if missing).

Mirrors pgsql-test (TS) `DbAdmin.createUserRole`. Granting `administrator`
is for the test harness only - never do this for a production app user.
"""
conn = self._get_admin_connection()
with conn.cursor() as cur:
for role in roles:
if self.role_exists(role):
continue
try:
cur.execute(sql.SQL("CREATE ROLE {} NOLOGIN").format(sql.Identifier(role)))
except psycopg2.errors.DuplicateObject:
pass
if not self.role_exists(user):
try:
cur.execute(
sql.SQL("CREATE ROLE {} LOGIN PASSWORD %s").format(sql.Identifier(user)),
(password,),
)
except psycopg2.errors.DuplicateObject:
pass
for role in roles:
self.grant_role(role, user)
logger.info(f"Created user role {user} with grants: {roles}")

def grant_connect(self, role_name: str, database: str) -> None:
"""
Grant CONNECT privilege on a database to a role.
Expand Down
57 changes: 37 additions & 20 deletions src/pgsql_test/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from typing import Any

import psycopg2
from psycopg2 import sql as pgsql
from psycopg2.extras import RealDictCursor

from pgsql_test.types import PgConfig, QueryResult
Expand Down Expand Up @@ -39,18 +40,25 @@ class PgTestClient:
client.close()
"""

def __init__(self, config: PgConfig, enhanced_errors: bool = True) -> None:
def __init__(
self,
config: PgConfig,
enhanced_errors: bool = True,
default_role: str | None = None,
) -> None:
"""
Initialize the test client.

Args:
config: PostgreSQL connection configuration
enhanced_errors: Whether to enhance error messages with PG details
default_role: Role restored by clear_context() (None = SET ROLE NONE)
"""
self._config = config
self._enhanced_errors = enhanced_errors
self._default_role = default_role
self._conn: psycopg2.extensions.connection | None = None
self._context: dict[str, str] = {}
self._context: dict[str, str | None] = {}
self._in_transaction = False
self._savepoint_name = "pgsql_test_savepoint"

Expand Down Expand Up @@ -111,6 +119,7 @@ def query(self, sql: str, params: tuple[Any, ...] | None = None) -> QueryResult:
conn = self.connection
try:
with conn.cursor() as cur:
self._apply_context(cur)
cur.execute(sql, params)
# Check if this is a SELECT-like query that returns rows
if cur.description is not None:
Expand Down Expand Up @@ -185,30 +194,40 @@ def execute(self, sql: str, params: tuple[Any, ...] | None = None) -> int:
result = self.query(sql, params)
return result.row_count

def set_context(self, context: dict[str, str]) -> None:
def set_context(self, context: dict[str, str | None]) -> None:
"""
Set PostgreSQL session context variables.
Set PostgreSQL context variables for RLS testing.

Useful for simulating RLS contexts in tests.
The `role` key becomes `SET LOCAL ROLE`; every other key becomes
`set_config(key, value, true)`. Both are transaction-local, so the
context is re-applied before every query (like the TS client) and
therefore persists across the transaction opened by before_each().
A value of None resets the setting (`SET LOCAL ROLE NONE` / NULL).

Args:
context: Dictionary of context variables to set
e.g., {"role": "authenticated", "jwt.claims.user_id": "123"}
context: e.g. {"role": "authenticated", "jwt.claims.user_id": "123"}
"""
self._context.update(context)
self._apply_context()

def clear_context(self) -> None:
"""Clear all context variables."""
self._context = {}
def get_context(self) -> dict[str, str | None]:
"""Return a copy of the current context settings."""
return dict(self._context)

def _apply_context(self) -> None:
"""Apply context variables to the current session."""
conn = self.connection
with conn.cursor() as cur:
for key, value in self._context.items():
# Use SET LOCAL so it only applies within the current transaction
cur.execute(f"SET LOCAL {key} = %s", (value,))
def clear_context(self) -> None:
"""Null out every context variable and restore the default role."""
self._context = {key: None for key in self._context}
self._context["role"] = self._default_role

def _apply_context(self, cur: Any) -> None:
"""Apply context variables on the given cursor (transaction-local)."""
for key, value in self._context.items():
if key == "role":
if value is None:
cur.execute("SET LOCAL ROLE NONE")
else:
cur.execute(pgsql.SQL("SET LOCAL ROLE {}").format(pgsql.Identifier(value)))
else:
cur.execute("SELECT set_config(%s, %s, true)", (key, value))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 bug · high

clear_context fails to reset RLS GUCs

In _apply_context (src/pgsql_test/client.py:230), a cleared context key with value None runs SELECT set_config(%s,%s,true) passing SQL NULL as the new value; PostgreSQL treats a NULL new_value as no-op (it never assigns NULL to a GUC), so the prior transaction-local setting from an earlier query in the same transaction stays active. As a result clear_context() (client.py:216) nulls the in-memory dict but does not actually clear the live RLS variables, so policies keyed on e.g. jwt.claims.user_id keep seeing the stale value across a context reset.

The same stale-state risk applies to any caller passing None in set_context() to mean reset (per its docstring).

📋 Prompt for AI Agents

In src/pgsql_test/client.py at line 230 in _apply_context(), handle a None value for non-role keys explicitly instead of passing it to set_config, since PostgreSQL set_config() with a NULL new_value is a no-op. Change the else branch so that when value is None you run cur.execute('SELECT set_config(%s, %s, true)', (key, '')) (or a RESET-style clear) to actually remove the transaction-local variable, and keep the non-None path passing the string normally. This ensures clear_context() genuinely clears RLS settings as its docstring promises.


def begin(self) -> None:
"""Begin a new transaction."""
Expand Down Expand Up @@ -256,8 +275,6 @@ def before_each(self) -> None:
"""
self.begin()
self.savepoint()
if self._context:
self._apply_context()

def after_each(self) -> None:
"""
Expand Down
Loading
Loading