Skip to content

feat(orm): with_count() correlated relationship counts#171

Open
tmgbedu wants to merge 3 commits into
mainfrom
task/orm-with-count-886
Open

feat(orm): with_count() correlated relationship counts#171
tmgbedu wants to merge 3 commits into
mainfrom
task/orm-with-count-886

Conversation

@tmgbedu

@tmgbedu tmgbedu commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements end-to-end with_count() (Laravel withCount) on the async ORM — the feature was previously unwired (BaseRelationship.get_with_count_query was NotImplementedError and the *Through/BelongsToMany overrides chained on the async count() coroutine, so nothing worked).

users = await User.query().with_count("articles").get()
users[0].articles_count            # -> int
User.query().with_count("articles", lambda q: q.where("published", 1))   # constrained count

Changes

  • QueryBuilder.select_count(column="*", alias=None) — synchronous COUNT aggregate builder (the query-building counterpart to the async count()), used to compose correlated COUNT subqueries.
  • QueryBuilder.with_count(relation, callback=None) + Model.with_count(...) — adds a correlated (SELECT COUNT(*) ...) AS {relation}_count column; the optional callback constrains the count subquery.
  • BaseRelationship.get_with_count_query — real default for direct relations (HasMany/HasOne/BelongsTo).
  • BelongsToMany / HasOneThrough / HasManyThrough — with-count subqueries now build the COUNT synchronously via select_count() (fixes the async-count() blocker, no un-awaited coroutine).
  • BelongsToMany __init__ — fixed the resolver lambda (it took a spurious arg, breaking get_builder()), so its correlated queries resolve.
  • Relabelled three factory/collection tests whose names misleadingly contained with_count.

Tests

tests/masoniteorm/sqlite/builder/test_sqlite_with_count.py — real sqlite end-to-end asserting actual counts for all four relationship types:

  • HasMany (Joe=1, Jane=0), HasManyThrough (1/0), HasOneThrough (all=1), BelongsToMany (store1=2, store2=0).
  • 0-related → 0; callback-constrained count; base-column seeding; Model.with_count classmethod; correlated-subquery SQL shape.

Fixtures use the ORM (Schema builder + Model.insert), not raw SQL.

Gates

  • uv run pytest --ignore=tests/masoniteorm/postgres --cov → 1829 passed / 7 skipped, coverage 79.23% (≥ fail_under 68).
  • ruff check + ruff format --check clean.

⚠️ Stacked PR

This branch is stacked on #170 (select_sub/add_select, which provides select_sub + table() + the migrated with-count call shape). The diff against main therefore includes #170's commit until it merges. Review/merge order: #170 first, then this. The with_count-specific commit is the top one (feat(orm): with_count() correlated relationship counts).

tmgbedu added 3 commits July 12, 2026 10:07
… table()

- select_sub(subquery, alias): the subquery-as-column primitive. Renders
  (subquery) AS alias with bindings in SELECT position; accepts a builder
  or a callable that builds one. Matches Laravel selectSub(query, as).
- add_select(*columns): Laravel's variadic column-adder. Appends plain
  string columns with a dedup guard; an associative {alias: subquery} entry
  (string key + queryable value) delegates to select_sub, seeding
  select({table}.*) first when nothing is selected yet. A string value
  under a string key stays a plain column.
- table(name): sets the target table on a bare builder; backs the callable
  form of select_sub.
- Migrate the *Through/BelongsToMany with-count callers from the old
  add_select(alias, callable) shape to select_sub(callable, alias).
- process_columns: route the subquery alias through subquery_alias_string()
  instead of a hardcoded AS (single source of alias emission; SQL unchanged).

Existing string-based select() is unchanged. Covered by grammar-level
to_sql()/to_qmark() with binding-order assertions across all four dialects,
plus a real sqlite suite incl. the Laravel addSelect({alias: subquery})
acceptance case. End-to-end relationship with-count remains gated on the
async count() fix (#886).
Replace raw conn.execute CREATE TABLE / INSERT with the framework's Schema
builder and Model.insert() for the categories/posts and accounts/logins
fixtures. Same data and same assertions (incl. the Laravel addSelect
last_login_at acceptance case).
Wire up relationship count subqueries end-to-end:

- QueryBuilder.select_count(column='*', alias=None): synchronous COUNT
  aggregate builder (the query-building counterpart to the async count()),
  used to compose correlated COUNT subqueries.
- QueryBuilder.with_count(relation, callback=None) + Model.with_count():
  adds a correlated '(SELECT COUNT(*) ...) AS {relation}_count' column.
- BaseRelationship.get_with_count_query: real default for direct relations
  (HasMany/HasOne/BelongsTo) — was NotImplementedError.
- BelongsToMany/HasOneThrough/HasManyThrough with-count queries now build
  the COUNT subquery synchronously via select_count() instead of chaining
  on the async count() coroutine (the #886 blocker).
- Fix BelongsToMany.__init__ resolver lambda (took a spurious arg, breaking
  get_builder()) so its correlated queries resolve.

Real sqlite end-to-end tests assert actual counts (0-related returns 0,
callback-constrained counts) for HasMany, HasManyThrough, HasOneThrough and
BelongsToMany; fixtures use ORM insert APIs. Relabel three factory/collection
tests whose names misleadingly contained 'with_count'.
@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@tmgbedu

tmgbedu commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

✅ Code Review: APPROVE (do not merge — stacked on #170; merge order #168#169#170#171)

Posting as a comment — GitHub blocks a formal approve on one's own PR.

Reviewed the top with_count commit (1384ecb) in isolation from the #170 base. Correctness, patterns, and test quality all hold up.

Correctness — verified independently

  • Correlated COUNT SQL is right. get_with_count_query builds related.select_count("*").where_column(f"{related_table}.{foreign_key}", f"{owner_table}.{local_key}")(SELECT COUNT(*) FROM articles WHERE articles.user_id = users.id) AS articles_count. Confirmed by the SQL-shape test and real end-to-end counts (Joe=1 / Jane=0) — it executes correctly on sqlite, not just plausible-looking.
  • Base-column seeding. if not builder._columns: builder.select("{table}.*") before select_sub — verified by test_has_many_count_seeds_base_columns.
  • The 3 Through/pivot overrides now compose COUNT synchronously via select_count("*") instead of chaining on the async count() coroutine — the #886 blocker. No un-awaited coroutine remains; all three return real counts.
  • BelongsToMany __init__ resolver fix is legitimate. get_builder() is self.fn()().get_builder() — calls self.fn() with no args, so the old lambda x: raised TypeError. The fix lambda: registry.Registry.resolve(fn) matches the BaseRelationship convention (line 9) and drops the accidental double-assignment.

Pattern consistency

  • with_count mirrors where_has/or_where_has exactly; select_count mirrors the existing aggregate/count shape. Model.with_count follows the with_/where delegation style. New code belongs.

Tests

Real sqlite, ORM-first fixtures (Schema + Model.insert), no hollow mocks — meets the PR #159 bar. Asserts actual counts across all four relationship types, plus 0-count→0, callback-constrained (1/0) and callback-excluding-all (0/0), classmethod, and correlated-subquery SQL shape. The three misleadingly-named factory/collection tests correctly relabeled (behavior unchanged).

Gates reproduced locally

  • test_sqlite_with_count.py: 9 passed.
  • models + sqlite + query suites: 397 passed / 6 skipped, no regressions.
  • ruff check + ruff format --check: clean.

Non-blocking note

Relationships declared without explicit keys (relying on set_keys convention defaults) would produce NULL-keyed correlation in with_count, since set_keys isn't invoked on class-level attribute access. This is a pre-existing limitation shared verbatim by where_has/has (same getattr(...class..., relation) path) — not introduced here; all fixtures/real usage pass explicit keys. Follow-up candidate, not a blocker.

LGTM. Merge left to PM per the stack order.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant