Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,26 @@ v0.3.1 (Unreleased)
### Query translation
* **`toStartOf*` date-time functions** via `EF.Functions`: `ToStartOfYear`, `ToStartOfQuarter`, `ToStartOfMonth`, `ToStartOfWeek` (with optional week `mode`), `ToStartOfDay`, `ToStartOfHour`, `ToStartOfMinute`, `ToStartOfSecond`, the fixed buckets `ToStartOfFiveMinutes` / `ToStartOfTenMinutes` / `ToStartOfFifteenMinutes`, and the general `ToStartOfInterval(source, value, unit)`. Each maps to the matching ClickHouse function and works in `GROUP BY`. Return types follow ClickHouse: the calendar buckets (`Year`/`Quarter`/`Month`/`Week`) return `Date`, the day/hour/minute buckets return `DateTime`, and `ToStartOfSecond` returns `DateTime64`. All accept `DateTime`/`DateTime64` columns, and the plain truncation functions also accept `DateOnly`; `ToStartOfInterval` requires a `DateTime`/`DateTime64` column on older ClickHouse, which rejects a `DateOnly` (Date/Date32) source with `Illegal type Date32 of 1st argument` for every unit; recent versions accept it. `ToStartOfInterval` takes a `ClickHouseInterval` unit (`Second`…`Year`) and emits `toStartOfInterval(source, toInterval<unit>(value))`; the unit must be a constant so it can be translated. The default `Date`/`DateTime` result types only span 1970–2149/2106, so ClickHouse narrows out-of-range values — enable `enable_extended_results_for_datetime_functions` (e.g. `set_enable_extended_results_for_datetime_functions=1` in the connection string) for range-preserving `Date32`/`DateTime64` results.

* **Standard `DateTime` members and methods now translate to SQL.** Previously the provider registered no date/time member translator, so only a direct comparison worked and every member threw `The LINQ expression ... could not be translated`. One shared translator serves `DateTime`, `DateTimeOffset` and `DateOnly`, because the ClickHouse function is the same for each. ([#55](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/55))
* **Components** — `.Year` → `toYear`, `.Month` → `toMonth`, `.Day` → `toDayOfMonth`, `.Hour` → `toHour`, `.Minute` → `toMinute`, `.Second` → `toSecond`, `.Millisecond` → `toMillisecond`, `.DayOfYear` → `toDayOfYear`. These ClickHouse functions return `UInt8`/`UInt16`, which the provider's integer mappings widen to `int` on read. `DateOnly` gets the date components only, matching the members it declares.
* **`.DayOfWeek`** → `toDayOfWeek(x, 2)`. Week mode 2 agrees with `System.DayOfWeek` exactly (Sunday 0 … Saturday 6), so no arithmetic correction is applied — the default mode 0 starts the week on Monday, which is why the mode argument is always sent. The result carries a number-backed enum mapping, because this provider maps a C# `enum` to a ClickHouse string and that mapping would otherwise render `x.DayOfWeek == DayOfWeek.Sunday` as a comparison against `'Sunday'`.
* **`.Date`** → `toStartOfDay`, which keeps the timezone of the source. Note that `toStartOfDay` returns a `DateTime`, whose range is 1970–2106, and ClickHouse **wraps** a value outside that window rather than reporting it — so `.Date` on a `DateTime64` column holding a pre-1970 date reads back wrong. Enable `enable_extended_results_for_datetime_functions` (for example `set_enable_extended_results_for_datetime_functions=1` in the connection string) to get a range-preserving `DateTime64` result. This is the same caveat that already applies to `EF.Functions.ToStartOfDay`.
* **`.TimeOfDay`** → `toTime64(x, 7)`; precision 7 is one .NET tick, so no part of the value is lost (`toTime` would drop the fraction). This is the one member here that needs a recent server: `toTime64` arrived with the `Time64` type in **ClickHouse 25.6**, and an earlier server answers `Function with name 'toTime64' does not exist`. Every other function in this entry works on 24.8 LTS.
* **`DateTime.UtcNow`** → `now64(7, 'UTC')`, **`DateTime.Now`** → `now64(7)` and **`DateTime.Today`** → `toStartOfDay(now())`. `today()` is not used for `.Today` because it returns a `Date`, whereas the member's type is `DateTime`.
* **`.AddYears(n)`** → `addYears` and **`.AddMonths(n)`** → `addMonths`. Both take an `int` in .NET, and ClickHouse clamps the day of month the same way .NET does, so `2026-01-31` plus one month gives `2026-02-28` in both. A constant beyond the bound .NET applies to the argument itself — 10 000 years or 120 000 months — is not translated, because ClickHouse saturates at the year 9999 where .NET raises `ArgumentOutOfRangeException`. Only the argument can be checked while translating; whether the *result* fits depends on the column value, and a result past 9999 still saturates on the server.
* **`.AddDays`/`.AddHours`/`.AddMinutes`/`.AddSeconds`/`.AddMilliseconds`** take a `double` in .NET. .NET splits the integral and fractional parts, scales each to whole **ticks** (100 ns), and truncates any fractional tick toward zero — so `AddSeconds(0.1234567)` adds exactly 1 234 567 ticks, while `AddMilliseconds(0.99995)` adds 9 999 ticks rather than one millisecond. The matching ClickHouse function takes a whole number of its own unit and discards the rest, so `addDays(x, 1.5)` would add only one day. A constant argument is therefore folded with the .NET algorithm during translation and then expressed in the coarsest unit that holds it exactly: a whole number of the unit emits the natural function (`addDays(x, 1)`), and otherwise `addMilliseconds` carries the exact count (`AddDays(1.5)` → `addMilliseconds(x, 129600000)`). Preferring the natural function keeps the store type of the source and keeps `Date`/`Date32` columns working, since `addMilliseconds` rejects those outright.
* A **sub-millisecond** offset, a **non-constant** offset, and a value **outside the `DateTime` range** are deliberately left untranslated rather than rounded to fit. Milliseconds are as fine as the translation goes, because `addNanoseconds` would express a tick exactly but promotes the result to `DateTime64(9)`, whose Int64 nanosecond count cannot span the `DateTime64` range — that would trade a rounding error for a silently wrong date. An untranslated call still gives the correct .NET value through client evaluation in a projection, and reports a clear reason in a predicate. `DateOnly.AddDays` takes an `int`, so it always emits `addDays`.
* A previously-unsupported Northwind query, `GroupJoin_aggregate_anonymous_key_selectors2`, now passes as a result of these translations; its provider-specific "not translatable" override is removed.
* For a `DateTimeOffset` property, component results use the timezone the column declares. The default store type pins that timezone to UTC; explicitly configured named and fixed-offset zones are preserved.
* **`Add*` translates only for a column whose timezone has one offset for every instant.** ClickHouse arithmetic follows the declared timezone, and a named zone with daylight saving disagrees with .NET in two ways. The calendar functions (`addDays`, `addMonths`, `addYears`) keep the wall clock but cannot produce the hour the clocks skip — on a `DateTime64(7, 'Europe/London')` column holding `2026-03-28 01:30`, `addDays(x, 1)` answers `00:30` where .NET answers `01:30`. The absolute functions (`addHours` … `addMilliseconds`, including the `addMilliseconds` fallback that a fractional `AddDays` uses) move the instant, so an interval crossing a transition shifts the wall clock by an hour. For `DateTimeOffset` there is a third difference: .NET preserves the instance offset, which the column's zone can change. So `Add*` is translated for `'UTC'` and `Fixed/UTC±HH:MM:SS` store types, and — for `DateTime`/`DateOnly` only — for a store type that declares no timezone, which the driver reads as a UTC wall clock. Everything else is client-evaluated in a projection and reports the limitation in a predicate. **Known limit:** a timezone-less `DateTime` column still has its calendar arithmetic evaluated in the server's `session_timezone`, which is invisible during translation; declare the timezone in the store type when that setting observes daylight saving.
* Not yet translated: `.Ticks`, `.AddTicks`, the `.Microsecond`/`.Nanosecond` members, and `DateTimeOffset.Now`. The ClickHouse-specific functions such as `dateDiff` and `dateTrunc` are tracked in [#58](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/58).
* **Behaviour change:** `DateTime.Now` and `DateTime.Today` in a *projection* used to be evaluated on the client; they now read the **server** clock. The value therefore follows the server's timezone rather than the client's, and comes back with `DateTimeKind.Unspecified` instead of `Local`. Use `DateTime.UtcNow` for an instant that does not depend on server configuration. `DateTimeOffset.UtcNow` also reads a UTC-pinned server clock; `DateTimeOffset.Now` stays on the client so its local offset is preserved. In a predicate the `DateTime` clock members were untranslatable before, so nothing changes there.

### Types
* **`DateTimeOffset` is now a supported property type**, and maps to `DateTime64(7, 'UTC')`. The provider previously had no mapping for it, so EF Core fell back to `DateTimeOffsetToStringConverter` and made a `String` column with no warning — which broke queries against a real `DateTime64` column with `TYPE_MISMATCH`, and stopped `SaveChanges` from writing the value at all. The store type pins the timezone to `'UTC'` so the instant does not depend on the server's `session_timezone`, and precision 7 is one .NET tick (100 ns), so the round trip is exact. `HasPrecision(n)` is honoured for a property with no `HasColumnType`, and keeps the UTC pin. Note that ClickHouse has no type that stores a UTC offset: `DateTime64` holds an instant, and a declared timezone only decides how that instant is rendered. The instant is kept and the offset is not, so a value read back carries the offset of the column's declared timezone. A non-UTC column such as `DateTime64(6, 'Asia/Tokyo')` reads correctly, as does a fixed-offset column such as `DateTime64(7, 'Fixed/UTC+05:30:00')`, and `DateTimeOffset` composes into `Array`, `Map` and `Tuple` columns. Where an instant cannot be reproduced exactly the read throws rather than returning a value that is quietly wrong: the repeated hour when clocks go back in a zone whose candidate offsets are both non-zero (`Europe/Paris`), and a value before 1900 in a named zone, where the offset is Local Mean Time to the second. Writing a value the store type cannot hold throws for the same reason — ClickHouse wraps it rather than reporting it, which matters for precision 8 and 9, whose ranges are narrower than `DateTimeOffset`. **Behaviour change:** a `DateTimeOffset` property that relied on the old `String` column now resolves to `DateTime64(7, 'UTC')` — add `HasConversion<string>()` to keep the previous shape. See the [DateTimeOffset section of the README](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore#datetimeoffset) for the daylight-saving and `MinValue`/`MaxValue` limits. ([#53](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/53))

### Bug fixes
* **Subtracting one date/time value from another no longer fails with an internal error.** `dt1 - dt2` gives a `TimeSpan`, which ClickHouse has no operator for — `dateDiff` returns a count of whole units instead. The expression used to reach type-mapping inference and fail with an `InvalidCastException` or a bare `No coercion operator is defined between types ...`, both of which name CLR types the user never wrote. The subtraction is now reported as not translatable, with the reason attached. In a projection EF Core can therefore fall back to the client and return the correct `TimeSpan`; in a predicate, where no fallback exists, the message explains why and what to do instead. Shifting a date by a `TimeSpan` gets its own message, because it does have a translatable equivalent: `x.AddDays(-7)` works where `x - TimeSpan.FromDays(7)` does not. ([#55](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/55))
* `ToStartOfWeek` now rejects row-dependent week modes during query translation, and `ToStartOfInterval` likewise rejects row-dependent interval sizes. ClickHouse requires these operands to be constant for the query; literals and captured query parameters remain supported.
* **Composite columns now convert their components on read.** `Array(T)`, `Map(K, V)` and `Tuple(...)` read the whole column through `GetValue`, so a component mapping's own read pipeline never ran, and any component whose CLR type differs from the driver's type threw `InvalidCastException`. This is not new with `DateTimeOffset` — `DateOnly[]`, `Dictionary<string, DateOnly>` and `Tuple<DateOnly, …>` were already affected, because `DateOnly` also arrives from the driver as a `DateTime`. The composite is now rebuilt component by component, with the same two steps EF Core applies to a scalar column: the mapping's data-reader conversion, then its `ValueConverter`. An `enum` component, a `List<T>` component and a nested composite therefore all read correctly, and a component that needs no conversion keeps the direct cast. **Known limit:** *writing* a component that needs a `ValueConverter` still does not work, because the bulk insert path passes model values to the driver without applying converters ([#54](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/54)) — an `enum` inside a composite is written as its raw ordinal.
* **`Array(Nullable(T))` DDL is no longer double-wrapped.** For a value-type element the store type came out as `Array(Nullable(Nullable(T)))`, which ClickHouse rejects with `Nested type Nullable(T) cannot be inside Nullable type`, so `EnsureCreated` and migrations both failed. The component mapping is resolved from a store type that already carries the wrapper, and `HasColumnType(...)` text is kept verbatim, so the nullable-element wrapper added a second one. It now adds the wrapper only when the inner store type does not already have one, including through `LowCardinality(Nullable(T))`. Reference-type elements were never affected.
Expand Down
Loading