Skip to content

v6: OffsetDateTime is serialised with toString(), which drops the seconds and is not RFC 3339 #605

Description

@dudanogueira

Summary

The client serialises OffsetDateTime with toString(), which omits the seconds when they are zero. 2024-03-01T00:00:00Z therefore goes on the wire as 2024-03-01T00:00Z, which is not RFC 3339 — partial-time requires hour ":" minute ":" second — and Weaviate rejects it.

Every timestamp on an exact minute boundary is affected, which is most timestamps written by hand. It breaks writes and filters alike.

Reproduction

OffsetDateTime round       = OffsetDateTime.parse("2024-03-01T00:00:00Z");
OffsetDateTime withSeconds = OffsetDateTime.parse("2024-03-01T00:00:01Z");

System.out.println(round);        // 2024-03-01T00:00Z    <- the seconds are gone
System.out.println(withSeconds);  // 2024-03-01T00:00:01Z

Against a collection with a single date property when:

FAIL  REST insert  round        -> invalid date property 'when' on class 'DBeaverDateProbe':
                                   requires a string with a RFC3339 formatted date,
                                   but the given value is '2024-03-01T00:00Z'
OK    REST insert  withSeconds
FAIL  gRPC batch   round        -> invalid date property 'when' on class 'DBeaverDateProbe':
                                   requires a string with a RFC3339 formatted date,
                                   but the given value is '2024-03-01T00:00Z'
OK    gRPC batch   withSeconds

and on the query side:

FAIL  Filter.property("when").gt(round)        -> trying parse time as RFC3339 string:
                                                  parsing time "2024-03-01T00:00Z" as
                                                  "2006-01-02T15:04:05Z07:00": cannot parse "Z" as ":"
OK    Filter.property("when").gt(withSeconds)  -> 2 rows
OK    Filter.property("when").gt("2024-03-01T00:00:00Z")   // String overload -> 2 rows
FAIL  Filter.createdAt().gt(round)             -> same parse error
OK    Filter.createdAt().gt(withSeconds)

The only difference between each passing and failing pair is whether the second happens to be zero, which isolates the cause.

Where it comes from

Disassembling every non-protocol class in client6-6.3.1-all.jar and grepping for java/time/OffsetDateTime.toString gives three call sites, one per path:

class path reached by
Filter$DateOperand gRPC search every date filter, plus Filter.createdAt() / Filter.lastUpdatedAt()
InsertManyRequest.marshalValue gRPC batch data.insertMany(...)Value.newBuilder().setStringValue(dt.toString())
DateUtil$CustomTypeAdapterFactory$1 REST / Gson data.insert(...) and object reads — JsonWriter.value(dt.toString())

OffsetDateTime.toString() delegates down to LocalTime.toString(), which emits HH:mm when both second and nano are zero. It is a display format, not a wire format.

Suggested fix

Format explicitly at all three sites instead of calling toString() — for example DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ssXXX"), or ISO_OFFSET_DATE_TIME applied to a value normalised to at least second precision, so the seconds are always present.

Workaround, and where there isn't one

For filters on a property, pass the RFC3339 String rather than an OffsetDateTime: Filter.property(p).gt("2024-03-01T00:00:00Z") takes the String overload and works (shown above).

There is no workaround on the metadata path — Filter.createdAt() and Filter.lastUpdatedAt() return DateProperty, whose comparison methods accept OffsetDateTime only. Same for writes, where the property map value has to be an OffsetDateTime to be recognised as a date at all.

Version

  • java-client 6.3.1
  • Weaviate 1.39.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions