Steps to reproduce
- Run a
dstack server 0.21.2 (or any version < 0.21.3).
- Use a 0.21.4 CLI against it.
- Apply any service configuration, e.g.:
type: service
name: httpbin
image: ghcr.io/psf/httpbin:0.10.2
replicas: 1
port: 8080
https: auto
auth: false
gateway: true
dstack apply -f .dstack/httpbin.yaml
Compatibility matrix observed:
| client |
server |
result |
| 0.21.3 |
0.21.2 |
ok |
| 0.21.4 |
0.21.3 |
ok |
| 0.21.4 |
0.21.2 |
fails |
Actual behaviour
Server validation error:
{'detail': [{'input': None,
'loc': ['body', 'run_spec', 'configuration', 'service', 'groups'],
'msg': 'Extra inputs are not permitted',
'type': 'extra_forbidden'}]}
The 0.21.4 client always serializes configuration.groups for services, including as
null for a homogeneous service. Servers before 0.21.3 have no such field and reject it
because request models are extra="forbid".
The regression can be reproduced without a running server by validating the client's
request payload against the 0.21.2 request schema:
run_spec.configuration.service.groups
Extra inputs are not permitted [type=extra_forbidden, input_value=None, input_type=NoneType]
Expected behaviour
Any service configuration that does not use the 0.21.3+ groups feature should apply
successfully against older servers, as it did with the 0.21.3 client. Per the client
compatibility convention documented in dstack/_internal/core/compatibility/runs.py,
new fields must be excluded from requests when they are not set.
dstack version
0.21.4 (client); server 0.21.2
Server logs
Not relevant — the request is rejected by request-body validation before it reaches any
run logic. The 422 response body is quoted under "Actual behaviour".
Additional information
ServiceConfigurationParams.groups was introduced in 0.21.3 by #4177, which renamed
service replica groups from the legacy replicas: [...] list to groups. That PR also
added a @model_serializer(mode="wrap") (_serialize_legacy_replica_groups) on
ServiceConfigurationParams that popped groups out of every dump and re-emitted it as
the legacy replicas: [{count: ...}] shape. Because it lived on the model, it applied in
both directions, so a 0.21.3 client never put groups on the wire — which is why
0.21.3 → 0.21.2 works.
#4248 removed that serializer and moved the rewrite to server/compatibility/runs.py,
gated on client_version < 0.21.3. That covers the server → old client direction, but
nothing replaced the client → old server direction, so 0.21.4 clients now always send
groups.
ServiceConfiguration.groups is the only field added since 0.21.2 across the service,
task and dev-environment configurations, RunSpec and ProfileParams, so this appears
to be the only source of 0.21.4 → 0.21.2 incompatibility.
Note that #4248 also dropped the ability to run a heterogeneous service (groups
actually set) against a pre-0.21.3 server, which the 0.21.3 client supported by
downgrading the value to the legacy replicas: [{count: ...}] shape. Restoring that is
not expressible with the client's exclude-mapping mechanism, which can only drop fields,
not rename them.
Steps to reproduce
dstackserver 0.21.2 (or any version < 0.21.3).Compatibility matrix observed:
Actual behaviour
The 0.21.4 client always serializes
configuration.groupsfor services, including asnullfor a homogeneous service. Servers before 0.21.3 have no such field and reject itbecause request models are
extra="forbid".The regression can be reproduced without a running server by validating the client's
request payload against the 0.21.2 request schema:
Expected behaviour
Any service configuration that does not use the 0.21.3+
groupsfeature should applysuccessfully against older servers, as it did with the 0.21.3 client. Per the client
compatibility convention documented in
dstack/_internal/core/compatibility/runs.py,new fields must be excluded from requests when they are not set.
dstack version
0.21.4 (client); server 0.21.2
Server logs
Not relevant — the request is rejected by request-body validation before it reaches any run logic. The 422 response body is quoted under "Actual behaviour".Additional information
ServiceConfigurationParams.groupswas introduced in 0.21.3 by #4177, which renamedservice replica groups from the legacy
replicas: [...]list togroups. That PR alsoadded a
@model_serializer(mode="wrap")(_serialize_legacy_replica_groups) onServiceConfigurationParamsthat poppedgroupsout of every dump and re-emitted it asthe legacy
replicas: [{count: ...}]shape. Because it lived on the model, it applied inboth directions, so a 0.21.3 client never put
groupson the wire — which is why0.21.3 → 0.21.2 works.
#4248 removed that serializer and moved the rewrite to
server/compatibility/runs.py,gated on
client_version < 0.21.3. That covers the server → old client direction, butnothing replaced the client → old server direction, so 0.21.4 clients now always send
groups.ServiceConfiguration.groupsis the only field added since 0.21.2 across the service,task and dev-environment configurations,
RunSpecandProfileParams, so this appearsto be the only source of 0.21.4 → 0.21.2 incompatibility.
Note that #4248 also dropped the ability to run a heterogeneous service (
groupsactually set) against a pre-0.21.3 server, which the 0.21.3 client supported by
downgrading the value to the legacy
replicas: [{count: ...}]shape. Restoring that isnot expressible with the client's exclude-mapping mechanism, which can only drop fields,
not rename them.