Skip to content

fix(llm): retry once more when structured output is cut off mid-response #618

Description

@zzwong

Problem

A structured-output task can fail both its first attempt and its single validation retry because the response stops before a complete JSON value is written (io.EOF / io.ErrUnexpectedEOF from the strict decoder). The standard "your JSON was invalid" retry prompt doesn't address a response that never finished, so the task fails after two attempts and the run aborts.

Proposed fix

When the retry still decodes as incomplete JSON, give it one more independently sampled attempt. A retry that fails on a real schema violation (complete but wrong JSON) keeps the current two-attempt budget.

Before landing

  • Check whether this failure still happens in live runs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions