.NET: [Bug]: AIAgent.RunStreamingAsync does not surface the final image from HostedImageGenerationTool

[marcominerva]

Description

When an agent uses HostedImageGenerationTool, AIAgent.RunStreamingAsync does not expose the completed image through AgentResponseUpdate.Contents.

If ImageGenerationOptions.StreamingCount is configured, Agent Framework surfaces only the lower-quality partial/intermediate images. The final image contained in ImageGenerationCallResponseItem.ImageResultBytes is not converted to an ImageGenerationToolResultContent.

If StreamingCount is not configured, no image is surfaced at all through the Agent Framework content abstraction.

The final image is still present in the underlying OpenAI streaming update, but retrieving it requires traversing:

AgentResponseUpdate.RawRepresentation
    -> ChatResponseUpdate.RawRepresentation
    -> StreamingResponseOutputItemDoneUpdate
    -> ImageGenerationCallResponseItem.ImageResultBytes

This requires application code to depend on provider-specific OpenAI response types and on the internal shape of nested RawRepresentation values, defeating the provider-independent Agent Framework abstraction.

Actual behavior

With StreamingCount = 3:

• Agent Framework exposes the partial images.
• Agent Framework does not expose the final image.
• The final image can only be recovered from the nested provider-specific raw update.

Without StreamingCount:

• Agent Framework exposes no image.
• The final image is still available in the nested provider-specific raw update.

The completion update normally has no corresponding AIContent, so consumers cannot retrieve the final image without the workaround shown below.

Expected behavior

When the underlying streaming response contains:

StreamingResponseOutputItemDoneUpdate
{
    Item: ImageGenerationCallResponseItem image
}

AIAgent.RunStreamingAsync should surface the completed image as an ImageGenerationToolResultContent containing a DataContent created from image.ImageResultBytes.

This should happen:

• regardless of whether StreamingCount is configured;
• in addition to any partial images;
• consistently with the non-streaming response path;
• without requiring consumers to inspect RawRepresentation.

It would also be useful for partial images and the final image to be clearly distinguishable using provider-independent metadata or content types.

Agent Framework impact

The underlying conversion problem appears to originate in Microsoft.Extensions.AI.OpenAI, but it directly affects the public Agent Framework streaming experience:

await foreach (AgentResponseUpdate update in agent.RunStreamingAsync(prompt, session))
{
    // The final generated image is absent from update.Contents.
}

Agent Framework currently provides no typed, provider-independent way to obtain the completed image from RunStreamingAsync. Applications must either:

• depend on OpenAI-specific streaming types and unwrap nested raw representations;
• use the non-streaming API and lose partial-image streaming; or
• receive no image when StreamingCount is not configured.

Related

• [Microsoft.Extensions.AI.OpenAI] Final image is never yielded when streaming a response that uses HostedImageGenerationTool dotnet/extensions#7758

That issue describes the underlying Microsoft.Extensions.AI.OpenAI behavior. This issue tracks the problem from the Agent Framework perspective, where AIAgent.RunStreamingAsync fails to expose the final image through AgentResponseUpdate.Contents.

Code Sample

#!/usr/bin/env dotnet

#:sdk Microsoft.NET.Sdk
#:property OutputType=Exe
#:property TargetFramework=net10.0
#:property ImplicitUsings=enable
#:property Nullable=enable
#:property NoWarn=$(NoWarn);MEAI001;OPENAI001;MAAI001
#:property PublishAot=false

#:package Microsoft.Agents.AI.OpenAI@1.22.0

using System.ClientModel;
using System.ClientModel.Primitives;
using System.Net.Mime;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Responses;

var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var apiKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY") ?? throw new InvalidOperationException("AZURE_OPENAI_KEY is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.5";
var imageDeploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_IMAGE_DEPLOYMENT_NAME") ?? "gpt-image-2";

await RunScenarioAsync(streamingCount: 3);
await RunScenarioAsync(streamingCount: null);

async Task RunScenarioAsync(int? streamingCount)
{
    Console.WriteLine(streamingCount is null
            ? "Scenario: StreamingCount is not configured"
            : $"Scenario: StreamingCount = {streamingCount}");

    var clientOptions = new OpenAIClientOptions
    {
        Endpoint = new Uri(endpoint)
    };

    // Azure OpenAI requires the image-generation deployment to be selected
    // independently from the chat deployment.
    clientOptions.AddPolicy(new ImageDeploymentPolicy(imageDeploymentName),
        PipelinePosition.PerCall);

    var chatClient = new OpenAIClient(new ApiKeyCredential(apiKey), clientOptions)
        .GetResponsesClient()
        .AsIChatClientWithStoredOutputDisabled(deploymentName);

    var imageOptions = new ImageGenerationOptions
    {
        ModelId = imageDeploymentName,
        Count = 1,
        ImageSize = new(1024, 1024),
        MediaType = MediaTypeNames.Image.Png
    };

    if (streamingCount is int count)
    {
        imageOptions.StreamingCount = count;
    }

    var agent = chatClient.AsAIAgent(new ChatClientAgentOptions
    {
        ChatOptions = new ChatOptions
        {
            Instructions = "Generate the image requested by the user.",
            Tools =
            [
                new HostedImageGenerationTool
                {
                    Options = imageOptions
                }
            ]
        }
    });

    var session = await agent.CreateSessionAsync();

    var frameworkPartialImageCount = 0;
    var frameworkFinalImageCount = 0;
    var rawFinalImageCount = 0;

    await foreach (var update in agent.RunStreamingAsync(
        "Generate a watercolor painting of a red bicycle next to a lake.",
        session))
    {
        // Images exposed by the provider-independent Agent Framework abstraction.
        foreach (var result in update.Contents.OfType<ImageGenerationToolResultContent>())
        {
            if (result.Outputs is null)
            {
                continue;
            }

            foreach (var image in result.Outputs.OfType<DataContent>())
            {
                var isPartial = image.AdditionalProperties?.ContainsKey("PartialImageIndex") == true;

                if (isPartial)
                {
                    frameworkPartialImageCount++;
                }
                else
                {
                    frameworkFinalImageCount++;
                }
            }
        }

        // Workaround: unwrap provider-specific raw representations to recover
        // the completed image that is absent from update.Contents.
        for (object? raw = update.RawRepresentation; raw is not null; raw = (raw as ChatResponseUpdate)?.RawRepresentation)
        {
            if (raw is StreamingResponseOutputItemDoneUpdate  { Item: ImageGenerationCallResponseItem image })
            {
                var mediaType = image.OutputFileFormat.HasValue
                    ? $"image/{image.OutputFileFormat.Value}"
                    : MediaTypeNames.Image.Png;

                var finalImage = new DataContent(
                    image.ImageResultBytes,
                    mediaType);

                finalImage.AdditionalProperties ??= [];
                finalImage.AdditionalProperties["prompt"] = image.RevisedPrompt;

                rawFinalImageCount++;

                Console.WriteLine(
                    $"Recovered final image from RawRepresentation: " +
                    $"{finalImage.Data.Length} bytes");
            }
        }
    }

    Console.WriteLine($"Partial images exposed by Agent Framework: {frameworkPartialImageCount}");
    Console.WriteLine($"Final images exposed by Agent Framework: {frameworkFinalImageCount}");
    Console.WriteLine($"Final images recovered from RawRepresentation: {rawFinalImageCount}");

    Console.WriteLine();
}

internal sealed class ImageDeploymentPolicy(string deployment) : PipelinePolicy
{
    private const string HeaderName = "x-ms-oai-image-generation-deployment";

    public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int index)
    {
        message.Request.Headers.Set(HeaderName, deployment);
        ProcessNext(message, pipeline, index);
    }

    public override ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int index)
    {
        message.Request.Headers.Set(HeaderName, deployment);
        return ProcessNextAsync(message, pipeline, index);
    }
}

Representative output:

Scenario: StreamingCount = 3
Recovered final image from RawRepresentation:
Partial images exposed by Agent Framework: 3
Final images exposed by Agent Framework: 0
Final images recovered from RawRepresentation: 1

Scenario: StreamingCount is not configured
Recovered final image from RawRepresentation:
Partial images exposed by Agent Framework: 0
Final images exposed by Agent Framework: 0
Final images recovered from RawRepresentation: 1

The RawRepresentation block is the workaround. Without that block, the application has no access to the final image while using AIAgent.RunStreamingAsync.

Package Versions

Microsoft.Agents.AI.OpenAI: 1.22.0

.NET Version

.NET 10.0, SDK 10.0.401

Additional Context

Relevant resolved packages:

Package	Version
Microsoft.Agents.AI.OpenAI	1.22.0
Microsoft.Extensions.AI	10.10.0
Microsoft.Extensions.AI.OpenAI	10.10.0
OpenAI	2.13.0
System.ClientModel	1.15.0

[github-actions]

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Repro: Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs::GetStreamingResponseAsync emits an empty ChatResponseUpdate when response.output_item.done contains an ImageGenerationCallResponseItem, and dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs:360-373 forwards it without typed image content. The trigger is a completed hosted image-generation item, regardless of whether partial-image streaming is enabled. Minimal repro: run HostedImageGenerationStreamingTests.cs with a mocked Responses API SSE completion and observe one raw final image but zero ImageGenerationToolResultContent instances.

• Failing test: dotnet/tests/Microsoft.Agents.AI.OpenAI.UnitTests/Extensions/HostedImageGenerationStreamingTests.cs
• Files examined: dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs, dotnet/src/Microsoft.Agents.AI.OpenAI/Extensions/OpenAIResponseClientExtensions.cs, dotnet/tests/OpenAIResponse.IntegrationTests/OpenAIResponseChatClientAgentRunStreamingTests.cs, dotnet/tests/Microsoft.Agents.AI.OpenAI.UnitTests/Extensions/OpenAIResponseClientExtensionsTests.cs
• Tests run: dotnet/tests/Microsoft.Agents.AI.OpenAI.UnitTests/Extensions/HostedImageGenerationStreamingTests.cs
• Reported version: 1.22.0
• Current version: 1.22.0