Skip to content

feat(bindings/go): add ReaderFrom and WriterTo support - #8258

Open
ClSlaid wants to merge 4 commits into
apache:mainfrom
ClSlaid:codex/go-copy
Open

feat(bindings/go): add ReaderFrom and WriterTo support#8258
ClSlaid wants to merge 4 commits into
apache:mainfrom
ClSlaid:codex/go-copy

Conversation

@ClSlaid

@ClSlaid ClSlaid commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #8248.

Rationale for this change

The default io.Copy loop uses a 32 KiB buffer. Small writes repeat the Go-to-native call and the native data allocation.

What changes are included in this PR?

  • Add Writer.ReadFrom and Reader.WriteTo. Each method uses one reusable 256 KiB buffer.
  • Use interface wrappers to prevent recursive calls. Use io.CopyBuffer to handle byte counts, EOF, and write errors.
  • Add tests for file sources, copy method selection, partial results, blocked destinations, buffer ownership, Seek, and Close.
  • Update the stream copy guide.

Are there any user-facing changes?

io.Copy can select the new methods. When io.CopyBuffer selects either method, that method uses its own buffer.

The copy methods leave both streams open. The caller must call Writer.Close to complete the write and get its metadata.

Validation

  • Unit tests, race tests, go vet, and golangci-lint v2.12.2 passed.
  • Native filesystem and memory tests passed with a C library built from the same source revision.
  • Existing filesystem Reader and Writer behavior tests passed.
  • The guide example compiled for Linux amd64 with CGO disabled.

A local benchmark copied a 16 MiB file 100 times on macOS arm64 (Apple M4, Go 1.27.1). Copy time includes stream creation and Close.

Upload metric Base (b6cf44f7) This change
Mean copy time 15.90 ms 10.70 ms
Native writes per copy 512 64
Go allocations per copy 4,220 638
Go allocated bytes per copy 206,375 288,839
Mean time to first native write 12.1 µs 46.7 µs

The larger buffer reduces call counts but increases allocated bytes and the delay before the first write. Go allocation metrics exclude Rust allocations.

AI Usage Statement

AI assisted the implementation, tests, performance analysis, and documentation. Network service throughput, multipart latency, and concurrent transfer memory use remain unmeasured.

@ClSlaid
ClSlaid marked this pull request as ready for review September 8, 2026 09:08
Comment thread bindings/go/stream.go

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't feel good to place them in a new file called stream.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: add efficient io.Copy paths to Go Reader and Writer

2 participants