feat: add external byte codecs by andreiltd · Pull Request #1792 · hyperlight-dev/hyperlight

andreiltd · 2026-09-03T13:09:47Z

See HIP document for motivation for those changes: #1791

This PR provides the codec and API groundwork for the optimization. Legacy stack based host-guest communication continues to embed byte payloads in flatbuffers. The patch

adds external value sources and sinks to function call and result codecs,
keeps embedded codecs available for the existing stack transport,
exposes ByteChunks parameters and typed guest returns through the C API.

Add external value sources and sinks while preserving embedded codecs. Expose chunk-preserving ByteChunks and typed guest returns through C. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>

Copilot

🟡 Changes recommended

The updated C guest path introduces a confirmed malloc leak and the new C API exposes an owned string without a matching deallocator, both of which must be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR lays the schema, codec, and C API groundwork for carrying byte payloads outside FlatBuffers, while preserving embedded byte encoding for the legacy stack-based transport.

Changes:

Add ByteChunks as a distinct logical byte type, plus FlatBuffer compatibility representations for embedded transport.
Introduce external byte value sources/sinks for function call and result codecs (encode markers in FlatBuffers, stream payloads out-of-band).
Update the guest C API to use typed hl_ReturnValue* results and expose ByteChunks parameters and host returns.

File summaries

File	Description
src/tests/rust_guests/simpleguest/src/main.rs	Extend fuzz guest to encode `ByteChunks` results.
src/tests/c_guests/c_simpleguest/main.c	Update C guest to return typed `hl_ReturnValue*` and use new constructors.
src/schema/function_types.fbs	Add FlatBuffer tables/unions/enums for external bytes + chunked bytes.
src/hyperlight_host/src/sandbox/snapshot/file/config.rs	Add `ByteChunks` support to snapshot JSON type mirroring + tests.
src/hyperlight_host/src/func/mod.rs	Re-export `Bytes` for chunk-preserving byte values.
src/hyperlight_guest_capi/src/types/vec.rs	Add borrowed slice constructor; simplify `copy_to_vec` implementation contract.
src/hyperlight_guest_capi/src/types/return_value.rs	New owned `FfiReturnValue` representation with tag + union + ownership.
src/hyperlight_guest_capi/src/types/parameter.rs	Add `ByteChunks` parameter support and switch to owner-backed borrowed views.
src/hyperlight_guest_capi/src/types/function_call.rs	Add owner-backed `OwnedFfiFunctionCall` to keep borrowed pointers valid.
src/hyperlight_guest_capi/src/types/byte_chunks.rs	New C-ABI types for borrowed/owned chunk descriptors and ownership helpers.
src/hyperlight_guest_capi/src/types.rs	Export new byte-chunk and return-value types.
src/hyperlight_guest_capi/src/return_value.rs	New C API entrypoints for constructing/reading typed return values.
src/hyperlight_guest_capi/src/lib.rs	Replace `flatbuffer` module export with `return_value`.
src/hyperlight_guest_capi/src/flatbuffer.rs	Remove legacy “flatbuffer result” C API helpers.
src/hyperlight_guest_capi/src/dispatch.rs	Update C guest dispatch to consume `hl_ReturnValue*` and encode via FlatBuffers.
src/hyperlight_guest_capi/README.md	Document `ByteChunks` semantics and the new `hl_result_from_*` contract.
src/hyperlight_guest_capi/include/macro.h	Update wrappers to return `hl_ReturnValue` via `hl_result_from_`.
src/hyperlight_guest_capi/cbindgen.toml	Rename/exports for `ReturnValue` and `ByteChunks` C-visible types.
src/hyperlight_guest_bin/src/guest_function/definition.rs	Add embedded encoding support for `ReturnValue::ByteChunks`.
src/hyperlight_component_util/src/hl.rs	Fix explicit `Vec::<u8>::new()` emission for empty results.
src/hyperlight_common/src/func/ret_type.rs	Add `Vec<Bytes>` as supported return type + round-trip test.
src/hyperlight_common/src/func/param_type.rs	Add `Vec<Bytes>` as supported parameter type + round-trip test.
src/hyperlight_common/src/func/mod.rs	Re-export `Bytes` from common func module.
src/hyperlight_common/src/flatbuffers/mod.rs	Wire new generated FlatBuffer modules into the tree.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/return_value_generated.rs	Regenerate union discriminants for new return value variants.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/return_value_box_generated.rs	Regenerate union accessors/verifier for new return value variants.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/return_type_generated.rs	Regenerate return type enum for `hlbytechunks`.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/parameter_value_generated.rs	Regenerate parameter value union discriminants for external/chunked bytes.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/parameter_type_generated.rs	Regenerate parameter type enum for `hlbytechunks`.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/parameter_generated.rs	Regenerate parameter union accessors/verifier for external/chunked bytes.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/hlsizeprefixedbytechunks_generated.rs	New generated table for embedded chunked returns.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/hlexternalbytes_generated.rs	New generated marker table for external byte payloads.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/hlbytechunks_generated.rs	New generated table for embedded chunked parameters.
src/hyperlight_common/src/flatbuffer_wrappers/util.rs	Add byte-chunk utilities + FlatBuffer serialization for chunked returns.
src/hyperlight_common/src/flatbuffer_wrappers/mod.rs	Export new external codec traits module.
src/hyperlight_common/src/flatbuffer_wrappers/function_types.rs	Add `ByteChunks` value/type support + external encode/decode for results.
src/hyperlight_common/src/flatbuffer_wrappers/function_call.rs	Add external encode/decode for byte parameters + embedded `ByteChunks` support.
src/hyperlight_common/src/flatbuffer_wrappers/codec.rs	New `ExternalValueSink` / `ExternalValueSource` traits.
CHANGELOG.md	Document new C API return model and `ByteChunks` exposure.

Review details

Files reviewed: 30/39 changed files
Comments generated: 2
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+#[unsafe(no_mangle)]
+pub extern "C" fn hl_get_host_return_value_as_String() -> *const c_char {
+    let string_value: String = take_last_host_return();
+
+    let c_string = CString::new(string_value).expect("Failed to create CString");
+    c_string.into_raw()
+}
+


  uint8_t *x = malloc(input.len);
  for (uintptr_t i = 0; i < input.len; i++) {
    x[i] = 0;
  }
-  return hl_flatbuffer_result_from_Bytes(x, input.len);
+  return hl_result_from_Bytes(x, input.len);


jsturtevant · 2026-09-03T21:41:39Z

+///
+/// `Vec<u8>` and `Vec<Bytes>` are distinct logical return types. `Vec<u8>`
+/// carries contiguous bytes, while `Vec<Bytes>` preserves local chunk
+/// ownership. Chunk boundaries are not application framing.


I've seen this comment a couple times and I don't really know what it means to be "application framing"?

My understanding of what @andreiltd is trying to say by this is that chunk boundaries are not semantically meaningful. That means you can't assume that one chunk == one message on the wire. Instead messages boundaries are independent of the way reads are chunked over the network.

jsturtevant · 2026-09-03T22:26:14Z

+    let marker = parameter
+        .value_as_hlexternalbytes()
+        .ok_or_else(|| anyhow!("Failed to get external byte parameter marker"))?;
+    let length = usize::try_from(marker.length()).map_err(|_| {


Does this length need to be bounded? (because it might allocate on the host later?)

ludfjig

this looks great

ludfjig · 2026-09-04T02:16:00Z

 }

 impl FfiVec {
+    pub(crate) fn from_mut_slice(value: &mut [u8]) -> Self {


maybe a comment on this could be useful

ludfjig · 2026-09-04T02:21:52Z

            }
            ParameterValue::VecBytes(v) => {
-                let leaked = unsafe { FfiVec::from_vec(v) };
+                let mut value = v;


tiny optional nit: slightly cleaner

Suggested change

let mut value = v;

ParameterValue::VecBytes(mut v) => {

let view = FfiVec::from_mut_slice(&mut v);

(

ParameterType::VecBytes,

FfiParameterValue { VecBytes: view },

FfiParameterOwner::VecBytes { _value: v },

)

}

yoshuawuyts · 2026-09-04T06:47:35Z

+pub trait ExternalValueSink<'a> {
+    /// Receive one contiguous byte value.
+    fn push_bytes(&mut self, value: &'a [u8]) -> Result<()>;
+
+    /// Receive one chunk-preserving byte value.
+    fn push_chunks(&mut self, value: &'a [Bytes]) -> Result<()>;
+}
+
+/// Supplies complete external byte values while a FlatBuffer is decoded.
+pub trait ExternalValueSource {
+    /// Take the next external value as contiguous bytes.
+    fn take_bytes(&mut self, length: usize) -> Result<Vec<u8>>;
+
+    /// Take the next external value as owned byte chunks.
+    fn take_chunks(&mut self, length: usize) -> Result<Vec<Bytes>>;
+
+    /// Finish decoding and reject any unused external values.
+    fn finish(&mut self) -> Result<()>;
+}


Could you elaborate on why Rust's standard Read and Write traits are insufficient for this purpose? I'm personally not a fan of the bytes crate, but I recognize the crate can occasionally beneficial. I would like to understand the motivation for us using it in this case, as the accompanying HIP does not appear to cover that part of the design.

yoshuawuyts

left some comments; I've reached out to @andreiltd offline to talk these through because that might be quickest

yoshuawuyts · 2026-09-04T06:57:45Z

 }

+/// Wrap contiguous bytes as one chunk without copying.
+pub fn byte_chunks_from_bytes(value: Bytes) -> Vec<Bytes> {


I'm a bit confused by this API? Bytes is a bit like an Arc<Vec<u8>>. What this seems to be doing is converting that to a Vec<Arc<Vec<u8>>>. I'm not sure why?

Also afaik it isn't right to call Bytes contiguous as they're internally represented by a rope structure. Not that that matters too much, but it doesn't make me less confused 😅

yoshuawuyts · 2026-09-04T07:18:39Z

+/// Materialize byte chunks as contiguous [`Bytes`].
+///
+/// This is O(1) for zero or one chunk and copies once for multiple chunks.
+pub fn byte_chunks_to_bytes(value: &[Bytes]) -> Bytes {


Actually: I can't help but feel that this is an anti-pattern, and we're doing something wrong here. I can see why we'd have a byte stream (e.g. impl Read), but a logical Vec<Vec<u8>> - that to me feels like something isn't quite right in our factoring?

The reason why I'm saying this also is because earlier on in the PR it's stated that chunk boundaries are unrelated to message framing.

feat: add external byte codecs and C guest ByteChunks API

fd830fd

Add external value sources and sinks while preserving embedded codecs. Expose chunk-preserving ByteChunks and typed guest returns through C. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>

andreiltd changed the title ~~external bytes~~ feat: add external byte codecs Sep 3, 2026

andreiltd added kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. area/performance Addresses performance labels Sep 3, 2026

andreiltd marked this pull request as ready for review September 3, 2026 14:56

Copilot AI lite review requested due to automatic review settings September 3, 2026 14:56

andreiltd requested review from danbugs, dblnz, devigned, jprendes, jsturtevant, ludfjig, simongdavies, squillace and syntactically as code owners September 3, 2026 14:56

andreiltd added the ready-for-review PR is ready for (re-)review label Sep 3, 2026

Copilot started reviewing on behalf of andreiltd September 3, 2026 14:59 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

jsturtevant reviewed Sep 3, 2026

View reviewed changes

ludfjig approved these changes Sep 4, 2026

View reviewed changes

yoshuawuyts reviewed Sep 4, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat: add external byte codecs - #1792

feat: add external byte codecs#1792
andreiltd wants to merge 1 commit into
mainfrom
external-bytes

andreiltd commented Sep 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

jsturtevant Sep 3, 2026

Uh oh!

yoshuawuyts Sep 4, 2026 •

edited

Loading

Uh oh!

jsturtevant Sep 3, 2026 •

edited

Loading

Uh oh!

ludfjig left a comment

Uh oh!

ludfjig Sep 4, 2026

Uh oh!

ludfjig Sep 4, 2026

Uh oh!

yoshuawuyts Sep 4, 2026

Uh oh!

yoshuawuyts left a comment

Uh oh!

yoshuawuyts Sep 4, 2026 •

edited

Loading

Uh oh!

yoshuawuyts Sep 4, 2026

Uh oh!

yoshuawuyts Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

-                let mut value = v;
+    ParameterValue::VecBytes(mut v) => {
+        let view = FfiVec::from_mut_slice(&mut v);
+        (
+            ParameterType::VecBytes,
+            FfiParameterValue { VecBytes: view },
+            FfiParameterOwner::VecBytes { _value: v },
+        )
+    }

Uh oh!

Conversation

andreiltd commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

jsturtevant Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

yoshuawuyts Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsturtevant Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ludfjig left a comment

Choose a reason for hiding this comment

Uh oh!

ludfjig Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

ludfjig Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

yoshuawuyts Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

yoshuawuyts left a comment

Choose a reason for hiding this comment

Uh oh!

yoshuawuyts Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yoshuawuyts Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

yoshuawuyts Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andreiltd commented Sep 3, 2026 •

edited

Loading

yoshuawuyts Sep 4, 2026 •

edited

Loading

jsturtevant Sep 3, 2026 •

edited

Loading

yoshuawuyts Sep 4, 2026 •

edited

Loading