feat: Floe binary row serialization format#2523
Merged
Merged
Conversation
norberttech
added
the
hackaton-07-2026
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 1.x #2523 +/- ##
============================================
+ Coverage 85.97% 86.27% +0.30%
- Complexity 22271 22924 +653
============================================
Files 1663 1739 +76
Lines 68436 70415 +1979
============================================
+ Hits 58840 60754 +1914
- Misses 9596 9661 +65 🚀 New features to boost your workflow:
|
stloyd
reviewed
Jul 11, 2026
- new Flow\Floe binary format: writer, reader, merger, schema evolution, extractor/loader + DSL - new flow-php/flow-php-ext Rust extension for native Floe encode/decode, with PHP fallback parity tests - cache layer rewritten on Floe: CacheIndex, streaming FilesystemCache, external sort buckets - CI workflow, nix packaging and codecov flag for the extension - documentation for Floe format and extension, upgrade notes
- install Composer dependencies before PHPT tests (bootstrap requires repo autoloader) - enable extension via php.ini fallback when PIE auto-enable fails - drop removed Flow\Serializer integration test dir from phpunit.xml.dist
- add missing ext/Makefile.frag so phpize/PIE builds the Rust library instead of shipping the dummy C stub - compare html fallback rows via re-encoded wire bytes; Dom\HTMLDocument forbids serialize()
- chain caught exceptions into KeyNotInCacheException to keep stacktraces - reuse a single FloeReader in FilesystemCache and FloeMerger instead of per-call construction - drop redundant array_values() in FloeMerger::merge(); reconcile() no longer depends on the first key being 0
norberttech
force-pushed
the
binary-serializer
branch
from
df6cfa1 to
00da815
Compare
stloyd
reviewed
Jul 11, 2026
- FilesystemBucketsCache batch size flows from config: externalSortBatchSize() on Config/CacheConfig builders, wired through SortingProcessor; also feeds bucket recover() reads - reuse a single FloeReader in FilesystemBucketsCache - simplify Rows::chunks() to array_chunk with inline adoption, drop fromList() (review feedback, perf-neutral at 1M rows)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Log
Added
flow-php/etl- Floe binary row serialization format with writer, reader, merger, extractor, loader and DSL.flow-php/flow-php-ext- new Rust-based PHP extension for native Floe encoding/decoding.flow-php/etl-Cache::read()for streaming cached Rows in batches.Fixed
flow-php/filesystem-MemoryStreamappends at end of stream after ranged reads.flow-php/types-EnumTypenormalization acceptsUnitEnum/BackedEnuminterfaces.Changed
flow-php/etl-Cachestores onlyRows; cache indexes are stored asRows.flow-php/etl- cache implementations and external sort buckets serialize with Floe instead of PHP serializer.flow-php/etl- serialize/unserialize transformers use Floe serializer.Removed
Deprecated
Security