From fe0f86a3f55b147d26562695bce36f54433a2fb6 Mon Sep 17 00:00:00 2001 From: hanliu0830 Date: Tue, 15 Sep 2026 17:09:35 +0800 Subject: [PATCH] [docs] Add Production Readiness Checklist --- .../operations/production-readiness.md | 113 ++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 website/docs/maintenance/operations/production-readiness.md diff --git a/website/docs/maintenance/operations/production-readiness.md b/website/docs/maintenance/operations/production-readiness.md new file mode 100644 index 0000000000..5bf7323100 --- /dev/null +++ b/website/docs/maintenance/operations/production-readiness.md @@ -0,0 +1,113 @@ +--- +title: Production Readiness Checklist +sidebar_position: 0 +--- + + + +# Production Readiness Checklist + +This checklist outlines the deployment and configuration recommendations for running Fluss in production. + +## ZooKeeper + +Fluss uses ZooKeeper as its central metadata store and for cluster coordination and configuration management. +For production, we recommend ZooKeeper 3.8 or later. Use the latest patch release of a supported release line from the +[official ZooKeeper downloads page](https://zookeeper.apache.org/releases/). Follow the upstream +[deployment](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_deployment) and +[configuration](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_configuration) documentation for your +chosen version, then apply the Fluss recommendations below. + +### Deployment + +- **Cluster size and placement:** A single node is suitable for development and testing. For production, use 3, 5, + or 7 voting nodes to tolerate 1, 2, or 3 node failures, respectively. Distribute nodes across physical hosts and + failure domains so a single failure cannot take down a majority of the ensemble. +- **Resources:** Use dedicated Linux hosts or virtual machines. Treat 1 vCPU and 4 GB of memory per node as a starting + point, and adjust resources for the metadata volume and expected workload. +- **Storage:** Use persistent data volumes separate from the operating-system disk. Place transaction logs on a + dedicated disk, separate from snapshots, so snapshot writes do not delay transaction-log writes. + +### Configuration + +The following settings are recommended for deploying Fluss. + +#### ZooKeeper configuration (zoo.cfg) + +The following example shows the recommended ZooKeeper settings for a three-node ensemble. Adjust the server list +for your deployment. Replace `` and `` with your persistent +volume mount paths, and ensure the volumes are mounted before starting ZooKeeper. +Follow the [official deployment guide](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_deployment) +for installation and each server's `myid` file. + +```text title="conf/zoo.cfg" +tickTime=2000 +initLimit=20 +syncLimit=10 + +# Replace the placeholders with your persistent volume mount paths. +dataDir=/zookeeper/data +dataLogDir=/zookeeper/log +clientPort=2181 + +4lw.commands.whitelist=srvr,stat,ruok,mntr,conf,isro + +autopurge.snapRetainCount=3 +autopurge.purgeInterval=1 + +server.1=zk1.example.com:2888:3888 +server.2=zk2.example.com:2888:3888 +server.3=zk3.example.com:2888:3888 +``` + +| Setting | Description | +| --- | --- | +| `tickTime` | Base time unit for heartbeats and timeouts, in milliseconds; `2000 ms = 2 s` in this example. | +| `initLimit` | Time allowed for a follower to connect and initially synchronize with the leader, in ticks; `20 × tickTime = 40 s` in this example. | +| `syncLimit` | Time allowed for ongoing follower synchronization, in ticks; `10 × tickTime = 20 s` in this example. Followers exceeding the limit are dropped. | +| `dataDir` | Stores snapshots, plus transaction logs if `dataLogDir` is unset. Use a directory on the mounted snapshot volume. | +| `dataLogDir` | Stores transaction logs. Use a directory on a dedicated transaction-log volume. | +| `clientPort` | Port accepting client connections. | +| `4lw.commands.whitelist` | Enabled four-letter commands. `*` enables all commands, including `wchc` and `wchp`, which can be expensive with many watches. | +| `autopurge.snapRetainCount` | Number of snapshots retained with the logs needed for recovery; minimum 3. Adjust the count to your retention needs. | +| `autopurge.purgeInterval` | Time between automatic cleanup runs, in hours; `1` runs cleanup once per hour. Adjust the interval to your cleanup needs. Purging removes old disk snapshots and transaction logs; it does not trim data inside znodes. | +| `server.N` | Members identified by server ID, with the quorum port followed by the leader-election port (`2888` and `3888` in this example). Use the same membership list on every server. | + +#### JVM configuration (java.env) + +The official `zkServer.sh` script loads `conf/java.env`; add the JVM option there as shown below. Other deployment +methods must supply it through their own server JVM-argument configuration. + +```bash title="conf/java.env" +SERVER_JVMFLAGS="$SERVER_JVMFLAGS -Djute.maxbuffer=104857600" +``` + +| Setting | Description | +| --- | --- | +| `jute.maxbuffer` | Maximum data size per znode, in bytes; `104857600 bytes = 100 MiB` in this example. | + +:::important Increase the ZooKeeper buffer limit (jute.maxbuffer) + +Fluss metadata can exceed ZooKeeper's default limit of about 1 MiB per znode, so the znode size limit needs to be +increased. Fluss has already raised the default client limit to 100 MiB through +[zookeeper.client.max-buffer-size](../configuration.md#zookeeper). The ZooKeeper servers should therefore be +configured with `jute.maxbuffer=104857600` (100 MiB) to match the client limit. + +:::