Skip to content

Fix replace-conf-from-env.sh word-splitting env values into junk config lines - #18680

Open
anishmehta24 wants to merge 1 commit into
apache:masterfrom
anishmehta24:fix/replace-conf-from-env-word-splitting
Open

anishmehta24 wants to merge 1 commit into
apache:masterfrom
anishmehta24:fix/replace-conf-from-env-word-splitting

Conversation

@anishmehta24

Copy link
Copy Markdown

Description

docker/src/main/DockerCompose/replace-conf-from-env.sh iterated over the unquoted output of env (for v in $(env)), so any variable whose value contains whitespace was split into words and every lowercase word was appended to iotdb-system.properties as its own line. With the reporter's -e "SOME_VAR=this is a multi word value" the file ended with

is
a
multi
word
value
_=/usr/bin/env

(_=/usr/bin/env came from bash's $_, which the ! 2w$key_name =~ ^_ test never actually filtered). The junk lines also shift the line numbers the sed edits rely on.

While reproducing it I hit a second, independent loss: when the key being overridden sits on the last line of the file, sed -i "${line_no}d" followed by sed -i "${line_no}a..." addresses a line that no longer exists, so the override is silently dropped (dn_rpc_port=7777 disappeared entirely in the repro below).

Changes:

  • replace_configs iterates over exported variable names (compgen -e) and reads each value through ${!key_name}, so values are never word-split. A value containing a newline is skipped with a message rather than corrupting the file. Names starting with _ are skipped as intended.
  • Paths and key=value pairs are quoted on the way into process_single; the key is taken with ${key_value%%=*}.
  • An existing, uncommented key is replaced in place with sed Nc; a commented one still gets the override appended after it; a missing key is appended with sed $a.

Verification

Ran the old and the new script against a 4-line properties file with IOTDB_HOME pointing at a temp dir and SOME_VAR="this is a multi word value" JAVA_TOOL_OPTIONS="-Xms1g -Xmx2g" dn_rpc_address=0.0.0.0 dn_rpc_port=7777 dn_new_key="a b c" in the environment.

Before:

# dn_metric_reporter_list=
dn_metric_prometheus_reporter_port=9092
# dn_rpc_address=127.0.0.1
dn_rpc_address=0.0.0.0
is
a
multi
word
value
_=/usr/bin/env

(dn_rpc_port=6667 is gone and 7777 never landed.)

After:

# dn_metric_reporter_list=
dn_metric_prometheus_reporter_port=9092
# dn_rpc_address=127.0.0.1
dn_rpc_address=0.0.0.0
dn_rpc_port=7777
dn_new_key=a b c

bash -n passes; the script only uses bash builtins plus grep/cut/sed as before, so no new image dependency.

Closes #18655

🤖 Generated with Claude Code

…ig lines

`replace_configs` iterated over the unquoted output of `env`, so any
variable whose value contains whitespace (JAVA_TOOL_OPTIONS, a
multi-word SOME_VAR, ...) was split into words, and every lowercase word
was appended to iotdb-system.properties as its own line. The junk also
shifted the line numbers the sed edits rely on, and `_=/usr/bin/env` from
bash slipped through the intended `_` filter.

Iterate over exported variable names (`compgen -e`) and read each value
through indirection instead, quote the paths and values passed to
process_single, and replace an existing key with `sed Nc` rather than
delete + append: the latter silently dropped the override when the key
was on the last line of the file.

Closes apache#18655
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] sbin/replace-conf-from-env.sh word-splits the whole environment and corrupts iotdb-system.properties

1 participant