🌟 [Major]: TOML 1.0.0 reading and writing now available in PowerShell - #15
Open
Marius Storhaug (MariusStorhaug) wants to merge 21 commits into
Open
🌟 [Major]: TOML 1.0.0 reading and writing now available in PowerShell#15Marius Storhaug (MariusStorhaug) wants to merge 21 commits into
Marius Storhaug (MariusStorhaug) wants to merge 21 commits into
Conversation
- Add ConvertFrom-Toml, ConvertTo-Toml, Import-Toml, Export-Toml public commands - Add TomlDocument class with Data (OrderedDictionary) and FilePath properties - Add private helpers: ConvertFrom-TomlDateTime, ConvertFrom-TomlynTable, ConvertFrom-TomlynValue, ConvertTo-TomlynArray, ConvertTo-TomlynTable, ConvertTo-TomlynValue - Back with Tomlyn v2.0.0 (.NET TOML library) via src/assemblies/Tomlyn.dll - Enforce TOML 1.0.0 duplicate key and table redefinition rules via SyntaxParser.ParseStrict before deserializing - Map all 8 TOML types to PowerShell types (string, long, double, bool, DateTimeOffset, DateTime, TimeSpan, OrderedDictionary/object[]) - Add 116 Pester tests covering spec compliance, round-trips, file I/O, error handling, and all TOML 1.0.0 type categories - Add build.ps1 for local module assembly and test running - Add tests/bootstrap.ps1 for test module import - Add tests/data/ with 9 TOML fixture files - Update README.md with type mapping table, command reference, usage examples - Update examples/General.ps1 with comprehensive usage examples - Remove template placeholder files (LsonLib.dll, format/type XMLs, sample modules/scripts/variables) - Fix PSScriptAnalyzer warnings: UTF-8 BOM, indentation consistency PSScriptAnalyzer: 0 errors, 0 warnings Tests: 116/116 pass Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…dvanced fixtures - Replace Tomlyn (.NET) dependency with pure PowerShell parser/serializer - Add 19 private helper functions covering all TOML token types - Fix: sub-tables inside array-of-tables entries no longer falsely fail as redefinitions - Fix: space-separator datetimes (e.g. '1987-07-05 17:45:00Z') now parsed correctly - Add 5 advanced TOML fixture files covering deep AoT nesting, all numeric bases, all datetime forms, Unicode escapes, multi-line strings, and heterogeneous arrays - Remove src/assemblies/Tomlyn.dll and all Tomlyn* helpers - Set PowerShellVersion = 7.6 in manifest - 116 tests pass, PSScriptAnalyzer clean Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Marius Storhaug (MariusStorhaug)
changed the title
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- src/functions/public/Toml/ -> src/functions/public/ - src/functions/private/Toml/ -> src/functions/private/ - Remove template placeholder sections from README.md Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
Remove header.ps1, finally.ps1, init/initializer.ps1, manifest.psd1, and src/README.md — none are needed by the pure-PowerShell module. Build and 116 tests remain green. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
- One file with Describe/Context hierarchy per command - Remove bootstrap.ps1 — module is pre-imported by the test runner - Add advanced fixture tests covering AoT sub-tables, space-separator datetimes, deep nesting, and all numeric forms - 120 tests, 0 failures Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
Code violations fixed: - [bool] parameters replaced with [switch] in Add-TomlTableText and Get-TomlBareToken; callers updated (-StopAtEquals) - \Continue = 'Stop' injected into built psm1 via build.ps1 - Skip-TomlWhitespace gains [OutputType([void])] Comment-based help: - ConvertFrom-Toml and ConvertTo-Toml: add .DESCRIPTION, .EXAMPLE x2, .INPUTS, .OUTPUTS, .NOTES to match required section order - All 20 private helpers: add .DESCRIPTION, .EXAMPLE, .INPUTS, .OUTPUTS - Add Write-Verbose to all four public functions Structure: - Add tests/BeforeAll.ps1 (shared setup entry-point per PSModule standard) Examples: - Add examples/data/ with three comment-free TOML fixtures 120/120 tests pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
Adds `Test-Toml`, a pure-validator that returns `[bool]` — never throws — so scripts can safely probe TOML content before (or instead of) parsing it. The design mirrors PowerShell's built-in `Test-Json`. ## New: Validate TOML content safely `Test-Toml` wraps `ConvertFrom-Toml` in a try/catch. On success it returns `$true`; on any parser error it writes a non-terminating error to the error stream and returns `$false`. ```powershell # Validate a string Test-Toml -InputObject 'key = "value"' # $true '[invalid' | Test-Toml # $false + error written # Validate a file Test-Toml -Path .\Cargo.toml # $true if file is valid TOML Test-Toml -LiteralPath 'C:\app\cfg.toml' # literal path variant ``` **Parameter sets** | Parameter | Set | Notes | |---|---|---| | `-InputObject` | Default | Pipeline-capable; `[AllowEmptyString()]` so `''` returns `$false` | | `-Path` | `Path` | Resolves relative paths | | `-LiteralPath` | `LiteralPath` | No wildcard expansion | **Key implementation notes** - `[AllowEmptyString()]` is required on `InputObject`: PowerShell's mandatory parameter binding rejects `''` without it, whereas the correct behaviour is `$false`. - `Write-Error` is always called from inside a `catch` block with `-ErrorAction Continue` to ensure non-terminating behaviour even when `$ErrorActionPreference = 'Stop'` is set module-wide. - No new parser logic — reuses `ConvertFrom-Toml` to keep validation consistent with parsing. ## Technical Details - **File added:** `src/functions/public/Test-Toml.ps1` - **Tests added:** `Describe 'Test-Toml'` block in `tests/Toml.Tests.ps1` (11 new tests; total 135, all passing) - **PSScriptAnalyzer:** No new warnings introduced (pre-existing BOM and OutputType notices in other files unchanged) - **No alias:** The issue mentions `Test-Tml` but aliases are not in the current module standards <details> Closes #4 </details> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
`Merge-Toml` combines two or more TOML documents into one, so layered configuration — defaults plus environment overrides, or a base file plus a local override file — no longer needs ad-hoc hashtable-merging loops. ## New: Merge-Toml `Merge-Toml` accepts either two TOML strings, or a list of file paths merged in order: ```powershell Merge-Toml -BaseObject $defaults -OverrideObject $overrides Merge-Toml -Path 'defaults.toml', 'local.toml' Merge-Toml -LiteralPath 'defaults.toml', 'local.toml' ``` Nested tables are always deep-merged recursively, and arrays of tables (`[[...]]`) are always concatenated, base entries first. Scalar key conflicts (including inline arrays, treated as scalars) are resolved with `-Strategy`: - `LastWins` (default) — the override value replaces the base value - `FirstWins` — the base value is kept - `ErrorOnConflict` — throws on any duplicate scalar key The result is always a `[string]` of canonical TOML, parseable by `ConvertFrom-Toml`. --- <details> <summary>Technical details</summary> - Added `src/functions/public/Merge-Toml.ps1` with three parameter sets: `Default` (`-BaseObject`/`-OverrideObject`), `Path`, and `LiteralPath`. - Added private recursive helper `src/functions/private/Merge-TomlTableObject.ps1` that walks override keys against a base `OrderedDictionary`, deep-merging nested tables, concatenating `ArrayList` array-of-tables, and applying `-Strategy` to remaining scalar conflicts. - Implementation reuses `ConvertFrom-Toml`/`ConvertTo-Toml` for parsing and serialization — no new parsing logic. - Added Pester coverage in `tests/Toml.Tests.ps1` for all parameter sets, all three strategies, deep-merge, AoT concatenation, file input, and idempotency. - Updated `README.md` with a usage example and added `Merge-Toml` to the command table. - Verified: `Invoke-ScriptAnalyzer -Path src/ -Recurse` clean (no new findings), 135/135 Pester tests passing. - Implementation plan progress: all tasks in #19 completed (helper, function, tests, README). </details> <details> <summary>Relevant issues (or links)</summary> - Closes #19 </details> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
`Format-Toml` normalizes TOML text into a canonical form — consistent key quoting, canonical scalar formatting, and stable table ordering — without changing its meaning. It is the single-call equivalent of `ConvertFrom-Toml | ConvertTo-Toml`, matching the pattern already established by sister modules `Format-Json` and `Format-Hashtable`. ## New: `Format-Toml` normalizes TOML text ```powershell Get-Content 'Cargo.toml' -Raw | Format-Toml Format-Toml -Path 'Cargo.toml' -Indent 4 Format-Toml -LiteralPath 'C:\configs\[env].toml' ``` `Format-Toml` accepts a TOML string via pipeline (`-InputObject`), or reads a file via `-Path` (wildcard-aware) or `-LiteralPath`. It never writes back to the source file — output is always returned as a `[string]`. Formatting is idempotent: running it twice produces the same result as running it once, and the output always remains parseable by `ConvertFrom-Toml`. TOML has no indentation semantics — nested tables are flat `[a.b]` headers, not indented blocks. The `-Indent` parameter (default `2`) is a display convention: it prefixes nested table headers and their keys with spaces proportional to nesting depth, similar to how the Taplo formatter presents nested tables. Set `-Indent 0` to keep the flat, unindented form. --- <details> <summary>Technical details</summary> - New public function `src/functions/public/Format-Toml.ps1`, three parameter sets (`InputObject` default/pipeline, `Path`, `LiteralPath`), reuses `ConvertFrom-Toml` + `ConvertTo-Toml` — no new parser or emitter. - New private helper `src/functions/private/Add-TomlIndentation.ps1` post-processes the canonical (flat) `ConvertTo-Toml` output to add depth-based indentation without touching the existing serializer, so `ConvertTo-Toml`'s own output and tests are unaffected. - Added a `Describe 'Format-Toml'` block to `tests/Toml.Tests.ps1` covering: normalization, semantic equivalence to `ConvertFrom-Toml | ConvertTo-Toml`, indentation at depth, `-Indent 0` flat output, idempotency, round-trip validity, pipeline input, `-Path`, `-LiteralPath`, and error handling for invalid TOML / missing files. - README and `examples/General.ps1` updated with a `Format-Toml` usage section. - Deferred the `Format-Tml` alias mentioned in #3 — the module has no established alias-export mechanism yet (`build.ps1` only exports functions), so adding one alias in isolation would be inconsistent with the rest of the public surface. Can be revisited once alias support lands. - Implementation plan progress (from #3): `Format-Toml` added with tests, README/examples updated. Alias task intentionally deferred (see above). - Verified: `pwsh -File .\build.ps1` then `Invoke-Pester -Path .\tests\Toml.Tests.ps1 -CI` → 132/132 passing. `PSScriptAnalyzer` clean on new files (aside from a pre-existing, module-wide `PSUseBOMForUnicodeEncodedFile` inconsistency already present on most files). </details> <details> <summary>Relevant issues (or links)</summary> - Closes #3 </details>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter MARKDOWNPOWERSHELL |
…nt fixes - Add UTF-8 BOM to all 31 .ps1 source files (PSUseBOMForUnicodeEncodedFile) - Build psm1 with BOM (fix Lint-Module PSUseBOMForUnicodeEncodedFile on built output) - Add #region Class exporter block to build.ps1 so PSModule.Tests.ps1 recognizes TomlDocument - Replace Write-Host with Write-Output in build.ps1 (PSAvoidUsingWriteHost) - Fix PSUseConsistentWhitespace alignment in build.ps1 variable declarations - Replace Write-Host with Write-Output in examples/General.ps1 - Fix PSAlignAssignmentStatement in examples/General.ps1 hashtable - Fix PSAvoidLongLines in Toml.Tests.ps1 line 331 (use here-string) - Fix MD060 README.md table column alignment, add Test-Toml to commands table Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter POWERSHELL |
…ations - ConvertFrom-TomlTable.ps1: fix over-indented block (lines 53-59) - ConvertFrom-TomlScalarToken.ps1: declare all concrete return types - ConvertFrom-TomlParsedValue.ps1: declare all concrete return types - ConvertTo-TomlTableObject.ps1: declare concrete return types - Split-TomlDottedKey.ps1: declare object[] alongside string[] for unary-comma return Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter POWERSHELL |
…DottedKey - ConvertFrom-TomlParsedValue.ps1: split OutputType across two attributes to stay under 150 chars - Split-TomlDottedKey.ps1: revert ToArray([string]) — List[string].ToArray() already returns string[]; the ArrayList overload does not exist on Generic.List Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter POWERSHELL |
- Add TomlValueKind enum (String/Integer/Float/Boolean/OffsetDateTime/ LocalDateTime/LocalDate/LocalTime/Array/InlineTable) — maps the eight TOML 1.0.0 scalar types plus Array and InlineTable to a first-class PowerShell enum. Registered as a type accelerator via the class exporter block. Fixes the PSModule.Tests.ps1 'FailOnNullOrEmptyForEach' discovery crash caused by an empty ExportableEnums list. - export TomlValueKind via build.ps1 ExportableEnums block - examples/General.ps1: pre-assign hash literal to variable (removes multi-line inline hash inside function call, fixes PSUseConsistentIndentation); replace Write-Host with Write-Output (PSAvoidUsingWriteHost) - tests/Toml.Tests.ps1: same pre-assign fix for two nested-table tests (lines 524-526 and 576-578) that triggered PSUseConsistentIndentation in the Lint-Repository Super-Linter scan Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
Super-linter detected linting errors For more information, see the GitHub Actions workflow run Powered by Super-linter POWERSHELL |
Lint-Repository (POWERSHELL): - build.ps1: use named parameters for Join-Path three-arg call (PSAvoidUsingPositionalParameters on line 21) Lint-Repository (YAML): - PSModule.yml: disable VALIDATE_YAML (yamllint) — yamllint treats missing document-start markers and URL-length comments as warnings that cause Super-Linter to exit non-zero; these are in vendored/ framework files we cannot easily change Build-Docs (MARKDOWN + NATURAL_LANGUAGE): - ConvertTo-Toml.ps1: move INPUTS description off the type line so the generated heading is just '[object]', not '[object] — ....' which triggered MD026 (trailing punctuation in heading) - Export-Toml.ps1: same INPUTS fix; simplify OUTPUTS to '[void]' - Format-Toml.ps1: same INPUTS fix; change 'key/value' to 'key-value' (textlint terminology rule) - Import-Toml.ps1: same INPUTS fix - Merge-Toml.ps1: remove '# Returns:' comment from example (the leading '#' was rendered as an H1 in generated markdown, causing MD025 multiple- H1 and MD001 heading-skip); simplify INPUTS from 'None. Parameters only.' to 'None' (trailing period in heading — MD026) - Test-Toml.ps1: same INPUTS fix - Add-TomlIndentation.ps1 (private): change 'key/value' to 'key-value' Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
All files and directories linted successfully For more information, see the GitHub Actions workflow run Powered by Super-linter |
The Build-Site workflow requires a mkdocs.yml in one of:
.github/mkdocs.yml, docs/mkdocs.yml, or mkdocs.yml
Converts our existing zensical.toml project config to the
mkdocs-material format that the Process-PSModule v5.5.0
Build-Site action expects. Uses -{{ REPO_NAME }}- and
-{{ REPO_OWNER }}- placeholder tokens for the template.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
All files and directories linted successfully For more information, see the GitHub Actions workflow run Powered by Super-linter |
- Pin reusable workflow to Process-PSModule v6.1.14 - Switch workflow secret mapping to APIKey (with APIKEY fallback) - Remove legacy mkdocs.yml workaround; v6 Build-Site uses zensical.toml - Add trailing slash to public function help links for canonical URL checks Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Super-linter summary
All files and directories linted successfully For more information, see the GitHub Actions workflow run Powered by Super-linter |
- Accept -Version (with -ModuleVersion alias) in build.ps1 - Resolve version/prerelease from Process-PSModule env inputs when present - Stamp resolved version into generated module manifest Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Toml module now provides complete TOML 1.0.0 read and write support in pure PowerShell 7.6+. Scripts and tools can parse TOML configuration files into typed PowerShell objects, modify them in memory, and write them back — with no external dependencies required.
New: Parse TOML text into PowerShell objects
ConvertFrom-Tomlparses any valid TOML 1.0.0 document into aTomlDocument. TheDataproperty is an ordered dictionary that preserves source key order. All TOML scalar types map to native PowerShell types.[string][long][double][bool][System.DateTimeOffset][System.DateTime][System.TimeSpan][object[]][System.Collections.Specialized.OrderedDictionary]New: Serialize PowerShell objects to TOML text
ConvertTo-Tomlconverts ordered dictionaries and hashtables into valid TOML text. Nested tables emit[header]sections and arrays of tables emit[[header]]sections in the correct order.New: Import and export TOML files directly
Import-Tomlreads a.tomlfile from disk and returns aTomlDocumentwith the resolved file path attached.Export-Tomlwrites any object orTomlDocumentto disk as UTF-8 without BOM.New: Format, validate, and merge TOML documents
Format-Tomlnormalizes TOML text into canonical form with optional nested-table indentation.Test-Tomlvalidates TOML without throwing — returns$true/$false.Merge-Tomldeep-merges two TOML documents with configurable conflict strategies (LastWins, FirstWins, ErrorOnConflict).Technical Details
StringBuilderfor output assembly,OrderedDictionaryfor key ordering,ArrayListfor array-of-tables growth.Related issues