Skip to content
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ The module exports:
| `Format-Yaml` | Normalize YAML streams without projecting representation nodes to PowerShell values. |
| `Import-Yaml` | Strictly decode and parse YAML files. |
| `Merge-Yaml` | Merge complete YAML streams without losing representation graph details. |
| `Remove-YamlEntry` | Remove selected entries or documents without losing representation graph details. |
| `Test-Yaml` | Test YAML syntax, tags, duplicate keys, and configured resource limits. |

## Parse YAML
Expand Down Expand Up @@ -217,6 +218,29 @@ creation, charged merge operations, and the resulting stream graph. Index,
fingerprint, candidate, alias-traversal, and equality work all consume the merge
operation budget. Alias and expanded-tag budgets are also enforced on the result.

## Remove YAML entries

`Remove-YamlEntry` removes mapping entries, sequence items, or whole documents directly from a deep-cloned representation graph. Input array elements and pipeline records are joined with LF as one YAML stream, matching `Format-Yaml`.

```powershell
$cleanYaml = Get-Content -Path '.\config.yaml' |
Remove-YamlEntry -Path @('/metadata/internalIdentifier', '/services/1/deprecated')
```

Paths use [RFC 6901 JSON Pointer](https://www.rfc-editor.org/rfc/rfc6901). An empty pointer selects a document root, `~0` addresses a tilde, and `~1` addresses a slash. Mapping tokens match only scalar YAML string keys by ordinal content, so numeric, complex, and unknown-tagged keys are never coerced or guessed. Sequence tokens must be `0` or a non-zero decimal index without signs or leading zeros.

Document zero is selected by default. Use `-DocumentIndex` for another zero-based document, or `-AllDocuments` to apply every path independently to every original document. `-IgnoreMissing` skips only unresolved document/path combinations.

```powershell
$withoutTemporaryData = Remove-YamlEntry $stream '/temporary' `
-AllDocuments -IgnoreMissing -Indent 4
$withoutSecondDocument = Remove-YamlEntry $stream '' -DocumentIndex 1
```

Every required target is resolved before mutation, duplicate logical targets are coalesced, and ancestors subsume descendants. Removing inside a shared mapping or sequence changes every alias to that node; removing an alias edge removes only that edge. Original sequence indexes and document roots are removed in descending order.

Output preserves unaffected tags, anchors, aliases, shared and cyclic identity, complex keys, mapping order, and document order. It is one deterministic string with LF line endings and no final newline; removing every document returns an empty string. Parser limits are mirrored, while cloning, pointer and mutation work, and output validation use independent resource ceilings.

## Export YAML files

`Export-Yaml` aggregates pipeline records like `ConvertTo-Yaml`, serializes the
Expand Down Expand Up @@ -265,6 +289,8 @@ limit violations. Unexpected runtime failures are not suppressed.
YAML 1.2 core schema.
- YAML stream merging compares effective tags and structural representation
values without projecting through PowerShell objects.
- YAML entry removal resolves JSON Pointers against an immutable representation
clone and mutates shared nodes by identity without object projection.
- Parsing defaults to depth 100, 100000 nodes, 1000 aliases, 1048576 decoded
characters per scalar, 1024 characters per expanded tag, 65536 cumulative
expanded tag characters, and 4096 digits per numeric scalar. The
Expand Down
5 changes: 5 additions & 0 deletions examples/General.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@ service:
$mergedYaml = Merge-Yaml -InputObject @($baseYaml, $overlayYaml)
$mergedYaml

# Remove selected YAML entries without projecting the representation graph.
$cleanedYaml = $mergedYaml |
Remove-YamlEntry -Path @('/service/image', '/service/ports/0')
$cleanedYaml

# Atomically export one file, then import it with strict decoding.
$configPath = Join-Path $env:TEMP 'yaml-example.yaml'
$config | Export-Yaml -Path $configPath -PassThru
Expand Down
37 changes: 37 additions & 0 deletions src/functions/private/Add-YamlRemovalWork.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
function Add-YamlRemovalWork {
<#
.SYNOPSIS
Charges deterministic operations to the YAML removal work budget.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory)]
[pscustomobject] $State,

[Parameter()]
[ValidateRange(1, 2147483647)]
[int] $Count = 1,

[Parameter(Mandatory)]
[string] $Operation,

[Parameter()]
[AllowNull()]
[pscustomobject] $Node
)

$State.Count = [long] $State.Count + $Count
if ($State.Count -le $State.MaxNodes) {
return
}

$exception = New-YamlRemovalException -Node $Node `
-ErrorId 'YamlRemovalWorkLimitExceeded' -Message (
"YAML removal operation '$Operation' exceeded the configured invocation work " +
"limit of $($State.MaxNodes) operations."
)
$exception.Data['YamlRemovalWorkCount'] = $State.Count
$exception.Data['YamlRemovalWorkLimit'] = $State.MaxNodes
$exception.Data['YamlRemovalWorkOperation'] = $Operation
throw $exception
}
61 changes: 61 additions & 0 deletions src/functions/private/Assert-YamlNodeKeyUnique.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
function Assert-YamlNodeKeyUnique {
<#
.SYNOPSIS
Indexes one representation key and confirms equality for fingerprint candidates.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory)]
[pscustomobject] $Node,

[Parameter(Mandatory)]
[AllowEmptyCollection()]
[System.Collections.Generic.Dictionary[string, object]] $Buckets,

[Parameter(Mandatory)]
[AllowEmptyCollection()]
[System.Collections.Generic.Dictionary[int, string]] $FingerprintCache,

[Parameter(Mandatory)]
[System.Security.Cryptography.HashAlgorithm] $FingerprintHasher,

[Parameter(Mandatory)]
[string] $DuplicateMessage,

[Parameter()]
[AllowNull()]
[pscustomobject] $RemovalWorkState,

[Parameter()]
[AllowNull()]
[pscustomobject] $EqualityState,

[Parameter()]
[AllowNull()]
[System.Collections.Generic.Dictionary[int, string]] $EqualityFingerprintCache
)

$fingerprint = Get-YamlNodeFingerprint -Node $Node `
-Active ([System.Collections.Generic.HashSet[int]]::new()) `
-Cache $FingerprintCache -Hasher $FingerprintHasher `
-RemovalWorkState $RemovalWorkState
$bucket = $null
if (-not $Buckets.TryGetValue($fingerprint, [ref] $bucket)) {
$bucket = [System.Collections.Generic.List[object]]::new()
$Buckets[$fingerprint] = $bucket
} elseif ($null -eq $EqualityState) {
throw (New-YamlException -Start $Node.Start -End $Node.End `
-ErrorId 'YamlDuplicateKey' -Message $DuplicateMessage)
} else {
foreach ($candidate in $bucket) {
if (Test-YamlMergeNodeEqual -Node $candidate -OtherNode $Node `
-State $EqualityState `
-LeftFingerprintCache $EqualityFingerprintCache `
-RightFingerprintCache $EqualityFingerprintCache) {
throw (New-YamlException -Start $Node.Start -End $Node.End `
-ErrorId 'YamlDuplicateKey' -Message $DuplicateMessage)
}
}
}
$bucket.Add($Node)
}
211 changes: 211 additions & 0 deletions src/functions/private/Assert-YamlRemovalGraph.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
function Assert-YamlRemovalGraph {
<#
.SYNOPSIS
Validates a removed representation graph and its resource budgets.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory)]
[AllowEmptyCollection()]
[object[]] $Documents,

[Parameter(Mandatory)]
[int] $Depth,

[Parameter(Mandatory)]
[int] $MaxNodes,

[Parameter(Mandatory)]
[int] $MaxAliases,

[Parameter(Mandatory)]
[int] $MaxScalarLength,

[Parameter(Mandatory)]
[int] $MaxTagLength,

[Parameter(Mandatory)]
[int] $MaxTotalTagLength,

[Parameter(Mandatory)]
[pscustomobject] $State
)

$visited = [System.Collections.Generic.HashSet[int]]::new()
$pending = [System.Collections.Generic.Stack[object]]::new()
foreach ($document in $Documents) {
$pending.Push([pscustomobject]@{ Node = $document; Depth = 1 })
}
$nodeCount = 0
$aliasCount = 0
$totalTagLength = 0L

while ($pending.Count -gt 0) {
$item = $pending.Pop()
$node = $item.Node
Add-YamlRemovalWork -State $State -Operation 'output validation' -Node $node
if (-not $visited.Add($node.Id)) {
continue
}

$nodeCount++
if ($nodeCount -gt $MaxNodes) {
throw (New-YamlRemovalException -Node $node `
-ErrorId 'YamlRemovalNodeLimitExceeded' -Message (
"The resulting YAML graph exceeds the configured limit of $MaxNodes nodes."
))
}
if ($item.Depth -gt $Depth) {
throw (New-YamlRemovalException -Node $node `
-ErrorId 'YamlRemovalDepthLimitExceeded' -Message (
"The resulting YAML graph exceeds the configured depth of $Depth."
))
}

if ($node.Kind -eq 'Alias') {
$aliasCount++
if ($aliasCount -gt $MaxAliases) {
throw (New-YamlRemovalException -Node $node `
-ErrorId 'YamlRemovalAliasLimitExceeded' -Message (
"The resulting YAML graph exceeds the configured limit of $MaxAliases aliases."
))
}
$pending.Push([pscustomobject]@{ Node = $node.Target; Depth = $item.Depth })
continue
}

$tagLength = ([string] $node.Tag).Length
if ($tagLength -gt $MaxTagLength) {
throw (New-YamlRemovalException -Node $node `
-ErrorId 'YamlRemovalTagLimitExceeded' -Message (
"A tag in the resulting YAML graph exceeds the configured limit of " +
"$MaxTagLength characters."
))
}
$totalTagLength += $tagLength
if ($totalTagLength -gt $MaxTotalTagLength) {
throw (New-YamlRemovalException -Node $node `
-ErrorId 'YamlRemovalTagLimitExceeded' -Message (
"The resulting YAML graph exceeds the configured cumulative tag limit of " +
"$MaxTotalTagLength characters."
))
}

if ($node.Kind -eq 'Scalar') {
if ((Get-YamlRuneCount -Text ([string] $node.Value)) -gt $MaxScalarLength) {
throw (New-YamlRemovalException -Node $node `
-ErrorId 'YamlRemovalScalarLimitExceeded' -Message (
"A scalar in the resulting YAML graph exceeds the configured limit of " +
"$MaxScalarLength characters."
))
}
continue
}

if ($node.Kind -eq 'Sequence') {
if ($node.Tag -cin @(
'tag:yaml.org,2002:omap',
'tag:yaml.org,2002:pairs'
)) {
foreach ($sequenceItem in $node.Items) {
$effectiveItem = Get-YamlRemovalNode -Node $sequenceItem -State $State
if ($effectiveItem.Kind -ne 'Mapping' -or
$effectiveItem.Entries.Count -ne 1) {
throw (New-YamlException -Start $sequenceItem.Start `
-End $sequenceItem.End -ErrorId 'YamlInvalidTaggedCollection' `
-Message (
"YAML tag '$($node.Tag)' requires a sequence of one-entry mappings."
))
}
}
}
for ($index = $node.Items.Count - 1; $index -ge 0; $index--) {
$pending.Push([pscustomobject]@{
Node = $node.Items[$index]
Depth = $item.Depth + 1
})
}
continue
}

if ($node.Tag -ceq 'tag:yaml.org,2002:set') {
foreach ($entry in $node.Entries) {
$effectiveValue = Get-YamlRemovalNode -Node $entry.Value -State $State
$resolvedValue = if ($effectiveValue.Kind -eq 'Scalar') {
(Resolve-YamlScalar -Node $effectiveValue).Value
} else {
[System.Management.Automation.Internal.AutomationNull]::Value
}
if ($effectiveValue.Kind -ne 'Scalar' -or $null -ne $resolvedValue) {
throw (New-YamlException -Start $entry.Value.Start -End $entry.Value.End `
-ErrorId 'YamlInvalidTaggedCollection' -Message (
'Every value in a YAML set must be null.'
))
}
}

}
for ($index = $node.Entries.Count - 1; $index -ge 0; $index--) {
$pending.Push([pscustomobject]@{
Node = $node.Entries[$index].Value
Depth = $item.Depth + 1
})
$pending.Push([pscustomobject]@{
Node = $node.Entries[$index].Key
Depth = $item.Depth + 1
})
}
}

$fingerprintCache = [System.Collections.Generic.Dictionary[int, string]]::new()
$fingerprintHasher = [System.Security.Cryptography.SHA256]::Create()
$fingerprintWorkState = [pscustomobject]@{
Count = 0L
MaxNodes = $MaxNodes
}
$equalityWorkState = [pscustomobject]@{
Count = 0L
MaxNodes = $MaxNodes
}
$equalityState = [pscustomobject]@{
MaxNodes = $MaxNodes
FingerprintHasher = $fingerprintHasher
WorkState = $equalityWorkState
MutationState = [pscustomobject]@{ Version = 0L }
IndexDependents = [System.Collections.Generic.Dictionary[int, object]]::new()
Cache = [System.Collections.Generic.Dictionary[string, bool]]::new(
[System.StringComparer]::Ordinal
)
InputIndex = 0
}
$equalityFingerprintCache = [System.Collections.Generic.Dictionary[int, string]]::new()
try {
foreach ($document in $Documents) {
try {
Test-YamlNodeGraph -Node $document `
-Visited ([System.Collections.Generic.HashSet[int]]::new()) `
-FingerprintCache $fingerprintCache -FingerprintHasher $fingerprintHasher `
-RemovalWorkState $fingerprintWorkState -EqualityState $equalityState `
-EqualityFingerprintCache $equalityFingerprintCache
} catch {
if ($_.Exception.Data.Contains('YamlErrorId') -and
$_.Exception.Data['YamlErrorId'] -ceq 'YamlMergeWorkLimitExceeded') {
$exception = New-YamlRemovalException -Node $document `
-ErrorId 'YamlRemovalWorkLimitExceeded' -Message (
'Post-removal duplicate-key graph comparison exceeded the configured ' +
"invocation work limit of $MaxNodes operations."
)
$exception.Data['YamlRemovalWorkCount'] = $equalityWorkState.Count
$exception.Data['YamlRemovalWorkLimit'] = $MaxNodes
$exception.Data['YamlRemovalWorkOperation'] = (
'duplicate-key graph comparison'
)
throw $exception
}
throw
}
}
} finally {
$fingerprintHasher.Dispose()
}
}
Loading
Loading