Skip to content

Commit 2baba2e

Browse files
committed
docs: address review comments
1 parent 919abfe commit 2baba2e

2 files changed

Lines changed: 36 additions & 2 deletions

File tree

‎docs/user_guidance/entity_relationships.md‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,29 @@ tags:
99
- Rejections
1010
---
1111

12-
Sometimes a user may choose to use the file transformation stage to `normalise` a heavily nested dataset into separate entities during the initial reading of data. This would be done by specifying different entities in the dataset section of the contract configuration in the `dischema` file. This allows for easier interaction when customising errors in the data contract or writing transformations in the business rules. However, if the dataset being processed requires more complex validation, for example removing orphaned records or implementing group rejections, then how to link normalised entities needs to be provided. This can be provided in the `entity_relationships` section of the `dischema`
12+
Sometimes a user may choose to use the file transformation stage to `normalise` a heavily nested dataset into separate entities during the initial reading of data. This would be done by specifying different entities in the dataset section of the contract configuration in the `dischema` file. This allows for easier interaction when customising errors in the data contract or writing transformations in the business rules.
13+
14+
`Normalising` assets can lead to more complex validations being required. For example in the dataset:
15+
16+
```mermaid
17+
erDiagram
18+
COUNTRY ||--o{ AIRPORT : ""
19+
AIRPORT ||--o{ FLIGHT : ""
20+
FLIGHT ||--o{ PASSENGER : ""
21+
AIRPORT ||--o{ STAFF_MEMBER : ""
22+
```
23+
24+
### Missing Parent Records
25+
26+
It could be that an airport record is deemed invalid and removed. Due to this, any flight records that linked to the now removed airport record are themselves invalid - a situation we refer to as a `missing_parent` issue, but are now existing in an entirely different entity.
27+
28+
### No Valid Mandatory Records
29+
30+
It could also be the case that staff records are a mandatory field for airport records. If all staff records for a particular airport record are removed during validation, this itself would invalidate the airport record - a situation we refer to as `no_valid_records` issue - but again the invalid airport record is in a different entity.
31+
32+
### Dischema
33+
34+
In order to perform these validations, how to link normalised entities needs to be provided. This can be specified in the `entity_relationships` section of the `dischema`.
1335

1436
## Entity Relationships Content
1537

@@ -21,12 +43,20 @@ To allow the DVE to link between normalised assets, the following information sh
2143

2244
There is also the functionality to customise errors related to either missing parent or group rejections:
2345

24-
- missing_parent_id_error_code: the error code to display if a record is rejected as it hs no valid parent record
46+
- missing_parent_id_error_code: the error code to display if a record is rejected as it has no valid parent record
2547
- missing_parent_id_error_message: the error message to display if a record is rejected as it hs no valid parent record
2648
- no_valid_records_error_code: the error code to display if parent records are removed due to no valid children in a mandatory field
2749
- no_valid_records_error_message: the error message to display if parent records are removed due to no valid children in a mandatory field
2850

51+
!!! note
52+
For root entities, you don't need to specify entity relationships - this will be inferred based on their absence.
53+
But you may wish to so that error codes and messages can be customised. Ensure that for root entities the parent_entity
54+
abd join_fields values are left blank.
55+
56+
2957
## Entity Hierarchy Object
3058

59+
60+
3161
The details provided in the entity_relationships section of the dischema are used to create an EntityHierarchy object.
3262
Please refer to [Advanced User Guidance: Entity Hierarchy](../advanced_guidance/package_documentation/entity_hierarchy.md).

‎zensical.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ nav = [
2525
{"File Transformation" = "user_guidance/file_transformation.md"},
2626
{"Data Contract" = "user_guidance/data_contract.md"},
2727
{"Business Rules" = "user_guidance/business_rules.md"},
28+
{"Entity Relationships" = "user_guidance/entity_relationships.md"}
2829
]},
2930
{"Backend Implementations" = [
3031
{"DuckDB" = "user_guidance/implementations/duckdb.md"},
@@ -193,6 +194,9 @@ options.custom_icons = ["overrides/.icons"]
193194
auto_append = ["includes/jargon_and_acronyms.md"]
194195

195196
[project.markdown_extensions.pymdownx.superfences]
197+
custom_fences = [
198+
{ name = "mermaid", class = "mermaid", format = "pymdownx.superfences.fence_code_format" },
199+
]
196200

197201
[project.markdown_extensions.pymdownx.tabbed]
198202
alternate_style = true

0 commit comments

Comments
 (0)