You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/user_guidance/entity_relationships.md
+32-2Lines changed: 32 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,29 @@ tags:
9
9
- Rejections
10
10
---
11
11
12
-
Sometimes a user may choose to use the file transformation stage to `normalise` a heavily nested dataset into separate entities during the initial reading of data. This would be done by specifying different entities in the dataset section of the contract configuration in the `dischema` file. This allows for easier interaction when customising errors in the data contract or writing transformations in the business rules. However, if the dataset being processed requires more complex validation, for example removing orphaned records or implementing group rejections, then how to link normalised entities needs to be provided. This can be provided in the `entity_relationships` section of the `dischema`
12
+
Sometimes a user may choose to use the file transformation stage to `normalise` a heavily nested dataset into separate entities during the initial reading of data. This would be done by specifying different entities in the dataset section of the contract configuration in the `dischema` file. This allows for easier interaction when customising errors in the data contract or writing transformations in the business rules.
13
+
14
+
`Normalising` assets can lead to more complex validations being required. For example in the dataset:
15
+
16
+
```mermaid
17
+
erDiagram
18
+
COUNTRY ||--o{ AIRPORT : ""
19
+
AIRPORT ||--o{ FLIGHT : ""
20
+
FLIGHT ||--o{ PASSENGER : ""
21
+
AIRPORT ||--o{ STAFF_MEMBER : ""
22
+
```
23
+
24
+
### Missing Parent Records
25
+
26
+
It could be that an airport record is deemed invalid and removed. Due to this, any flight records that linked to the now removed airport record are themselves invalid - a situation we refer to as a `missing_parent` issue, but are now existing in an entirely different entity.
27
+
28
+
### No Valid Mandatory Records
29
+
30
+
It could also be the case that staff records are a mandatory field for airport records. If all staff records for a particular airport record are removed during validation, this itself would invalidate the airport record - a situation we refer to as `no_valid_records` issue - but again the invalid airport record is in a different entity.
31
+
32
+
### Dischema
33
+
34
+
In order to perform these validations, how to link normalised entities needs to be provided. This can be specified in the `entity_relationships` section of the `dischema`.
13
35
14
36
## Entity Relationships Content
15
37
@@ -21,12 +43,20 @@ To allow the DVE to link between normalised assets, the following information sh
21
43
22
44
There is also the functionality to customise errors related to either missing parent or group rejections:
23
45
24
-
- missing_parent_id_error_code: the error code to display if a record is rejected as it hs no valid parent record
46
+
- missing_parent_id_error_code: the error code to display if a record is rejected as it has no valid parent record
25
47
- missing_parent_id_error_message: the error message to display if a record is rejected as it hs no valid parent record
26
48
- no_valid_records_error_code: the error code to display if parent records are removed due to no valid children in a mandatory field
27
49
- no_valid_records_error_message: the error message to display if parent records are removed due to no valid children in a mandatory field
28
50
51
+
!!! note
52
+
For root entities, you don't need to specify entity relationships - this will be inferred based on their absence.
53
+
But you may wish to so that error codes and messages can be customised. Ensure that for root entities the parent_entity
54
+
abd join_fields values are left blank.
55
+
56
+
29
57
## Entity Hierarchy Object
30
58
59
+
60
+
31
61
The details provided in the entity_relationships section of the dischema are used to create an EntityHierarchy object.
32
62
Please refer to [Advanced User Guidance: Entity Hierarchy](../advanced_guidance/package_documentation/entity_hierarchy.md).
0 commit comments