数据验证¶

验证方式¶

要生成数据验证摘要，您可以与 验证资源。数据集资源针对数据输入进行了优化 用于验证数据集/表单的客户端，可以像这样访问：

获取 /api/33/validation/dataSet/QX4ZTUbOt3a.json?pe=201501&ou=DiszpKrYNg8

除了基于数据集验证规则外，还有两种 执行验证的其他方法：自定义验证和 预定验证。

第一个路径变量是引用数据集的标识符 证实。支持 XML 和 JSON 资源表示。这 响应包含违反验证规则。这将延长 在即将到来的版本中有更多的验证类型。

要检索与特定数据集相关的验证规则， 意思是所有数据元素都是一部分的带有公式的验证规则 的特定数据集，您可以向 validationRules 资源如下：

GET /api/validationRules?dataSet=<dataset-id>

验证规则有左边和右边，也就是 根据运营商比较有效性。有效的运算符 值见下表。

表：运算符

左边和右边的表达式是数学表达式 其中可以包含对数据元素和类别选项的引用 以下格式的组合：

$ {<dataelement-id>。 <catoptcombo-id>}

左侧和右侧表达式有一个 missing 值 战略。这是指系统应该如何处理数据值 缺少数据元素/类别选项组合引用 在公式中是否应该检查验证规则 为有效性或跳过。有效的缺失值策略见于 下表。

表：缺失值策略

验证结果{ #webapi_validation_results }¶

验证结果是在执行期间发现的违规的持久结果 验证分析。如果您在开始时选择“持久结果”或 安排验证分析，发现的任何违规将存储在 数据库。当结果存储在数据库中时，它将被使用 对于 3 件事：

1. 根据存储的结果生成分析。

2. 未生成通知的持久结果将这样做， 一次。

3. 跟踪结果是否产生了 通知。

4. 跳过运行时已经检查过的规则 验证分析。

这意味着如果你不坚持你的结果，你将无法 为验证结果生成分析，如果选中，结果将 每次找到并运行验证时生成通知 分析可能会更慢。

查询验证结果{ #query-validation-results }¶

持久化的验证结果可以在下面查看 端点：

获取 /api/33/validationResults

您还可以使用验证结果 ID 检查单个结果 在这个端点：

GET /api/33/validationResults/<id>

验证结果也可以通过以下属性过滤：

• 组织单位：ou = <UID>
• 验证规则：vr = <UID>
• 期间：pe = <ISO-expression>

上面的每个过滤器属性可以多次出现，例如：

获取 /api/36/validationResults?ou=jNb63DIHuwU&ou=RzgSFJ9E46G

同一过滤器的多个值与OR组合，结果必须匹配给定值之一。

如果使用了一个以上的过滤器属性，则将它们与AND组合在一起，结果必须与每个属性的值之一匹配。

对于时段过滤器，匹配结果必须与任何指定的时段重叠。

此外，验证结果还可以按其创建日期进行过滤：

GET /api/36/validationResults?createdDate=<date>

该过滤器可以与其他任何过滤器结合使用。

触发验证结果通知{ #trigger-validation-result-notifications }¶

验证结果每天发送给适当的用户一次， 但也可以使用以下命令手动触发以按需运行 API端点：

POST / api / 33 / validation / sendNotifications

使用此端点仅发送未发送的结果。

删除验证结果{ #delete-validation-results }¶

验证结果可以通过ID手动删除，

删除/ api / 36 / validationResults / <id>

或使用过滤器

删除/ api / 36 / validationResults？ <filters>

支持的过滤器参数包括：

• ou = <UID>以匹配组织单位的所有验证结果；提供多个参数时，多个单元组合或
• vr = <UID>以匹配验证规则的所有验证结果；提供多个参数时，多个规则组合或
• pe = <ISO-expression>以匹配与与指定时期重叠的时期相关的所有验证结果
• created = <ISO-expression>以匹配在规定时间内创建的所有验证结果
• notificationSent=<boolean> 仅匹配已发送或未发送通知的验证结果

对于时段过滤器,匹配结果必须与任何指定的时段重叠。

一些例子：

要删除 2020 年第一季度与 UID 为NqwvaQC1ni4的组织单位相关的所有验证结果，请使用：

DELETE /api/36/validationResults?ou=NqwvaQC1ni4&pe=2020Q1

要删除在2019年第1周创建的且已发送通知的所有验证结果，请使用：

DELETE /api/36/validationResults?created=2019W1&notificationSent=true

任何删除操作都需要_执行维护任务_权限。

离群值检测¶

异常值检测端点允许检测聚合数据值中的异常值。

GET / api / 36 / outlierDetection

该端点支持两种用于检测离群值的算法：

• ** Z分数：** Z分数定义为分数与平均值之间的绝对偏差除以标准偏差。必须使用z分数算法指定一个阈值参数，该阈值参数表示与平均值之间的标准偏差，以定义异常值的上限和下限。
• 修改后的 Z 分数： 与 z 分数相同，只是它使用中位数而不是均值作为集中趋势的度量。参数与 Z 分数相同。
• Supported filter parameters include:
• 无效数值： 该算法可检测无法转换为数值的数据值。这种情况通常发生在输入的数据值位数过多时。

ou = <UID>以匹配组织单位的所有验证结果;提供多个参数时,多个单元组合或

vr = <UID>以匹配验证规则的所有验证结果;提供多个参数时,多个规则组合或¶

Start date for the timespan

| eventCreatedPeriod | Parameter name | Accept: application/json | Attribute identifiers | | --------------- | ------------------------------------------------------------ | --------- | ----------------------------------------- | | Table: Query parameters | DELETE /api/36/validationResults?ou=NqwvaQC1ni4&pe=2020Q1 | Follow-up data analysis | 要基于最小/最大值来识别数据离群值: | | Any delete operation will require the authority Perform maintenance tasks. | 支持的查询参数等于基于 std dev 的异常值 上面描述的分析*资源。 | Follow-up data analysis | 组合维度。相反,响应将包含项目 | | dueDate | At least one data set or data element, start date and end date or period, and at least one organisation unit must be defined. | json { "name": "Case notification", "notificationTrigger": "ENROLLMENT", "subjectTemplate": "Case notification V{org_unit_name}", "displaySubjectTemplate": "Case notification V{org_unit_name}", "notifyUsersInHierarchyOnly": false, "sendRepeatable": false, "notificationRecipient": "ORGANISATION_UNIT_CONTACT", "notifyParentOrganisationUnitOnly": false, "displayMessageTemplate": "Case notification A{h5FuguPFF2j}", "messageTemplate": "Case notification A{h5FuguPFF2j}", "deliveryChannels": [ "EMAIL" ] } | Parameter | | lastUpdatedDate | 请求查询参数 { #request-query-parameters } | json { "name": "Case notification", "notificationTrigger": "ENROLLMENT", "subjectTemplate": "Case notification V{org_unit_name}", "displaySubjectTemplate": "Case notification V{org_unit_name}", "notifyUsersInHierarchyOnly": false, "sendRepeatable": false, "notificationRecipient": "ORGANISATION_UNIT_CONTACT", "notifyParentOrganisationUnitOnly": false, "displayMessageTemplate": "Case notification A{h5FuguPFF2j}", "messageTemplate": "Case notification A{h5FuguPFF2j}", "deliveryChannels": [ "EMAIL" ] } | Parameter | | Filter on whether the current user can manage the returned users through the managed user group relationships. | 强制的 | json { "name": "Case notification", "notificationTrigger": "ENROLLMENT", "subjectTemplate": "Case notification V{org_unit_name}", "displaySubjectTemplate": "Case notification V{org_unit_name}", "notifyUsersInHierarchyOnly": false, "sendRepeatable": false, "notificationRecipient": "ORGANISATION_UNIT_CONTACT", "notifyParentOrganisationUnitOnly": false, "displayMessageTemplate": "Case notification A{h5FuguPFF2j}", "messageTemplate": "Case notification A{h5FuguPFF2j}", "deliveryChannels": [ "EMAIL" ] } | GET / api / 36 / outlierDetection?ds = BfMAe6Itzgt&ds = QX4ZTUbOt3a &ou = O6uvpzGd5pu&ou = fdc6uOvgoji&startDate = 2020-01-01&endDate = 2020-12-31 &algorithm = MIN_MAX | | Date used as basis for relative periods. | 不 [] | Program notification template | z_score、min_max、mod_z_score、invalid_numeric`。 | | Organisation unit, organisation unit level or groups (can be combined) | 不 [] | Program notification template | 开始日期 | | 间隔的开始日期,以检查异常值。 | 是的 | Program notification template | Parameter | | 检查异常值的时间间隔的结束日期。 | 是的 | Program notification template | Parameter | | End date for interval to check for outliers | 是的| Program notification template | 算法 | | Sort the records on the value column | 不 [] | Program notification template | 临界点 |

Threshold for outlier values. Z_SCORE and MOD_Z_SCORE algorithm only.

不

数值,大于零。默认值:3.0。

dataStartDate¶

Start date for interval for mean and std dev calculation. Z_SCORE and MOD_Z_SCORE algorithm only.

不

日期(yyyy-MM-dd)。

dataEndDate

End date for interval for mean and std dev calculation. Z_SCORE and MOD_Z_SCORE algorithm only.

不

日期(yyyy-MM-dd)。

订购

Field to order by. Z_SCORE and MOD_Z_SCOREalgorithm only.

不

MEAN_ABS_DEV,Z_SCORE

maxResults

输出的最大限制。¶

不

GET / api / 36 / outlierDetection?ds = BfMAe6Itzgt&ds = QX4ZTUbOt3a
  &ou = O6uvpzGd5pu&ou = fdc6uOvgoji&startDate = 2020-01-01&endDate = 2020-12-31

| The manual merge is suitable when there are resolvable conflicts or when not all the data needs to be moved during the merge. For example, if an attribute has different values in both tracked entities , the user can specify whether to keep the original value or move over the duplicate's value. Since the manual merge involves the user explicitly requesting to move data, there are some additional checks: | Parameter name | | ---------- | ------------------------------------------------------------ | | Any delete operation will require the authority Perform maintenance tasks. | 组合维度。相反,响应将包含项目 | | 获取前10个离群值: | GET / api / 36 / outlierDetection?ds = BfMAe6Itzgt &ou = O6uvpzGd5pu&startDate = 2020-01-01&endDate = 2020-12-31 &maxResults = 10 | | E7228 | GET / api / 36 / outlierDetection?ds = BfMAe6Itzgt &ou = O6uvpzGd5pu&startDate = 2020-01-01&endDate = 2020-12-31 &dataStartDate = 2018-01-01&dataEndDate = 2020-12-31 | | Filter on whether the current user can manage the returned users through the managed user group relationships. | GET / api / 36 / outlierDetection?ds = BfMAe6Itzgt&ds = QX4ZTUbOt3a &ou = O6uvpzGd5pu&ou = fdc6uOvgoji&startDate = 2020-01-01&endDate = 2020-12-31 &algorithm = MIN_MAX | | 回应格式 { #response-format } | 支持以下响应格式。 | | 数据元素,可以多次指定。 | API格式 | | JSON格式 | / api / 36 / outlierDetection.json或Accept:application / json(默认格式) | | 可选类别组合 | / api / 36 / outlierDetection.csv或接受:application / csv | | 响应包含以下字段: | 领域 | | 日期时间 | 德 | | Response { #webapi_analytics_outlier_detection_response } | 取消命名 | | 数据元素名称。 | 聚乙烯 | | 期间ISO标识符。 | 欧 | | 组织单位标识符。 | ouName | | 组织单位名称。 | 可可 | | 类别选项组合标识符。 | cocName | | Table: Program Stage Query Criteria | 冠捷 |

属性选项组合标识符。

aocName

属性选项组合名称。

价值¶

数据值。

信息¶

E2200

必须至少指定一个数据元素

E2201¶

必须指定开始日期和结束日期

E2202

persist

E2203

| eventCreatedPeriod | Parameter name | Validation rule | 检索和删除项目通知模板 | 仪表板资源将提供仪表板列表。请记住 仪表板对象是共享的,因此列表将受 当前已验证的用户。您可以检索有关一个的更多信息 特定的仪表板,请点击其链接,类似于: | |---|---|---|---| | E2206 | Indicator type | Response format | subjectTemplate | E2208 | | Filter on whether the current user can manage the returned users through the managed user group relationships. | 类别 | Response format |Tracked Entity UIDS | | | dueDate | 要运行验证规则并检索违规: | ds | subjectTemplate | 昨天 | | lastUpdatedDate | 描述 | ds | subjectTemplate | 昨天 | | vrg | Validation rule group | 假 | 假 | subjectTemplate | | 欧 | 组织单位 | 假 | 假 | subjectTemplate | | Sort the records on the value column | Date (yyyy-MM-dd) | 假| subjectTemplate | 500 |

结束日期

End date for the time span

Date (yyyy-MM-dd)¶

假

Today

persist

Whether to persist violations in the system

Sample output:

[{
    "validationRuleId": "kgh54Xb9LSE",
    "validationRuleDescription": "Malaria outbreak",
    "organisationUnitId": "DiszpKrYNg8",
    "organisationUnitDisplayName": "Ngelehun CHC",
    "organisationUnitPath": "/ImspTQPwCqd/O6uvpzGd5pu/YuQRtpLP10I/DiszpKrYNg8",
    "organisationUnitAncestorNames": "Sierra Leone / Bo / Badjia / ",
    "periodId": "201901",
    "periodDisplayName": "January 2019",
    "attributeOptionComboId": "HllvX50cXC0",
    "attributeOptionComboDisplayName": "default",
    "importance": "MEDIUM",
    "leftSideValue": 10.0,
    "operator": ">",
    "rightSideValue": 14.0
}, {
    "validationRuleId": "ZoG4yXZi3c3",
    "validationRuleDescription": "ANC 2 cannot be higher than ANC 1",
    "organisationUnitId": "DiszpKrYNg8",
    "organisationUnitDisplayName": "Ngelehun CHC",
    "organisationUnitPath": "/ImspTQPwCqd/O6uvpzGd5pu/YuQRtpLP10I/DiszpKrYNg8",
    "organisationUnitAncestorNames": "Sierra Leone / Bo / Badjia / ",
    "periodId": "201901",
    "periodDisplayName": "January 2019",
    "attributeOptionComboId": "HllvX50cXC0",
    "attributeOptionComboDisplayName": "default",
    "importance": "MEDIUM",
    "leftSideValue": 22.0,
    "operator": "<=",
    "rightSideValue": 19.0
}]
``` | Response format |
| 根据平均值的标准偏差识别数据异常值
价值: |     GET /api/dataAnalysis/stdDevOutlier | 支持以下查询参数: |

### Table: Standard deviation outlier analysis query parameters

查询参数

描述

选项

### 欧

组织单位

ID

开始日期

Start date for the timespan

| 默认值  | Parameter name                                                  | ```
Accept: application/json
``` | Attribute identifiers                   |
| ---------- | ------------------------------------------------------------ | --------- | ----------------------------------------- |
| Filter on whether the current user can manage the returned users through the managed user group relationships.         | 强制的          | ```json
{
    "name": "Case notification",
    "notificationTrigger": "ENROLLMENT",
    "subjectTemplate": "Case notification V{org_unit_name}",
    "displaySubjectTemplate": "Case notification V{org_unit_name}",
    "notifyUsersInHierarchyOnly": false,
    "sendRepeatable": false,
    "notificationRecipient": "ORGANISATION_UNIT_CONTACT",
    "notifyParentOrganisationUnitOnly": false,
    "displayMessageTemplate": "Case notification A{h5FuguPFF2j}",
    "messageTemplate": "Case notification A{h5FuguPFF2j}",
    "deliveryChannels": [
        "EMAIL"
    ]
}
```       | ```
GET / api / 36 / outlierDetection?ds = BfMAe6Itzgt&ds = QX4ZTUbOt3a
  &ou = O6uvpzGd5pu&ou = fdc6uOvgoji&startDate = 2020-01-01&endDate = 2020-12-31
  &algorithm = MIN_MAX
```             |
| Table: Query parameters         | ```
DELETE /api/36/validationResults?ou=NqwvaQC1ni4&pe=2020Q1
```                   | Follow-up data analysis    | 要基于最小/最大值来识别数据离群值:                      |
| Any delete operation will require the authority _Perform maintenance tasks_.         | 支持的查询参数等于基于 *std dev 的异常值
上面描述的分析*资源。               | Follow-up data analysis    | 组合维度。相反,响应将包含项目                  |
| dueDate  | At least one data set or data element, start date and end date or period, and at least one organisation unit must be defined.               | Follow-up data analysis    | Parameter                        |
| lastUpdatedDate    | 请求查询参数 { #request-query-parameters }                  | Follow-up data analysis    | Parameter                        |
| E7228         | 是的                                               | Follow-up data analysis    | ds                        |
| 数据集,可以多次指定。     | 不 [*]                                                  | Follow-up data analysis    | 德                        |
| 数据元素,可以多次指定。        | 不 [*]     | Program notification template        | 开始日期         |
| Sort the records on the value column | 不 [*]                                    | Program notification template        | 结束日期  |

检查异常值的时间间隔的结束日期。

不 [*]

日期(yyyy-MM-dd)。


## 聚乙烯

ISO period ID.

### 不 [*]
Period ISO ID.

peType

ISO period.

不 [*]

Period ISO string.

可可

| Category option combos, can be specified multiple times. | Parameter name                                                                                                                   |
| -------- |-------------------------------------------------------------------------------------------------------------------------------|
| Category option combo identifier.     | maxResults                                                                                  |
| 输出的最大限制。  | 不 |
| Integer, greater than zero. Default: 50.   | [*]  You must specify either data sets with the `ds` parameter, which will include all data elements in the data sets, _or_ specify data elements with the `de` parameter.
     Equally, either `startDate` and `endDate` _or_ `period` must be specified.                               |
| The `startDate` and `endDate` parameters refer to the time interval for which you want to detect outliers.
If a period `pe` is provided instead the interval start and end is that of the period. | 如果未提供选项组合`coc`,则考虑所有数值类型的数据元素。                                           |

数据的完整性 { #webapi_data_integrity } 

数据管理模块的数据完整性功能通过网络应用项目接口提供。
可通过网络 API 使用。本节介绍如何运行
数据完整性流程和检索结果。具体
详细信息请参阅用户手册。

Listing the available data integrity checks { #webapi_data_integrity_list }

A description of the available checks is returned by a request to:

    GET /api/dataIntegrity

The `name` member of the returned check elements is the identifier used for the
`checks` parameter to declare the set of checks to run.

> **Note**
> 
> Each check will indicate whether it may require significant time and resources to complete with the `isSlow` field. 
> Users should be cautious about running these
> checks on production systems as they could lead to decreased performance. 
> These checks can be run individually, but will 
> not be run unless specifically requested.

Checks are grouped semantically by the `section` member and categorised in 
one of four `severity` levels:

Severity

描述

### INFO

Indicates that this is for information only.

- WARNING
- A warning indicates that this may be a problem, but not necessarily an error. It is however recommended to triage these issues.

SEVERE

An error that should be fixed but which may not necessarily lead to the system not functioning.

CRITICAL

An error that must be fixed and which may lead to end-user error or system crashes.

The available checks can be filtered using the `checks` parameter.

    GET /api/dataIntegrity?checks=<pattern1>,<pattern2>

One or more exact names or patterns using `*` as a wildcard can be provided.

Additional results can be filtered using a `section` parameter.

    GET /api/dataIntegrity?section=Categories

The `section` filter will return all exact matches which have the specified section. 

Furthermore, to filter (select) only checks marked as `isSlow` use `slow=true`,
    GET /api/dataIntegrity?slow=true

或仅过滤(选择)不通过数据库查询执行的检查 
(项目化检查)使用 `programmatic=true`:

    GET /api/dataIntegrity?programmatic=true

The `slow`, `programmatic` and `section` filters can be combined in which case
all conditions must be met.

Running data integrity summaries { #webapi_data_integrity_run_summary }

Since version 2.38, data integrity checks have two levels of specificity: 

a `summary` level that provides an overview of the number of issues

a `details` level that provides a list of issues pointing to individual data integrity violations.

To trigger a summary analysis for a set of checks run:

###     POST /api/dataIntegrity/summary?checks=<name1>,<name2>

这将触发一个异步运行检查的作业。单个检查结果
将在检查完成后立即返回应用项目缓存。

Alternatively the list of checks can also be given as BODY of the POST request.
This can be useful if the list becomes to long to be used in the URL.
 - To fetch the data integrity summary of the triggered check(s) use:
 -     GET /api/dataIntegrity/summary?checks=<name1>,<name2>
 - When the `checks` parameter is omitted, all checks are fetched from the server cache.

The response is a "map" of check results, one for each check that has completed already.
This information is cached for one hour or until the check is rerun.

To wait for the summary to be available in the cache a `timeout` in milliseconds can be added:

    GET /api/dataIntegrity/summary?checks=<name1>,<name2>&timeout=500

### An example of a summary response could look like: 

```json
{
  "<name1>": {
    "name": "<name1>",
    "displayName": "<displayName1>",
    "startTime": "2023-01-11T06:12:56.436",
    "finishedTime": "2023-01-11T06:12:57.021",
    "section": "...",
    "severity": "WARNING",
    "description": "...",
    "count": 12,
    "percentage": 2.3
  },
  "<name2>": {
    "name": "<name2>",
    "displayName": "<displayName2>",
    "startTime": "2023-01-11T06:12:57.345",
    "finishedTime": "2023-01-11T06:12:58.007",
    "section": "...",
    "severity": "WARNING",
    "description": "...",
    "count": 4,
    "percentage": 5.1
  }
}

Each summary response will contain the name, section, severity, description and optionally an introduction and recommendation.
Each summary contains the number of issues found in the count field. When possible, an optional percentage field will provide the percentage of objects with data integrity issues when compared to all objects of the same type. The startTime field indicates when the check was initiated. Using the finishedTime the duration which was required to execute the check can be calculated.

Should a check analysis fail due to programming error or unforeseen data inconsistencies both the summary and the details will have an error field describing the error that occurred. The count of any checks which failed will be set to -1. No percentage will be returned in such cases.

{
  "<name1>": {
    "name": "<name1>",
    "displayName": "<displayName1>",
    "finishedTime": "2022-02-15 14:55",
    "section": "...",
    "severity": "WARNING",
    "description": "...",
    "error": "what has happened",
    "issues": []
  }
}

Note

Each metadata check is run asynchronously on the server. Results will be returned as soon as each check completes. The safest way to ensure that you have retrieved the latest set of results which has been requested is to compare the timestamp of when the request was made with the finishedTime in the response.

To get a list of the names of checks that are currently being performed by the server use:

GET /api/dataIntegrity/summary/running

To get a list of the names of checks for which results are available already use:

GET /api/dataIntegrity/summary/completed

Retrieving completed checks as Prometheus metrics {#webapi_data_integrity_metrics}

Metadata integrity checks which are present in the cache, can be retrieved in the Prometheus metrics format by making a request to : GET /api/dataIntegrity/metrics

The response should return a plain text format in the Prometheus plain text exposition format. Several metrics are available for each data integrity check.

Count: A count of the number of issues identified by the metadata check.

Percentage: When available, provides a percentage of the objects which have been identified by the check relative to a baseline. For instance, if check "Organisation units with trailing spaces" has a percent of 2.13, the percentage is calculated by dividing the number of organisation units with trailing spaces by the total number of organisation units. Note that this percentage may not be available for all metadata integrity checks.

Duration: Number of milliseconds that the check took to execute the last time it was run.¶

An example of the output of this endpoint is provided below:

# HELP dhis_data_integrity_check_count Data integrity check counts { #help-dhis_data_integrity_check_count-data-integrity-check-counts } 
# TYPE dhis_data_integrity_check_count gauge { #type-dhis_data_integrity_check_count-gauge } 
dhis_data_integrity_check_count{check="orgunits_invalid_geometry"} 1
dhis_data_integrity_check_count{check="user_groups_scarce"} 0
dhis_data_integrity_check_count{check="indicator_no_analysis"} 13
# HELP dhis_data_integrity_check_percentage Data integrity check percentages { #help-dhis_data_integrity_check_percentage-data-integrity-check-percentages } 
# TYPE dhis_data_integrity_check_percentage gauge { #type-dhis_data_integrity_check_percentage-gauge } 
dhis_data_integrity_check_percentage{check="orgunits_invalid_geometry"} 0.13054830287206268
dhis_data_integrity_check_percentage{check="user_groups_scarce"} 0.0
dhis_data_integrity_check_percentage{check="indicator_no_analysis"} 16.0
# HELP dhis_data_integrity_check_duration Data integrity check durations { #help-dhis_data_integrity_check_duration-data-integrity-check-durations } 
# TYPE dhis_data_integrity_check_duration gauge { #type-dhis_data_integrity_check_duration-gauge } 
dhis_data_integrity_check_duration{check="orgunits_invalid_geometry"} 11
dhis_data_integrity_check_duration{check="user_groups_scarce"} 1
dhis_data_integrity_check_duration{check="indicator_no_analysis"} 0

Data integrity checks which are not currently in the cache will not be returned by this endpoint. A request would need to be made to the /summary endpoint to trigger the checks to be run or alternatively through a scheduled job.

Running data integrity details { #webapi_data_integrity_run_details } - To run a selection of details checks first trigger them using a POST request: POST /api/dataIntegrity/details?checks=, - Similar to the summary the list of checks can also be given as the POST body. - Then fetch the results from the cache using:

GET /api/dataIntegrity/details?checks=,&timeout=500¶

When the checks parameter is not provided, all checks which have not been marked as isSlow will be scheduled to be run on the server.

peType

The /details response returns a map similar to the summary, but does not contain a count or percentage. Instead, a list of issues is returned.

{
  "<name1>": {
    "name": "<name1>",
    "displayName": "<displayName1>",
    "startTime": "2023-01-11T06:12:56.436",
    "finishedTime": "2023-01-11T06:12:57.021",
    "section": "...",
    "severity": "WARNING",
    "description": "...",
    "issuesIdType": "<object-type-plural>",
    "isSlow": false,
    "issues": [{
      "id": "<id-or-other-identifier>",
      "name": "<name-of-the-id-obj>",
      "comment": "optional plain text description or hint of the issue",
      "refs": ["<id1>", "<id2>"]
    }]
  },
  "<name2>": {
    "name": "<name2>",
    "displayName": "<displayName2>",
    "startTime": "2023-01-11T06:12:57.345",
    "finishedTime": "2023-01-11T06:12:58.007",
    "section": "...",
    "severity": "WARNING",
    "description": "...",
    "issuesIdType": "<object-type-plural>",
    "isSlow": false,
    "issues": []
  }
}

Each issue will always have id and name members. Often the issuesIdType is available to indicate the type of objects the id refers to. If the issuesIdType is not available, the id often is not available either and the name is used for an aggregate key of an issue that has no object equivalent.

The comment and refs fields are optional for each issue. A comment may provide more context or insight into why this particular issue is regarded to be a data integrity problem. The refs list may also give the identifiers of other objects that contributed to the violation. The finishedTime field shows when the particular check finished processing on the server. The cache will store the result of each completed check for one hour.

Tip

A set of checks can also be specified using wild-cards. To include all checks with element in the name use checks=*element*. Like full names such patterns can be used in a comma-separated list and be mixed with full names as well. Duplicates will be eliminated. Also a check can be given by its code. A code consists of the first letters of each word in the name as upper case letter. For example, orgunits_invalid_geometry has the code OIG.

Similar to the summary a set of names of the currently performed and the already completed details checks can be obtained using:

GET /api/dataIntegrity/details/running
GET /api/dataIntegrity/details/completed

Custom Data Integrity Checks¶

Users of DHIS2 can now create and supply their own Data Integrity Checks. This can be useful if users want to avail of this functionality and extend upon the supplied set of core data integrity checks.

Tip

Users are also encouraged to share their custom checks with others by opening a pull request in the dhis2-core repository containing their .yaml file(s). Please select platform-backend as reviewer to put the PR on our radar early on. The team will take care of checking and linking the check correctly, so it becomes part of the provided suite of checks with the next release.

An example of a custom check could be for determining if certain users are members of specific user groups. This type of check would be very specific to an implementation, and not generally applicable across all installs. These types of metadata checks can be used to extend the default checks which are included with DHIS2.

Custom checks can be implemented by satisfying the following requirements, each of which we will go into detail:

08:29:57.729  WARN o.h.d.d.DataIntegrityYamlReader: Failed to load data integrity check from YAML. Error message `{DHIS2_HOME}/custom-data-integrity-checks.yaml (No such file or directory)

The custom-data-integrity-checks.yaml file should list all of your custom data integrity checks. As an example, it could look something like this:

checks:
  - categories/my_custom_check.yaml
  - users/my_user_group_check.yaml
  - base_check.yaml

Check names in this file can be preceded with a directory name for logical grouping. From the 3 example checks listed¶

above, the directory structure should look like this:

├── DHIS2_HOME
│   ├── dhis.conf
│   ├── custom-data-integrity-checks.yaml
│   ├── custom-data-integrity-checks
│   │   ├── categories
│   │   │   ├── my_custom_check.yaml
│   │   ├── users
│   │   │   ├── my_user_group_check.yaml
│   │   ├── base_check.yaml

Each data integrity check name and code must be unique. If there are any clashes then the violating custom check will not be loaded.

Note

System data integrity checks are always loaded first. Any name or code clashes resulting from custom checks will not affect these core system checks. An example data integrity check yaml file is located here for reference. Note the name property.

The data integrity code is calculated dynamically by using the first letter of each word in the name. Some examples:

名称

码¶

my_custom_check

the order of the check when displayed in the UI¶

summary_sql

yes

an SQL query which should return a single result which represents the total count of issues

details_sql¶

yes

an SQL query which should return a list of identified objects from this particular issue. Should return at least uid and name¶

details_id_type

yes

a short string which identifies the section of the details SQL

severity

yes

level of severity of the issue. One of [INFO, WARNING, SEVERE, CRITICAL]

introduction

yes

Property of the attribute option combos to use to map the complete registrations.

方案 id | name | code | attribute:ID

Property of all objects including data sets, org units and attribute option combos, to use to map the complete registrations.

preheatCache¶

false | true

Whether to save changes on the server or just return the import summary.

createdDuration

JSON 格式的有效载荷如下所示,其中attributes指元数据属性,groupSets指组织单位组集,dataItems指数据元素、指标、数据集和计划指标:

orgUnit

Organisation unit identifier, can be specified multiple times. Not applicable if orgUnitGroup is given.

orgUnitGroup¶

Organisation unit group identifier, can be specified multiple times. Not applicable if orgUnit is given.

yes

Whether to include the children in the hierarchy of the organisation units

yes