1.17. Data values

This section is about sending and reading data values.

/api/26/dataValueSets

1.17.1. Sending data values

A common use-case for system integration is the need to send a set of data values from a third-party system into DHIS. In this example we will use the DHIS2 demo on http://play.dhis2.org/demo as basis and we recommend that you follow the provided links with a web browser while reading (log in with admin/district as username/password). We assume that we have collected case-based data using a simple software client running on mobile phones for the Mortality <5 years data set in the community of Ngelehun CHC (in Badjia chiefdom, Bo district) for the month of January 2014. We have now aggregated our data into a statistical report and want to send that data to the national DHIS2 instance.

The resource which is most appropriate for our purpose of sending data values is the dataValueSets resource. A data value set represents a set of data values which have a logical relationship, usually from being captured off the same data entry form. We follow the link to the HTML representation which will take us to http://play.dhis2.org/demo/api/24/dataValueSets. The format looks like this:

<dataValueSet xmlns="http://dhis2.org/schema/dxf/2.0" dataSet="dataSetID"
  completeDate="date" period="period" orgUnit="orgUnitID" attributeOptionCombo="aocID">
  <dataValue dataElement="dataElementID" categoryOptionCombo="cocID" value="1" comment="comment1"/>
  <dataValue dataElement="dataElementID" categoryOptionCombo="cocID" value="2" comment="comment2"/>
  <dataValue dataElement="dataElementID" categoryOptionCombo="cocID" value="3" comment="comment3"/>
</dataValueSet>

JSON is supported in this format:

{
  "dataSet": "dataSetID",
  "completeDate": "date",
  "period": "period",
  "orgUnit": "orgUnitID",
  "attributeOptionCombo", "aocID",
  "dataValues": [
    { "dataElement": "dataElementID", "categoryOptionCombo": "cocID", "value": "1", "comment": "comment1" },
    { "dataElement": "dataElementID", "categoryOptionCombo": "cocID", "value": "2", "comment": "comment2" },
    { "dataElement": "dataElementID", "categoryOptionCombo": "cocID", "value": "3", "comment": "comment3" }
  ]
}

CSV is supported in this format:

"dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"
"dataElementID","period","orgUnitID","cocID","aocID","1","username","2015-04-01","comment1"
"dataElementID","period","orgUnitID","cocID","aocID","2","username","2015-04-01","comment2"
"dataElementID","period","orgUnitID","cocID","aocID","3","username","2015-04-01","comment3"

Note: Please refer to the date and period section above for time formats.

From the example we can see that we need to identify the period, the data set, the org unit (facility) and the data elements for which to report.

To obtain the identifier for the data set we return to the entry point at http://play.dhis2.org/demo/api/24 and follow the embedded link pointing at the dataSets resource located at http://play.dhis2.org/demo/api/24/dataSets. From there we find and follow the link to the Mortality < 5 years data set which leads us to http://play.dhis2.org/demo/api/24/dataSets/pBOMPrpg1QX. The resource representation for the Mortality < 5 years data set conveniently advertises links to the data elements which are members of it. From here we can follow these links and obtain the identifiers of the data elements. For brevity we will only report on three data elements: Measles with id f7n9E0hX8qk, Dysentery with id Ix2HsbDMLea and Cholera with id eY5ehpbEsB7.

What remains is to get hold of the identifier of the facility (org unit). The dataSet representation conveniently provides link to org units which report on it so we search for Ngelehun CHC and follow the link to the HTML representation at http://play.dhis2.org/demo/api/24/organisationUnits/DiszpKrYNg8, which tells us that the identifier of this org unit is DiszpKrYNg8.

From our case-based data we assume that we have 12 cases of measles, 14 cases of dysentery and 16 cases of cholera. We have now gathered enough information to be able to put together the XML data value set message:

<dataValueSet xmlns="http://dhis2.org/schema/dxf/2.0" dataSet="pBOMPrpg1QX"
  completeDate="2014-02-03" period="201401" orgUnit="DiszpKrYNg8">
  <dataValue dataElement="f7n9E0hX8qk" value="12"/>
  <dataValue dataElement="Ix2HsbDMLea" value="14"/>
  <dataValue dataElement="eY5ehpbEsB7" value="16"/>
</dataValueSet>

In JSON format:

{
  "dataSet": "pBOMPrpg1QX",
  "completeData": "2014-02-03",
  "period": "201401",
  "orgUnit": "DiszpKrYNg8",
  "dataValues": [
    { "dataElement": "f7n9E0hX8qk", "value": "1" },
    { "dataElement": "Ix2HsbDMLea", "value": "2" },
    { "dataElement": "eY5ehpbEsB7", "value": "3" }
  ]
}

To perform functional testing we will use the cURL tool which provides an easy way of transferring data using HTTP. First we save the data value set XML content in a file called datavalueset.xml . From the directory where this file resides we invoke the following from the command line:

curl -d @datavalueset.xml "https://play.dhis2.org/demo/api/26/dataValueSets" 
  -H "Content-Type:application/xml" -u admin:district -v

For sending JSON content you must set the content-type header accordingly:

curl -d @datavalueset.json "https://play.dhis2.org/demo/api/26/dataValueSets" 
  -H "Content-Type:application/json" -u admin:district -v

The command will dispatch a request to the demo Web API, set application/xml as the content-type and authenticate using admin/district as username/password. If all goes well this will return a 200 OK HTTP status code. You can verify that the data has been received by opening the data entry module in DHIS2 and select the org unit, data set and period used in this example.

The API follows normal semantics for error handling and HTTP status codes. If you supply an invalid username or password, 401 Unauthorized is returned. If you supply a content-type other than application/xml, 415 Unsupported Media Type is returned. If the XML content is invalid according to the DXF namespace, 400 Bad Request is returned. If you provide an invalid identifier in the XML content, 409 Conflict is returned together with a descriptive message.

1.17.2. Sending bulks of data values

The previous example showed us how to send a set of related data values sharing the same period and organisation unit. This example will show us how to send large bulks of data values which don't necessarily are logically related.

Again we will interact with the with http://play.dhis2.org/demo/api/24/dataValueSets resource. This time we will not specify the dataSet and completeDate attributes. Also, we will specify the period and orgUnit attributes on the individual data value elements instead of on the outer data value set element. This will enable us to send data values for various periods and org units:

<dataValueSet xmlns="http://dhis2.org/schema/dxf/2.0">
  <dataValue dataElement="f7n9E0hX8qk" period="201401" orgUnit="DiszpKrYNg8" value="12"/>
  <dataValue dataElement="f7n9E0hX8qk" period="201401" orgUnit="FNnj3jKGS7i" value="14"/>
  <dataValue dataElement="f7n9E0hX8qk" period="201402" orgUnit="DiszpKrYNg8" value="16"/>
  <dataValue dataElement="f7n9E0hX8qk" period="201402" orgUnit="Jkhdsf8sdf4" value="18"/>
</dataValueSet>

In JSON format:

{
  "dataValues": [
    { "dataElement": "f7n9E0hX8qk", "period": "201401", "orgUnit": "DiszpKrYNg8", "value": "12" },
    { "dataElement": "f7n9E0hX8qk", "period": "201401", "orgUnit": "FNnj3jKGS7i", "value": "14" },
    { "dataElement": "f7n9E0hX8qk", "period": "201402", "orgUnit": "DiszpKrYNg8", "value": "16" },
    { "dataElement": "f7n9E0hX8qk", "period": "201402", "orgUnit": "Jkhdsf8sdf4", "value": "18" }
  ]
}

In CSV format:

"dataelement","period","orgunit","categoryoptioncombo","attributeoptioncombo","value"
"f7n9E0hX8qk","201401","DiszpKrYNg8","bRowv6yZOF2","bRowv6yZOF2","1"
"Ix2HsbDMLea","201401","DiszpKrYNg8","bRowv6yZOF2","bRowv6yZOF2","2"
"eY5ehpbEsB7","201401","DiszpKrYNg8","bRowv6yZOF2","bRowv6yZOF2","3"

We test by using cURL to send the data values in XML format:

curl -d @datavalueset.xml "https://play.dhis2.org/demo/api/26/dataValueSets" 
  -H "Content-Type:application/xml" -u admin:district -v

Note that when using CSV format you must use the binary data option to preserve the line-breaks in the CSV file:

curl --data-binary @datavalueset.csv "https://play.dhis2.org/demo/24/api/dataValueSets" 
  -H "Content-Type:application/csv" -u admin:district -v

The data value set resource provides an XML response which is useful when you want to verify the impact your request had. The first time we send the data value set request above the server will respond with the following import summary:

<importSummary>
  <dataValueCount imported="2" updated="1" ignored="1"/>
  <dataSetComplete>false</dataSetComplete>
</importSummary>

This message tells us that 3 data values were imported, 1 data value was updated while zero data values were ignored. The single update comes as a result of us sending that data value in the previous example. A data value will be ignored if it references a non-existing data element, period, org unit or data set. In our case this single ignored value was caused by the last data value having an invalid reference to org unit. The data set complete element will display the date of which the data value set was completed, or false if no data element attribute was supplied.

1.17.3. Import parameters

The import process can be customized using a set of import parameters:

Table 1.26. Import parameters

ParameterValues (default first)Description
dataElementIdSchemeid | name | code | attribute:IDProperty of the data element object to use to map the data values.
orgUnitIdSchemeid | name | code | attribute:IDProperty of the org unit object to use to map the data values.
categoryOptionComboIdSchemeid | name | code | attribute:IDProperty of the category option combo and attribute option combo objects to use to map the data values.
idSchemeid | name | code| attribute:IDProperty of all objects including data elements, org units and category option combos, to use to map the data values.
preheatCachefalse | trueIndicates whether to preload metadata caches before starting to import data values, will speed up large import payloads with high metadata cardinality.
dryRunfalse | trueWhether to save changes on the server or just return the import summary.
importStrategyCREATE | UPDATE | CREATE_AND_UPDATE | DELETESave objects of all, new or update import status on the server.
skipExistingCheckfalse | trueSkip checks for existing data values. Improves performance. Only use for empty databases or when the data values to import do not exist already.
asyncfalse | trueIndicates whether the import should be done asynchronous or synchronous. The former is suitable for very large imports as it ensures that the request does not time out, although it has a significant performance overhead. The latter is faster but requires the connection to persist until the process is finished.

All parameters are optional and can be supplied as query parameters in the request URL like this:

/api/26/dataValueSets?dataElementIdScheme=code&orgUnitIdScheme=name&dryRun=true&importStrategy=CREATE

They can also be supplied as XML attributes on the data value set element like below. XML attributes will override query string parameters.

<dataValueSet xmlns="http://dhis2.org/schema/dxf/2.0" dataElementIdScheme="code"
  orgUnitIdScheme="name" dryRun="true" importStrategy="CREATE">
  ..
</dataValueSet>

Note that the preheatCache parameter can have huge impact for performance. For small import files, leaving it to false will be fast. For large import files which contain a large number of distint data elements and organisation units, setting it to true will be orders of magnitude faster.

1.17.3.1. Identifier schemes

Regarding the id schemes, by default the identifiers used in the XML messages uses the DHIS2 stable object identifiers referred to as uid. In certain interoperability situations we might experience that external system decides the identifiers of the objects. In that case we can use the code property of the organisation units and other objects to set fixed identifiers. When importing data values we hence need to reference the code property instead of the identfier property of these metadata objects. Identifier schemes can be specified in the XML message as well as in the request as query parameters. To specify it in the XML payload you can do this:

<dataValueSet xmlns="http://dhis2.org/schema/dxf/2.0" 
  dataElementIdScheme="CODE" orgUnitIdScheme="UID" idScheme="CODE">
  ..
</dataValueSet>

The parameter table above explains how the id schemes can be specified as query parameters. The following rules apply for what takes precedence:

  • Id schemes defined in the XML or JSON payload take precedence over id schemes defined as URL query parameters.

  • Specific id schemes including dataElementIdScheme and orgUnitIdScheme take precedence over the general idScheme.

  • The default id scheme is UID, which will be used if no explicit id scheme is defined.

The following identifier schemes are availabe.

  • uid (default)

  • code

  • name

  • attribute (followed by UID of attribute)

The attribute option is special and refers to meta-data attributes which have been marked as "unique". When using this option, "attribute" must be immediately followed by the uid of the attribute, e.g. "attributeDnrLSdo4hMl".

1.17.4. CSV data value format

The following section describes the CSV format used in DHIS2. The first row is assumed to be a header row and will be ignored during import.

Table 1.27. CSV format of DHIS2

ColumnRequiredDescription
Data elementYesRefers to ID by default, can also be name and code based on selected id scheme
PeriodYesIn ISO format
Org unitYesRefers to ID by default, can also be name and code based on selected id scheme
Category option comboNoRefers to ID
Attribute option comboNoRefers to ID (from version 2.16)
ValueNoData value
Stored byNoRefers to username of user who entered the value
Last updatedNoDate in ISO format
CommentNoFree text comment
Follow upNotrue or false


An example of a CSV file which can be imported into DHIS2 is seen below.

"dataelement","period","orgunit","categoryoptioncombo","attroptioncombo","value","storedby","timestamp"
"DUSpd8Jq3M7","201202","gP6hn503KUX","Prlt0C1RF0s",,"7","bombali","2010-04-17"
"DUSpd8Jq3M7","201202","gP6hn503KUX","V6L425pT3A0",,"10","bombali","2010-04-17"
"DUSpd8Jq3M7","201202","OjTS752GbZE","V6L425pT3A0",,"9","bombali","2010-04-06"

1.17.5. Generating data value set template

To generate a data value set template for a certain data set you can use the /api/dataSets/<id>/dataValueSet resource. XML and JSON response formats are supported. Example:

/api/26/dataSets/BfMAe6Itzgt/dataValueSet.json

The parameters you can use to further adjust the output are described below:

Table 1.28. Data values query parameters

Query parameterRequiredDescription
periodNoPeriod to use, will be included without any checks.
orgUnitNoOrganisation unit to use, supports multiple orgUnits, both id and code can be used.
commentNoShould comments be include, default: Yes.
orgUnitIdSchemeNoOrganisation unit scheme to use, supports id | code.
dataElementIdSchemeNoData-element scheme to use, supports id | code.

1.17.6. Reading data values

This section explains how to retrieve data values from the Web API by interacting with the dataValueSets resource. Data values can be retrieved in XML, JSON and CSV format. Since we want to read data we will use the GET HTTP verb. We will also specify that we are interested in the XML resource representation by including an Accept HTTP header with our request. The following query parameters are required:

Table 1.29. Data value set query parameters

ParameterDescription
dataSetData set identifier. Can be repeated any number of times.
dataElementGroupData element group identifier. Can be repeated any number of times.
periodPeriod identifier in ISO format. Can be repeated any number of times.
startDateStart date for the time span of the values to export.
endDateEnd date for the time span of the values to export.
orgUnitOrganisation unit identifier. Can be repeated any number of times.
childrenWhether to include the children in the hierarchy of the organisation units.
orgUnitGroupOrganisation unit group identifier. Can be repeated any number of times.
attributeOptionComboAttribute option combo identifier. Can be repeated any number of times.
includeDeletedWhether to include deleted data values.
lastUpdatedInclude only data values which are updated since the given time stamp.
lastUpdatedDurationInclude only data values which are updated within the given duration. The format is <value><time-unit>, where the supported time units are "d" (days), "h" (hours), "m" (minutes) and "s" (seconds).
limitThe max number of results in the response.
idSchemeProperty of meta data objects to use for data values in response.
dataElementIdSchemeProperty of the data element object to use for data values in response.
orgUnitIdSchemeProperty of the org unit object to use for data values in response.
categoryOptionComboIdSchemeProperty of the category option combo and attribute option combo objects to use for data values in response.
dataSetIdSchemeProperty of the data set object to use in the response.

The following response formats are supported:

  • xml (application/xml)

  • json (application/json)

  • csv (application/csv)

  • adx (application/xml+adx)

Assuming that we have posted data values to DHIS2 according to the previous section called "Sending data values" we can now put together our request for a single data value set and request it using cURL:

curl "https://play.dhis2.org/demo/api/26/dataValueSets?dataSet=pBOMPrpg1QX&period=201401&orgUnit=DiszpKrYNg8"
  -H "Accept:application/xml" -u admin:district -v

We can also use the start and end dates query parameters to request a larger bulk of data values. I.e. you can also request data values for multiple data sets and org units and a time span in order to export larger chunks of data. Note that the period query parameter takes presedence over the start and end date parameters. An exampe looks like this:

curl "https://play.dhis2.org/demo/api/26/dataValueSets?dataSet=pBOMPrpg1QX&dataSet=BfMAe6Itzgt
  &startDate=2013-01-01&endDate=2013-01-31&orgUnit=YuQRtpLP10I&orgUnit=vWbkYPRmKyS&children=true" 
  -H "Accept:application/xml" -u admin:district -v

To retrieve data values which have been created or updated within the last 10 days you can make a request like this:

https://play.dhis2.org/demo/api/26/dataValueSets?dataSet=pBOMPrpg1QX&orgUnit=DiszpKrYNg8&lastUpdatedDuration=10d

The response will look like this:

<?xml version='1.0' encoding='UTF-8'?>
<dataValueSet xmlns="http://dhis2.org/schema/dxf/2.0" dataSet="pBOMPrpg1QX"
  completeDate="2014-01-02" period="201401" orgUnit="DiszpKrYNg8">
<dataValue dataElement="eY5ehpbEsB7" period="201401" orgUnit="DiszpKrYNg8"
  categoryOptionCombo="bRowv6yZOF2" value="10003"/>
<dataValue dataElement="Ix2HsbDMLea" period="201401" orgUnit="DiszpKrYNg8"
  categoryOptionCombo="bRowv6yZOF2" value="10002"/>
<dataValue dataElement="f7n9E0hX8qk" period="201401" orgUnit="DiszpKrYNg8"
  categoryOptionCombo="bRowv6yZOF2" value="10001"/>
</dataValueSet>

You can request the data in JSON format like this:

https://play.dhis2.org/demo/api/26/dataValueSets.json?dataSet=pBOMPrpg1QX&period=201401&orgUnit=DiszpKrYNg8

The response will look something like this:

{
  "dataSet": "pBOMPrpg1QX",
  "completeData": "2014-02-03",
  "period": "201401",
  "orgUnit": "DiszpKrYNg8",
  "dataValues": [
    { "dataElement": "eY5ehpbEsB7", "categoryOptionCombo": "bRowv6yZOF2", "period": "201401",
      "orgUnit": "DiszpKrYNg8", "value": "10003" },
    { "dataElement": "Ix2HsbDMLea", "categoryOptionCombo": "bRowv6yZOF2", "period": "201401",
      "orgUnit": "DiszpKrYNg8", "value": "10002" },
    { "dataElement": "f7n9E0hX8qk", "categoryOptionCombo": "bRowv6yZOF2", "period": "201401",
      "orgUnit": "DiszpKrYNg8", "value": "10001" }
  ]
}

Note that data values are softly deleted, i.e. a deleted value has the deleted property set to true instead of being permanently deleted. This is useful when integrating multiple systems in order to communicate deletions. You can include deleted values in the response like this:

/api/26/dataValueSets.json?dataSet=pBOMPrpg1QX&period=201401&orgUnit=DiszpKrYNg8&includeDeleted=true

You can also request data in CSV format like this:

https://play.dhis2.org/demo/api/26/dataValueSets.csv?dataSet=pBOMPrpg1QX&period=201401&orgUnit=DiszpKrYNg8

The response will look like this:

dataelement,period,orgunit,categoryoptioncombo,attributeoptioncombo,value,storedby,lastupdated,comment,followup
f7n9E0hX8qk,201401,DiszpKrYNg8,bRowv6yZOF2,bRowv6yZOF2,12,system,2015-04-05T19:58:12.000,comment1,false
Ix2HsbDMLea,201401,DiszpKrYNg8,bRowv6yZOF2,bRowv6yZOF2,14,system,2015-04-05T19:58:12.000,comment2,false
eY5ehpbEsB7,201401,DiszpKrYNg8,bRowv6yZOF2,bRowv6yZOF2,16,system,2015-04-05T19:58:12.000,comment3,false
FTRrcoaog83,201401,DiszpKrYNg8,bRowv6yZOF2,bRowv6yZOF2,12,system,2014-03-02T21:45:05.519,comment4,false

The following constraints apply to the data value sets resource:

  • At least one data set must be specified.

  • Either at least one period or a start date and end date must be specified.

  • At least one organisation unit must be specified.

  • Organisation units must be within the hierarchy of the organisation units of the authenticated user.

  • Limit cannot be less than zero.

1.17.7. Sending, reading and deleting individual data values

This example will show how to send individual data values to be saved in a request. This can be achieved by sending a POST request to the dataValues resource:

https://play.dhis2.org/demo/api/26/dataValues

The following query parameters are supported for this resource:

Table 1.30. Data values query parameters

Query parameterRequiredDescription
deYesData element identifier
peYesPeriod identifier
ouYesOrganisation unit identifier
coNoCategory option combo identifier, default will be used if omitted
ccNo (must combine with cp)Attribute combo identifier
cpNo (must combine with cc)Attribute option identifiers, separated with ; for multiple values
valueNoData value
commentNoData comment
followUpNoFollow up on data value, will toggle the current boolean value

If any of the identifiers given are invalid, if the data value or comment are invalid or if the data is locked, the response will contain the 409 Conflict status code and descriptive text message. If the operation lead to a saved or updated value, 200 OK will be returned. An example of a request looks like this:

curl "https://play.dhis2.org/demo/api/26/dataValues?de=s46m5MS0hxu
  &pe=201301&ou=DiszpKrYNg8&co=Prlt0C1RF0s&value=12"
  -X POST -u admin:district -v

This resource also allows a special syntax for associating the value to an attribute option combination. This can be done by sending the identifier of the attribute combination, together with the identifier(s) of the attribute option(s) which the value represents within the combination. An example looks like this:

curl "https://play.dhis2.org/demo/api/26/dataValues?de=s46m5MS0hxu&ou=DiszpKrYNg8
  &pe=201308&cc=dzjKKQq0cSO&cp=wbrDrL2aYEc;btOyqprQ9e8&value=26"
  -X POST -u admin:district -v

You can retrieve a data value with a request using the GET method. The value, comment and followUp params are not applicable in this regard:

curl "https://play.dhis2.org/demo/api/26/dataValues?de=s46m5MS0hxu
  &pe=201301&ou=DiszpKrYNg8&co=Prlt0C1RF0s"
  -X GET -u admin:district -v

You can delete a data value with a request using the DELETE method.

1.17.7.1. Working with file data values

When dealing with data values which have a data element of type file there is some deviation from the method described above. These data values are special in that the contents of the value is a UID reference to a FileResource object instead of a self-contained constant. These data values will behave just like other data values which store text content, but should be handled differently in order to produce meaningful input and output.

The process of storing one of these data values roughly goes like this:

  1. Upload the file to the /api/26/fileResources endpoint as described in the file resource section.

  2. Retrieve the 'id' property of the returned FileResource.

  3. Store the retrieved id as the value to the data value using any of the methods described above.

Only one-to-one relationships between data values and file resources are allowed. This is enforced internally so that saving a file resource id in several data values is not allowed and will return an error. Deleting the data value will delete the referenced file resource. Direct deletion of file resources are not possible.

The data value can now be retrieved as any other but the returned data will be the UID of the file resource. In order to retrieve the actual contents (meaning the file which is stored in the file resource mapped to the data value) a GET request must be made to api/dataValues/files mirroring the query parameters as they would be for the data value itself. The dataValues/files endpoint only supports GET requests.

It is worth noting that due to the underlying storage mechanism working asynchronously the file content might not be immediately ready for download from the dataValues/files endpoint. This is especially true for large files which might require time consuming uploads happening in the background to a an external file store (depending on the system configuration). Retrieving the file resource meta-data from the api/fileResources/<id> endpoint allows checking the storageStatus of the content before attempting to download it.