Serialization

The power-grid-model provides tools for serialization and deserialization for datasets.

Visit the Python API Reference and C API Reference for the full documentation of the serialization tools.

Serialization format

Currently, two serialization formats are provided:

JSON serialization format specification

The JSON serialization format is a generic format and supports all dataset types and structures. Rather than providing a full schema, this documentation provides the features by object.

JSON schema root object

The format consists of a PowerGridModelRoot JSON object containing metadata and the actual data.

  • PowerGridModelRoot: Object

    • version: string containing the schema version (required, current version is "1.0")

    • type: string containing the dataset type, e.g. "input", "update", etc.

    • is_batch: boolean flag that describes whether the dataset is a batch or not.

    • attributes: Attributes containing specified attributes per component type (e.g.: "node").

    • data: Dataset containing the actual dataset.

JSON schema attributes object

Attributes contains specified attributes per Component type (e.g.: "node"). It is only required for those components that contain HomogeneousComponentData objects and that data needs to follow the attributes listed in this object. It may be empty if for data for all instances certain component is InhomogeneousComponentData. It reduces compression when a dataset largely follows the exact same pattern.

For example, for an "update" dataset that contains only updates to the "from_status" attribute of "branch" components, it may be {"branch": ["from_status"]}.

JSON schema component

A Component string contains the component name (see also the Components reference). E.g.: "node"

JSON schema component attributes

A ComponentAttributes array contains the desired Attributes for a specific Component.

JSON schema attribute

A Attribute string contains the name of an attribute of a component (see also the Components reference). E.g.: "id".

JSON schema dataset object

The Dataset object is either a SingleDataset if the is_batch field in the PowerGridModelRoot object is false, or a BatchDataset otherwise.

JSON schema single dataset object

A SingleDataset object contains the ComponentDataset for each component.

JSON schema batch dataset object

A BatchDataset is an array containing SingleDataset objects for all batch scenarios.

NOTE: The actual deserialized data representation may be sparse or dense, depending on the contents. Regardless of whether the deserialized data representation data is sparse or dense, the serialization format remains the same.

JSON schema component dataset object

A ComponentDataset is an array of ComponentDatas per component of the same type.

NOTE: The actual deserialized data representation may be row based or columnar, depending on the data_filter provided at deserialization (Check for example). Regardless of whether the deserialized data representation data is row based or columnar, the serialization format remains the same.

JSON schema component data object

A ComponentData object is either a HomogeneousComponentData object or an InhomogeneousComponentData object

JSON schema homogeneous component data object

A HomogeneousComponentData object contains the actual values of a certain component following the exact order of the attributes listed in the attributes field in the PowerGridModelRoot object.

JSON schema inhomogeneous component data object

An InhomogeneousComponentData object contains actual values per attribute of a certain component. Contrary to the HomogeneousComponentData, it lists the names of the attributes for which the values are specified, so the attributes may be in arbitrary order and do not have to follow the schema listed in the attributes field in the PowerGridModelRoot object.

JSON schema attribute value

The AttributeValue contains the actual value of an attribute. The value can be any of null if it is not provided, or any of int32_t, int8_t, double, RealValueInput or RealValueOutput. The type is listed for each attribute in Components.

JSON schema null (absence of value)

NOTE: This is the JSON-specific version of absence of value. For msgpack, refer to absence of value for msgpack.

NOTE: any nan values for concrete number types represent absence of value and are represented by null in the JSON schema.

JSON schema int32_t

An int32_t is an integer number value usually representing an ID. It may be in the inclusive range \(\left[-2^{31},+2^{31} - 1\right]\). The type is listed for each attribute in Components.

NOTE: the special value -2147483648 represents absence of value and may also be represented by null in the JSON schema.

JSON schema int8_t

An int8_t integer number value usually representing an enumeration value or a discrete setting. It may be in the inclusive range \(\left[-2^{7},+2^{7} - 1\right]\). The type is listed for each attribute in Components.

NOTE: the special value -128 represents absence of value and may also be represented by null in the JSON schema.

JSON schema double

NOTE: This is the JSON-specific version of double. For msgpack, refer to double for msgpack.

A double floating point number or string value, usually representing a real. If it is a string, it shall be either "inf" or "+inf" for positive infinity, or "-inf" for negative infinity. Any other values are unsupported. The type is listed for each attribute in Components.

NOTE: the special value nan represents absence of value and is represented by null in the JSON schema.

JSON schema RealValueInput

A RealValueInput of which the data format depends on the type of calculation. For symmetric components, it is a double. For asymmetric components, it is an Array of size 3 containing double or null values. The type is listed for each attribute in Components.

JSON schema RealValueOutput

A RealValueOutput of which the data format depends on the type of calculation. For symmetric calculations, it is a double. For asymmetric calculations, it is an Array of size 3 containing double or null values. The type is listed for each attribute in Components.

JSON schema single dataset example

The following example contains an input dataset. The nodes and sym_loads are represented using HomogeneousComponentData, the lines are represented using InomogeneousComponentData, while the sources are represented using a mixture of HomogeneousComponentData and InomogeneousComponentData.

{
  "version": "1.0",
  "type": "input",
  "is_batch": false,
  "attributes": {
    "node": ["id", "u_rated"],
    "sym_load": ["id", "node", "status", "type", "p_specified", "q_specified"],
    "source": ["id", "node", "status", "u_ref", "sk"]
  },
  "data": {
    "node": [
      [1, 10.5e3],
      [2, 10.5e3],
      [3, 10.5e3],
      [100, "inf"],
      [101, "+inf"],
      [102, "-inf"]
    ],
    "line": [
      {
        "id": 4,
        "from_node": 1,
        "to_node": 2,
        "from_status": 1,
        "to_status": 1,
        "r1": 0.11,
        "x1": 0.12,
        "c1": 4e-05,
        "tan1": 0.1,
        "i_n": 500.0
      },
      {
        "id": 5,
        "from_node": 2,
        "to_node": 3,
        "from_status": 1,
        "to_status": 1,
        "r1": 0.15,
        "x1": 0.16,
        "c1": 5e-05,
        "tan1": 0.12,
        "i_n": 550.0
      }
    ],
    "source": [
      [15, 1, 1, 1.03, 1e20],
      [16, 1, 1, 1.04, null],
      {
        "id": 17,
        "node": 1,
        "status": 1,
        "u_ref": 1.03,
        "sk": 1e10,
        "rx_ratio": 0.2
      }
    ],
    "sym_load": [
      [7, 2, 1, 0, 1.01e6, 0.21e6],
      [8, 3, 1, 0, 1.02e6, 0.22e6]
    ]
  }
}

JSON schema batch dataset example

The following example contains a batch update dataset containing two sym_loads and one asym_load. Not every scenario updates all components and attributes, reducing the total amount of data necessary to represent the batch dataset.

{
  "version": "1.0",
  "type": "update",
  "is_batch": true,
  "attributes": {
    "sym_load": ["id", "p_specified", "q_specified"],
    "asym_load": ["id", "p_specified"]
  },
  "data": [
    {
      "sym_load": [
        [7, 20.0, 50.0]
      ],
      "asym_load": [
        [9, [100.0, null, 200.0]]
      ]
    },
    {
      "asym_load": [
        [9, null]
      ]
    },
    {
      "sym_load": [
        [7, null, 10.0],
        {
          "id": 8,
          "status": 0
        }
      ],
      "asym_load": [
        {
          "id": 9,
          "q_specified": [70.0, 80.0, 90.0]
        }
      ]
    }
  ]
}

msgpack serialization format specification

The msgpack serialization format is a compressed version of the JSON serialization format and all features supported for JSON are also supported for msgpack.

msgpack schema nil (absence of value)

NOTE: This is the msgpack-specific version of absence of value. For JSON, refer to absence of value for JSON.

NOTE: any nan values for concrete number types represent absence of value are represented by nil in the msgpack schema.

msgpack schema double

NOTE: This is the msgpack-specific version of double. For JSON, refer to double for JSON.

A double floating point number value usually representing a real. Infinities are represented using the IEEE 754 standard for double-precision floating point values representation for infinities. Any other values are unsupported. The type is listed for each attribute in Components.

NOTE: the special value nan represents absence of value and may also be represented by nil in the msgpack schema.