7.4 Examples
This section is non-normative.
7.4.1 Metadata Examples
The following sections contain examples of metadata provided by a hypothetical FOCUS data provider called ACME to supply the required reference between the FOCUS dataset and the schema metadata. Provider implementations will vary on how the metadata is disseminated; however, the provider's chosen metadata delivery approach should be able to support the structure represented in this example.
In this example, the provider supports delivery of FOCUS data via file export to a data storage system. It uses JSON as the format for providing the metadata. The provider delivers data every 12 hours into a path structure described below:
| Type of data | Path |
|---|---|
| Export location | /FOCUS |
| Metadata location | /FOCUS/metadata |
| Cost data location | /FOCUS/data |
Here are some metadata examples for various scenarios:
7.4.1.1 Data Generator Metadata
7.4.1.1.1 Scenario
Acme provides metadata about the data generator as a part of their FOCUS data export. They provide the relevant data via the Data Generator schema object.
7.4.1.1.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/data_generator.json.
The updated data generator related metadata could look like this:
{
"DataGenerator": "Acme"
}
7.4.1.2 Schema Metadata
7.4.1.2.1 Scenario
ACME has only provided one Schema for their FOCUS data export. ACME provides a directory of schemas and each schema is a single file. Acme's provides a file representing the schema for the data they provide.
7.4.1.2.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-1234-abcde-12345-abcde-12345.json.
The updated schema related metadata could look like this:
{
"SchemaId": "1234-abcde-12345-abcde-12345",
"FocusVersion": "1.0",
"CreationDate": "2024-01-01T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
}
]
}
7.4.1.3 Schema Metadata to FOCUS Data Reference
7.4.1.3.1 Scenario
ACME makes a change to the Schema of their data exports. For each FOCUS data export, ACME includes a metadata reference to the schema object. Because multiple files are provided in each export, Acme has elected to include a metadata file in each export folder that includes the FOCUS schema reference that applies to the data export files within that folder. When the schema changes, they include the new Schema ID in their export metadata file of the new folder.
7.4.1.3.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/data/export1-metadata.json
The export metadata could look like this:
{
"SchemaId":"1234-abcde-12345-abcde-12345",
"data_location":
[
{
"filepath": "/FOCUS/data/export1/export1-part1.csv",
"total_bytes": 9010387,
"total_rows": 4450
},
{
"filepath": "/FOCUS/data/export1/export1-part2.csv",
"total_bytes": 9010387,
"total_rows": 4450
},
{
"filepath": "/FOCUS/data/export1/export1-part3.csv",
"total_bytes": 9010387,
"total_rows": 4450
},
{
"filepath": "/FOCUS/data/export1/export1-part4.csv",
"total_bytes": 9010387,
"total_rows": 4450
}
]
}
New metadata can be provided at a location such as /FOCUS/data/export2-metadata.json.
The new export metadata could look like this:
{
"SchemaId":"23456-abcde-23456-abcde-23456",
"data_location":
[
{
"filepath": "/FOCUS/data/export2/export2-part1.csv",
"total_bytes": 9010387,
"total_rows": 4450
},
{
"filepath": "/FOCUS/data/export2/export2-part2.csv",
"total_bytes": 9010387,
"total_rows": 4450
},
{
"filepath": "/FOCUS/data/export2/export2-part3.csv",
"total_bytes": 9010387,
"total_rows": 4450
},
{
"filepath": "/FOCUS/data/export2/export2-part4.csv",
"total_bytes": 9010387,
"total_rows": 4450
}
]
}
7.4.1.4 Adding New Columns
7.4.1.4.1 Scenario
ACME has decided add additional columns to their FOCUS data export. The new columns are x_awesome_column1, x_awesome_column2, and x_awesome_column3. The provider creates a new Schema object to represent the new schema, this schema object has a unique SchemaId. The subsequent data exports that use the new schema include the new schema's id as a reference to their corresponding schema object.
7.4.1.4.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-23456-abcde-23456-abcde-23456.json.
The updated schema related metadata could look like this:
{
"SchemaId": "23456-abcde-23456-abcde-23456",
"FocusVersion": "1.0",
"CreationDate": "2024-02-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["awecorp", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "x_awesome_column2",
"DataType": "DATETIME"
},
{
"ColumnName": "x_awesome_column3",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
}
]
}
For an example of how ACME ensures the schema metadata reference requirement is met see: Schema Metadata to FOCUS Data Reference
7.4.1.5 Removing Columns
7.4.1.5.1 Scenario
ACME has decided to remove columns from their FOCUS data export. The column removed is x_awesome_column3. The provider creates a new Schema object to represent the new schema, with a unique SchemaId.
7.4.1.5.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-34567-abcde-34567-abcde-34567.json.
The updated schema related metadata could look like this:
{
"SchemaId": "34567-abcde-34567-abcde-34567",
"FocusVersion": "1.0",
"CreationDate": "2024-03-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "x_awesome_column2",
"DataType": "DATETIME"
}
]
}
For an example of how ACME ensures the schema metadata reference requirement is met see: Schema Metadata to FOCUS Data Reference
7.4.1.6 Changing Column Metadata
7.4.1.6.1 Scenario
ACME has decided to change the datatype of column x_awesome_column1 from a string to a number. ACME creates a new Schema object with the modification to x_awesome_column2.
7.4.1.6.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-67891-abcde-67891-abcde-67891.json.
The updated schema related metadata could look like this:
{
"SchemaId": "67891-abcde-67891-abcde-67891",
"FocusVersion": "1.0",
"CreationDate": "2024-06-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "x_awesome_column2",
"DataType": "DATETIME"
}
]
}
For an example of how ACME ensures the schema metadata reference requirement is met see: Schema Metadata to FOCUS Data Reference
7.4.1.7 Provider Metadata Error Correction
7.4.1.7.1 Scenario
ACME has discovered that while their export includes the column x_awesome_column3, the Schema metadata does not include this column. In this case, the provider fixes the metadata in the existing schema object and does not need to create a new schema object. Reference metadata remains the same.
7.4.1.7.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-34567-abcde-34567-abcde-34567.json.
The updated schema related metadata could look like this:
{
"SchemaId": "34567-abcde-34567-abcde-34567",
"FocusVersion": "1.0",
"CreationDate": "2024-03-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "x_awesome_column2",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
}
]
}
7.4.1.8 FOCUS Version Changed
7.4.1.8.1 Scenario
ACME's previous exports used FOCUS version 1.0. They are now going to adopt FOCUS version 1.1. It is required that they create a new schema metadata object which specifies the new FOCUS version via the FOCUS Version property - regardless of schema changes. In this example, the new FOCUS version adoption doesn't include columns changes. This is to illustrate that FOCUS version changes are independent of column changes, however, this scenario is unlikely.
7.4.1.8.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-45678-abcde-45678-abcde-45678.json.
The updated schema related metadata could look like this:
{
"SchemaId": "45678-abcde-45678-abcde-45678",
"FocusVersion": "1.1",
"CreationDate": "2024-04-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "x_awesome_column2",
"DataType": "DATETIME"
}
]
}
For an example of how ACME ensures the schema metadata reference requirement is met see: Schema Metadata to FOCUS Data Reference
7.4.1.9 FOCUS Version Changed by Provider Using Provider Version
7.4.1.9.1 Scenario
ACME specifies the optional metadata property Provider Version in their Schema object. Their provider version 2.2 supported FOCUS version 1.0. They are now going to adopt FOCUS Version 1.1 which requires that they update their Provider Version when updating the FOCUS Version. They create a new schema object designating that both properties have changed. In this example, the adoption of the new FOCUS version doesn't include additional columns. This is to illustrate that Provider Version can change independent of column changes; however, this scenario is unlikely.
The provider creates a new schema object to represent the new schema. The provider includes both the new FOCUS Version and Provider Version in the schema object.
7.4.1.9.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-45678-abcde-45678-abcde-45678.json.
The updated schema related metadata could look like this:
{
"SchemaId": "45678-abcde-45678-abcde-45678",
"FocusVersion": "1.1",
"ProviderVersion": "2.3",
"name": "New Columns",
"CreationDate": "2024-04-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "x_awesome_column2",
"DataType": "DATETIME"
}
]
}
For reference, the prior schema object looked like this:
{
"SchemaId": "34567-abcde-34567-abcde-34567",
"FocusVersion": "1.0",
"ProviderVersion": "2.2",
"CreationDate": "2024-04-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "x_awesome_column2",
"DataType": "DATETIME"
}
]
}
For an example of how ACME ensures the schema metadata reference requirement is met see: Schema Metadata to FOCUS Data Reference
7.4.1.10 Data Changed by Provider Using Provider Version
7.4.1.10.1 Scenario
ACME specifies the optional metadata property Provider Version in their Schema object. They made a change to the FOCUS dataset they produce that does not adopt a new FOCUS Version, nor make a change the included columns but does impact values in the data. This example illustrates that Provider Version changes are independent of column changes, however provider version changes may include column changes.
The provider creates a new schema object to represent the new schema. The provider includes both the FOCUS Version and Provider Version in the schema object.
7.4.1.10.2 Supplied Metadata
Metadata can be provided at a location such as /FOCUS/metadata/schemas/schema-56789-abcde-56789-abcde-56789.json.
The updated schema related metadata could look like this:
{
"SchemaId": "56789-abcde-56789-abcde-56789",
"FocusVersion": "1.1",
"ProviderVersion": "2.4",
"CreationDate": "2024-05-02T12:01:03.083z",
"ColumnDefinition": [
{
"ColumnName": "BillingAccountId",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "BillingAccountName",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "ChargePeriodStart",
"DataType": "DATETIME"
},
{
"ColumnName": "ChargePeriodEnd",
"DataType": "DATETIME"
},
{
"ColumnName": "BilledCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "EffectiveCost",
"DataType": "DECIMAL",
"NumericPrecision": 20,
"NumberScale": 10
},
{
"ColumnName": "Tags",
"DataType": "JSON",
"ProviderTagPrefixes": ["acme", "ac"]
},
{
"ColumnName": "x_awesome_column1",
"DataType": "STRING",
"StringMaxLength": 64,
"StringEncoding": "UTF-8"
},
{
"ColumnName": "x_awesome_column2",
"DataType": "DATETIME"
}
]
}
For an example of how ACME ensures the schema metadata reference requirement is met see: Schema Metadata to FOCUS Data Reference