Model Validation

Model Validation#

The purpose of this tutorial is to learn about model validation and how BuildingMOTIF can give useful feedback for fixing a model that has failed validation.

Note

This tutorial has the following learning objectives:

  1. validating the model against an ontology to ensure that the model is a valid Brick model

  2. validating the model against a manifest, which contains the metadata requirements for a specific model

  3. validating the model against a use case for a specific application

Validating a model is the process of ensuring that the model is both correct (uses the ontologies correctly) and semantically sufficient (it contains sufficient metadata to execute the desired applications or enable the desired use cases). Validation is always done with respect to sets of Shapes using the Shapes Constraint Language (SHACL)[1].

Setup#

We create an in-memory BuildingMOTIF instance, load the model from the previous tutorial, and load some libraries to create the manifest with. The constraints.ttl library we load is a special library with some custom constraints defined that are helpful for writing manifests.

from rdflib import Namespace
from buildingmotif import BuildingMOTIF
from buildingmotif.dataclasses import Model, Library
from buildingmotif.namespaces import BRICK # import this to make writing URIs easier

# in-memory instance
bm = BuildingMOTIF("sqlite://")

# create the namespace for the building
BLDG = Namespace('urn:bldg/')

# create the building model
model = Model.create(BLDG, description="This is a test model for a simple building")

# load tutorial 1 model
model.graph.parse("tutorial1_model.ttl", format="ttl")

# load libraries included with the python package
constraints = Library.load(ontology_graph="../../buildingmotif/libraries/constraints/constraints.ttl")

# load libraries excluded from the python package (available from the repository)
brick = Library.load(ontology_graph="../../libraries/brick/Brick-subset.ttl")
g36 = Library.load(directory="../../libraries/ashrae/guideline36")

/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pyshacl/extras/__init__.py:46: Warning: Extra "js" is not satisfied because requirement pyduktape2 is not installed.
  warn(Warning(f"Extra \"{extra_name}\" is not satisfied because requirement {req} is not installed."))

2025-06-16 23:34:31,906 | root |  WARNING: An ontology could not resolve a dependency on https://brickschema.org/schema/1.4/Brick (Name: https://brickschema.org/schema/1.4/Brick). Check this is loaded into BuildingMOTIF

Model Validation - Ontology#

BuildingMOTIF organizes Shapes into Shape Collections. The shape collection associated with a library (if there is one) can be retrieved with the get_shape_collection method. Below, we use Brick’s shape collection to ensure that the model is using Brick correctly:

# pass a list of shape collections to .validate()
validation_result = model.validate([brick.get_shape_collection()])
print(f"Model is valid? {validation_result.valid}")

Model is valid? True

Success! The model is valid according to the Brick ontology.

Model Validation - Manifest#

Writing a Manifest#

For now, we will write a manifest file directly; in the future, BuildingMOTIF will contain features that make manifests easier to write. Here is the header of a manifest file. This should also suffice for most of your own manifests.

@prefix brick: <https://brickschema.org/schema/Brick#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix constraint: <https://nrel.gov/BuildingMOTIF/constraints#> .
@prefix : <urn:my_site_constraints/> .

: a owl:Ontology ;
    owl:imports <https://brickschema.org/schema/1.3/Brick>,
                <https://nrel.gov/BuildingMOTIF/constraints>,
                <urn:ashrae/g36> .

We will now add a constraint stating that the model should contain exactly 1 Brick AHU.

:ahu-count a sh:NodeShape ;
    sh:message "need 1 AHU" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:AHU .

This basic structure can be changed to require different numbers of different Brick classes. Just don’t forget to change the name of the shape (:ahu-count, above) when you copy-paste!

Attention

As an exercise, try writing shapes that require the model to contain the following.

  • (1) Brick Supply_Fan

  • (1) Brick Damper

  • (1) Brick Cooling_Coil

  • (1) Brick Heating_Coil

Hint

:fan-count a sh:NodeShape ;
    sh:message "need 1 supply fan" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Supply_Fan .

:damper-count a sh:NodeShape ;
    sh:message "need 1 damper" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Damper .

:clg-coil-count a sh:NodeShape ;
    sh:message "need 1 cooling coil" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Cooling_Coil .

:htg-coil-count a sh:NodeShape ;
    sh:message "need 1 heating coil" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Heating_Coil .

Put all of the above in a new file called tutorial1_manifest.ttl. We’ll also add a shape called sz-vav-ahu-control-sequences, which is a use case shape to validate the model against in the next section.

The following block of code puts all of the above in the tutorial1_manifest.ttl file for you:

with open("tutorial1_manifest.ttl", "w") as f:
    f.write("""
@prefix brick: <https://brickschema.org/schema/Brick#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix constraint: <https://nrel.gov/BuildingMOTIF/constraints#> .
@prefix : <urn:my_site_constraints/> .

: a owl:Ontology ;
    owl:imports <https://brickschema.org/schema/1.3/Brick>,
                <https://nrel.gov/BuildingMOTIF/constraints>,
                <urn:ashrae/g36> .

:ahu-count a sh:NodeShape ;
    sh:message "need 1 AHU" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:AHU .

:fan-count a sh:NodeShape ;
    sh:message "need 1 supply fan" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Supply_Fan .

:damper-count a sh:NodeShape ;
    sh:message "need 1 damper" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Damper .

:clg-coil-count a sh:NodeShape ;
    sh:message "need 1 cooling coil" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Cooling_Coil .

:htg-coil-count a sh:NodeShape ;
    sh:message "need 1 heating coil" ;
    sh:targetNode : ;
    constraint:exactCount 1 ;
    constraint:class brick:Heating_Coil .

""")

Adding the Manifest to the Model#

We associate the manifest with our model so that BuildingMOTIF knows that we want validate the model against these specific shapes. We can always update this manifest, or validate our model against other shapes; however, validating a model against its manifest is the most common use case, so this is treated specially in BuildingMOTIF.

# load manifest into BuildingMOTIF as its own library!
manifest = Library.load(ontology_graph="tutorial1_manifest.ttl")
# set it as the manifest for the model
model.update_manifest(manifest.get_shape_collection())

2025-06-16 23:34:34,027 | root |  WARNING: An ontology could not resolve a dependency on urn:ashrae/g36 (Name: urn:ashrae/g36). Check this is loaded into BuildingMOTIF

Validating the Model#

We can now ask BuildingMOTIF to validate the model against the manifest and ask BuildingMOTIF for some details if it fails. By default, BuildingMOTIF will include all shape collections imported by the manifest (owl:imports). BuildingMOTIF will complain if the manifest requires ontologies that have not yet been loaded into BuildingMOTIF; this is why we are careful to load in the Brick and Guideline36 libraries at the top of this tutorial.

validation_result = model.validate()
print(f"Model is valid? {validation_result.valid}")

# print reasons
for entity, errors in validation_result.diffset.items():
    print(entity)
    for err in errors:
        print(f" - {err.reason()}")

2025-06-16 23:34:34,260 | buildingmotif.dataclasses.shape_collection |  WARNING: Could not resolve import of urn:ashrae/g36 from Libraries (Name: urn:ashrae/g36). Trying shape collections

2025-06-16 23:34:34,266 | buildingmotif.dataclasses.shape_collection |  WARNING: Could not resolve import of urn:ashrae/g36 from Libraries. Trying shape collections

2025-06-16 23:34:34,366 | buildingmotif.dataclasses.shape_collection |  WARNING: Could not resolve import of https://brickschema.org/schema/1.4/Brick from Libraries (Name: https://brickschema.org/schema/1.4/Brick). Trying shape collections

2025-06-16 23:34:34,374 | buildingmotif.dataclasses.shape_collection |  WARNING: Could not resolve import of https://brickschema.org/schema/1.4/Brick from Libraries. Trying shape collections

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[5], line 1
----> 1 validation_result = model.validate()
      2 print(f"Model is valid? {validation_result.valid}")
      4 # print reasons

File ~/work/BuildingMOTIF/BuildingMOTIF/buildingmotif/dataclasses/model.py:218, in Model.validate(self, shape_collections, error_on_missing_imports, shacl_engine)
    194 """Validates this model against the given list of ShapeCollections.
    195 If no list is provided, the model will be validated against the model's "manifest".
    196 If a list of shape collections is provided, the manifest will *not* be automatically
   (...)
    215 :rtype: ValidationContext
    216 """
    217 compiled_model = self.compile(shape_collections or [self.get_manifest()])
--> 218 return compiled_model.validate(error_on_missing_imports)

File ~/work/BuildingMOTIF/BuildingMOTIF/buildingmotif/dataclasses/compiled_model.py:145, in CompiledModel.validate(self, error_on_missing_imports)
    143 # aggregate shape graphs
    144 for sc in self.shape_collections:
--> 145     shapeg += sc.resolve_imports(
    146         error_on_missing_imports=error_on_missing_imports
    147     ).graph
    148 # inline sh:node for interpretability
    149 shapeg = rewrite_shape_graph(shapeg)

File ~/work/BuildingMOTIF/BuildingMOTIF/buildingmotif/dataclasses/shape_collection.py:137, in ShapeCollection.resolve_imports(self, recursive_limit, error_on_missing_imports)
    120 """Resolves `owl:imports` to as many levels as requested.
    121 
    122 By default, all `owl:imports` are recursively resolved. This limit can
   (...)
    134 :rtype: ShapeCollection
    135 """
    136 resolved_namespaces: Set[rdflib.URIRef] = set()
--> 137 resolved = _resolve_imports(
    138     self.graph,
    139     recursive_limit,
    140     resolved_namespaces,
    141     error_on_missing_imports=error_on_missing_imports,
    142 )
    143 new_sc = ShapeCollection.create()
    144 new_sc.add_graph(resolved)

File ~/work/BuildingMOTIF/BuildingMOTIF/buildingmotif/dataclasses/shape_collection.py:577, in _resolve_imports(graph, recursive_limit, seen, error_on_missing_imports)
    574             raise Exception("Could not resolve import of %s", ontology)
    575         continue
--> 577     dependency = _resolve_imports(
    578         sc_to_add.graph,
    579         recursive_limit - 1,
    580         seen,
    581         error_on_missing_imports=error_on_missing_imports,
    582     )
    583     new_g += dependency
    584 return new_g

File ~/work/BuildingMOTIF/BuildingMOTIF/buildingmotif/dataclasses/shape_collection.py:574, in _resolve_imports(graph, recursive_limit, seen, error_on_missing_imports)
    572 if sc_to_add is None:
    573     if error_on_missing_imports:
--> 574         raise Exception("Could not resolve import of %s", ontology)
    575     continue
    577 dependency = _resolve_imports(
    578     sc_to_add.graph,
    579     recursive_limit - 1,
    580     seen,
    581     error_on_missing_imports=error_on_missing_imports,
    582 )

Exception: ('Could not resolve import of %s', rdflib.term.URIRef('https://brickschema.org/schema/1.4/Brick'))

Tip on supplying extra shape collections

We can also provide a list of shape collections directly to Model.validate; BuildingMOTIF will use these shape collections to validate the model instead of the manifest.

# gather shape collections into a list for ease of use
shape_collections = [
    brick.get_shape_collection(),
    constraints.get_shape_collection(),
    manifest.get_shape_collection(),
    g36.get_shape_collection(),
]

# pass a list of shape collections to .validate()
validation_result = model.validate(shape_collections)
print(f"Model is valid? {validation_result.valid}")

# print reasons
for entity, errors in validation_result.diffset.items():
    print(entity)
    for err in errors:
        print(f" - {err.reason()}")

Fixing the Model#

One of the reasons the model is failing is we don’t have a heating coil required by the manifest, which we forgot to add in the previous tutorial. It’s also failing the use case validation, which we’ll cover in the next section. To fix the manifest validation, use the equipment templates in the Brick library to create a heating coil, add it to the model, and connect it to the AHU using RDFLib’s graph.add() method.

# ahu name
ahu_name = "Core_ZN-PSC_AC"

# get template
htg_coil_template = brick.get_template_by_name(BRICK.Heating_Coil)

# add htg coil
htg_coil_name = f"{ahu_name}-Htg_Coil"
htg_coil_binding = {"name": BLDG[htg_coil_name]}
htg_coil_graph = htg_coil_template.evaluate(htg_coil_binding)
model.add_graph(htg_coil_graph)

# connect htg coil
model.graph.add((BLDG[ahu_name], BRICK.hasPart, BLDG[htg_coil_name]))

# print model to confirm component was added and connected
print(model.graph.serialize())

We can see that the heating coil was added to the model and connected to the AHU so let’s check if the manifest validation failure was fixed.

validation_result = model.validate()
print(f"Model is valid? {validation_result.valid}")

# print reasons
for entity, errors in validation_result.diffset.items():
    print(entity)
    for err in errors:
        print(f" - {err.reason()}")

Success! Our model is now valid.

Model Validation - Use Case#

Finding Use Case Shapes#

We can use a couple methods to search the libraries for shapes we might want to use. Let’s start by asking the g36 library for any system specifications it knows about. This library represents ASHRAE Guideline 36 High-Performance Sequences of Operation for HVAC Systems[2]. A system specification will specify all of the metadata required for an entity to run control sequences associated with that system type.

from buildingmotif.namespaces import BMOTIF
shapes = g36.get_shape_collection()
for shape in shapes.get_shapes_of_definition_type(BMOTIF["System_Specification"]):
    print(shape)

The model represents the Small Office Commercial Prototype Building model, which has single zone packaged AHUs, so we’re interested in validating it against Section 4.8 of Guideline 36 for single zone variable air volume (VAV) AHUs.

Let’s update our manifest to include the requirement that AHUs must match the “single zone AHU” shape from G36:

model.get_manifest().graph.parse(data="""
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix brick: <https://brickschema.org/schema/Brick#> .
@prefix : <urn:my_site_constraints/> .
:sz-vav-ahu-control-sequences a sh:NodeShape ;
    sh:message "AHUs must match the single-zone VAV AHU shape" ;
    sh:targetClass brick:AHU ;
    sh:node <urn:ashrae/g36/4.8/sz-vav-ahu/sz-vav-ahu> .
""")

Validating the Model#

Now we can run validation to see if our AHU is ready to run the “single zone AHU” control sequence:

validation_result = model.validate()
print(f"Model is valid? {validation_result.valid}")

The AHU fails validation because it doesn’t match the sz-vav-ahu-control-sequences requirements. Take a look at the first bit of output, which is the official SHACL validation report text format. This can be difficult to interpret without a background in SHACL, so BuildingMOTIF provides a more easily understood version of the same information.

# SHACL validation report
print(validation_result.report_string)

# separator
print("-"*79)

Here is BuildingMOTIF’s interpretation of that report.

# BuildingMOTIF output
print("Model is invalid for these reasons:")
for entity, errors in validation_result.diffset.items():
    print(entity)
    for err in errors:
        print(f" - {err.reason()}")

The model is failing because the AHU doesn’t have the required points. We could find those templates manually, evaluate them, and add the resulting graphs to the model. However, this can be a bit tedious. To address this issue, BuildingMOTIF can find those templates automatically for us. We’ll cover this feature in the next tutorial so let’s save the model.

#save model
model.graph.serialize(destination="tutorial2_model.ttl")
Contents