Data

Given the complexity of the ComStock software workflow, and the big-data skill set and computing hardware required, the pathway for professionals and researchers to use ComStock successfully is to interact with the pre-created results, rather than running the ComStock modeling tool. This section provides information about accessing ComStock data, and a list of published datasets.

Data Access Platforms, Structure and Contents

Published Datasets

These datasets describe the timeseries energy consumption of the U.S. commercial building stock at the end-use level. For details on how it was created and validated, please see the project’s final report. See the Data Access Platforms, Structure and Contents section for more details about the data.

Each End Use Savings Shape dataset release is introduced with a webinar presentation. Access the webinar recordings and slides, below:

ComStock dataset releases are summarized in the following table with links for accessing the aggregate results.

  ComStock End Use Savings Shape 2024 Release 1 - 2018 Weather ComStock End Use Savings Shape 2023 Release 2 - 2018 Weather ComStock End Use Savings Shape 2023 Release 1 - 2018 Weather ComStock End Use Load Profiles - 2018 Weather ComStock End Use Load Profiles - Typical Weather
OEDI Name 2024/comstock_amy2018_release_1 2023/comstock_amy2018_release_2 2023/comstock_amy2018_release_1 2021/comstock_amy2018_release_1 2021/comstock_tmy3_release_1
Data Viewer Links
Annual and Timeseries Energy		 by_state by_state by_state by_state,
by_puma_northeast,
by_puma_midwest,
by_puma_south,
by_puma_west		 by_state,
by_puma_northeast,
by_puma_midwest,
by_puma_south,
by_puma_west
Data Table with
Characteristics and
Annual Energy Use		 metadata metadata metadata metadata metadata
OpenEI Data Lake suppl_data_dict suppl_data_dict suppl_data_dict suppl_data_dict suppl_data_dict
Publication Date March-24 Sept-23 March-23 Oct-21 Oct-21
Release # 2024_1 2023_2 2023_1 2021_1 2021_1
Building Stock
Represented		 U.S. commercial sector circa 2018 U.S. commercial sector circa 2018 U.S. commercial sector circa 2018 U.S. commercial sector circa 2018 U.S. commercial sector circa 2018
Upgrades Available 1. Variable Speed HP RTU, Electric Backup
2. Variable Speed HP RTU, Electric Backup, Energy Recovery
3. Variable Speed HP RTU, Original Heating Fuel Backup
4. Standard Performance HP RTU, Electric Backup
5. VRF with DOAS
6. VRF with 25pct Upsizing Allowance
7. DOAS HP Minisplits
8. HP Boiler, Electric Backup
9. HP Boiler, Gas Backup
10. Air Side Economizers for AHUs
11. Demand Control Ventilation
12. Energy Recovery for AHUs
13. Advanced RTU Controls
14. Unoccupied AHU Control
15. Hydronic Water-to-Water Geothermal Heat Pump
16. Packaged Water-to-Air Geothermal Heat Pump
17. Console Water-to-Air Geothermal Heat Pump
18. Comprehensive Geothermal Heat Pump Package, Hydronic GHP, Packaged GHP, or Console GHP
19. Demand Flexibility, Thermostat Control, Load Shed
20. Demand Flexibility, Thermostat Control, Load Shift
21. LED Lighting
22. Electric Kitchen Equipment
23. Wall Insulation
24. Roof Insulation
25. Secondary Windows
26. Window Film
27. New Windows
28. Package 1, Wall & Roof Insulation + New Windows
29. Package 2, LED Lighting + Variable Speed HP RTU or HP Boilers
30. Package 3, LED Lighting + Standard Performance HP RTU or HP Boilers
31. Package 4, Package 1 + Package 2
32. Package 5, Variable Speed HP RTU or HP Boilers + Economizer + DCV + Energy Recovery
 1. HP RTU, Electric Backup;
2. HP RTU, Original Heating Fuel Backup;
3. VRF with DOAS;
4. DOAS HP Minisplits;
5. HP Boiler, Electric Backup;
6. HP Boiler, Gas Backup;
8. Demand Control Ventilation;
9. Energy Recovery;
10. LED Lighting;
11. Wall Insulation;
12. Roof Insulation;
13. Secondary Windows;
14. Window Film;
15. New Windows;
16. Package 1, Wall & Roof Insulation + New Windows;
17. Package 2, LED Lighting + HP RTU or HP Boilers;
18. Package 3, Package 1 + Package 2		 1. HP-RTU
2. DOAS HP Minisplits
3. Heat Pump Boiler
5. LED Lighting
6. Exterior Wall Insulation
7. Roof Insulation
8. Secondary Windows
9.Window Film
10. New Windows		 None None
Weather Year amy2018 amy2018 amy2018 amy2018 tmy3

Data Access Platforms, Structure and Contents

At the most fundamental level, the ComStock dataset is a collection of end-use load profiles of approximately 350,000 building energy models. The output of each building energy model is 1 year of energy consumption in 15-minute intervals, separated into end-use categories.

Accessing national ComStock building load profiles in the full dataset requires big-data skills that make the full dataset inaccessible for most users. To support many use cases, aggregate load profiles for the following geographic resolutions are published for ComStock releases:

  • 16 ASHRAE/International Energy Conservation Code climate zones
  • 5 U.S. Department of Energy Building America climate zones
  • 8 Electric System independent system operator and regional transmission organization regions
  • 2,400+ U.S. Census Public Use Microdata Areas
  • 3,000+ U.S. counties.

Data Access Platforms

The following table summarizes the various ways to access and use ComStock data.

The dataset has been formatted to be accessible in four main ways to meet the needs of many different users and use cases.

Metadata: Files of individual model characteristics together with annual results, commonly referred to as the “metadata” file

Load Profiles: Timeseries load profiles (individual building and pre-aggregated) in downloadable spreadsheets

Data Viewer: A web-based data viewer, customizable time scales and aggregations

Full Database: A detailed format that can be queried with big data tools

Aggregate ComStock datasets can be accessed via the Open Energy Initiative (OpenEI) Data Lake and the ComStock data viewer. There are two versions of the datasets published with each release: one with actual weather data (AMY), and another with typical weather data (TMY3). Note: The TMY3 15-minute energy data should not be used for larger geographies because weather events are not regionally aligned.

For information on how to query the full ComStock dataset, please refer to this documentation.

Please note, there are separate public datasets available for residential and commercial building stocks. 

Open Energy Initiative (OEDI) Data Lake

OpenEI is an energy information portal, and is developed and maintained by the National Renewable Energy Laboratory with funding and support from the U.S. Department of Energy and a network of International Partners & Sponsors. The OpenEI data lake contains comprehensive aggregate data for ComStock releases. This includes metadata and timeseries energy consumption results (baseline and upgrades, if applicable), individual building energy models, weather files, geographic information, and data dictionaries.

The ComStock release directory structure of the data lake is summarized in the table, below. For more detailed information about the contents of the ComStock OpenEI data lake, visit the README.

ComStock Data Viewer

The ComStock data viewer exists to quickly filter, slice, combine, visualize, and download the results in custom ways. This platform is available at comstock.nrel.gov. Multiple geographic views of the datasets on the data viewer have been created: by state, and by Census region by PUMA.

OEDI Directory Structure and Contents

Name Contents
building_energy_models Building energy models, in OpenStudio format, that were run to create the dataset.
geographic_information Information on various geographies used in the dataset provided for convenience. Includes map files showing the shapes of the geographies (states, PUMAs) used for partitioning and a lookup table mapping between census tracts and various other geographies.
metadata Building characteristics (age, area, HVAC system type, etc.) for each of the building energy models run to create the timeseries data and annual energy results. Descriptions of the characteristics are included in data_dictionary.tsv, enumeration_dictionary.tsv, and upgrade_dictionary.tsv.
timeseries_aggregates Aggregate end-use load profiles by building type and geography that can be opened and analyzed in Excel, python, or other common data analysis tools.
timeseries_aggregates_metadata Building characteristics for timeseries_aggregates building energy models. Follows the same format at metadata.
timeseries_individual_buildings The raw individual building timeseries data. This is a large number of individual files!
weather Key weather data used as an input to run the building energy models to create the dataset.
citation.txt Citation to use when referencing this work.
data_dictionary.tsv Describes the column names found in the metadata and timeseries data files.
enumeration_dictionary.tsv Expands the definitions of the enumerations used in the metadata files.
upgrade_dictionary.tsv Expands the definitions of the upgrades.

Dataset Naming Convention

ComStock releases on OpenEI data lake and the data viewer use the following naming convention.

         <dataset type>_<weather data>_<year of publication>_release_<release number>
 example:   comstock   _   amy2018    _         2021        _release_       1
  result:   comstock_amy2018_2021_release_1
  • dataset type
    • resstock = residential buildings stock
    • comstock = commercial building stock
  • weather data
    • amy2018 = actual meteorological year 2018 (2018 weather data from NOAA ISD, NSRDB, and MesoWest)
    • tmy3 = typical weather from 1991-2005 (see this publication for details)
  • year of publication
    • 2021 = dataset was published in 2021
    • 2022 = dataset was published in 2022
    • etc.
  • release
    • release_1 = first release of the dataset during the year of publication
    • release_2 = second release of the dataset during the year of publication
    • etc.

Field Naming Convention

The field naming convention is fairly simple. At the highest level there is – “in.” for inputs, “out.” for outputs, “calc.” for calculated fields, then a handful of columns that provide simulation information.

For the “out.” prefix there is a second level that includes – fuel type, emissions, model parameter and statistic fields, and site energy. The “in.” prefix does not have a second level.

The third level of “out.” is where you’ll find the end uses.

Finally, units are denoted by a “..” with the unit following.

ComStock, Copyright (c) 2019-2023, Alliance for Sustainable Energy, LLC, and other contributors. All rights reserved.