reVX.plexos.utilities.DataCleaner

class DataCleaner(plexos_meta, profiles, name_map=None)[source]

Bases: object

Class for custom Plexos data cleaning procedures.

Parameters:
  • plexos_meta (pd.DataFrame) – Plexos meta data including the built capacity at each plexos node.

  • profiles (np.ndarray) – 2D timeseries array of generation profiles. Number of columns must match the length of the meta data.

  • name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Methods

merge_extent(new_meta, new_profiles[, name_map])

Merge a new set of plexos node aggregation data into the self attr.

merge_multiple_extents(meta_list, profile_list)

Merge multiple plexos extents into the self attrs.

merge_small([capacity_threshold])

Merge small plexos buildout nodes into closest bigger nodes.

pre_filter_plexos_meta(plexos_meta[, name_map])

Pre-filter the plexos meta data to drop bad node names and duplicate lat/lons.

reduce_df(df, cols[, name_map])

Reduce a df to just certain columns.

rename_cols(df[, name_map])

Parameters:
  • df (pd.DataFrame) -- Input df with bad or inconsistent column names.

Attributes

PLEXOS_META_COLS

REEDS_NAME_MAP

REV_NAME_MAP

static rename_cols(df, name_map=None)[source]
Parameters:
  • df (pd.DataFrame) – Input df with bad or inconsistent column names.

  • name_map (dictionary, optional) – Column rename mapping, by default None -> {‘lat’: ‘latitude’, ‘lon’: ‘longitude’}

Parameters:

df (pd.DataFrame) – Same as inputs but with better col names.

classmethod reduce_df(df, cols, name_map=None)[source]

Reduce a df to just certain columns.

Parameters:
  • df (pd.DataFrame) – Dataframe to reduce.

  • cols (list | tuple) – List of column names to keep.

  • name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Returns:

df (pd.DataFrame) – Dataframe with only cols if the input df had all cols.

classmethod pre_filter_plexos_meta(plexos_meta, name_map=None)[source]

Pre-filter the plexos meta data to drop bad node names and duplicate lat/lons.

Parameters:
  • plexos_meta (pd.DataFrame) – Plexos meta data.

  • name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Returns:

plexos_meta (pd.DataFrame) – Filtered plexos meta data.

merge_small(capacity_threshold=20.0)[source]

Merge small plexos buildout nodes into closest bigger nodes.

Parameters:

capacity_threshold (float) – Capacity threshold, nodes with built capacities less than this will be merged into bigger nodes.

Returns:

  • meta (pd.DataFrame) – New plexos node meta data with updated built capacities.

  • profiles (np.ndarray) – New profiles with big nodes having absorbed additional generation from bigger nodes.

merge_extent(new_meta, new_profiles, name_map=None)[source]

Merge a new set of plexos node aggregation data into the self attr.

Parameters:
  • new_meta (pd.DataFrame) – A new set of Plexos node meta data to be merged into the meta in self.

  • new_profiles (np.ndarray) – A new set of plexos node profiles corresponding to new_meta to be merged into the profiles in self where the meta data overlaps with common nodes.

  • name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

merge_multiple_extents(meta_list, profile_list, name_map=None)[source]

Merge multiple plexos extents into the self attrs.

Parameters:
  • meta_list (list) – List of new meta data extents to merge into self.

  • profile_list (list) – List of new gen profile to merge into self.

  • name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Returns:

  • meta (pd.DataFrame) – Merged plexos node meta data.

  • profiles (np.ndarray) – New profiles with merged profiles for matching nodes.