reVX.plexos.utilities.DataCleaner

class DataCleaner(plexos_meta, profiles, name_map=None)

Bases: object

Class for custom Plexos data cleaning procedures.

Parameters:

• plexos_meta (pd.DataFrame) – Plexos meta data including the built capacity at each plexos node.

• profiles (np.ndarray) – 2D timeseries array of generation profiles. Number of columns must match the length of the meta data.

• name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Methods

merge_extent(new_meta, new_profiles[, name_map])	Merge a new set of plexos node aggregation data into the self attr.
merge_multiple_extents(meta_list, profile_list)	Merge multiple plexos extents into the self attrs.
merge_small([capacity_threshold])	Merge small plexos buildout nodes into closest bigger nodes.
pre_filter_plexos_meta(plexos_meta[, name_map])	Pre-filter the plexos meta data to drop bad node names and duplicate lat/lons.
reduce_df(df, cols[, name_map])	Reduce a df to just certain columns.
rename_cols(df[, name_map])	Parameters: df (pd.DataFrame) -- Input df with bad or inconsistent column names.

Attributes

PLEXOS_META_COLS
REEDS_NAME_MAP
REV_NAME_MAP

static rename_cols(df, name_map=None)

Parameters:

• df (pd.DataFrame) – Input df with bad or inconsistent column names.

• name_map (dictionary, optional) – Column rename mapping, by default None -> {‘lat’: ‘latitude’, ‘lon’: ‘longitude’}

Parameters:

df (pd.DataFrame) – Same as inputs but with better col names.

classmethod reduce_df(df, cols, name_map=None)

Reduce a df to just certain columns.

Parameters:

• df (pd.DataFrame) – Dataframe to reduce.

• cols (list | tuple) – List of column names to keep.

• name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Returns:

df (pd.DataFrame) – Dataframe with only cols if the input df had all cols.

classmethod pre_filter_plexos_meta(plexos_meta, name_map=None)

Pre-filter the plexos meta data to drop bad node names and duplicate lat/lons.

Parameters:

• plexos_meta (pd.DataFrame) – Plexos meta data.

• name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Returns:

plexos_meta (pd.DataFrame) – Filtered plexos meta data.

merge_small(capacity_threshold=20.0)

Merge small plexos buildout nodes into closest bigger nodes.

Parameters:

capacity_threshold (float) – Capacity threshold, nodes with built capacities less than this will be merged into bigger nodes.

Returns:

• meta (pd.DataFrame) – New plexos node meta data with updated built capacities.

• profiles (np.ndarray) – New profiles with big nodes having absorbed additional generation from bigger nodes.

merge_extent(new_meta, new_profiles, name_map=None)

Merge a new set of plexos node aggregation data into the self attr.

Parameters:

• new_meta (pd.DataFrame) – A new set of Plexos node meta data to be merged into the meta in self.

• new_profiles (np.ndarray) – A new set of plexos node profiles corresponding to new_meta to be merged into the profiles in self where the meta data overlaps with common nodes.

• name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

merge_multiple_extents(meta_list, profile_list, name_map=None)

Merge multiple plexos extents into the self attrs.

Parameters:

• meta_list (list) – List of new meta data extents to merge into self.

• profile_list (list) – List of new gen profile to merge into self.

• name_map (dictionary, optional) – Column rename mapping, by default None -> {‘gid’: ‘sc_gid’}

Returns:

• meta (pd.DataFrame) – Merged plexos node meta data.

• profiles (np.ndarray) – New profiles with merged profiles for matching nodes.