marmot.plottingmodules.plotutils.plot_data_helper.PlotDataStoreAndProcessor

class PlotDataStoreAndProcessor(AGG_BY: str, ordered_gen: List[str], marmot_solutions_folder: pathlib.Path, gen_names_dict: Optional[dict] = None, tech_subset: Optional[List[str]] = None, **_)

Bases: dict

Methods used to assist with the creation of Marmot plots

Collection of Methods to assist with creation of figures, including getting and formatting data and modifying dataframes

PlotDataStoreAndProcessor inherits the python class ‘dict’ so acts like a dictionary and stores the formatted data when retrieved by the get_formatted_data method.

Parameters

• AGG_BY (str) – Informs region type to aggregate by when creating plots.

• ordered_gen (List[str]) – Ordered list of generator technologies to plot, order defines the generator technology position in stacked bar and area plots.

• marmot_solutions_folder (Path) – Directory containing Marmot solution outputs.

• gen_names_dict (dict, optional) – Mapping dictionary to rename generator technologies. Default is None.

• tech_subset (List[str], optional) – Tech subset category to plot. The tech_subset value should be a column in the ordered_gen_categories.csv. If left None all techs will be plotted Defaults to None.

Methods

add_battery_gen_to_df(df, scenario, zone_input)	Adds Battery generation to the passed dataframe.
add_curtailment_to_df(df, scenario, ...[, ...])	Adds curtailment to the passed Dataframe as a new column
assign_curtailment_techs(df, vre_techs)	Assign technologies to Marmot's Curtailment property (generator_Curtailment).
capacity_energy_unitconversion(df, Scenarios)	Unitconversion for capacity and energy figures.
create_categorical_tech_index(df[, axis])	Creates categorical index based on generators.
df_process_gen_inputs(df)	Processes generation data into a pivot table.
get_formatted_data(properties)	Get data from formatted h5 file or csv property input files.
include_net_imports(gen_df, load_series[, ...])	Adds net imports to total and timeseries generation plots.
insert_custom_data_columns(existing_df, ...)	Insert custom columns into existing DataFrame before plotting.
process_extra_properties(extra_properties, ...)	Processes a list of extra properties and saves them into a single dataframe.
rename_gen_techs(df)	Renames generator technologies based on the gen_names.csv file.
year_scenario_grouper(df, scenario[, ...])	Special groupby method to group dataframes by Scenario or Year-Scenario.

get_formatted_data(properties: List[tuple]) → list

Get data from formatted h5 file or csv property input files.

Adds data to dictionary with scenario name as key

Parameters

properties (List[tuple]) – list of tuples containing required plexos property information

Returns

If 1 in list required data is missing.

Return type

list

rename_gen_techs(df: pandas.core.frame.DataFrame) → pandas.core.frame.DataFrame

Renames generator technologies based on the gen_names.csv file.

Parameters

df (pd.DataFrame) – Dataframe to process.

Returns

Processed DataFrame with renamed techs.

Return type

pd.DataFrame

assign_curtailment_techs(df: pandas.core.frame.DataFrame, vre_techs: list) → pandas.core.frame.DataFrame

Assign technologies to Marmot’s Curtailment property (generator_Curtailment).

Parameters

• df (pd.DataFrame) – Dataframe to process.

• vre_techs (list) – List of vre tech names, or technologies that should be included in curtailment calculations.

Returns

Dataframe containing only specified curtailment technologies.

Return type

pd.DataFrame

df_process_gen_inputs(df: pandas.core.frame.DataFrame) → pandas.core.frame.DataFrame

Processes generation data into a pivot table.

Also calls rename_gen_techs() to rename technologies Technology names will be columns, Timeseries as index

Parameters

df (pd.DataFrame) – Dataframe to process.

Returns

Transformed Dataframe.

Return type

pd.DataFrame

create_categorical_tech_index(df: pandas.core.frame.DataFrame, axis=0) → pandas.core.frame.DataFrame

Creates categorical index based on generators.

Parameters

df (pd.DataFrame) – Dataframe to process.

Returns

Processed DataFrame.

Return type

pd.DataFrame

include_net_imports(gen_df: pandas.core.frame.DataFrame, load_series: pandas.core.series.Series, unsereved_energy: pandas.core.series.Series = Series([], dtype: float64)) → pandas.core.frame.DataFrame

Adds net imports to total and timeseries generation plots.

Net imports are calculated as load - total generation

Parameters

• gen_df (pd.DataFrame) – generation dataframe

• load_series (pd.Series) – load series

• unsereved_energy (pd.Series) – unsereved energy series, (optional)

Returns

Dataframe with net imports included

Return type

pd.DataFrame

process_extra_properties(extra_properties: List[str], scenario: str, zone_input: str, agg: str, data_resolution: str = '') → pandas.core.frame.DataFrame

Processes a list of extra properties and saves them into a single dataframe.

Use with properties that should be aggregated to a zonal/regional aggregation such as; Load, Demand and Unsereved Energy.

Parameters

• extra_properties (List[str]) – list of extra property names to retrieve from formatted data file and process

• scenario (str) – scenario to pull data from

• zone_input (str) – zone to subset by.

• agg_by (str) – Area aggregtaion, zone or region.

• data_resolution (str, optional) – Specifies the data resolution to pull from the formatted data and plot. Defaults to “”.

Returns

Dataframe of extra properties with timeseries index.

Return type

pd.DataFrame

add_curtailment_to_df(df: pandas.core.frame.DataFrame, scenario: str, zone_input: str, vre_techs: list, data_resolution: str = '') → pandas.core.frame.DataFrame

Adds curtailment to the passed Dataframe as a new column

Parameters

• df (pd.DataFrame) – DataFrame to add curtailment column to

• scenario (str) – scenario to pull data from

• zone_input (str) – zone to subset by

• vre_techs (list) – List of vre tech names, or technologies that should be included in curtailment calculations.

• data_resolution (str, optional) – Specifies the data resolution to pull from the formatted data and plot. Defaults to “”.

Returns

DataFrame with added curtailment column.

Return type

pd.DataFrame

add_battery_gen_to_df(df: pandas.core.frame.DataFrame, scenario: str, zone_input: str, data_resolution: str = '') → pandas.core.frame.DataFrame

Adds Battery generation to the passed dataframe.

Parameters

• df (pd.DataFrame) – DataFrame to add battery generation to.

• scenario (str) – scenario to pull data from

• zone_input (str) – zone to subset by

• data_resolution (str, optional) – Specifies the data resolution to pull from the formatted data and plot. Defaults to “”.

Returns

DataFrame with added battery gen column.

Return type

pd.DataFrame

static year_scenario_grouper(df: pandas.core.frame.DataFrame, scenario: str, groupby: str = 'Scenario', additional_groups: Optional[list] = None, **kwargs) → pandas.core.frame.DataFrame.groupby

Special groupby method to group dataframes by Scenario or Year-Scenario.

New in version 0.10.0.

Grouping by Year-Scenario is useful for multi year results sets where examining results by year is of interest.

This method is a wrapper around pd.DataFrame.groupby and takes all the same arguments.

Parameters

• df (pd.DataFrame) – DataFrame to group

• scenario (str) – name of the scenario to groupby

• groupby (str, optional) – Groupby ‘Scenario’ or ‘Year-Scenario’. If Year-Scenario is chosen the year is extracted from the DatetimeIndex and appended to the scenario name. Defaults to ‘Scenario’.

• additional_groups (list, optional) – List of any additional columns to groupby. Defaults to None.

• **kwargs – These parameters will be passed to pandas.DataFrame.groupby function.

Raises

• ValueError – If df.index is not of type type pd.DatetimeIndex or type pd.MultiIndex with level timestamp.

• ValueError – If additional_groups is not a list

Returns

Returns a groupby object that contains

information about the groups.

Return type

DataFrameGroupBy

static insert_custom_data_columns(existing_df: pandas.core.frame.DataFrame, custom_data_file_path: pathlib.Path) → pandas.core.frame.DataFrame

Insert custom columns into existing DataFrame before plotting.

Custom data is loaded from passed custom_data_file_path, the custom data file must be a csv. Default position of new columns is at the end of the existing DataFrame. Specific positions can be selected by including a row with index label ‘column_position’. Corresponding column positions can then be included. -1 can be passed to insert the column at the end of the DataFrame (rightmost position).

New rows can also be included but their position can not be changed and are appended to end of DataFrame.

NaN values are returned as 0

Parameters

• existing_df (pd.DataFrame) – DataFrame to modify

• custom_data_file_path (Path) – path to custom data file

Returns

DataFrame with the newly inserted columns

Return type

pd.DataFrame

static capacity_energy_unitconversion(df: pandas.core.frame.DataFrame, Scenarios: List[str], sum_values: bool = False) → dict

Unitconversion for capacity and energy figures.

Takes a pd.DataFrame as input and will then determine the max value in the frame.

If sum_values is True, either rows or columns will be summated before determining max value. The axis is chosen automatically based on where the scenario entries or datetime index is located. If correct axis cannot be determined axis 0 (rows) will be summed. This setting should mainly be set to True when potting stacked bar and area plots.

Parameters

• df (pd.DataFrame) – pandas dataframe

• Scenarios (List[str]) – List of scenarios being processed.

• sum_values (bool, optional) – Sum axis values if True. Should be set to True for stacked bar and area plots. Defaults to False.

Returns

Dictionary containing divisor and units.

Return type

dict