gdxpds package

Subpackages

Submodules

gdxpds.gdx module

Backend functionality for reading and writing GDX files. The GdxFile and GdxSymbol classes are full-featured interfaces for going between the GDX format and pandas DataFrames, including translation between GDX and numpy special values.

gdxpds.gdx.replace_df_column(df, colname, new_col)[source]

Utility function that replaces df[colname] with new_col. Special care is taken for the case when df has multiple columns named ‘*’, since this causes pandas to crash.

Parameters:

  • df (pandas.DataFrame) – edited in place by this function

  • colname (str) – name of column in df whose data is to be replaced

  • new_col (vector, list, pandas.Series) – new column data for df[colname]

exception gdxpds.gdx.GdxError(H, msg)[source]

Bases: Error

class gdxpds.gdx.GdxFile(gams_dir=None, lazy_load=True)[source]

Bases: MutableSequence, NeedsGamsDir

cleanup()[source]
clone()[source]

Returns a new GdxFile containing clones of the GdxSymbols in this GdxFile. The clone will not be associated with a filename. The clone’s GdxSymbols will not have indexes. The clone will be ready to write to a new file.

Return type:

GdxFile

property empty

Returns True if this GdxFile object does not contain any symbols.

Return type:

bool

property H

GDX object handle

property filename

Filename this GdxFile is associated with, if any

Return type:

None or str

property version

GDX file version

property producer

What program wrote the GDX file

property num_elements

Total number of records present in this file, summed over all symbols.

Return type:

int

read(filename)[source]

Opens gdx file at filename and reads meta-data. If not self.lazy_load, also loads all symbols.

Throws an Error if not self.empty.

Throws a GdxError if any calls to gdxcc fail.

Parameters:

filename (pathlib.Path or str) –

write(filename)[source]

Writes this GdxFile to filename

Parameters:

filename (pathlib.Path or str) –

__getitem__(key)[source]

Supports list-like indexing and symbol-based indexing

Parameters:

key (int or str) – If int, the index into the list of symbols. If str, the name of the symbol to be accessed.

Return type:

GdxSymbol

__setitem__(key, value)[source]

Supports overwriting or adding a GdxSymbol via a list-like interface

Parameters:

  • key (int) – Must be an index into the list of symbols, within range(len(self)+1)

  • value (GdxSymbol) –

insert(key, value)[source]

Inserts value at position key

Parameters:

  • key (int) – Must be an index into the list of symbols, within range(len(self)+1)

  • value (GdxSymbol) –

keys()[source]

List of symbol names obtained by iterating through this GdxFile

Return type:

list of str

class gdxpds.gdx.GamsDataType(value)[source]

Bases: Enum

An enumeration.

Set = 0
Parameter = 1
Variable = 2
Equation = 3
Alias = 4
class gdxpds.gdx.GamsVariableType(value)[source]

Bases: Enum

An enumeration.

Unknown = 0
Binary = 1
Integer = 2
Positive = 3
Negative = 4
Free = 5
SOS1 = 6
SOS2 = 7
Semicont = 8
Semiint = 9
class gdxpds.gdx.GamsEquationType(value)[source]

Bases: Enum

An enumeration.

Equality = 53
GreaterThan = 54
LessThan = 55
NothingEnforced = 56
External = 57
Conic = 58
class gdxpds.gdx.GamsValueType(value)[source]

Bases: Enum

An enumeration.

Level = 0
Marginal = 1
Lower = 2
Upper = 3
Scale = 4
gdxpds.gdx.GAMS_VALUE_COLS_MAP = defaultdict(<function <lambda>>, {<GamsDataType.Variable: 2>: [('Level', 0), ('Marginal', 1), ('Lower', 2), ('Upper', 3), ('Scale', 4)], <GamsDataType.Equation: 3>: [('Level', 0), ('Marginal', 1), ('Lower', 2), ('Upper', 3), ('Scale', 4)]})

List of value columns provided for each GamsValueType

gdxpds.gdx.GAMS_VALUE_DEFAULTS = {<GamsValueType.Level: 0>: 0.0, <GamsValueType.Marginal: 1>: 0.0, <GamsValueType.Lower: 2>: -inf, <GamsValueType.Upper: 3>: inf, <GamsValueType.Scale: 4>: 1.0}

Default values for each GamsValueType

gdxpds.gdx.GAMS_VARIABLE_DEFAULT_LOWER_UPPER_BOUNDS = {<GamsVariableType.Unknown: 0>: (-inf, inf), <GamsVariableType.Binary: 1>: (0.0, 1.0), <GamsVariableType.Integer: 2>: (0.0, inf), <GamsVariableType.Positive: 3>: (0.0, inf), <GamsVariableType.Negative: 4>: (-inf, 0.0), <GamsVariableType.Free: 5>: (-inf, inf), <GamsVariableType.SOS1: 6>: (0.0, inf), <GamsVariableType.SOS2: 7>: (0.0, inf), <GamsVariableType.Semicont: 8>: (1.0, inf), <GamsVariableType.Semiint: 9>: (1.0, inf)}

Default lower and upper bounds for each GamsVariableType

class gdxpds.gdx.GdxSymbol(name, data_type, dims=0, file=None, index=None, description='', variable_type=None, equation_type=None)[source]

Bases: object

clone()[source]

Create a copy of this GdxSymbol

Return type:

GdxSymbol

property name

Name of this GdxSymbol

Return type:

str

property data_type

GAMS data type of this GdxSymbol

Return type:

GamsDataType

property variable_type

Only not none if data_type == GamsDataType.Variable

Return type:

None or GamsDataType.Variable

property equation_type

Only not none if data_type == GamsDataType.Equation

Return type:

None or GamsDataType.Equation

property value_cols

List of (name, GamsValueType.value) tuples that describe the value columns in the dataframe, that is, the columns that follow the self.dims columns.

Return type:

list of (str, int)

property value_col_names

List of value column names, that is, the columns that follow the self.dims columns.

Return type:

list of str

get_value_col_default(value_col_name)[source]
property file

GdxFile file that contains this GdxSymbol, if any

Return type:

None or GdxFile

property index

Index of this GdxSymbol in its GdxFile, if any

Return type:

None or int

property loaded

Whether the data for this symbol has been loaded

Return type:

bool

property full_typename
property dims

List of dimension names over which this symbol is defined. If the GdxSymbol was constructed with dims set to an integer, all dimension names will be the wildcard ‘*’.

Returns:

length of list is equal to num_dims

Return type:

list of str

property num_dims

Number of dimensions over which this symbol is defined

Return type:

int

property dataframe

Data table for this symbol. Dim columns are followed by value columns, left to right.

Return type:

pd.DataFrame

property num_records

Number of rows in the data table, per the DataFrame if loaded, or per GAMS.

Return type:

int

load(load_set_text=False)[source]

Loads this GdxSymbol from its file, thereby popluating dataframe.

Parameters:

load_set_text (bool) – If True (default is False) and this symbol is a GamsDataType.Set, loads the GDX Text field into the dataframe rather than a c_bool.

unload()[source]

Drops this GdxSymbol’s dataframe

write(index=None)[source]

Writes this GdxSymbol to its file

gdxpds.gdx.append_set(gdx_file, set_name, df, cols=None, dim_names=None, description=None)[source]

Convenience function that appends set_name to gdx_file as a GamsDataType.Set GdxSymbol using data in df.

Parameters:

  • gdx_file (GdxFile) – file to which new GdxSymbol is to be added

  • set_name (str) – name of the GdxSymbol to be added

  • df (pandas.DataFrame) – dataframe or data that can be used to construct a dataframe containing the set data. assumes that all columns define dimensions (there is no ‘Value’ column)

  • cols (None or list of str) – if not None, these are the columns in df to be used for the set definition

  • dim_names (None or list of str) – if provided, the columns of a copy of df (or of df[cols]) will be renamed to these names, because the dimension names are taken from the final dataframe, these will also be the dimension names

  • description (None or str) – passed directly to GdxSymbol

gdxpds.gdx.append_parameter(gdx_file, param_name, df, cols=None, dim_names=None, description=None)[source]

Convenience function that appends param_name to gdx_file as a GamsDataType.Parameter GdxSymbol using data in df.

Parameters:

  • gdx_file (GdxFile) – file to which new GdxSymbol is to be added

  • param_name (str) – name of the GdxSymbol to be added

  • df (pandas.DataFrame) – dataframe or data that can be used to construct a dataframe containing the parameter data. assumes that the last selected column is the ‘Value’ column

  • cols (None or list of str) – if not None, these are the columns in df to be used for the parameter definition (dimension columns followed by value column)

  • dim_names (None or list of str) – if provided, the columns of a copy of df (or of df[cols]) will be renamed to these names + [‘Value’]. because the dimension names are taken from the final dataframe, these will also be the dimension names

  • description (None or str) – passed directly to GdxSymbol

gdxpds.read_gdx module

class gdxpds.read_gdx.Translator(gdx_file, gams_dir=None, lazy_load=False)[source]

Bases: object

property gams_dir
property gdx_file
property gdx
property dataframes
property symbols
property data_types
dataframe(symbol_name, load_set_text=False)[source]
gdxpds.read_gdx.to_dataframes(gdx_file, gams_dir=None, load_set_text=False)[source]

Primary interface for converting a GAMS GDX file to pandas DataFrames.

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

  • load_set_text (bool) – If True (default is False), then for every symbol that is a Set, loads the GDX Text field into the dataframe rather than a c_bool.

Returns:

Returns a dict of Pandas DataFrames, one item for each symbol in the GDX file, keyed with the symbol name.

Return type:

dict of str to pd.DataFrame

gdxpds.read_gdx.list_symbols(gdx_file, gams_dir=None)[source]

Returns the list of symbols available in gdx_file.

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

Returns:

List of symbol names

Return type:

list of str

gdxpds.read_gdx.get_data_types(gdx_file, gams_dir=None)[source]

Returns a dict of the symbols’ GamsDataTypes.

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

Returns:

dict of str to – Map of symbol names to the corresponding :py:class:GamsDataType`

Return type:

py:class:GamsDataType`

gdxpds.read_gdx.to_dataframe(gdx_file, symbol_name, gams_dir=None, old_interface=True, load_set_text=False)[source]

Interface for getting the data for a single symbol

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • symbol_name (str) – Name of the symbol whose data are to be read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

  • old_interface (bool) – Whether to use the old interface and return a dict, or the new interface, and simply return a pd.DataFrame

  • load_set_text (bool) – If True (default is False) and symbol_name is a Set, loads the GDX Text field into the dataframe rather than a c_bool.

Returns:

If old_interface (the default), returns a dict with a single entry, where the key is symbol_name and the value is the corresponding pd.DataFrame. Otherwise (if not old_interface), returns just the pd.DataFrame.

Return type:

dict of str to pd.DataFrame OR pd.DataFrame

gdxpds.special module

gdxpds.special.NUMPY_SPECIAL_VALUES = [None, nan, inf, -inf, 2.220446049250313e-16]

List of numpy special values in gdxGetSpecialValues order, i.e., [None, np.nan, np.inf, -np.inf, np.finfo(float).eps]

gdxpds.special.convert_gdx_to_np_svs(df, num_dims)[source]

Converts GDX special values to the corresponding numpy versions.

Parmeters

dfpandas.DataFrame

a GdxSymbol DataFrame as it was read directly from GDX

num_dimsint

the number of columns in df that list the dimension values for which the symbol value is non-zero / non-default

returns:

copy of df for which all GDX special values have been converted to their numpy equivalents

rtype:

pandas.DataFrame

gdxpds.special.is_np_eps(val)[source]
Parameters:

val (numeric) – value to test

Returns:

True if val is equal to eps (np.finfo(float).eps), False otherwise

Return type:

bool

gdxpds.special.is_np_sv(val)[source]
Parameters:

val (numeric) – value to test

Returns:

True if val is NaN, eps, or is in NUMPY_SPECIAL_VALUES; False otherwise

Return type:

bool

gdxpds.special.convert_np_to_gdx_svs(df, num_dims)[source]

Converts numpy special values to the corresponding GDX versions.

Parmeters

dfpandas.DataFrame

a GdxSymbol DataFrame in pandas/numpy form

num_dimsint

the number of columns in df that list the dimension values for which the symbol value is non-zero / non-default

returns:

copy of df for which all numpy special values have been converted to their GDX equivalents

rtype:

pandas.DataFrame

gdxpds.special.pd_isnan(val)[source]

Utility function for identifying None or NaN (which are indistinguishable in pandas).

Parameters:

val (numeric) – value to test

Returns:

True if val is a GDX encoded special value that maps to None or numpy.nan; False otherwise

Return type:

bool

gdxpds.special.pd_val_equal(val1, val2)[source]

Utility function used to test special value conversions.

Parameters:

  • val1 (float or None) – first value to compare

  • val2 (float or None) – second value to compare

Returns:

True if val1 and val2 are equal in the sense of == or they are both NaN/None. The values that map to None and np.nan are assumed to be equal because pandas cannot be relied upon to make the distinction.

Return type:

bool

gdxpds.special.gdx_isnan(val, gdxf)[source]

Utility function for equating the GDX special values that map to None or NaN (which are indistinguishable in pandas).

Parameters:

  • val (numeric) – value to test

  • gdxf (GdxFile) – GDX file containing the value. Provides np_to_gdx_svs map.

Returns:

True if val is a GDX encoded special value that maps to None or numpy.nan; False otherwise

Return type:

bool

gdxpds.special.gdx_val_equal(val1, val2, gdxf)[source]

Utility function used to test special value conversions.

Parameters:

  • val1 (float or GDX special value) – first value to compare

  • val2 (float or GDX special value) – second value to compare

  • gdxf (GdxFile) – GDX file containing val1 and val2

Returns:

True if val1 and val2 are equal in the sense of == or they are equivalent GDX-format special values. The values that map to None and np.nan are assumed to be equal because pandas cannot be relied upon to make the distinction.

Return type:

bool

gdxpds.special.load_specials(gams_dir_finder)[source]

Load special values

Needs to be called after gdxcc is loaded. Populates the module attributes SPECIAL_VALUES, GDX_TO_NP_SVS, and NP_TO_GDX_SVS.

Parameters:

gams_dir_finder (gdxpds.tools.GamsDirFinder) –

gdxpds.tools module

exception gdxpds.tools.Error[source]

Bases: Exception

Base class for all Exceptions raised by this package.

class gdxpds.tools.GamsDirFinder(gams_dir=None)[source]

Bases: object

Class for finding and accessing the system’s GAMS directory.

The find function first looks for the ‘GAMS_DIR’ environment variable. If that is unsuccessful, it next uses ‘which gams’ for POSIX systems, and the default install location, ‘C:/GAMS’, for Windows systems. In the latter case it prefers the largest version number.

You can always specify the GAMS directory directly, and this class will attempt to clean up your input. (Even on Windows, the GAMS path must use ‘/’ rather than ‘’.)

gams_dir_cache = 'C:/GAMS/34'
property gams_dir

The GAMS directory on this system.

class gdxpds.tools.NeedsGamsDir(gams_dir=None)[source]

Bases: object

Mix-in class that asserts that a GAMS directory is needed and provides the methods required to find and access it.

property gams_dir

The GAMS directory whose value has either been directly set or has been found using the GamsDirFinder class.

Return type:

str

gdxpds.write_gdx module

class gdxpds.write_gdx.Translator(dataframes, gams_dir=None)[source]

Bases: object

property dataframes
property gams_dir
property gdx
save_gdx(path, gams_dir=None)[source]
gdxpds.write_gdx.to_gdx(dataframes, path=None, gams_dir=None)[source]

Creates a gdxpds.gdx.GdxFile from dataframes and optionally writes it to path

Parameters:

  • dataframes (dict of str to pd.DataFrame) – symbol name to pd.DataFrame dict to be compiled into a single gdx file. Each DataFrame is assumed to represent a single set or parameter. The last column must be the parameter’s value, or the set’s listing of True/False, and must be labeled as (case insensitive) ‘value’.

  • path (None or pathlib.Path or str) – If provided, the gdx file will be written to this path

  • gams_dir (None or pathlib.Path or str) –

Return type:

gdxpds.gdx.GdxFile

Module contents

gdxpds.load_gdxcc(gams_dir=None)[source]

Method to initialize GAMS, especially to load required libraries that can sometimes conflict with other packages.

Parameters:

gams_dir (None or str) – if not None, directory containing the GAMS executable

gdxpds.to_dataframes(gdx_file, gams_dir=None, load_set_text=False)[source]

Primary interface for converting a GAMS GDX file to pandas DataFrames.

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

  • load_set_text (bool) – If True (default is False), then for every symbol that is a Set, loads the GDX Text field into the dataframe rather than a c_bool.

Returns:

Returns a dict of Pandas DataFrames, one item for each symbol in the GDX file, keyed with the symbol name.

Return type:

dict of str to pd.DataFrame

gdxpds.to_dataframe(gdx_file, symbol_name, gams_dir=None, old_interface=True, load_set_text=False)[source]

Interface for getting the data for a single symbol

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • symbol_name (str) – Name of the symbol whose data are to be read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

  • old_interface (bool) – Whether to use the old interface and return a dict, or the new interface, and simply return a pd.DataFrame

  • load_set_text (bool) – If True (default is False) and symbol_name is a Set, loads the GDX Text field into the dataframe rather than a c_bool.

Returns:

If old_interface (the default), returns a dict with a single entry, where the key is symbol_name and the value is the corresponding pd.DataFrame. Otherwise (if not old_interface), returns just the pd.DataFrame.

Return type:

dict of str to pd.DataFrame OR pd.DataFrame

gdxpds.list_symbols(gdx_file, gams_dir=None)[source]

Returns the list of symbols available in gdx_file.

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

Returns:

List of symbol names

Return type:

list of str

gdxpds.get_data_types(gdx_file, gams_dir=None)[source]

Returns a dict of the symbols’ GamsDataTypes.

Parameters:

  • gdx_file (pathlib.Path or str) – Path to the GDX file to read

  • gams_dir (None or pathlib.Path or str) – optional path to GAMS directory

Returns:

dict of str to – Map of symbol names to the corresponding :py:class:GamsDataType`

Return type:

py:class:GamsDataType`

gdxpds.to_gdx(dataframes, path=None, gams_dir=None)[source]

Creates a gdxpds.gdx.GdxFile from dataframes and optionally writes it to path

Parameters:

  • dataframes (dict of str to pd.DataFrame) – symbol name to pd.DataFrame dict to be compiled into a single gdx file. Each DataFrame is assumed to represent a single set or parameter. The last column must be the parameter’s value, or the set’s listing of True/False, and must be labeled as (case insensitive) ‘value’.

  • path (None or pathlib.Path or str) – If provided, the gdx file will be written to this path

  • gams_dir (None or pathlib.Path or str) –

Return type:

gdxpds.gdx.GdxFile