Uncategorized

escaping pd.set_option hell

Every one of my python scripts, Jupyter notebooks, IPython console sessions, etc has long began with the omnipresent imports and settings override:

#!/usr/bin/env python3
# *_* coding: utf-8 *_*

# author: Graham Duncan
# date: 2020-06-18

"""
module docstring - short summary

If the description is long, the first line should be a short summary that makes sense on its own,
separated from the rest by a newline
"""



import numpy as np
import pandas as pd

pd.set_option('display.width', 1500)
pd.set_option('display.max_columns', 25)

This is a whole lot of shit, and the IDE won’t collapse it all together (and never collapses the settings overrides which is annoying.

So I did a thing, which is probably the wrong thing, but No Ragrets (so far):

I found the declarations of the default settings and just hard coded overrides in (mind you, I already did the actually smart thing of editing startup files for IPython and Jupyter, so don’t @ me.

The file of interest is in the core module of your pandas package, and the file is named config_init.py.

/your-path-to-python-interpreter/python3.8/site-packages/pandas/core/config_init.py

So mine, for instance, is:

/Libary/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pandas/core/config_init.py

For any (probably ill-advised) fellow travelers out there, if you want to give it a shot..the place where the magic happens is:

with cf.config_prefix("display"):
    cf.register_option("precision", 6, pc_precision_doc, validator=is_nonnegative_int)
    cf.register_option(
        "float_format",
        None,
        float_format_doc,
        validator=is_one_of_factory([None, is_callable]),
    )
    cf.register_option("column_space", 12, validator=is_int)
    cf.register_option(
        "max_info_rows",
        1690785,
        pc_max_info_rows_doc,
        validator=is_instance_factory((int, type(None))),
    )
    cf.register_option("max_rows", 60, pc_max_rows_doc, validator=is_nonnegative_int)
    cf.register_option(
        "min_rows",
        10,
        pc_min_rows_doc,
        validator=is_instance_factory([type(None), int]),
    )
    cf.register_option("max_categories", 8, pc_max_categories_doc, validator=is_int)

    def _deprecate_negative_int_max_colwidth(key):
        value = cf.get_option(key)
        if value is not None and value < 0:
            warnings.warn(
                "Passing a negative integer is deprecated in version 1.0 and "
                "will not be supported in future version. Instead, use None "
                "to not limit the column width.",
                FutureWarning,
                stacklevel=4,
            )

    cf.register_option(
        # FIXME: change `validator=is_nonnegative_int`
        # in version 1.2
        "max_colwidth",
        50,
        max_colwidth_doc,
        validator=is_instance_factory([type(None), int]),
        cb=_deprecate_negative_int_max_colwidth,
    )
    if is_terminal():
        max_cols = 25  # automatically determine optimal number of columns
    else:
        max_cols = 25  # cannot determine optimal number of columns
    cf.register_option(
        "max_columns", max_cols, pc_max_cols_doc, validator=is_nonnegative_int
    )
    cf.register_option(
        "large_repr",
        "truncate",
        pc_large_repr_doc,
        validator=is_one_of_factory(["truncate", "info"]),
    )
    cf.register_option("max_info_columns", 100, pc_max_info_cols_doc, validator=is_int)
    cf.register_option(
        "colheader_justify", "right", colheader_justify_doc, validator=is_text
    )
    cf.register_option("notebook_repr_html", True, pc_nb_repr_h_doc, validator=is_bool)
    cf.register_option("pprint_nest_depth", 3, pc_pprint_nest_depth, validator=is_int)
    cf.register_option("multi_sparse", True, pc_multi_sparse_doc, validator=is_bool)
    cf.register_option("expand_frame_repr", True, pc_expand_repr_doc)
    cf.register_option(
        "show_dimensions",
        "truncate",
        pc_show_dimensions_doc,
        validator=is_one_of_factory([True, False, "truncate"]),
    )
    cf.register_option("chop_threshold", None, pc_chop_threshold_doc)
    cf.register_option("max_seq_items", 100, pc_max_seq_items)
    cf.register_option(
        "width", 1500, pc_width_doc, validator=is_instance_factory([type(None), int])
    )
    cf.register_option(
        "memory_usage",
        True,
        pc_memory_usage_doc,
        validator=is_one_of_factory([None, True, False, "deep"]),
    )
    cf.register_option(
        "unicode.east_asian_width", False, pc_east_asian_width_doc, validator=is_bool
    )
    cf.register_option(
        "unicode.ambiguous_as_wide", False, pc_east_asian_width_doc, validator=is_bool
    )
    cf.register_option("latex.repr", False, pc_latex_repr_doc, validator=is_bool)
    cf.register_option("latex.escape", True, pc_latex_escape, validator=is_bool)
    cf.register_option("latex.longtable", False, pc_latex_longtable, validator=is_bool)
    cf.register_option(
        "latex.multicolumn", True, pc_latex_multicolumn, validator=is_bool
    )
    cf.register_option(
        "latex.multicolumn_format", "l", pc_latex_multicolumn, validator=is_text
    )
    cf.register_option("latex.multirow", False, pc_latex_multirow, validator=is_bool)
    cf.register_option(
        "html.table_schema",
        False,
        pc_table_schema_doc,
        validator=is_bool,
        cb=table_schema_cb,
    )
    cf.register_option("html.border", 1, pc_html_border_doc, validator=is_int)
    cf.register_option(
        "html.use_mathjax", True, pc_html_use_mathjax_doc, validator=is_bool
    )

You can see I just edited the inline default for max_cols = 25 and the display.width attribute to 1500.

Granted this is probably a bad idea since you’ll have to change this anytime you switch interpreters, use a new venv, or upgrade your pandas version.

But those all sound like problems for another day ¯\_(ツ)_/¯

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s