plans.root

A set of primitive classes used in other modules.

Classes

Collection(base_object[, name, alias])	A collection of primitive MbaE instances.
DataSet([name, alias])	The core DataSet base class.
FileSys([name, alias])	Handles files and folder organization
MbaE([name, alias])	Mba'e in Guarani means Thing.
Note([name, alias])
RecordTable([name, alias])	The base class for RecordTable.

class plans.root.MbaE(name='MyMbaE', alias=None)

Bases: object

Mba’e in Guarani means Thing.

Important

Mba’e is the origin. The very-basic almost-zero level class. Deeper than here is only the Python built-in object class.

Examples:

Here’s how to use the MbaE class:

Import MbaE:

# import the object
from plans.root import MbaE

MbaE instantiation

# MbaE instantiation
mb = MbaE(name="Algo", alias="al")

Retrieve metadata (not all attributes)

# Retrieve metadata (not all attributes)
dc = mb.get_metadata()
print(dc)

Retrieve metadata in a pandas.DataFrame

# Retrieve metadata in a :class:`pandas.DataFrame`
df = mb.get_metadata_df()
print(df.to_string(index=False))

Set new values for metadata

# Set new values for metadata
dc = {"Name": "Algo2", "Alias": "al2"}
mb.set(dict_setter=dc)

Boot attributes from csv file:

# Boot attributes from csv file:
mb.boot(bootfile="path/to/bootfile.csv")

__init__(name='MyMbaE', alias=None)

_create_alias()

If alias is None, it takes the first and last characters from name

_set_fields()

get_metadata()

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

get_metadata_df()

Get a pandas.DataFrame created from the metadata dictionary.

Returns:

a pandas.DataFrame with listed metadata

Return type:

pandas.DataFrame

setter(dict_setter)

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

boot(bootfile)

Boot basic attributes from a csv table.

Parameters:

bootfile (str) – file path to csv table with booting information.

Notes

Expected bootfile format:

field;value
name;ResTia
alias;Ra
...;...

export_metadata(folder, filename)

Export object metadata to destination file.

Parameters:

• folder (str) – path to folder

• filename (str) – file name without extension

export(folder, filename)

Export object resources to destination file.

Parameters:

• folder (str) – path to folder

• filename (str) – file name without extension

save()

Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

class plans.root.Collection(base_object, name='MyCollection', alias='Col0')

Bases: MbaE

A collection of primitive MbaE instances.

Useful for large scale manipulations in MbaE-based objects. Expected to have custom methods and attributes downstream.

Main Attributes

• catalog (pandas.DataFrame): A catalog containing metadata of the objects in the test_collection.

• collection (dict): A dictionary containing the objects in the Collection.

• name (str): The name of the Collection.

• alias (str): The name of the Collection.

• baseobject: The class of the base object used to initialize the Collection.

Main Methods

• __init__(self, base_object, name=”myCatalog”): Initializes a new Collection with a base object.

• update(self, details=False): Updates the Collection catalog.

• append(self, new_object): Appends a new object to the Collection.

• remove(self, name): Removes an object from the Collection.

Examples

Here’s how to use the Collection class:

Import objects:

# import MbaE-based object
from plans.root import MbaE

# import Collection
from plans.root import Collection

Instantiate Collection:

# instantiate Collection object
c = Collection(base_object=MbaE, name="Collection")

Append a new object to the Collection:

# append a new object
m1 = MbaE(name="Thing1", alias="al1")
c.append(m1)  # use .append()

Append extra objects:

# append extra objects
m2 = MbaE(name="Thing2", alias="al2")
c.append(m2)  # use .append()
m3 = MbaE(name="Res", alias="r")
c.append(m3)  # use .append()

Print the catalog pandas.DataFrame:

# print catalog :class:`pandas.DataFrame`
print(c.catalog)

Print the collection dict:

# print collection dict
print(c.collection)

Remove an object by using object name:

# remove object by object name
c.remove(name="Thing1")

Apply MbaE-based methods for Collection

# -- apply MbaE methods for Collection

# reset metadata
c.set(dict_setter={"Name": "Coll", "Alias": "C1"})

# Boot attributes from csv file:
c.boot(bootfile="/content/metadata_coll.csv")

__init__(base_object, name='MyCollection', alias='Col0')

Initialize the Collection object.

Parameters:

• base_object (MbaE) – MbaE-based object for collection

• name (str) – unique object name

• alias (str) – unique object alias.

_set_fields()

get_metadata()

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

update(details=False)

Update the Collection catalog.

Parameters:

details (bool) – Option to update catalog details, defaults to False.

append(new_object)

Append a new object to the Collection.

Parameters:

new_object (object) – Object to append.

Important

The object is expected to have a .get_metadata() method that returns a dictionary with metadata keys and values.

remove(name)

Remove an object from the Collection by the name.

Parameters:

name (str) – Name attribute of the object to remove.

class plans.root.DataSet(name='MyDataSet', alias='DS0')

Bases: MbaE

The core DataSet base class.

Notes

Expected to hold one pandas.DataFrame. This is a Dummy class. Expected to be implemented downstream for custom applications.

Examples

Import Dataset

# import Dataset
from plans.root import DataSet

Instantiate Dataset Object

# instantiate DataSet object
ds = DataSet(name="DataSet_1", alias="DS1")

Set object and load data

# set object and load data.
# Note: this dummy object expects "RM", "P", and "TempDB" as columns in data
ds.set(
    dict_setter={
        "Name": "DataSet_2",
        "Alias": "DS2",
        "Color": "red",
        "Source": "",
        "Description": "This is DataSet Object",
        "File_Data": "/content/data_ds1.csv"
    },
    load_data=True
)

Check data

# check data :class:`pandas.DataFrame`
print(ds.data.head())

Reload new data from file

# re-load new data from file
ds.load_data(file_data="/content/data_ds2.csv")

Get view

# get basic visual
ds.view(show=True)

Customize view specifications

# customize view parameters via the view_specs attribute:
ds.view_specs["title"] = "My Custom Title"
ds.view(show=True)

Save the view

# save the figure
ds.view_specs["folder"] = "path/to/folder"
ds.view_specs["filename"] = "my_visual"
ds.view_specs["fig_format"] = "png"
ds.view(show=False)

__init__(name='MyDataSet', alias='DS0')

_set_fields()

_set_view_specs()

Set view specifications in a dict.

get_metadata()

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

update()

Refresh all mutable attributes based on data (including paths).

setter(dict_setter, load_data=True)

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

load_data(file_data)

Load data from file.

Parameters:

file_data (str) – file path to data.

export(folder, filename, data_suffix=None)

Export object resources (e.g., data and metadata).

Parameters:

• folder (str) – path to folder

• filename (str) – file name without extension

• data_suffix (Union[str, None]) – suffix for file names

view(show=True)

Get the basic visualization.

Parameters:

show (bool) – option for showing instead of saving.

Note

Uses values in the view_specs() attribute for plotting.

static dc2df(dc, name='main')

class plans.root.FileSys(name='MyFS', alias='FS0')

Bases: DataSet

Handles files and folder organization

Notes

This class is useful for complex folder structure setups and controlling the status of expected file.

Warning

This is a Dummy class. Expected to be implemented downstream for custom applications.

__init__(name='MyFS', alias='FS0')

_set_fields()

get_metadata()

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

update()

Refresh all mutable attributes based on data (including paths).

setter(dict_setter, load_data=True)

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

load_data(file_data)

Load data from file.

Parameters:

file_data (str) – file path to data.

setup()

This method sets up the FileSys structure (default folders and files)

Danger

This method overwrites all existing default files.

setup_root_folder()

Make the root folder for file system. Skip if exists.

setup_subfolders()

Make all subfolders expected in the file system. Skip if exists.

setup_templates()

Copy all template files to default files in the file system.

Danger

This method overwrites all existing default files.

backup(dst_folder, version_id=None)

Backup project in a zip code

Parameters:

• dst_folder (str or Path) – path to destination folder

• version_id (str) – suffix label for versioning. if None, a timestamp is created.

static archive_folder(src_dir, dst_dir)

Archive to a zip folder

Parameters:

• src_dir (str) – source directory

• dst_dir (str) – destination directory

static check_file_status(files)

Static method for file existing checkup

Parameters:

files (list) – iterable with file paths

Returns:

list status (‘ok’ or ‘missing’)

Return type:

list

static copy_batch(dst_pattern, src_pattern)

Util static method for batch-copying pattern-based files.

Note

Pattern is expected to be a prefix prior to * suffix.

Parameters:

• dst_pattern (str) – destination path with file pattern. Example: path/to/dst_file_*.csv

• src_pattern (str) – source path with file pattern. Example: path/to/src_file_*.csv

static get_file_size_mb(file_path)

Util for getting the file size in MB

Parameters:

file_path (str) – path to file

Returns:

file size in MB

Return type:

float

class plans.root.Note(name='MyNote', alias='Nt1')

Bases: MbaE

__init__(name='MyNote', alias='Nt1')

_set_fields()

get_metadata()

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

load_metadata()

load_data()

load()

save()

Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

to_file(file_path, cleanup=True)

Export Note to markdown

Parameters:

file_path (str) – path to file

Returns:

Return type:

static remove_excessive_blank_lines(file_path)

static parse_metadata(note_file)

Extracts YAML metadata from the header of a Markdown file.

Parameters:

note_file – str, path to the Markdown file

Returns:

dict, extracted YAML metadata

static parse_yaml(yaml_content)

Parses YAML content into a dictionary.

Parameters:

yaml_content – str, YAML content as string

Returns:

dict, parsed YAML content

static metadata_to_list(metadata_dict)

static data_to_list(data_dict)

static parse_note(file_path)

static list_by_pattern(md_dict, patt_type='tag')

Retrieve a list of patterns from the note dictionary.

Parameters:

• md_dict (dict) – Dictionary containing note sections.

• patt_type (str) – Type of pattern to search for, either “tag” or “related”. Defaults to “tag”.

Returns:

List of found patterns or None if no patterns are found.

Return type:

list or None

class plans.root.RecordTable(name='MyRecordTable', alias='RcT')

Bases: DataSet

The base class for RecordTable.

A Record is expected to keep adding stamped records in order to keep track of large inventories, catalogs, etc. All records are expected to have a unique Id. It is considered to be a relational table.

Examples

Instantiate RecordTable Object

# Instantiate RecordTable object
rt = RecordTable(name="RecTable_1", alias="RT1")

Setup custom columns for the data

# Setup custom columns for the data
rt.columns_data_main = ["Name", "Size"]  # main data
rt.columns_data_extra = ["Type"]  # extra data
rt.columns_data_files = ["File_P"]  # file-related
rt.columns_data = rt.columns_data_main + rt.columns_data_extra + rt.columns_data_files

Set Object Metadata and Load Data

# Set object metadata and load data.
# Note: this dummy object expects the following columns in data
rt.set(
    dict_setter={
        "Name": "RecTable_01",
        "Alias": "RT01",
        "Color": "red",
        "Source": "-",
        "Description": "This is RecordTable Object",
        "File_Data": "/content/data_rt1.csv"
    },
    load_data=True
)

Check Data

# Check data :class:`pandas.DataFrame`
print(rt.data.head())

Load More Data from Other File

# Load more new data from other file
rt.load_data(file_data="/content/data_rt2.csv")

Insert New Record

# Insert new record from incoming dict
d2 = {
    "Name": "k",
    "Size": 177,
    "Type": 'inputs',
    "File_P": "/filee.pdf",
}
rt.insert_record(dict_rec=d2)

Edit Record

# Edit record based on ``RecId`` and new dict
d = {
    "Size": 34,
    "Name": "C"
}
rt.edit_record(rec_id="Rec0002", dict_rec=d)

Archive a Record

# Archive a record in the RT, that is ``RecStatus`` = ``Off``
rt.archive_record(rec_id="Rec0003")

Get a Record Dict by ID

# Get a record dict by id
d = rt.get_record(rec_id="Rec0001")
print(d)

Get a Record DataFrame by ID

# Get a record :class:`pandas.DataFrame` by id
df = rt.get_record_df(rec_id="Rec0001")
print(df.to_string(index=False))

Load Record Data from CSV

# Load record data from a ``csv`` file to a dict
d = rt.load_record_data(file_record_data="/content/rec_rt2.csv")
print(d)

Export a Record to CSV

# Export a record from the table to a ``csv`` file
f = rt.export_record(
    rec_id="Rec0001",
    folder_export="/content",
    filename="export_rt2"
)
print(f)

__init__(name='MyRecordTable', alias='RcT')

Initialize the object.

Parameters:

• name (str) – unique object name

• alias (str) – unique object alias. If None, it takes the first and last characters from name

_set_fields()

_set_base_columns()

Set base columns names.

Note

Base method. See downstream classes for actual implementation.

_set_data_columns()

Set specifics data columns names.

Note

Base method. See downstream classes for actual implementation.

_set_operator()

Set the builtin operator for automatic column calculations.

Note

Base method. See downstream classes for actual implementation.

_get_organized_columns()

Return the organized columns (base + data columns)

Returns:

organized columns (base + data columns)

Return type:

list

_last_id_int()

Compute the last ID integer in the record data table.

Returns:

last Id integer from the record data table.

Return type:

int

_next_recid()

Get the next record id string based on the existing ids.

Returns:

next record id

Return type:

str

_filter_dict_rec(input_dict)

Filter inputs record dictionary based on the expected table data columns.

Parameters:

input_dict (dict) – inputs record dictionary

Returns:

filtered record dictionary

Return type:

dict

update()

Refresh all mutable attributes based on data (including paths).

save()

Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

export(folder_export=None, filename=None, filter_archive=False)

Export the RecordTable data.

Parameters:

• folder_export (str) – folder to export

• filename (str) – file name (name alone, without file extension)

• filter_archive (bool) – option for exporting only records with RecStatus = On

Returns:

file path is export is successfull (1 otherwise)

Return type:

str or int

setter(dict_setter, load_data=True)

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

refresh_data()

Refresh data method for the object operator. Performs spreadsheet-like formulas for columns.

load_data(file_data)

Load data from file.

Parameters:

file_data (str) – file path to data.

set_data(input_df, append=True, inplace=True)

Set RecordTable data from incoming pandas.DataFrame.

Parameters:

• input_df (pandas.DataFrame) – incoming pandas.DataFrame

• append (bool) – option for appending the pandas.DataFrame to existing data. Default True

• inplace (bool) – option for overwrite data. Else return pandas.DataFrame. Default True

Notes

It handles if the pandas.DataFrame has or not the required RT columns Base Method.

insert_record(dict_rec)

Insert a record in the RT

Parameters:

dict_rec (dict) – inputs record dictionary

edit_record(rec_id, dict_rec, filter_dict=True)

Edit RT record

Parameters:

• rec_id (str) – record id

• dict_rec (dict) – incoming record dictionary

• filter_dict (bool) – option for filtering incoming record

archive_record(rec_id)

Archive a record in the RT, that is RecStatus = Off

Parameters:

rec_id (str) – record id

get_record(rec_id)

Get a record dict by id

Parameters:

rec_id (str) – record id

Returns:

record dictionary

Return type:

dict

get_record_df(rec_id)

Get a record pandas.DataFrame by id

Parameters:

rec_id (str) – record id

Returns:

record dictionary

Return type:

dict

load_record_data(file_record_data, input_field='Field', input_value='Value')

Load record data from a csv file to a dict

Note

This method does not insert the record data to the RecordTable.

Parameters:

• file_record_data (str) – file path to csv file.

• input_field – Name of Field column in the file.

• input_value – Name of Value column in the file.

Returns:

record dictionary

Return type:

dict

export_record(rec_id, filename=None, folder_export=None)

Export a record from the table to a csv file.

Parameters:

• rec_id (str) – record id

• filename (str) – file name (name alone, without file extension)

• folder_export (str) – folder to export

Returns:

path to exported file

Return type:

str

static get_timestamp()

Return a string timestamp

Returns:

full timestamp text %Y-%m0-%d %H:%M:%S

Return type:

str

static timedelta_disagg(timedelta)

Util static method for dissaggregation of time delta

Parameters:

timedelta (pandas.TimeDelta) – TimeDelta object from pandas

Returns:

dictionary of time delta

Return type:

dict

static timedelta_to_str(timedelta, dct_struct)

Util static method for string conversion of timedelta

Parameters:

• timedelta (pandas.TimeDelta) – TimeDelta object from pandas

• dct_struct (dict) – Dictionary of string strucuture. Ex: {‘Expected days’: ‘Days’}

Returns:

text of time delta

Return type:

str

static running_time(start_datetimes, kind='raw')

Util static method for computing the runnning time for a list of starting dates

Parameters:

• start_datetimes (list) – List of starting dates

• kind (str) – mode for output format (‘raw’, ‘human’ or ‘age’)

Returns:

list of running time

Return type:

list