plans.root#

This module provides a set of the primitive classes used.

Overview#

# todo [major docstring improvement] – overview Mauris gravida ex quam, in porttitor lacus lobortis vitae. In a lacinia nisl. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.

Example#

# todo [major docstring improvement] – examples Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla mollis tincidunt erat eget iaculis. Mauris gravida ex quam, in porttitor lacus lobortis vitae. In a lacinia nisl.

import numpy as np
print("Hello World!")

Mauris gravida ex quam, in porttitor lacus lobortis vitae. In a lacinia nisl. Mauris gravida ex quam, in porttitor lacus lobortis vitae. In a lacinia nisl.

Classes

`Collection`(base_object[, name, alias])	A collection of primitive `MbaE` objects with associated metadata.
`DataSet`([name, alias])	The core `DataSet` base class.
`FileSys`([name, alias])	The core `FileSys` base class.
`MbaE`([name, alias])	Mba'e in Guarani means Thing.
`Note`([name, alias])
`RecordTable`([name, alias])	The base class for `RecordTable`.

class plans.root.MbaE(name='MyMbaE', alias=None)[source]#

Bases: object

Mba’e in Guarani means Thing.

Important

Mba’e is the origin. The very-basic almost-zero level object. Deeper than here is only the Python builtin object class.

Examples:

Here’s how to use the MbaE class:

Import MbaE:

# import the object
from plans.root import MbaE

MbaE instantiation

# MbaE instantiation
mb = MbaE(name="Algo", alias="al")

Retrieve metadata (not all attributes)

# Retrieve metadata (not all attributes)
dc = mb.get_metadata()
print(dc)

Retrieve metadata in a pandas.DataFrame

# Retrieve metadata in a :class:`pandas.DataFrame`
df = mb.get_metadata_df()
print(df.to_string(index=False))

Set new values for metadata

# Set new values for metadata
dc = {"Name": "Algo2", "Alias": "al2"}
mb.set(dict_setter=dc)

Boot attributes from csv file:

# Boot attributes from csv file:
mb.boot(bootfile="path/to/bootfile.csv")

__init__(name='MyMbaE', alias=None)[source]#

Initialize the MbaE object.

Parameters:

name (str) – unique object name
alias (str) – unique object alias. If None, it takes the first and last characters from name

_create_alias()[source]#: If alias is None, it takes the first and last characters from name

_set_fields()[source]#: Set fields names

get_metadata()[source]#

Get a dictionary with object metadata.

Returns:: dictionary with all metadata
Return type:: dict

Warning

Metadata does not necessarily include all object attributes.

get_metadata_df()[source]#

Get a pandas.DataFrame created from the metadata dictionary.

Returns:: a pandas.DataFrame with listed metadata
Return type:: pandas.DataFrame

setter(dict_setter)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:: dict_setter (dict) – incoming dictionary with attribute values

boot(bootfile)[source]#

Boot basic attributes from a csv table.

Parameters:: bootfile (str) – file path to csv table with booting information.

Notes

Expected bootfile format:

field;value
name;ResTia
alias;Ra
...;...

export_metadata(folder, filename)[source]#

Export object metadata to destination file.

Parameters:

folder (str) – path to folder
filename (str) – file name without extension

export(folder, filename)[source]#

Export object resources to destination file.

Parameters:

folder (str) – path to folder
filename (str) – file name without extension

save()[source]#: Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

class plans.root.Collection(base_object, name='MyCollection', alias='Col0')[source]#

Bases: MbaE

A collection of primitive MbaE objects with associated metadata. Useful for large scale manipulations in MbaE-based objects. Expected to have custom methods and attributes downstream.

Main Attributes

catalog (pandas.DataFrame): A catalog containing metadata of the objects in the test_collection.
collection (dict): A dictionary containing the objects in the Collection.
name (str): The name of the Collection.
alias (str): The name of the Collection.
baseobject: The class of the base object used to initialize the Collection.

Main Methods

__init__(self, base_object, name=”myCatalog”): Initializes a new Collection with a base object.
update(self, details=False): Updates the Collection catalog.
append(self, new_object): Appends a new object to the Collection.
remove(self, name): Removes an object from the Collection.

Examples

Here’s how to use the Collection class:

Import objects:

# import MbaE-based object
from plans.root import MbaE

# import Collection
from plans.root import Collection

Instantiate Collection:

# instantiate Collection object
c = Collection(base_object=MbaE, name="Collection")

Append a new object to the Collection:

# append a new object
m1 = MbaE(name="Thing1", alias="al1")
c.append(m1)  # use .append()

Append extra objects:

# append extra objects
m2 = MbaE(name="Thing2", alias="al2")
c.append(m2)  # use .append()
m3 = MbaE(name="Res", alias="r")
c.append(m3)  # use .append()

Print the catalog pandas.DataFrame:

# print catalog :class:`pandas.DataFrame`
print(c.catalog)

Print the collection dict:

# print collection dict
print(c.collection)

Remove an object by using object name:

# remove object by object name
c.remove(name="Thing1")

Apply MbaE-based methods for Collection

# -- apply MbaE methods for Collection

# reset metadata
c.set(dict_setter={"Name": "Coll", "Alias": "C1"})

# Boot attributes from csv file:
c.boot(bootfile="/content/metadata_coll.csv")

__init__(base_object, name='MyCollection', alias='Col0')[source]#

Initialize the Collection object.

Parameters:

base_object (MbaE) – MbaE-based object for collection
name (str) – unique object name
alias (str) – unique object alias. If None, it takes the first and last characters from name

_set_fields()[source]#: Set fields names

get_metadata()[source]#

Get a dictionary with object metadata.

Note

Metadata does not necessarily inclue all object attributes.

Returns:: dictionary with all metadata
Return type:: dict

update(details=False)[source]#

Update the Collection catalog.

Parameters:: details (bool) – Option to update catalog details, defaults to False.

append(new_object)[source]#

Append a new object to the Collection.

Parameters:: new_object (object) – Object to append.

Important

The object is expected to have a .get_metadata() method that returns a dictionary with metadata keys and values.

remove(name)[source]#

Remove an object from the Collection by the name.

Parameters:: name (str) – Name attribute of the object to remove.

class plans.root.DataSet(name='MyDataSet', alias='DS0')[source]#

Bases: MbaE

The core DataSet base class.

Notes

Expected to hold one pandas.DataFrame. This is a Base and Dummy object. Expected to be implemented downstream for custom applications.

Examples

Import Dataset

# import Dataset
from plans.root import DataSet

Instantiate Dataset Object

# instantiate DataSet object
ds = DataSet(name="DataSet_1", alias="DS1")

Set object and load data

# set object and load data.
# Note: this dummy object expects "RM", "P", and "TempDB" as columns in data
ds.set(
    dict_setter={
        "Name": "DataSet_2",
        "Alias": "DS2",
        "Color": "red",
        "Source": "",
        "Description": "This is DataSet Object",
        "File_Data": "/content/data_ds1.csv"
    },
    load_data=True
)

Check data

# check data :class:`pandas.DataFrame`
print(ds.data.head())

Reload new data from file

# re-load new data from file
ds.load_data(file_data="/content/data_ds2.csv")

Get view

# get basic visual
ds.view(show=True)

Customize view specifications

# customize view parameters via the view_specs attribute:
ds.view_specs["title"] = "My Custom Title"
ds.view(show=True)

Save the view

# save the figure
ds.view_specs["folder"] = "path/to/folder"
ds.view_specs["filename"] = "my_visual"
ds.view_specs["fig_format"] = "png"
ds.view(show=False)

__init__(name='MyDataSet', alias='DS0')[source]#

Initialize the DataSet object.

Parameters:

name (str) – unique object name
alias (str) – unique object alias. If None, it takes the first and last characters from name

_set_fields()[source]#: Set fields names

_set_view_specs()[source]#: Set view specifications.

get_metadata()[source]#

Get a dictionary with object metadata.

Returns:: dictionary with all metadata
Return type:: dict

Warning

Metadata does not necessarily include all object attributes.

update()[source]#: Refresh all mutable attributes based on data (including paths).

setter(dict_setter, load_data=True)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:: dict_setter (dict) – incoming dictionary with attribute values

load_data(file_data)[source]#

Load data from file.

Parameters:: file_data (str) – file path to data.

export(folder, filename, data_suffix=None)[source]#

Export object resources (e.g., data and metadata).

Parameters:

folder (str) – path to folder
filename (str) – file name without extension
data_suffix (Union[str, None]) – suffix for file names

view(show=True)[source]#

Get the basic visualization.

Parameters:: show (bool) – option for showing instead of saving.

Note

Uses values in the view_specs() attribute for plotting.

static dc2df(dc, name='main')[source]#

class plans.root.Note(name='MyNote', alias='Nt1')[source]#

Bases: MbaE

__init__(name='MyNote', alias='Nt1')[source]#

Initialize the MbaE object.

Parameters:

name (str) – unique object name
alias (str) – unique object alias. If None, it takes the first and last characters from name

_set_fields()[source]#: Set fields names

get_metadata()[source]#

Get a dictionary with object metadata.

Note

Metadata does not necessarily inclue all object attributes.

Returns:: dictionary with all metadata
Return type:: dict

load_metadata()[source]#

load_data()[source]#

load()[source]#

save()[source]#: Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

to_file(file_path, cleanup=True)[source]#

Export Note to markdown

Parameters:: file_path (str) – path to file
Returns:
Return type:

static remove_excessive_blank_lines(file_path)[source]#

static parse_metadata(note_file)[source]#

Extracts YAML metadata from the header of a Markdown file.

Parameters:: note_file – str, path to the Markdown file
Returns:: dict, extracted YAML metadata

static parse_yaml(yaml_content)[source]#

Parses YAML content into a dictionary.

Parameters:: yaml_content – str, YAML content as string
Returns:: dict, parsed YAML content

static metadata_to_list(metadata_dict)[source]#

static data_to_list(data_dict)[source]#

static parse_note(file_path)[source]#

static list_by_pattern(md_dict, patt_type='tag')[source]#

Retrieve a list of patterns from the note dictionary.

Parameters:

md_dict (dict) – Dictionary containing note sections.
patt_type (str) – Type of pattern to search for, either “tag” or “related”. Defaults to “tag”.

Returns:

List of found patterns or None if no patterns are found.

Return type:

list or None

class plans.root.RecordTable(name='MyRecordTable', alias='RcT')[source]#

Bases: DataSet

The base class for RecordTable.

A Record is expected to keep adding stamped records in order to keep track of large inventories, catalogs, etc. All records are expected to have a unique Id. It is considered to be a relational table.

Examples

Instantiate RecordTable Object

# Instantiate RecordTable object
rt = RecordTable(name="RecTable_1", alias="RT1")

Setup custom columns for the data

# Setup custom columns for the data
rt.columns_data_main = ["Name", "Size"]  # main data
rt.columns_data_extra = ["Type"]  # extra data
rt.columns_data_files = ["File_P"]  # file-related
rt.columns_data = rt.columns_data_main + rt.columns_data_extra + rt.columns_data_files

Set Object Metadata and Load Data

# Set object metadata and load data.
# Note: this dummy object expects the following columns in data
rt.set(
    dict_setter={
        "Name": "RecTable_01",
        "Alias": "RT01",
        "Color": "red",
        "Source": "-",
        "Description": "This is RecordTable Object",
        "File_Data": "/content/data_rt1.csv"
    },
    load_data=True
)

Check Data

# Check data :class:`pandas.DataFrame`
print(rt.data.head())

Load More Data from Other File

# Load more new data from other file
rt.load_data(file_data="/content/data_rt2.csv")

Insert New Record

# Insert new record from incoming dict
d2 = {
    "Name": "k",
    "Size": 177,
    "Type": 'inputs',
    "File_P": "/filee.pdf",
}
rt.insert_record(dict_rec=d2)

Edit Record

# Edit record based on ``RecId`` and new dict
d = {
    "Size": 34,
    "Name": "C"
}
rt.edit_record(rec_id="Rec0002", dict_rec=d)

Archive a Record

# Archive a record in the RT, that is ``RecStatus`` = ``Off``
rt.archive_record(rec_id="Rec0003")

Get a Record Dict by ID

# Get a record dict by id
d = rt.get_record(rec_id="Rec0001")
print(d)

Get a Record DataFrame by ID

# Get a record :class:`pandas.DataFrame` by id
df = rt.get_record_df(rec_id="Rec0001")
print(df.to_string(index=False))

Load Record Data from CSV

# Load record data from a ``csv`` file to a dict
d = rt.load_record_data(file_record_data="/content/rec_rt2.csv")
print(d)

Export a Record to CSV

# Export a record from the table to a ``csv`` file
f = rt.export_record(
    rec_id="Rec0001",
    folder_export="/content",
    filename="export_rt2"
)
print(f)

__init__(name='MyRecordTable', alias='RcT')[source]#

Initialize the object.

Parameters:

name (str) – unique object name
alias (str) – unique object alias. If None, it takes the first and last characters from name

_set_fields()[source]#: Set fields names

_set_base_columns()[source]#: Set base columns names.

Note

Base method. See downstream classes for actual implementation.

_set_data_columns()[source]#: Set specifics data columns names.

Note

Base method. See downstream classes for actual implementation.

_set_operator()[source]#: Set the builtin operator for automatic column calculations.

Note

Base method. See downstream classes for actual implementation.

_get_organized_columns()[source]#

Return the organized columns (base + data columns)

Returns:: organized columns (base + data columns)
Return type:: list

_last_id_int()[source]#

Compute the last ID integer in the record data table.

Returns:: last Id integer from the record data table.
Return type:: int

_next_recid()[source]#

Get the next record id string based on the existing ids.

Returns:: next record id
Return type:: str

_filter_dict_rec(input_dict)[source]#

Filter inputs record dictionary based on the expected table data columns.

Parameters:: input_dict (dict) – inputs record dictionary
Returns:: filtered record dictionary
Return type:: dict

update()[source]#: Refresh all mutable attributes based on data (including paths).

save()[source]#: Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

export(folder_export=None, filename=None, filter_archive=False)[source]#

Export the RecordTable data.

Parameters:

folder_export (str) – folder to export
filename (str) – file name (name alone, without file extension)
filter_archive (bool) – option for exporting only records with RecStatus = On

Returns:

file path is export is successfull (1 otherwise)

Return type:

str or int

setter(dict_setter, load_data=True)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:: dict_setter (dict) – incoming dictionary with attribute values

refresh_data()[source]#: Refresh data method for the object operator. Performs spreadsheet-like formulas for columns.

load_data(file_data)[source]#

Load data from file.

Parameters:: file_data (str) – file path to data.

set_data(input_df, append=True, inplace=True)[source]#

Set RecordTable data from incoming pandas.DataFrame.

Parameters:

input_df (pandas.DataFrame) – incoming pandas.DataFrame
append (bool) – option for appending the pandas.DataFrame to existing data. Default True
inplace (bool) – option for overwrite data. Else return pandas.DataFrame. Default True

Notes

It handles if the pandas.DataFrame has or not the required RT columns Base Method.

insert_record(dict_rec)[source]#

Insert a record in the RT

Parameters:: dict_rec (dict) – inputs record dictionary

edit_record(rec_id, dict_rec, filter_dict=True)[source]#

Edit RT record

Parameters:

rec_id (str) – record id
dict_rec (dict) – incoming record dictionary
filter_dict (bool) – option for filtering incoming record

archive_record(rec_id)[source]#

Archive a record in the RT, that is RecStatus = Off

Parameters:: rec_id (str) – record id

get_record(rec_id)[source]#

Get a record dict by id

Parameters:: rec_id (str) – record id
Returns:: record dictionary
Return type:: dict

get_record_df(rec_id)[source]#

Get a record pandas.DataFrame by id

Parameters:: rec_id (str) – record id
Returns:: record dictionary
Return type:: dict

load_record_data(file_record_data, input_field='Field', input_value='Value')[source]#

Load record data from a csv file to a dict

Note

This method does not insert the record data to the RecordTable.

Parameters:

file_record_data (str) – file path to csv file.
input_field – Name of Field column in the file.
input_value – Name of Value column in the file.

Returns:

record dictionary

Return type:

dict

export_record(rec_id, filename=None, folder_export=None)[source]#

Export a record from the table to a csv file.

Parameters:

rec_id (str) – record id
filename (str) – file name (name alone, without file extension)
folder_export (str) – folder to export

Returns:

path to exported file

Return type:

str

static get_timestamp()[source]#

Return a string timestamp

Returns:: full timestamp text %Y-%m0-%d %H:%M:%S
Return type:: str

static timedelta_disagg(timedelta)[source]#

Util static method for dissaggregation of time delta

Parameters:: timedelta (pandas.TimeDelta) – TimeDelta object from pandas
Returns:: dictionary of time delta
Return type:: dict

static timedelta_to_str(timedelta, dct_struct)[source]#

Util static method for string conversion of timedelta

Parameters:

timedelta (pandas.TimeDelta) – TimeDelta object from pandas
dct_struct (dict) – Dictionary of string strucuture. Ex: {‘Expected days’: ‘Days’}

Returns:

text of time delta

Return type:

str

static running_time(start_datetimes, kind='raw')[source]#

Util static method for computing the runnning time for a list of starting dates

Parameters:

start_datetimes (list) – List of starting dates
kind (str) – mode for output format (‘raw’, ‘human’ or ‘age’)

Returns:

list of running time

Return type:

list

class plans.root.FileSys(name='MyFS', alias='FS0')[source]#

Bases: DataSet

The core FileSys base class. Handles files and folder organization,

Notes

This class is useful for complex folder structure setups and controlling the status of expected file.

Warning

This is a Base and Dummy object. Expected to be implemented downstream for custom applications.

__init__(name='MyFS', alias='FS0')[source]#

Initialize the FileSys object.

Parameters:

folder_base (str) – path to File System folder location
name (str) – unique class name
alias (str) – unique object alias. If None, it takes the first and last characters from name

_set_fields()[source]#: Set fields names

get_metadata()[source]#

Get a dictionary with object metadata.

Returns:: dictionary with all metadata
Return type:: dict

Warning

Metadata does not necessarily include all object attributes.

update()[source]#: Refresh all mutable attributes based on data (including paths).

setter(dict_setter, load_data=True)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:: dict_setter (dict) – incoming dictionary with attribute values

load_data(file_data)[source]#

Load data from file.

Parameters:: file_data (str) – file path to data.

setup()[source]#: This method sets up all the FileSys structure (default folders and files)

Danger

This method overwrites all existing default files.

setup_root_folder()[source]#: Make the root folder for file system. Skip if exists.

setup_subfolders()[source]#: Make all subfolders expected in the file system. Skip if exists.

setup_templates()[source]#: Copy all template files to default files in the file system.

Danger

This method overwrites all existing default files.

backup(dst_folder, version_id=None)[source]#

Backup project in a zip code

Parameters:

dst_folder (str or Path) – path to destination folder
version_id (str) – suffix label for versioning. if None, a timestamp is created.

static archive_folder(src_dir, dst_dir)[source]#

Archive to a zip folder

Parameters:

src_dir (str) – source directory
dst_dir (str) – destination directory

static check_file_status(files)[source]#

Static method for file existing checkup

Parameters:: files (list) – iterable with file paths
Returns:: list status (‘ok’ or ‘missing’)
Return type:: list

static copy_batch(dst_pattern, src_pattern)[source]#

Util static method for batch-copying pattern-based files.

Note

Pattern is expected to be a prefix prior to * suffix.

Parameters:

dst_pattern (str) – destination path with file pattern. Example: path/to/dst_file_*.csv
src_pattern (str) – source path with file pattern. Example: path/to/src_file_*.csv

static get_file_size_mb(file_path)[source]#

Util for getting the file size in MB

Parameters:: file_path (str) – path to file
Returns:: file size in MB
Return type:: float

plans.root#

Overview#

Example#

This Page