babilonia.root#

Foundational base classes shared across the babilonia package.

Defines the class hierarchy rooted at MbaE, from which Collection, DataSet, FileSys, RecordTable, and Note derive. Downstream modules inherit these primitives rather than Python builtins directly.

Classes

Collection(base_object[, name, alias])

A collection of primitive MbaE instances.

DataSet([name, alias])

The core DataSet base class.

FileSys([name, alias])

Handles files and folder organization

MbaE([name, alias])

Mba'e in Guarani means Thing.

Note([name, alias])

RecordTable([name, alias])

The base class for RecordTable.
class babilonia.root.MbaE(name='MyMbaE', alias=None)[source]#

Bases: object

Mba’e in Guarani means Thing.

Important

Mba’e is the origin. The very-basic almost-zero level class. Deeper than here is only the Python built-in object class.

Examples:

Here’s how to use the MbaE class:

Import MbaE:

# import the object
from plans.root import MbaE

MbaE instantiation

# MbaE instantiation
mb = MbaE(name="Algo", alias="al")

Retrieve metadata (not all attributes)

# Retrieve metadata (not all attributes)
dc = mb.get_metadata()
print(dc)

Retrieve metadata in a pandas.DataFrame

# Retrieve metadata in a :class:`pandas.DataFrame`
df = mb.get_metadata_df()
print(df.to_string(index=False))

Set new values for metadata

# Set new values for metadata
dc = {"Name": "Algo2", "Alias": "al2"}
mb.set(dict_setter=dc)

Boot attributes from csv file:

# Boot attributes from csv file:
mb.boot(bootfile="path/to/bootfile.csv")
get_metadata()[source]#

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

get_metadata_df()[source]#

Get a pandas.DataFrame created from the metadata dictionary.

Returns:

a pandas.DataFrame with listed metadata

Return type:

pandas.DataFrame

setter(dict_setter)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

boot(bootfile)[source]#

Boot basic attributes from a csv table.

Parameters:

bootfile (str) – file path to csv table with booting information.

Notes

Expected bootfile format:

field;value
name;ResTia
alias;Ra
...;...
export_metadata(folder, filename)[source]#

Export object metadata to destination file.

Parameters:

  • folder (str) – path to folder

  • filename (str) – file name without extension

export(folder, filename)[source]#

Export object resources to destination file.

Parameters:

  • folder (str) – path to folder

  • filename (str) – file name without extension

save()[source]#

Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

class babilonia.root.Collection(base_object, name='MyCollection', alias='Col0')[source]#

Bases: MbaE

A collection of primitive MbaE instances.

Useful for large scale manipulations in MbaE-based objects. Expected to have custom methods and attributes downstream.

Main Attributes

  • catalog (pandas.DataFrame): A catalog containing metadata of the objects in the test_collection.

  • collection (dict): A dictionary containing the objects in the Collection.

  • name (str): The name of the Collection.

  • alias (str): The name of the Collection.

  • baseobject: The class of the base object used to initialize the Collection.

Main Methods

  • __init__(self, base_object, name=”myCatalog”): Initializes a new Collection with a base object.

  • update(self, details=False): Updates the Collection catalog.

  • append(self, new_object): Appends a new object to the Collection.

  • remove(self, name): Removes an object from the Collection.

Examples

Here’s how to use the Collection class:

Import objects:

# import MbaE-based object
from plans.root import MbaE

# import Collection
from plans.root import Collection

Instantiate Collection:

# instantiate Collection object
c = Collection(base_object=MbaE, name="Collection")

Append a new object to the Collection:

# append a new object
m1 = MbaE(name="Thing1", alias="al1")
c.append(m1)  # use .append()

Append extra objects:

# append extra objects
m2 = MbaE(name="Thing2", alias="al2")
c.append(m2)  # use .append()
m3 = MbaE(name="Res", alias="r")
c.append(m3)  # use .append()

Print the catalog pandas.DataFrame:

# print catalog :class:`pandas.DataFrame`
print(c.catalog)

Print the collection dict:

# print collection dict
print(c.collection)

Remove an object by using object name:

# remove object by object name
c.remove(name="Thing1")

Apply MbaE-based methods for Collection

# -- apply MbaE methods for Collection

# reset metadata
c.set(dict_setter={"Name": "Coll", "Alias": "C1"})

# Boot attributes from csv file:
c.boot(bootfile="/content/metadata_coll.csv")
get_metadata()[source]#

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

update(details=False)[source]#

Update the Collection catalog.

Parameters:

details (bool) – Option to update catalog details, defaults to False.

append(new_object)[source]#

Append a new object to the Collection.

Parameters:

new_object (object) – Object to append.

Important

The object is expected to have a .get_metadata() method that returns a dictionary with metadata keys and values.

remove(name)[source]#

Remove an object from the Collection by the name.

Parameters:

name (str) – Name attribute of the object to remove.

class babilonia.root.DataSet(name='MyDataSet', alias='DS0')[source]#

Bases: MbaE

The core DataSet base class.

Notes

Expected to hold one pandas.DataFrame. This is a Dummy class. Expected to be implemented downstream for custom applications.

Examples

Import Dataset

# import Dataset
from plans.root import DataSet

Instantiate Dataset Object

# instantiate DataSet object
ds = DataSet(name="DataSet_1", alias="DS1")

Set object and load data

# set object and load data.
# Note: this dummy object expects "RM", "P", and "TempDB" as columns in data
ds.set(
    dict_setter={
        "Name": "DataSet_2",
        "Alias": "DS2",
        "Color": "red",
        "Source": "",
        "Description": "This is DataSet Object",
        "File_Data": "/content/data_ds1.csv"
    },
    load_data=True
)

Check data

# check data :class:`pandas.DataFrame`
print(ds.data.head())

Reload new data from file

# re-load new data from file
ds.load_data(file_data="/content/data_ds2.csv")

Get view

# get basic visual
ds.view(show=True)

Customize view specifications

# customize view parameters via the view_specs attribute:
ds.view_specs["title"] = "My Custom Title"
ds.view(show=True)

Save the view

# save the figure
ds.view_specs["folder"] = "path/to/folder"
ds.view_specs["filename"] = "my_visual"
ds.view_specs["fig_format"] = "png"
ds.view(show=False)
get_metadata()[source]#

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

update()[source]#

Refresh all mutable attributes based on data (including paths).

setter(dict_setter, load_data=True)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

load_data(file_data)[source]#

Load data from file.

Parameters:

file_data (str) – file path to data.

export(folder, filename, data_suffix=None)[source]#

Export object resources (e.g., data and metadata).

Parameters:

  • folder (str) – path to folder

  • filename (str) – file name without extension

  • data_suffix (Union[str, None]) – suffix for file names

view(show=True)[source]#

Get the basic visualization.

Parameters:

show (bool) – option for showing instead of saving.

Note

Uses values in the view_specs() attribute for plotting.

static dc2df(dc, name='main')[source]#
class babilonia.root.FileSys(name='MyFS', alias='FS0')[source]#

Bases: DataSet

Handles files and folder organization

Notes

This class is useful for complex folder structure setups and controlling the status of expected file.

Warning

This is a Dummy class. Expected to be implemented downstream for custom applications.

get_metadata()[source]#

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

update()[source]#

Refresh all mutable attributes based on data (including paths).

setter(dict_setter, load_data=True)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

load_data(file_data)[source]#

Load data from file.

Parameters:

file_data (str) – file path to data.

setup()[source]#

This method sets up the FileSys structure (default folders and files)

Danger

This method overwrites all existing default files.

setup_root_folder()[source]#

Make the root folder for file system. Skip if exists.

setup_subfolders()[source]#

Make all subfolders expected in the file system. Skip if exists.

setup_templates()[source]#

Copy all template files to default files in the file system.

Danger

This method overwrites all existing default files.

backup(dst_folder, version_id=None)[source]#

Backup project in a zip code

Parameters:

  • dst_folder (str or Path) – path to destination folder

  • version_id (str) – suffix label for versioning. if None, a timestamp is created.

static archive_folder(src_dir, dst_dir)[source]#

Archive to a zip folder

Parameters:

  • src_dir (str) – source directory

  • dst_dir (str) – destination directory

static check_file_status(files)[source]#

Static method for file existing checkup

Parameters:

files (list) – iterable with file paths

Returns:

list status (‘ok’ or ‘missing’)

Return type:

list

static copy_batch(dst_pattern, src_pattern)[source]#

Util static method for batch-copying pattern-based files.

Note

Pattern is expected to be a prefix prior to * suffix.

Parameters:

  • dst_pattern (str) – destination path with file pattern. Example: path/to/dst_file_*.csv

  • src_pattern (str) – source path with file pattern. Example: path/to/src_file_*.csv

static get_file_size_mb(file_path)[source]#

Util for getting the file size in MB

Parameters:

file_path (str) – path to file

Returns:

file size in MB

Return type:

float

class babilonia.root.Note(name='MyNote', alias='Nt1')[source]#

Bases: MbaE

get_metadata()[source]#

Get a dictionary with object metadata.

Returns:

dictionary with all metadata

Return type:

dict

Warning

Metadata does not necessarily include all object attributes.

load_metadata()[source]#
load_data()[source]#
load()[source]#
save()[source]#

Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

to_file(file_path, cleanup=True)[source]#

Export Note to markdown

Parameters:

file_path (str) – path to file

Returns:

Return type:

static remove_excessive_blank_lines(file_path)[source]#
static parse_metadata(note_file)[source]#

Extracts YAML metadata from the header of a Markdown file.

Parameters:

note_file – str, path to the Markdown file

Returns:

dict, extracted YAML metadata

static parse_yaml(yaml_content)[source]#

Parses YAML content into a dictionary.

Parameters:

yaml_content – str, YAML content as string

Returns:

dict, parsed YAML content

static metadata_to_list(metadata_dict)[source]#
static data_to_list(data_dict)[source]#
static parse_note(file_path)[source]#
static list_by_pattern(md_dict, patt_type='tag')[source]#

Retrieve a list of patterns from the note dictionary.

Parameters:

  • md_dict (dict) – Dictionary containing note sections.

  • patt_type (str) – Type of pattern to search for, either “tag” or “related”. Defaults to “tag”.

Returns:

List of found patterns or None if no patterns are found.

Return type:

list or None

class babilonia.root.RecordTable(name='MyRecordTable', alias='RcT')[source]#

Bases: DataSet

The base class for RecordTable.

A Record is expected to keep adding stamped records in order to keep track of large inventories, catalogs, etc. All records are expected to have a unique Id. It is considered to be a relational table.

Examples

Instantiate RecordTable Object

# Instantiate RecordTable object
rt = RecordTable(name="RecTable_1", alias="RT1")

Setup custom columns for the data

# Setup custom columns for the data
rt.columns_data_main = ["Name", "Size"]  # main data
rt.columns_data_extra = ["Type"]  # extra data
rt.columns_data_files = ["File_P"]  # file-related
rt.columns_data = rt.columns_data_main + rt.columns_data_extra + rt.columns_data_files

Set Object Metadata and Load Data

# Set object metadata and load data.
# Note: this dummy object expects the following columns in data
rt.set(
    dict_setter={
        "Name": "RecTable_01",
        "Alias": "RT01",
        "Color": "red",
        "Source": "-",
        "Description": "This is RecordTable Object",
        "File_Data": "/content/data_rt1.csv"
    },
    load_data=True
)

Check Data

# Check data :class:`pandas.DataFrame`
print(rt.data.head())

Load More Data from Other File

# Load more new data from other file
rt.load_data(file_data="/content/data_rt2.csv")

Insert New Record

# Insert new record from incoming dict
d2 = {
    "Name": "k",
    "Size": 177,
    "Type": 'inputs',
    "File_P": "/filee.pdf",
}
rt.insert_record(dict_rec=d2)

Edit Record

# Edit record based on ``RecId`` and new dict
d = {
    "Size": 34,
    "Name": "C"
}
rt.edit_record(rec_id="Rec0002", dict_rec=d)

Archive a Record

# Archive a record in the RT, that is ``RecStatus`` = ``Off``
rt.archive_record(rec_id="Rec0003")

Get a Record Dict by ID

# Get a record dict by id
d = rt.get_record(rec_id="Rec0001")
print(d)

Get a Record DataFrame by ID

# Get a record :class:`pandas.DataFrame` by id
df = rt.get_record_df(rec_id="Rec0001")
print(df.to_string(index=False))

Load Record Data from CSV

# Load record data from a ``csv`` file to a dict
d = rt.load_record_data(file_record_data="/content/rec_rt2.csv")
print(d)

Export a Record to CSV

# Export a record from the table to a ``csv`` file
f = rt.export_record(
    rec_id="Rec0001",
    folder_export="/content",
    filename="export_rt2"
)
print(f)
update()[source]#

Refresh all mutable attributes based on data (including paths).

save()[source]#

Save object resources to sourced files.

Danger

This method overwrites the sourced data file.

export(folder_export=None, filename=None, filter_archive=False)[source]#

Export the RecordTable data.

Parameters:

  • folder_export (str) – folder to export

  • filename (str) – file name (name alone, without file extension)

  • filter_archive (bool) – option for exporting only records with RecStatus = On

Returns:

file path is export is successfull (1 otherwise)

Return type:

str or int

setter(dict_setter, load_data=True)[source]#

Set selected attributes based on an incoming dictionary.

Parameters:

dict_setter (dict) – incoming dictionary with attribute values

refresh_data()[source]#

Refresh data method for the object operator. Performs spreadsheet-like formulas for columns.

load_data(file_data)[source]#

Load data from file.

Parameters:

file_data (str) – file path to data.

set_data(input_df, append=True, inplace=True)[source]#

Set RecordTable data from incoming pandas.DataFrame.

Parameters:

  • input_df (pandas.DataFrame) – incoming pandas.DataFrame

  • append (bool) – option for appending the pandas.DataFrame to existing data. Default True

  • inplace (bool) – option for overwrite data. Else return pandas.DataFrame. Default True

Notes

It handles if the pandas.DataFrame has or not the required RT columns Base Method.

insert_record(dict_rec)[source]#

Insert a record in the RT

Parameters:

dict_rec (dict) – inputs record dictionary

edit_record(rec_id, dict_rec, filter_dict=True)[source]#

Edit RT record

Parameters:

  • rec_id (str) – record id

  • dict_rec (dict) – incoming record dictionary

  • filter_dict (bool) – option for filtering incoming record

archive_record(rec_id)[source]#

Archive a record in the RT, that is RecStatus = Off

Parameters:

rec_id (str) – record id

get_record(rec_id)[source]#

Get a record dict by id

Parameters:

rec_id (str) – record id

Returns:

record dictionary

Return type:

dict

get_record_df(rec_id)[source]#

Get a record pandas.DataFrame by id

Parameters:

rec_id (str) – record id

Returns:

record dictionary

Return type:

dict

load_record_data(file_record_data, input_field='Field', input_value='Value')[source]#

Load record data from a csv file to a dict

Note

This method does not insert the record data to the RecordTable.

Parameters:

  • file_record_data (str) – file path to csv file.

  • input_field – Name of Field column in the file.

  • input_value – Name of Value column in the file.

Returns:

record dictionary

Return type:

dict

export_record(rec_id, filename=None, folder_export=None)[source]#

Export a record from the table to a csv file.

Parameters:

  • rec_id (str) – record id

  • filename (str) – file name (name alone, without file extension)

  • folder_export (str) – folder to export

Returns:

path to exported file

Return type:

str

static get_timestamp()[source]#

Return a string timestamp

Returns:

full timestamp text %Y-%m0-%d %H:%M:%S

Return type:

str

static timedelta_disagg(timedelta)[source]#

Util static method for dissaggregation of time delta

Parameters:

timedelta (pandas.TimeDelta) – TimeDelta object from pandas

Returns:

dictionary of time delta

Return type:

dict

static timedelta_to_str(timedelta, dct_struct)[source]#

Util static method for string conversion of timedelta

Parameters:

  • timedelta (pandas.TimeDelta) – TimeDelta object from pandas

  • dct_struct (dict) – Dictionary of string strucuture. Ex: {‘Expected days’: ‘Days’}

Returns:

text of time delta

Return type:

str

static running_time(start_datetimes, kind='raw')[source]#

Util static method for computing the runnning time for a list of starting dates

Parameters:

  • start_datetimes (list) – List of starting dates

  • kind (str) – mode for output format (‘raw’, ‘human’ or ‘age’)

Returns:

list of running time

Return type:

list