treevalue.tree.tree

TreeValue

class treevalue.tree.tree.TreeValue

Overview:

Base framework of tree value. And if the fast functions and operators are what you need, please use FastTreeValue in treevalue.tree.general. The TreeValue class is a light-weight framework just for DIY.

__bool__()

self != 0

__contains__()

Check if attribute is in this tree value.

Parameters:

key – Key to check.

Returns:

key is existed or not in this tree value.

Example:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> 'a' in t  # True
>>> 'b' in t  # True
>>> 'c' in t  # False

__delattr__()

Delete attribute from tree value.

Parameters:

item – Name of the attribute.

Example:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> del t.a
>>> t
<TreeValue 0x7fd2251ae320>
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fd2553f00b8>
    ├── 'c' --> 3
    └── 'd' --> 4
>>> del t.x.c
>>> t
<TreeValue 0x7fd2251ae320>
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fd2553f00b8>
    └── 'd' --> 4

__delitem__()

Delete item from current TreeValue.

Parameters:

key – Key object.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> del t['a']
>>> del t['x']['c']
>>> t
<TreeValue 0x7f11704c53c8>
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f11704c5438>
    └── 'd' --> 4

__eq__()

Check the equality of two tree values.

Parameters:

other – Another tree value object.

Returns:

Equal or not.

Example:

>>> from treevalue import TreeValue, clone, FastTreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t == TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})      # True
>>> t == TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 5}})      # False
>>> t == FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})  # False (type not match)

__getattribute__()

Get item from this tree value.

Parameters:

item – Name of attribute.

Returns:

Target attribute value; if item is exist in keys(), return its value.

Example:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t.a
1
>>> t.b
2
>>> t.x
<TreeValue 0x7fd2553e3fd0>
├── 'c' --> 3
└── 'd' --> 4
>>> t.x.c
3
>>> t.keys  # this is a method
<bound method TreeValue.keys of <TreeValue 0x7fd2553f00b8>>
>>> t.ff  # this key is not exist
AttributeError: Attribute 'ff' not found.

__getitem__()

Get item from this tree value.

Parameters:

key – Item object.

Returns:

Target object value.

Example:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t['a']
1
>>> t['b']
2
>>> t['x']['c']
3

__getstate__()

Serialize operation, can support pickle.dumps.

Returns:

A treevalue.tree.common.TreeStorage object, used for serialization.

Examples:

>>> import pickle
>>> from treevalue import TreeValue
>>>
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3}})
>>> bin_ = pickle.dumps(t)  # dump it to binary
>>> pickle.loads(bin_)      #  TreeValue({'a': 1, 'b': 2, 'x': {'c': 3}})

__hash__()

Hash value of current object.

Returns:

Hash value of current object.

Note

If the structure and hash values of two tree values are exactly the same, their hash value is guaranteed to be the same.

__init__()

Constructor of TreeValue.

Parameters:

data – Original data to init a tree value, should be a treevalue.tree.common.TreeStorage, TreeValue or a dict.

Example:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})  # create an instance
>>> t
<TreeValue 0x7fbcf70750b8>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fbcf70750f0>
    ├── 'c' --> 3
    └── 'd' --> 4
>>> t2 = TreeValue(t)  # create a new treevalue with the same storage of t
>>> t2
<TreeValue 0x7fbcf70750b8>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fbcf70750f0>
    ├── 'c' --> 3
    └── 'd' --> 4
>>> t2 is t
False

__iter__()

Get iterator of this tree value.

Returns:

An iterator for the keys.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': 3})
>>> for key in t:
...     print(key)
a
b
x

Note

The method __iter__()’s bahaviour should be similar to dict.__iter__.

__len__()

Get count of the keys.

Returns:

Count of the keys.

Example:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> len(t)    # 3
>>> len(t.x)  # 2

__repr__()

Get representation format of tree value.

Returns:

Represenation string.

Example:

>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t
<TreeValue 0x7f672fc53320>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f672fc53390>
    ├── 'c' --> 3
    └── 'd' --> 4

__reversed__()

Get the reversed iterator of tree value.

Returns:

A reversed iterator for the keys.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': 3})
>>> for key in reversed(t):
...     print(key)
x
b
a

Note

Only available in python 3.8 or higher version.

__setattr__()

Set sub node to this tree value.

Parameters:

• key – Name of the attribute.

• value – Sub value.

Example:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t.a = 3  # set a value
>>> t
<TreeValue 0x7fd2553f0048>
├── 'a' --> 3
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fd2553f0080>
    ├── 'c' --> 3
    └── 'd' --> 4
>>> t.b = {'x': 1, 'y': 2}  # set a dict
>>> t
<TreeValue 0x7fd2553f0048>
├── 'a' --> 3
├── 'b' --> {'x': 1, 'y': 2}
└── 'x' --> <TreeValue 0x7fd2553f0080>
    ├── 'c' --> 3
    └── 'd' --> 4
>>> t.b = TreeValue({'x': 1, 'y': 2})  # set a tree
>>> t
<TreeValue 0x7fd2553f0048>
├── 'a' --> 3
├── 'b' --> <TreeValue 0x7fd2553e3fd0>
│   ├── 'x' --> 1
│   └── 'y' --> 2
└── 'x' --> <TreeValue 0x7fd2553f0080>
    ├── 'c' --> 3
    └── 'd' --> 4

__setitem__()

Set item to current TreeValue object.

Parameters:

• key – Key object.

• value – Value object.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t['a'] = 11
>>> t['x']['c'] = 30
>>> t
<TreeValue 0x7f11704c5358>
├── 'a' --> 11
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f11704c52e8>
    ├── 'c' --> 30
    └── 'd' --> 4

__setstate__(state)

Deserialize operation, can support pickle.loads.

Parameters:

state – treevalue.tree.common.TreeStorage object, which should be used when deserialization.

Examples:

>>> import pickle
>>> from treevalue import TreeValue
>>>
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3}})
>>> bin_ = pickle.dumps(t)  # dump it to binary
>>> pickle.loads(bin_)      #  TreeValue({'a': 1, 'b': 2, 'x': {'c': 3}})

__str__()

Return str(self).

_attr_extern(key)

Overview:

External protected function for support the unfounded attributes. Default is raise a KeyError.

Arguments:

• key (str): Attribute name.

Returns:

• return (:obj:): Anything you like,

  and if it is not able to validly return anything, just raise an exception here.

_delitem_extern(key)

External function for supporting __delitem__() operation.

Parameters:

key – Key object.

Raises:

KeyError – Raise KeyError in default case.

_detach()

Detach the inner treevalue.tree.common.TreeStorage object.

Returns:

treevalue.tree.common.TreeStorage object.

Warning

This is an inner method of TreeValue, which is not recommended to be actually used. If you need to instantiate a new TreeValue with the same storage, just pass this TreeValue object as the argument of the __init__().

_getitem_extern(key)

External protected function for support the __getitem__() operation. Default behaviour is raising a KeyError.

Parameters:

key – Item object.

Returns:

Anything you like, this depends on your implements.

Raises:

KeyError – If it is not able to validly return anything, just raise an KeyError here.

_setitem_extern(key, value)

External function for supporting __setitem__() operation.

Parameters:

• key – Key object.

• value – Value object.

Raises:

NotImplementedError – When not implemented, raise NotImplementedError.

clear()

Clear all the items in this treevalue object.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t
<TreeValue 0x7fd2553f0048>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fd2553e3fd0>
    ├── 'c' --> 3
    └── 'd' --> 4
>>> t.clear()
>>> t
<TreeValue 0x7fd2553f0048>

get(key, default=None)

Get item from the tree node.

Parameters:

• key – Item’s name.

• default – Default value when this item is not found, default is None.

Returns:

Item’s value.

Note

The method get() will never raise KeyError, like the behaviour in dict.get.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t.get('a')
1
>>> t.get('x')
<TreeValue 0x7f488a65f0b8>
├── 'c' --> 3
└── 'd' --> 4
>>> t.get('f')  # key not exist
None
>>> t.get('f', 123)
123

items()

Get pairs of keys and values of this treevalue object, like the items.

Returns:

A mapping object for tree value’s items.

Examples:

>>> from treevalue import TreeValue
>>> 
>>> t = TreeValue({'a': 1, 'b': 3, 'c': '233'})
>>> t.items()
treevalue_items([('a', 1), ('b', 3), ('c', '233')])
>>> len(t.items())
3
>>> list(t.items())
[('a', 1), ('b', 3), ('c', '233')]
>>> list(reversed(t.items()))  # only supported on python3.8+
[('c', '233'), ('b', 3), ('a', 1)]
>>> ('a', 1) in t.items()
True
>>> ('c', '234') in t.values()
False

Note

reversed() is only available in python 3.8 or higher versions.

keys()

Get keys of this treevalue object, like the dict.

Returns:

A mapping object for tree value’s keys.

Examples:

>>> from treevalue import TreeValue
>>> 
>>> t = TreeValue({'a': 1, 'b': 3, 'c': '233'})
>>> t.keys()
treevalue_keys(['a', 'b', 'c'])
>>> len(t.keys())
3
>>> list(t.keys())
['a', 'b', 'c']
>>> list(reversed(t.keys()))  # only available in python3.8+
['c', 'b', 'a']
>>> 'a' in t.keys()
True
>>> 'f' in t.keys()
False

Note

reversed() is only available in python 3.8 or higher versions.

pop(key, default=<SingletonMark 'get_no_default'>)

Pop item from the tree node.

Parameters:

• key – Item’s name.

• default – Default value when this item is not found, default is _GET_NO_DEFAULT which means just raise KeyError when not found.

Returns:

Item’s value.

Raises:

KeyError – When key is not exist and default is not given, raise KeyError.

Note

The method pop() will raise KeyError when key is not found, like the behaviour in dict.pop.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t.pop('a')
1
>>> t.pop('x')
<TreeValue 0x7f488a65f080>
├── 'c' --> 3
└── 'd' --> 4
>>> t
<TreeValue 0x7f488a65f048>
└── 'b' --> 2
>>> t.pop('f')
KeyError: 'f'
>>> t.pop('f', 123)
123

popitem()

Pop item (with a key and its value) from the tree node.

Returns:

Popped item.

Raises:

KeyError – When current treevalue is empty.

Note

The method popitem() will raise KeyError when empty, like the behaviour in dict.popitem.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t.popitem()
('x', <TreeValue 0x7f488a65f0b8>
├── 'c' --> 3
└── 'd' --> 4
)
>>> t.popitem()
('b', 2)
>>> t.popitem()
('a', 1)
>>> t.popitem()
KeyError: 'popitem(): TreeValue is empty.'

setdefault(key, default=None)

Set the default to this treevalue and return it if the key is not exist, otherwise just return the existing value of key.

Parameters:

• key – Items’ name.

• default – Default value of the key, None will be used when not given.

Returns:

The newest value of the key.

Note

The behaviour of method setdefault() is similar to dict.setdefault.

Examples:

>>> from treevalue import TreeValue, delayed
>>> t = TreeValue({'a': 1, 'b': 3, 'c': '233'})
>>> t.setdefault('d', 'dsflgj')  # set new value
'dsflgj'
>>> t
<TreeValue 0x7efe31576048>
├── 'a' --> 1
├── 'b' --> 3
├── 'c' --> '233'
└── 'd' --> 'dsflgj'
>>> t.setdefault('ff')  # default value - None
>>> t
<TreeValue 0x7efe31576048>
├── 'a' --> 1
├── 'b' --> 3
├── 'c' --> '233'
├── 'd' --> 'dsflgj'
└── 'ff' --> None
>>> t.setdefault('a', 1000)  # existing key
1
>>> t
<TreeValue 0x7efe31576048>
├── 'a' --> 1
├── 'b' --> 3
├── 'c' --> '233'
├── 'd' --> 'dsflgj'
└── 'ff' --> None
>>> t.setdefault('g', delayed(lambda: 1))  # delayed value
1
>>> t
<TreeValue 0x7efe31576048>
├── 'a' --> 1
├── 'b' --> 3
├── 'c' --> '233'
├── 'd' --> 'dsflgj'
├── 'ff' --> None
└── 'g' --> 1

update(__update_dict=None, **kwargs)

Overview:

Update items in current treevalue.

Parameters:

• __update_dict – Dictionary object for updating.

• kwargs – Arguments for updating.

Note

The method update() is similar to dict.update.

Examples:

>>> from treevalue import TreeValue
>>> t = TreeValue({'a': 1, 'b': 3, 'c': '233'})
>>> t.update({'a': 10, 'f': 'sdkj'})  # with dict
>>> t
<TreeValue 0x7fa31f5ba048>
├── 'a' --> 10
├── 'b' --> 3
├── 'c' --> '233'
└── 'f' --> 'sdkj'
>>> t.update(a=100, ft='fffff')  # with key-word arguments
>>> t
<TreeValue 0x7fa31f5ba048>
├── 'a' --> 100
├── 'b' --> 3
├── 'c' --> '233'
├── 'f' --> 'sdkj'
└── 'ft' --> 'fffff'
>>> t.update(TreeValue({'f': {'x': 1}, 'b': 40}))  # with TreeValue
>>> t
<TreeValue 0x7fa31f5ba048>
├── 'a' --> 100
├── 'b' --> 40
├── 'c' --> '233'
├── 'f' --> <TreeValue 0x7fa31f5ba278>
│   └── 'x' --> 1
└── 'ft' --> 'fffff'

values()

Get value of this treevalue object, like the dict.

Returns:

A mapping object for tree value’s values.

Examples:

>>> from treevalue import TreeValue
>>> 
>>> t = TreeValue({'a': 1, 'b': 3, 'c': '233'})
>>> t.values()
treevalue_values([1, 3, '233'])
>>> len(t.values())
3
>>> list(t.values())
[1, 3, '233']
>>> list(reversed(t.values()))  # only supported on python3.8+
['233', 3, 1]
>>> 1 in t.values()
True
>>> 'fff' in t.values()
False

Note

reversed() is only available in python 3.8 or higher versions.

delayed

treevalue.tree.tree.delayed(func, *args, **kwargs)

Overview:

Use delayed function in treevalue. The given func will not be called until its value is accessed, and it will be only called once, after that the delayed node will be replaced by the actual value.

Parameters:

• func – Delayed function.

• args – Positional arguments.

• kwargs – Key-word arguments.

Examples:

>>> from treevalue import TreeValue, delayed
>>> def f(x):
...     print('f is called, x is', x)
...     return x ** x
...
>>> t = TreeValue({'a': delayed(f, 2), 'x': delayed(f, 3)})
>>> t.a
f is called, x is 2
4
>>> t.x
f is called, x is 3
27
>>> t.a
4
>>> t.x
27
>>> t = TreeValue({'a': delayed(f, 2), 'x': delayed(f, 3)})
>>> print(t)
f is called, x is 2
f is called, x is 3
<TreeValue 0x7f672fc53550>
├── 'a' --> 4
└── 'x' --> 27
>>> print(t)
<TreeValue 0x7f672fc53550>
├── 'a' --> 4
└── 'x' --> 27

jsonify

treevalue.tree.tree.jsonify(val)

Overview:

Dump TreeValue object to json data.

Arguments:

• tree (TreeValue): Tree value object or tree storage object.

Returns:

• json (dict): Dumped json data.

Example:

>>> jsonify(TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}))  # {'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}

clone

treevalue.tree.tree.clone(t, copy_value=None)

Overview:

Create a fully clone of the given tree.

Arguments:

• tree (_TreeValue): Tree value object

• copy_value (Union[None, bool, Callable, Any]): Deep copy value or not, default is None which means do not deep copy the values. If deep copy is required, just set it to True.

Returns:

• tree (_TreeValue): Cloned tree value object.

Example:

>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> clone(t.x)  # TreeValue({'c': 3, 'd': 4})

typetrans

treevalue.tree.tree.typetrans(t, return_type)

Overview:

Transform tree value object to another tree value type. Attention that in this function, no copy will be made, the original tree value and the transformed tree value are using the same space area.

Arguments:

• tree (TreeValue): Tree value object

• return_type (Type[_TreeValue]): Target tree value type

Returns:

• tree (_TreeValue): Transformed tree value object.

Example:

>>> t = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> typetrans(t, TreeValue)  # TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})

walk

treevalue.tree.tree.walk(tree)

Overview:

Walk the values and nodes in the tree. The order of walk is not promised, if you need the ordered walking result, just use function sorted at the outer side of walk().

Arguments:

• tree: Tree value object to be walked.

Returns:

• iter: Iterator to walk the given tree, contains 2 items, the 1st one is the full path of the node, the 2nd one is the value.

Examples:

>>> from treevalue import TreeValue, walk
>>> tv1 = TreeValue({'a': 1, 'b': 2, 'c': {'x': 2, 'y': 2}})
>>> for k, v in walk(tv1):
...     print(k, v)
() <TreeValue 0x7f672fc53390>
├── 'a' --> 1
├── 'b' --> 2
└── 'c' --> <TreeValue 0x7f672fc53320>
    ├── 'x' --> 2
    └── 'y' --> 2
('a',) 1
('b',) 2
('c',) <TreeValue 0x7f672fc53320>
├── 'x' --> 2
└── 'y' --> 2
('c', 'x') 2
('c', 'y') 2

flatten

treevalue.tree.tree.flatten(tree)

Overview:

Flatten the key-value pairs in the tree.

Arguments:

• tree (TreeValue): Tree object to be flatten.

Returns:

• flatted (list): Flatted tree, a list of tuple with (path, value).

Note

The result of function flatten() is guaranteed to be ordered, which means it obey one of the tree traversal order. But please note that the order of key values under the same subtree is not guaranteed.

flatten_values

treevalue.tree.tree.flatten_values(tree)

Overview:

Flatten the values in the tree.

Arguments:

• tree (TreeValue): Tree object to be flatten.

Returns:

• flatted (list): Flatted tree, a list of values.

flatten_keys

treevalue.tree.tree.flatten_keys(tree)

Overview:

Flatten the keys in the tree.

Arguments:

• tree (TreeValue): Tree object to be flatten.

Returns:

• flatted (list): Flatted tree, a list of paths.

unflatten

treevalue.tree.tree.unflatten(pairs, return_type=None)

Overview:

Unflatten the given pairs of tree’s data.

Arguments:

• pairs: Data pairs, should be a iterable object with items of (path, value).

• return_type: Return type of unflatted tree, default is None which means use the default

  TreeValue class.

Returns:

• tree (TreeValue): Unflatted tree object.

Note

It is recommended to pass an ordered iterable object in pairs, this will improve the speed performance of function unflatten().

Because of this, it is a good idea to keep the flatten()’s result’s order when executing your own processing logic.

mapping

treevalue.tree.tree.mapping(tree, func, delayed=False)

Overview:

Do mapping on every value in this tree.

Arguments:

• tree (_TreeValue): Tree value object

• func (Callable): Function for mapping

Note

There are 3 different patterns of given func:

• lambda :v, which means map all the original value to the new v.

• lambda v: f(v), which means map the original value to a new value based on given v.

• lambda v, p: f(v, p), which means map the original value and full path (in form of tuple) to a new values based on given v and given p.

When the given func is a python function (with __code__ attribute), its number of arguments will be detected automatically, and use the correct way to call it when doing mapping, otherwise it will be directly used with the pattern of lambda v: f(v).

Returns:

• tree (_TreeValue): Mapped tree value object.

Example:

>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> mapping(t, lambda x: x + 2)  # TreeValue({'a': 3, 'b': 4, 'x': {'c': 5, 'd': 6}})
>>> mapping(t, lambda: 1)        # TreeValue({'a': 1, 'b': 1, 'x': {'c': 1, 'd': 1}})
>>> mapping(t, lambda x, p: p)   # TreeValue({'a': ('a',), 'b': ('b',), 'x': {'c': ('x', 'c'), 'd': ('x', 'd')}})

mask

treevalue.tree.tree.mask(tree, mask_, remove_empty=True)

Overview:

Filter the element in the tree with a mask

Arguments:

• tree (_TreeValue): Tree value object

• mask_ (TreeValue): Tree value mask object

• remove_empty (bool): Remove empty tree node automatically, default is True.

Returns:

• tree (_TreeValue): Filtered tree value object.

Example:

>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> mask(t, TreeValue({'a': True, 'b': False, 'x': False}))                    # TreeValue({'a': 1})
>>> mask(t, TreeValue({'a': True, 'b': False, 'x': {'c': True, 'd': False}}))  # TreeValue({'a': 1, 'x': {'c': 3}})

filter_

treevalue.tree.tree.filter_(tree, func, remove_empty=True)

Overview:

Filter the element in the tree with a predict function.

Arguments:

• tree (_TreeValue): Tree value object

• func (Callable): Function for filtering

• remove_empty (bool): Remove empty tree node automatically, default is True.

Note

There are 3 different patterns of given func:

• lambda :v, which means map all the original value to the new v.

• lambda v: f(v), which means map the original value to a new value based on given v.

• lambda v, p: f(v, p), which means map the original value and full path (in form of tuple) to a new values based on given v and given p.

When the given func is a python function (with __code__ attribute), its number of arguments will be detected automatically, and use the correct way to call it when doing filter, otherwise it will be directly used with the pattern of lambda v: f(v).

Returns:

• tree (_TreeValue): Filtered tree value object.

Example:

>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> filter_(t, lambda x: x < 3)                  # TreeValue({'a': 1, 'b': 2})
>>> filter_(t, lambda x: x < 3, False)           # TreeValue({'a': 1, 'b': 2, 'x': {}})
>>> filter_(t, lambda x: x % 2 == 1)             # TreeValue({'a': 1, 'x': {'c': 3}})
>>> filter_(t, lambda x, p: p[0] in {'b', 'x'})  # TreeValue({'b': 2, 'x': {'c': 3, 'd': 4}})

union

treevalue.tree.tree.union(*trees, return_type=None, inherit=True, mode='strict', missing=<SingletonMark 'missing_not_allow'>, delayed=False)

Overview:

Union tree values together.

Arguments:

• trees (_TreeValue): Tree value objects.

• return_type (Optional[Type[_ClassType]]): Return type of the wrapped function, default is TreeValue.

• inherit (bool): Allow inheriting in wrapped function, default is True.

• mode (str): Mode of the wrapping, default is strict.

• missing (Union[Any, Callable]): Missing value or lambda generator of when missing, default is MISSING_NOT_ALLOW, which means raise KeyError when missing detected.

• delayed (bool): Enable delayed mode or not, the calculation will be delayed when enabled, default is False, which means to all the calculation at once.

Returns:

• result (TreeValue): Unionised tree value.

Example:

>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> tx = mapping(t, lambda v: v % 2 == 1)
>>> union(t, tx)  # TreeValue({'a': (1, True), 'b': (2, False), 'x': {'c': (3, True), 'd': (4, False)}})

subside

treevalue.tree.tree.subside(value, dict_=True, list_=True, tuple_=True, return_type=None, inherit=True, mode='strict', missing=<SingletonMark 'missing_not_allow'>, delayed=False)

Overview:

Drift down the structures (list, tuple, dict) down to the tree’s value.

Arguments:

• value (:obj:): Original value object, may be nested dict, list or tuple.

• dict_ (bool): Enable dict subside, default is True.

• list_ (bool): Enable list subside, default is True.

• tuple_ (bool): Enable list subside, default is True.

• return_type (Optional[Type[_ClassType]]): Return type of the wrapped function, will be auto detected when there is exactly one tree value type in this original value, otherwise the default will be TreeValue.

• inherit (bool): Allow inherit in wrapped function, default is True.

• mode (str): Mode of the wrapping, default is strict.

• missing (Union[Any, Callable]): Missing value or lambda generator of when missing, default is MISSING_NOT_ALLOW, which means raise KeyError when missing detected.

• delayed (bool): Enable delayed mode or not, the calculation will be delayed when enabled, default is False, which means to all the calculation at once.

Returns:

• return (_TreeValue): Subsided tree value.

Example:

>>> data = {
>>>     'a': TreeValue({'a': 1, 'b': 2}),
>>>     'x': {
>>>         'c': TreeValue({'a': 3, 'b': 4}),
>>>         'd': [
>>>             TreeValue({'a': 5, 'b': 6}),
>>>             TreeValue({'a': 7, 'b': 8}),
>>>         ]
>>>     },
>>>     'k': '233'
>>> }
>>>
>>> tree = subside(data)
>>> # tree should be --> TreeValue({
>>> #    'a': raw({'a': 1, 'k': '233', 'x': {'c': 3, 'd': [5, 7]}}),
>>> #    'b': raw({'a': 2, 'k': '233', 'x': {'c': 4, 'd': [6, 8]}}),
>>> #}), all structures above the tree values are subsided to the bottom of the tree.

rise

treevalue.tree.tree.rise(tree, dict_=True, list_=True, tuple_=True, template=None)

Overview:

Make the structure (dict, list, tuple) in value rise up to the top, above the tree value.

Arguments:

• tree (_TreeValue): Tree value object

• dict_ (bool): Enable dict rise, default is True.

• list_ (bool): Enable list rise, default is True.

• tuple_ (bool): Enable list rise, default is True.

• template (:obj:): Rising template, default is NO_RISE_TEMPLATE, which means auto detect.

Returns:

• risen (:obj:): Risen value.

Example:

>>> t = TreeValue({'x': raw({'a': [1, 2], 'b': [2, 3]}), 'y': raw({'a': [5, 6, 7], 'b': [7, 8]})})
>>> dt = rise(t)
>>> # dt will be {'a': <TreeValue 1>, 'b': [<TreeValue 2>, <TreeValue 3>]}
>>> # TreeValue 1 will be TreeValue({'x': [1, 2], 'y': [5, 6, 7]})
>>> # TreeValue 2 will be TreeValue({'x': 2, 'y': 7})
>>> # TreeValue 3 will be TreeValue({'x': 3, 'y': 8})
>>>
>>> t2 = TreeValue({'x': raw({'a': [1, 2], 'b': [2, 3]}), 'y': raw({'a': [5, 6], 'b': [7, 8]})})
>>> dt2 = rise(t2)
>>> # dt2 will be {'a': [<TreeValue 1>, <TreeValue 2>], 'b': [<TreeValue 3>, <TreeValue 4>]}
>>> # TreeValue 1 will be TreeValue({'x': 1, 'y': 5})
>>> # TreeValue 2 will be TreeValue({'x': 2, 'y': 6})
>>> # TreeValue 3 will be TreeValue({'x': 2, 'y': 7})
>>> # TreeValue 4 will be TreeValue({'x': 3, 'y': 8})
>>>
>>> dt3 = rise(t2, template={'a': None, 'b': None})
>>> # dt3 will be {'a': <TreeValue 1>, 'b': <TreeValue 2>}
>>> # TreeValue 1 will be TreeValue({'x': [1, 2], 'y': [5, 6]})
>>> # TreeValue 2 will be TreeValue({'x': [2, 3], 'y': [7, 8]})

reduce_

treevalue.tree.tree.reduce_(tree, func)

Overview

Reduce the tree to value.

Arguments:

• tree (_TreeValue): Tree value object

• func (:obj:): Function for reducing

Returns:

• result (:obj:): Reduce result

Examples:

>>> from functools import reduce
>>>
>>> t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> reduce_(t, lambda **kwargs: sum(kwargs.values()))  # 10, 1 + 2 + (3 + 4)
>>> reduce_(t, lambda **kwargs: reduce(lambda x, y: x * y, list(kwargs.values())))  # 24, 1 * 2 * (3 * 4)

graphics

treevalue.tree.tree.graphics(*trees, title: Optional[str] = None, cfg: Optional[dict] = None, dup_value: Union[bool, Callable, type, Tuple[Type, …]] = False, repr_gen: Optional[Callable] = None, node_cfg_gen: Optional[Callable] = None, edge_cfg_gen: Optional[Callable] = None) → graphviz.graphs.Digraph

Overview:

Draw graph by tree values. Multiple tree values is supported.

Args:

• trees: Given tree values, tuples of Tuple[TreeValue, str] or tree values are both accepted.

• title (Optional[str]): Title of the graph.

• cfg (Optional[dict]): Configuration of the graph.

• dup_value (Union[bool, Callable, type, Tuple[Type, ...]]): Value duplicator, set True to make value with same id use the same node in graph, you can also define your own node id algorithm by this argument. Default is False which means do not use value duplicator.

• repr_gen (Optional[Callable]): Representation format generator, default is None which means using repr function.

• node_cfg_gen (Optional[Callable]): Node configuration generator, default is None which means no configuration.

• edge_cfg_gen (Optional[Callable]): Edge configuration generator, default is None which means no configuration.

Returns:

• graph (Digraph): Generated graph of tree values.

Here is an example of graphics function. The source code is

import numpy as np

from treevalue import FastTreeValue, graphics, TreeValue


class MyFastTreeValue(FastTreeValue):
    pass


if __name__ == '__main__':
    t = MyFastTreeValue({
        'a': 1,
        'b': np.array([[5, 6], [7, 8]]),
        'x': {
            'c': 3,
            'd': 4,
            'e': np.array([[1, 2], [3, 4]])
        },
    })
    t2 = TreeValue({'ppp': t.x, 'x': {'t': t, 'y': t.x}})

    g = graphics(
        (t, 't'), (t2, 't2'),
        title="This is a demo of 2 trees.",
        cfg={'bgcolor': '#ffffff00'},
    )
    g.render('graphics.dat.gv', format='svg')

The generated graphviz source code should be

// This is a demo of 2 trees.
digraph this_is_a_demo_of_2_trees {
	graph [bgcolor="#ffffff00" label="This is a demo of 2 trees."]
	node_7f7e7ed50b20 [label=t color="#3eb24899" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman bold" penwidth=3 shape=diamond style=filled]
	node_7f7e7edfe250 [label=t2 color="#5b3eb299" fillcolor="#9673ff99" fontcolor="#0d0033ff" fontname="Times-Roman bold" penwidth=3 shape=diamond style=filled]
	value__node_7f7e7ed50b20__a [label=1 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e7ed50b20 -> value__node_7f7e7ed50b20__a [label=a arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	value__node_7f7e7ed50b20__b [label="array([[5, 6],
       [7, 8]])" color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e7ed50b20 -> value__node_7f7e7ed50b20__b [label=b arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	node_7f7e7edd4ee0 [label="t.x" color="#3eb24899" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman bold" penwidth=1.5 shape=ellipse style=filled]
	node_7f7e7ed50b20 -> node_7f7e7edd4ee0 [label=x arrowhead=vee arrowsize=1.0 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	node_7f7e7edfe250 -> node_7f7e7edd4ee0 [label=ppp arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
	node_7f7e7edeb460 [label="t2.x" color="#5b3eb299" fillcolor="#9673ff99" fontcolor="#0d0033ff" fontname="Times-Roman bold" penwidth=1.5 shape=ellipse style=filled]
	node_7f7e7edfe250 -> node_7f7e7edeb460 [label=x arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
	value__node_7f7e7edd4ee0__c [label=3 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e7edd4ee0 -> value__node_7f7e7edd4ee0__c [label=c arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	value__node_7f7e7edd4ee0__d [label=4 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e7edd4ee0 -> value__node_7f7e7edd4ee0__d [label=d arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	value__node_7f7e7edd4ee0__e [label="array([[1, 2],
       [3, 4]])" color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e7edd4ee0 -> value__node_7f7e7edd4ee0__e [label=e arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	node_7f7e7edeb460 -> node_7f7e7ed50b20 [label=t arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
	node_7f7e7edeb460 -> node_7f7e7edd4ee0 [label=y arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
}

The graph should be

Also, graphics function can support value duplication. For if the value nodes are using the same object, they will be displayed in the same node of the generated graph, such as the source code below

import numpy as np

from treevalue import FastTreeValue, graphics


class MyFastTreeValue(FastTreeValue):
    pass


if __name__ == '__main__':
    t = MyFastTreeValue({
        'a': [4, 3, 2, 1],
        'b': np.array([[5, 6], [7, 8]]),
        'x': {
            'c': np.array([[5, 7], [8, 6]]),
            'd': {'a', 'b', 'c'},
            'e': np.array([[1, 2], [3, 4]])
        },
    })
    t1 = MyFastTreeValue({
        'aa': t.a,
        'bb': np.array([[5, 6], [7, 8]]),
        'xx': {
            'cc': t.x.c,
            'dd': t.x.d,
            'ee': np.array([[1, 2], [3, 4]])
        },
    })

    g = graphics(
        (t, 't'), (t1, 't1'),
        (MyFastTreeValue({'a': t, 'b': t1, 'c': [1, 2], 'd': t1.xx}), 't2'),
        # Here is the dup value, with several types
        # np.ndarray and list type will use the same value node,
        # but set type is not in this tuple, so will not share the same node.
        dup_value=(np.ndarray, list),
        title="This is a demo of 2 trees with dup value.",
        cfg={'bgcolor': '#ffffff00'},
    )
    g.render('graphics_dup_value.dat.gv', format='svg')

The graph of the case with dup_value should be

The return value’s type of function graphics is class graphviz.dot.Digraph, from the opensource library graphviz, for further information of this project and graphviz.dot.Digraph’s usage, take a look at:

• Official site of Graphviz.

• User Guide of Graphviz.

• API Reference of Graphviz.

dump

treevalue.tree.tree.dump(t: _TreeType, file, compress=None)

Overview:

Dump tree value to file.

Arguments:

• t (_TreeType): Tree value object.

• file: Target dump file.

• compress: Compress object, may be compression function, tuple of functions or module (compress and decompress required).

dumps

treevalue.tree.tree.dumps(t: _TreeType, compress=None) → bytes

Overview:

Dump tree value to file.

Arguments:

• t (_TreeType): Tree value object.

• compress: Compress object, may be compression function, tuple of functions or module (compress and decompress required).

Returns:

• data (bytes): Dumped binary data.

load

treevalue.tree.tree.load(file, type_: Type[_TreeType] = <class 'treevalue.tree.tree.tree.TreeValue'>, decompress: Optional[Callable] = None) → _TreeType

Overview:

Load tree value object from file.

Arguments:

• file: Original file.

• type_ (Type[_TreeType]): Type of tree value, default is TreeValue.

• decompress (Optional[Callable]): Decompress function, default is None which means do not do any decompression.

Returns:

• tree (_TreeType): Tree value object.

loads

treevalue.tree.tree.loads(data: Union[bytes, bytearray], type_: Optional[Type[_TreeType]] = <class 'treevalue.tree.tree.tree.TreeValue'>, decompress: Optional[Callable] = None) → _TreeType

Overview:

Load tree value object from file.

Arguments:

• data (Union[bytes, bytearray]): Binary data.

• type_ (Type[_TreeType]): Type of tree value, default is TreeValue.

• decompress (Optional[Callable]): Decompress function, default is None which means do not do any decompression.

Returns:

• tree (_TreeType): Tree value object.

penetrate

treevalue.tree.tree.penetrate(decorator, **kwargs)

Overview:

Penetrate TreeValue object through decorated function, such as jax.jit.

Parameters:

• decorator – Original decorator

• kwargs – Other keyword arguments, will be passed into decorator.

Note

penetrate() can be used on jax.jit.

>>> import jax
>>> import numpy as np
>>> from treevalue import FastTreeValue, PENETRATE_SESSIONID_ARGNAME, penetrate
>>>
>>> @penetrate(jax.jit, static_argnames=PENETRATE_SESSIONID_ARGNAME)
... def double(x):
...     return x * 2
>>>
>>> t = FastTreeValue({
...     'a': np.random.randint(0, 10, (2, 3)),
...     'b': {
...         'x': 233,
...         'y': np.random.randn(2, 3)
...     }
... })
>>>
>>> t
<FastTreeValue 0x7ff8facde8d0>
├── 'a' --> array([[0, 0, 7],
│                  [9, 6, 4]])
└── 'b' --> <FastTreeValue 0x7ff8deb3d110>
    ├── 'x' --> 233
    └── 'y' --> array([[ 0.86424466,  0.62416234, -0.76929206],
                       [ 1.16229066, -0.28098265,  0.1849025 ]])
>>> double(t)
WARNING:jax._src.lib.xla_bridge:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
<FastTreeValue 0x7ff8dea57410>
├── 'a' --> DeviceArray([[ 0,  0, 14],
│                        [18, 12,  8]], dtype=int32)
└── 'b' --> <FastTreeValue 0x7ff8dea57350>
    ├── 'x' --> DeviceArray(466, dtype=int32, weak_type=True)
    └── 'y' --> DeviceArray([[ 1.7284893,  1.2483246, -1.5385841],
                             [ 2.3245814, -0.5619653,  0.369805 ]], dtype=float32)
>>> double(t + 1)
<FastTreeValue 0x7ff8dea57810>
├── 'a' --> DeviceArray([[ 2,  2, 16],
│                        [20, 14, 10]], dtype=int32)
└── 'b' --> <FastTreeValue 0x7ff8dea57890>
    ├── 'x' --> DeviceArray(468, dtype=int32, weak_type=True)
    └── 'y' --> DeviceArray([[3.7284894 , 3.2483246 , 0.46141586],
                             [4.324581  , 1.4380347 , 2.369805  ]], dtype=float32)