Advanced Usage

In this part, Some of the advanced usages of TreeValue will be introduced one by one with sample code and graph to explain them.

For further information or usage of TreeValue and FastTreeValue, you may take a look at the following pages:

• TreeValue and its utilities: treevalue.tree.tree.

• func_treelize and other wrappers: treevalue.tree.func.

• FastTreeValue and its operators, methods: treevalue.tree.general.

Function Modes

In the basic usage description in Make function tree supported, we can see that a common function which only support the calculation of the common values can be wrapped to support the tree-based calculation painlessly.

Here is the documentation of function treevalue.tree.func.func_treelize.

treevalue.tree.func.func_treelize(mode: Union[str, treevalue.tree.func.func.TreeMode] = 'strict', return_type: Optional[Type[_ClassType]] = <class 'treevalue.tree.tree.tree.TreeValue'>, inherit: bool = True, missing: Union[Any, Callable] = <SingletonMark 'missing_not_allow'>, subside: Optional[Union[Mapping, bool]] = None, rise: Optional[Union[Mapping, bool]] = None)

Overview:

Wrap a common function to tree-supported function.

Arguments:

• mode (Union[str, TreeMode]): Mode of the wrapping (string or TreeMode both okay), default is strict.

• return_type (Optional[Type[_ClassType]]): Return type of the wrapped function, default is TreeValue.

• inherit (bool): Allow inherit in wrapped function, default is True.

• missing (Union[Any, Callable]): Missing value or lambda generator of when missing, default is MISSING_NOT_ALLOW, which means raise KeyError when missing detected.

• subside (Union[Mapping, bool, None]): Subside enabled to function’s arguments or not, and subside configuration, default is None which means do not use subside. When subside is True, it will use all the default arguments in subside function.

• rise (Union[Mapping, bool, None]): Rise enabled to function’s return value or not, and rise configuration, default is None which means do not use rise. When rise is True, it will use all the default arguments in rise function. (Not recommend to use auto mode when your return structure is not so strict.)

Returns:

• decorator (Callable): Wrapper for tree-supported function.

Example:

>>> @func_treelize()
>>> def ssum(a, b):
>>>     return a + b  # the a and b will be integers, not TreeValue
>>>
>>> t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t2 = TreeValue({'a': 11, 'b': 22, 'x': {'c': 33, 'd': 5}})
>>> ssum(1, 2)    # 3
>>> ssum(t1, t2)  # TreeValue({'a': 12, 'b': 24, 'x': {'c': 36, 'd': 9}})

In the arguments listed above, 3 of them are key arguments:

• mode, Tree process mode of the calculation. Will be introduced in this section.

• inherit, Inheriting mode of the calculation on the tree. Will be introduced in Inheriting on Trees.

• missing, Missing processor of the calculation on the tree. Will be introduced in Process Missing Values.

The mode argument is the most important argument in function func_treelize. For it depends the basic logic of the graph calculation.

The type of mode argument is TreeMode, which documentation is like this.

enum treevalue.tree.func.TreeMode(value)

Overview:

Four mode of the tree calculation

Member Type:

int

Valid values are as follows:

STRICT = <TreeMode.STRICT: 1>

Strict mode, which means the keys should be one to one in every trees.

LEFT = <TreeMode.LEFT: 2>

Left mode, the keys of the result is relied on the left value.

INNER = <TreeMode.INNER: 3>

Inner mode, the keys of the result is relied on the intersection of the trees’ key set.

OUTER = <TreeMode.OUTER: 4>

Outer mode, the keys of the result is relied on the union of the trees’ key set.

In this part, all of the 4 modes will be introduced with details and samples one by one.

Strict Mode

Strict mode is the most frequently-used mode in real cases. It is also the default value of the mode argument in the treelize functions.

To be simple, strict mode need the keys of a node in the tree be strictly mapped one by one, missing or overage of the keys are both forbidden in strict mode, which means KeyError will be raised.

For example, in the trees in strict_demo_1, if we need to added t1 and t2 together, t3 will be the result. Because all the keys (a, b, x of the root nodes, and c, d, e of the x node) can be mapped one by one.

In another situation, when the keys are not so regular like that in strict_demo_1, like the trees in strict_demo_2. If we try to add t1 and t2 in strict mode, KeyError will be raised due to the missing of key a in t1 and key f in t2.x.

Here is a real code example of strict mode.

from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='strict')
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t4, t2):', plus(t4, t2))

The stdout and stderr should like below, a KeyError is raised at the second print statement.

plus(t1, t2): <TreeValue 0x7f608937adc0 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7f608937af10 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/strict_demo.demox.py", line 15, in <module>
    print('plus(t4, t2):', plus(t4, t2))
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 138, in _recursion
    )) for key in sorted(_MODE_PROCESSORS[mode].get_key_set(*args, **kwargs))
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/strict.py", line 12, in get_key_set
    raise KeyError(
KeyError: "Argument keys not match in strict mode, key set of argument 1 is ('b', 'x') but 2 in ('a', 'b', 'x')."

Note

How does the treelized function work?

Here is another example which show the actual calculation process of the wrapped function.

import numpy as np

from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='strict')
def plus(a, b):
    print("Current a and b:", type(a), a, type(b), b)
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t2 = FastTreeValue({
        'a': np.array([1, 3, 5, 7]),
        'b': np.array([2, 5, 8, 11]),
        'x': {
            'c': 3,
            'd': 4.8,
            'e': np.array([[2, 3], [5, 6]]),
        }
    })
    t3 = FastTreeValue({
        'a': np.array([1, 3, 5, 7]),
        'b': [2, 5, 8, 11],
        'x': {
            'c': 3,
            'd': 4.8,
            'e': np.array([[2, 3], [5, 6]]),
        }
    })

    print('plus(t1, t2):', plus(t1, t2))
    print()
    print('plus(t1, t3):', plus(t1, t3))
    print()

In this code, once the original function plus is called, the actual value of argument a and b will be printed to stdout.

In the plus(t1, t2), all the calculation can be carried on because add operator between primitive int and np.ndarray is supported, so no error occurs and the result of + between int and np.ndarray will be one of the values of the result tree.

But in plus(t1, t3), when get the result of plus(t1, t3).b, it will try to add t1.b and t2.b together, but there is no implement of operator + between primitive int and list objects, so TypeError will be raised when do this calculation.

The complete stdout and stderr should be like this.

Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7]
Current a and b: <class 'int'> 22 <class 'numpy.ndarray'> [ 2  5  8 11]
Current a and b: <class 'int'> 30 <class 'int'> 3
Current a and b: <class 'int'> 48 <class 'float'> 4.8
Current a and b: <class 'int'> 54 <class 'numpy.ndarray'> [[2 3]
 [5 6]]
plus(t1, t2): <TreeValue 0x7f6e17d48ee0 keys: ['a', 'b', 'x']>
├── 'a' --> array([12, 14, 16, 18])
├── 'b' --> array([24, 27, 30, 33])
└── 'x' --> <TreeValue 0x7f6e17d4b310 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52.8
    └── 'e' --> array([[56, 57],
       [59, 60]])


Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7]
Current a and b: <class 'int'> 22 <class 'list'> [2, 5, 8, 11]

Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 35, in <module>
    print('plus(t1, t3):', plus(t1, t3))
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 129, in _recursion
    return func(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 9, in plus
    return a + b
TypeError: unsupported operand type(s) for +: 'int' and 'list'

In strict mode, missing or overage of keys are not tolerated at all. So in some of the cases, especially when you can not suppose that all the trees involved in calculation has exactly the same structure. Based on this demand, left mode, inner mode and outer mode is designed, and they will be introduced with details and examples in the next 3 sections.

Left Mode

In left mode, the result tree’s key set will be aligned to the first tree from the left.

In the trees in left_demo_1, t3 is the plus result of t1 and t2. We can see that t2.a is ignored because of t1.a ‘s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of t1.

But in the left_demo_2 (actually it is exactly the same as strict_demo_2), KeyError will be raised because t2.x.f` not found in tree t2`, so the result \ can not be aligned to tree ``t1, calculation failed.

When you decorate a function as left mode, the decorated function will try to find the first tree from left. If there is no less than 1 positional arguments detected, the first positional argument will be used as the left tree. Otherwise, the tree with the smallest lexicographic key will be assigned as the left tree. Obviously, As least 1 positional or key-word-based argument should be in the arguments.

Here is a real code example of left mode.

from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='left')
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t4, t2):', plus(t4, t2))

The stdout and stderr should like below, a KeyError is raised at the second print statement.

plus(t1, t2): <TreeValue 0x7f87a5a52dc0 keys: ['b', 'x']>
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7f87a5a52f10 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/left_demo.demox.py", line 15, in <module>
    print('plus(t4, t2):', plus(t4, t2))
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 136, in <genexpr>
    *(item(key) for item in pargs),
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 113, in _get_from_key
    raise KeyError("Missing is off, key {key} not found in {item}.".format(
KeyError: "Missing is off, key 'f' not found in <FastTreeValue 0x7f87a5aaca30 keys: ['c', 'd', 'e']>."

In the code example above, the result is aligned to tree a in wrapped function plus. So the first plus calculation will get the proper result, but the second one will fail.

Inner Mode

In inner mode, the result tree’s key set will be aligned to the intersection set of all the tree’s key set in arguments.

In the trees in inner_demo_1, t3 is the plus result of t1 and t2. We can see that t2.a and t1.x.f is ignored because of t1.a and t2.x.f’s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of the intersection between tree t1 and t2.

Note

In most cases of inner mode, the calculation will be processed properly because of the intersection operation on the key set and key missing or overage are both avoided completely.

But the shortage of inner mode is the loss of some information, for keys and its value or subtrees will be directly ignored without any warning. So before using inner mode, please confirm that you know its running logic and the missing of some nodes.

Here is a real code example of inner mode.

from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='inner')
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t4, t2):', plus(t4, t2))

The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.

plus(t1, t2): <TreeValue 0x7fc5bca65dc0 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7fc5bca65f10 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

plus(t4, t2): <TreeValue 0x7fc5bd37a4f0 keys: ['b', 'x']>
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7fc5bd37a640 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

Outer Mode

In outer mode, the result tree’s key set will be aligned to the union set of the tree’s key set in arguments.

In the trees in outer_demo_1, t3 is the plus result of t1 and t2, with the missing value of 0. We can see that all the values in tree t1 and t2 are involved in this calculation, and because of the missing of t1.a and t2.x.f they will be actually treated as 0 in calculation process. Finally no error will be raised, and sum of the tree nodes’ value will form the result tree t3.

Note

In outer mode, it is strongly recommended to set the value of argument missing. When missing’s value is not set and there are key missing in some of the argument trees, KeyError will be raised due to this missing and value of missing.

For further information and examples of missing argument, take a look at Process Missing Values.

Here is a real code example of inner mode.

from treevalue import FastTreeValue, func_treelize


# missing value is very important when use outer mode
@func_treelize(mode='outer', missing=0)
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})

    print('plus(t1, t2):', plus(t1, t2))

The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.

plus(t1, t2): <TreeValue 0x7fe73c2fbb50 keys: ['a', 'b', 'x']>
├── 'a' --> 11
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7fe73c2fbc10 keys: ['c', 'd', 'e', 'f']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    ├── 'e' --> 59
    └── 'f' --> 6

Inheriting on Trees

Inheriting is another important feature of treelize functions. It will be introduced in this section with necessary examples.

In some cases, we need to calculation between values with different dimensions, such as in numpy, you can add primitive integer with a ndarray object, like this

import os

import numpy as np

if __name__ == '__main__':
    ar1 = np.array([[1, 2], [3, 4]])
    print('ar1 + 9:', ar1 + 9, sep=os.linesep)

The result will be like below, result of ar1 + 9 is actually the same as the result of expression ar1 + np.array([[9, 9], [9, 9]]).

ar1 + 9:
[[10 11]
 [12 13]]

In treelize functions, you can achieve this processing way by use the inherit argument. Considering this situation can be find almost anywhere, so the default value of inherit argument is True, which means the inheriting option is on by default.

We can see the example in inherit_demo_1, the tree t1 and tree t2 have different structures, and cannot be processed by any modes in Function Modes, because t1.x is a primitive integer value but t2.x is a subtree node, their types are structurally different.

But when inheriting is enabled, this adding operation can still be carried on, for the value of t1.x (is 9 in inherit_demo_1) will be applied to the whole subtree t2.x, like the result tree t3 shows. The result of tree t3.x can be considered as the sum of primitive integer 9 and subtree t2.x.

Based on the structural properties above, in the example of inherit_demo_2, a primitive value can be directly added with a complete tree, like the result tree t2 = t1 + 5 shows.

When inheriting is disabled, all the cases with primitive value and tree node at the same path position will cause TypeError. For example, when inheriting is disabled, inherit_demo_1 and inherit_demo_2 will failed, but strict_demo_1 will still success because no primitive value appears at the same path position of tree node.

Here is a real code example of inheriting.

from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='strict')
def plus(a, b):
    return a + b


@func_treelize(mode='strict', inherit=False)
def plusx(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': 9})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t2, 5):', plus(t2, 5))
    print()

    print('plusx(t1, t2):', plusx(t1, t2))
    print()

The stdout and stderr should be like below, the the calculation of plus function and the first calculation of plusx function will be processed properly, but the last one will failed due to the disablement of inheriting.

plus(t1, t2): <TreeValue 0x7f1cca8fdbe0 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7f1cca8fdd90 keys: ['c', 'd', 'e']>
    ├── 'c' --> 39
    ├── 'd' --> 57
    └── 'e' --> 63

plus(t2, 5): <TreeValue 0x7f1cca8ff070 keys: ['a', 'b', 'x']>
├── 'a' --> 16
├── 'b' --> 27
└── 'x' --> <TreeValue 0x7f1cca8ff400 keys: ['c', 'd', 'e']>
    ├── 'c' --> 35
    ├── 'd' --> 53
    └── 'e' --> 59

Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/inherit_demo.demox.py", line 22, in <module>
    print('plusx(t1, t2):', plusx(t1, t2))
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 131, in _recursion
    pargs = [_value_wrap(item, index) for index, item in enumerate(args)]
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 131, in <listcomp>
    pargs = [_value_wrap(item, index) for index, item in enumerate(args)]
  File "/tmp/tmpuz3cygjh/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 121, in _value_wrap
    raise TypeError("Inherit is off, tree value expected but {type} found in args {index}.".format(
TypeError: Inherit is off, tree value expected but 'int' found in args 0.

Note

In most cases, disablement of inheriting is not recommended because it will cause some errors. Disable inheriting can increase the structural strictness of calculation, but all the cross-dimensional tree operation will be forbidden as well.

So before disabling inheriting, make sure you know well about what you are going to do.

Process Missing Values

In some cases of left mode and outer mode, some of keys in result tree can not find the node nor value at the corresponding path position in some of the argument trees. In order the make these cases be processable, missing value can be used by setting value or lambda expression in missing argument.

Just like the example outer_demo_1 mentioned forwards, when t1.a and t2.x.f not found in trees, the given missing value 0 will be used instead.

Another case is the missing_demo_1 which is based on left mode. In this case, t2.a is ignore due to the property of left mode, t2.x.f is missing but t1.x.f can be found, so t2.x.f will use the missing value 0. Finally, t3 will be the result of left addition of t1 and t2, with missing value of 0.

Considering another complex case, when the values of tree are primitive lists, like the missing_demo_2 which is based on outer mode. At this time, if we need to do addition with missing value supported, we can set the missing value to an empty list. The result is like below.

Note

In missing_demo_2, lambda :[] will be the value of missing argument in actual code.

In missing argument, its value will be automatically executed when it is callable, and its execution’s return value will be the actual missing value.

By passing a lambda expression in, you can construct a new object everytime missing value is required, and the duplication of the missing value’s object will be avoided. So it is not recommended to use missing argument like [], {} or myobject() in python code, but lambda :[], lambda :{} and lambda :myobject() are better practices.

Here is a real code example of missing_value.

import os

from treevalue import func_treelize, FastTreeValue


@func_treelize(mode='outer', missing=lambda: [])
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({
        'b': [2, 3],
        'x': {
            'c': [5],
            'd': [7, 11, 13],
            'e': [17, 19],
        }
    })
    t2 = FastTreeValue({
        'a': [23],
        'b': [29, 31],
        'x': {
            'c': [37],
            'd': [41, 43],
        }
    })

    print('plus(t1, t2):', plus(t1, t2), sep=os.linesep)

The stdout (no error occurred) should be like below, the calculation of plus function will be processed properly, like the result in missing_demo_2.

plus(t1, t2):
<TreeValue 0x7fa4a5672b50 keys: ['a', 'b', 'x']>
├── 'a' --> [23]
├── 'b' --> [2, 3, 29, 31]
└── 'x' --> <TreeValue 0x7fa4a5672ca0 keys: ['c', 'd', 'e']>
    ├── 'c' --> [5, 37]
    ├── 'd' --> [7, 11, 13, 41, 43]
    └── 'e' --> [17, 19]

Note

Missing value will be only applied in left mode and outer mode.

In strict mode, missing or overage of keys are absolutely not tolerated, missing value will make no sense and a RuntimeWarning will be logged.

In inner mode, missing value will never be actually in use because all the key sets of final result are intersection set of argument trees. There will never be any key missing cases at all when inner mode is used, a RuntimeWarning will be logged with the usage of missing value in inner mode.

Functional Utilities

In the primitive python, some convenient functional operations are supported, such as

if __name__ == '__main__':
    # map function
    print("Result of map:", list(map(lambda x: x + 1, [2, 3, 5, 7])))

    # filter function
    print("Result of filter:", list(filter(lambda x: x % 4 == 3, [2, 3, 5, 7])))

The result should be

Result of map: [3, 4, 6, 8]
Result of filter: [3, 7]

Actually, functional operations are supported in tree values as well, and they will be introduced in this section.

Mapping

Mapping function is similar to the primitive map function. The relation between TreeValue and mapping function is like that between list and map function.

Here is a simple real code example

from treevalue import mapping, FastTreeValue

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})

    print('mapping(t, lambda x: x ** x + 2):')
    print(mapping(t, lambda x: x ** x + 2))

The mapped tree result should be like below.

mapping(t, lambda x: x ** x + 2):
<FastTreeValue 0x7f38ce4fe700 keys: ['a', 'b', 'x']>
├── 'a' --> 3
├── 'b' --> 6
└── 'x' --> <FastTreeValue 0x7f38ce4fe850 keys: ['c', 'd']>
    ├── 'c' --> 29
    └── 'd' --> 258

For further definition or source code implement of function mapping, take a look at mapping.

Filter

Filter function is similar to the primitive filter function. The relation between TreeValue and filter_ function is like that between list and filter function .

Here is a simple real code example

from treevalue import FastTreeValue, filter_

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    print('filter_(t, lambda x: x % 2 == 1):')
    print(filter_(t, lambda x: x % 2 == 1))

    print('filter_(t, lambda x: x % 2 == 1, remove_empty=False):')
    print(filter_(t, lambda x: x % 2 == 1, remove_empty=False))

The filtered tree result should be like below. When remove_empty is disabled, the empty tree node will be kept to make sure the structure of the result tree is similar to the unfiltered tree.

filter_(t, lambda x: x % 2 == 1):
<FastTreeValue 0x7f00c75ae760 keys: ['a', 'x']>
├── 'a' --> 1
└── 'x' --> <FastTreeValue 0x7f00c75af5e0 keys: ['c']>
    └── 'c' --> 3

filter_(t, lambda x: x % 2 == 1, remove_empty=False):
<FastTreeValue 0x7f00c75c1b80 keys: ['a', 'x', 'y']>
├── 'a' --> 1
├── 'x' --> <FastTreeValue 0x7f00c75c16d0 keys: ['c']>
│   └── 'c' --> 3
└── 'y' --> <FastTreeValue 0x7f00c75c1970 keys: []>

Note

Actually, the code above is equal to the code below. The filter_ function can be seen as a combination of mask function and mapping function.

from treevalue import FastTreeValue, mapping, mask

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    print('mask(t, mapping(t, lambda x: x % 2 == 1)):')
    print(mask(t, mapping(t, lambda x: x % 2 == 1)))

    print('mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False):')
    print(mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False))

The result should be like below, exactly the same as the code above.

mask(t, mapping(t, lambda x: x % 2 == 1)):
<FastTreeValue 0x7f5ec467b760 keys: ['a', 'x']>
├── 'a' --> 1
└── 'x' --> <FastTreeValue 0x7f5ec467d5e0 keys: ['c']>
    └── 'c' --> 3

mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False):
<FastTreeValue 0x7f5ec468fb80 keys: ['a', 'x', 'y']>
├── 'a' --> 1
├── 'x' --> <FastTreeValue 0x7f5ec468f6d0 keys: ['c']>
│   └── 'c' --> 3
└── 'y' --> <FastTreeValue 0x7f5ec468f970 keys: []>

For further information about mask function, take a look at Mask.

For further definition or source code implement of function filter_, take a look at filter_.

Mask

Mask function allow your choice in one tree, by another tree of true-or-false flags. A simple code example is

from treevalue import FastTreeValue, mask

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})
    m = FastTreeValue({'a': True, 'b': False, 'x': {'c': True, 'd': True}, 'y': {'e': False, 'f': False}})

    print('mask(t, m):')
    print(mask(t, m))

    print('mask(t, m, remove_empty=False):')
    print(mask(t, m, remove_empty=False))

The result will be like below, values mapped with the tree m which mask flag is False are all deleted. Also, remove_empty argument is supported.

mask(t, m):
<FastTreeValue 0x7f7dbd6d3c10 keys: ['a', 'x']>
├── 'a' --> 1
└── 'x' --> <FastTreeValue 0x7f7dbd6d3700 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

mask(t, m, remove_empty=False):
<FastTreeValue 0x7f7dbd6d1a90 keys: ['a', 'x', 'y']>
├── 'a' --> 1
├── 'x' --> <FastTreeValue 0x7f7dbd6d32e0 keys: ['c', 'd']>
│   ├── 'c' --> 3
│   └── 'd' --> 4
└── 'y' --> <FastTreeValue 0x7f7dbd6d3190 keys: []>

For further definition or source code implement of function mask, take a look at mask.

Reduce

By using reduce_ function, you can get some calculation result based on the tree structure. Its meaning is similar to primitive reduce_ function, but its base structure is TreeValue instead of sequence or iterator. For example, we can get the sum and multiply accumulation of the values in the tree.

from functools import reduce
from operator import __mul__

from treevalue import reduce_, FastTreeValue


def multi(items):
    return reduce(__mul__, items, 1)


if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    print("Sum of t:", reduce_(t, lambda **kwargs: sum(kwargs.values())))
    print("Multiply of t:", reduce_(t, lambda **kwargs: multi(kwargs.values())))

The result should be like below.

Sum of t: 24
Multiply of t: 1152

If we use reduce_ function with mapping function, huffman weight sum can also be easily calculated with the code below.

from treevalue import reduce_, FastTreeValue, mapping

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    weights = mapping(t, lambda v, p: v * len(p))
    print("Weight tree:", weights)
    print("Huffman weight sum of t:",
          reduce_(weights, lambda **kwargs: sum(kwargs.values())))

The result should be like below.

Weight tree: <FastTreeValue 0x7f15080581c0 keys: ['a', 'b', 'x', 'y']>
├── 'a' --> 1
├── 'b' --> 2
├── 'x' --> <FastTreeValue 0x7f1508058070 keys: ['c', 'd']>
│   ├── 'c' --> 6
│   └── 'd' --> 8
└── 'y' --> <FastTreeValue 0x7f1508058340 keys: ['e', 'f']>
    ├── 'e' --> 12
    └── 'f' --> 16

Huffman weight sum of t: 45

Besides, we can easily calculate sum of the np.ndarray objects’ bytes size, like the following code.

import numpy as np

from treevalue import reduce_, FastTreeValue

if __name__ == '__main__':
    t = FastTreeValue({
        'a': np.identity(3),
        'b': np.array([[1, 2], [3, 4]]),
        'x': {
            'c': np.zeros(4),
            'd': np.array([[5, 6, 7], [8, 9, 10]])
        },
    })

    print("Size tree:", t.nbytes)
    print("Total bytes of arrays in t:",
          reduce_(t.nbytes, lambda **kwargs: sum(kwargs.values())))

The result should be like below.

Size tree: <FastTreeValue 0x7f9ba39d0bb0 keys: ['a', 'b', 'x']>
├── 'a' --> 72
├── 'b' --> 32
└── 'x' --> <FastTreeValue 0x7f9ba39d0c40 keys: ['c', 'd']>
    ├── 'c' --> 32
    └── 'd' --> 48

Total bytes of arrays in t: 184

For further definition or source code implement of function reduce_, take a look at reduce_.

Structural Utilities

In order to process some structured data, especially when you need to process a sort of TreeValue objects which is in the primitive collections or mappings, the structural utilities are designed, and they will be introduced in this section with examples.

Union

The union function is similar to the primitive zip function some ways, it can combine a list of TreeValue objects as one TreeValue object which leaf values are tuples, like the simple example below.

from treevalue import FastTreeValue, union

if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t3 = FastTreeValue({'a': -13, 'b': -7, 'x': {'c': -5, 'd': -3, 'e': -2}})
    t4 = FastTreeValue({'a': -13, 'b': -7, 'x': 8})

    print("union(t1, t2):")
    print(union(t1, t2))

    print("union(t1, t2, t3):")
    print(union(t1, t2, t3))

    print("union(t1, t2, t3, t4):")
    print(union(t1, t2, t3, t4))

The result should be like below, the leaf values are unionised as tuples.

union(t1, t2):
<FastTreeValue 0x7f6d2f748eb0 keys: ['a', 'b', 'x']>
├── 'a' --> (1, 11)
├── 'b' --> (2, 22)
└── 'x' --> <FastTreeValue 0x7f6d300612e0 keys: ['c', 'd', 'e']>
    ├── 'c' --> (3, 30)
    ├── 'd' --> (4, 48)
    └── 'e' --> (5, 54)

union(t1, t2, t3):
<FastTreeValue 0x7f6d30061ca0 keys: ['a', 'b', 'x']>
├── 'a' --> (1, 11, -13)
├── 'b' --> (2, 22, -7)
└── 'x' --> <FastTreeValue 0x7f6d30061e50 keys: ['c', 'd', 'e']>
    ├── 'c' --> (3, 30, -5)
    ├── 'd' --> (4, 48, -3)
    └── 'e' --> (5, 54, -2)

union(t1, t2, t3, t4):
<FastTreeValue 0x7f6d30061be0 keys: ['a', 'b', 'x']>
├── 'a' --> (1, 11, -13, -13)
├── 'b' --> (2, 22, -7, -7)
└── 'x' --> <FastTreeValue 0x7f6d30061a90 keys: ['c', 'd', 'e']>
    ├── 'c' --> (3, 30, -5, 8)
    ├── 'd' --> (4, 48, -3, 8)
    └── 'e' --> (5, 54, -2, 8)

Note

In union function (actually subside function has the same property), all the trees to be unionised need to have the same structure. You can just consider it as strict mode with inheriting is enabled.

For further information and arguments of function union, take a look at union.

Subside

The subside function can transform the collections or mapping nested TreeValue objects into one TreeValue object which leaf values has the dispatch’s structure. The union function mentioned above is also based on the subside function. Function subside will greatly simplify the code when the structured data need to be calculated. Just like this following example code.

from treevalue import TreeValue, subside

if __name__ == '__main__':
    t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})
    t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}})
    t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}})

    st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]}
    print("subside(st):")
    print(subside(st))

The result should be like below, the leaf values are combined as the dispatch structure of st.

subside(st):
<TreeValue 0x7f3b18a76b80 keys: ['a', 'b', 'x']>
├── 'a' --> {'first': (1, -14), 'second': [6, {'x': 0, 'y': 3}]}
├── 'b' --> {'first': (2, 9), 'second': [0, {'x': -17, 'y': 9}]}
└── 'x' --> <TreeValue 0x7f3b18a76dc0 keys: ['c', 'd']>
    ├── 'c' --> {'first': (3, 3), 'second': [-5, {'x': -8, 'y': 11}]}
    └── 'd' --> {'first': (4, 8), 'second': [17, {'x': 15, 'y': -17}]}

Note

In function subside, only primitive list, tuple and dict (or subclasses) objects will be subsided to leaf values.

For further information and arguments of function subside, take a look at subside.

Rise

Function rise can be seen as the inverse operation of function subside, it will try to extract the greatest common structure of the leaf values, and rise them up to the dispatch above the TreeValue objects. Like the following example code.

from treevalue import TreeValue, subside, rise

if __name__ == '__main__':
    # The same demo as the subside docs
    t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})
    t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}})
    t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}})
    st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]}
    tx = subside(st)

    # Rising process
    st2 = rise(tx)
    assert st2 == st
    print('st2:', st2)

    print("st2['first'][0]:")
    print(st2['first'][0])

    print("st2['first'][1]:")
    print(st2['first'][1])

    print("st2['second'][0]:")
    print(st2['second'][0])

    print("st2['second'][1]['x']:")
    print(st2['second'][1]['x'])

    print("st2['second'][1]['y']:")
    print(st2['second'][1]['y'])

The result should be like below, the subsided tree st can be extract back to the original structure.

st2: {'first': (<TreeValue 0x7fb826236eb0 keys: ['a', 'b', 'x']>, <TreeValue 0x7fb8262369a0 keys: ['a', 'b', 'x']>), 'second': [<TreeValue 0x7fb8262362e0 keys: ['a', 'b', 'x']>, {'x': <TreeValue 0x7fb826234fd0 keys: ['a', 'b', 'x']>, 'y': <TreeValue 0x7fb8262348b0 keys: ['a', 'b', 'x']>}]}
st2['first'][0]:
<TreeValue 0x7fb826236eb0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fb8262366d0 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

st2['first'][1]:
<TreeValue 0x7fb8262369a0 keys: ['a', 'b', 'x']>
├── 'a' --> -14
├── 'b' --> 9
└── 'x' --> <TreeValue 0x7fb826236a00 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 8

st2['second'][0]:
<TreeValue 0x7fb8262362e0 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 0
└── 'x' --> <TreeValue 0x7fb826236160 keys: ['c', 'd']>
    ├── 'c' --> -5
    └── 'd' --> 17

st2['second'][1]['x']:
<TreeValue 0x7fb826234fd0 keys: ['a', 'b', 'x']>
├── 'a' --> 0
├── 'b' --> -17
└── 'x' --> <TreeValue 0x7fb826236040 keys: ['c', 'd']>
    ├── 'c' --> -8
    └── 'd' --> 15

st2['second'][1]['y']:
<TreeValue 0x7fb8262348b0 keys: ['a', 'b', 'x']>
├── 'a' --> 3
├── 'b' --> 9
└── 'x' --> <TreeValue 0x7fb826234b50 keys: ['c', 'd']>
    ├── 'c' --> 11
    └── 'd' --> -17

Note

In function rise, only primitive list, tuple and dict (or subclasses) objects will join the rising operation.

Actually, when template argument is not assigned, the rise function will try to find the greatest common structure and rise them to the dispatch. The following rules will be strictly followed when doing this:

• If the values in current level have the same type ( must all be list, all be tuple or all be dict), the rising function will carry on the find sub structure, otherwise values in current level will be treated as atomic values.

• If all of them are dicts, and they have exactly the same key sets with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.

• If all of them are lists, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.

• If all of them are tuples, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values. (Actually, this rule is the same as the last one which is about lists.)

Considering this automatic structure finding process, if you only want to extract some of the structure (make sure the extraction will not be too deep or too shallow, and make sure the result will have the same structure as your expectation), you can assign value in template arguments. Like the example code below.

from treevalue import TreeValue, subside, rise

if __name__ == '__main__':
    # The same demo as the subside docs
    t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})
    t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}})
    t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}})
    st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]}
    tx = subside(st)

    # Rising process, with template
    # only the top-leveled dict will be extracted,
    # neither tuple, list nor low-leveled dict will not be extracted
    # because they are not defined in `template` argument
    st2 = rise(tx, template={'first': None, 'second': [None, None]})
    print('st2:', st2)

    print("st2['first']:")
    print(st2['first'])

    print("st2['second'][0]:")
    print(st2['second'][0])

    print("st2['second'][1]:")
    print(st2['second'][1])

The result should be like below, the subsided tree st can be extract back to the structure of dict with only first and second keys.

st2: {'first': <TreeValue 0x7ffb7b9a96a0 keys: ['a', 'b', 'x']>, 'second': [<TreeValue 0x7ffb7b9a9ac0 keys: ['a', 'b', 'x']>, <TreeValue 0x7ffb7b9a98e0 keys: ['a', 'b', 'x']>]}
st2['first']:
<TreeValue 0x7ffb7b9a96a0 keys: ['a', 'b', 'x']>
├── 'a' --> (1, -14)
├── 'b' --> (2, 9)
└── 'x' --> <TreeValue 0x7ffb7b9af2b0 keys: ['c', 'd']>
    ├── 'c' --> (3, 3)
    └── 'd' --> (4, 8)

st2['second'][0]:
<TreeValue 0x7ffb7b9a9ac0 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 0
└── 'x' --> <TreeValue 0x7ffb7b9a9ee0 keys: ['c', 'd']>
    ├── 'c' --> -5
    └── 'd' --> 17

st2['second'][1]:
<TreeValue 0x7ffb7b9a98e0 keys: ['a', 'b', 'x']>
├── 'a' --> {'x': 0, 'y': 3}
├── 'b' --> {'x': -17, 'y': 9}
└── 'x' --> <TreeValue 0x7ffb7b9a9850 keys: ['c', 'd']>
    ├── 'c' --> {'x': -8, 'y': 11}
    └── 'd' --> {'x': 15, 'y': -17}

For further information and arguments of function rise, take a look at rise.

Tree Utilities

In this section, utilities with TreeValue class and its objects themselves will be introduced with examples.

Jsonify

With the usage of jsonify function, you can transform a TreeValue object to json-formatted data.

For example, the following real code

import json

from treevalue import TreeValue, raw, jsonify

if __name__ == '__main__':
    t = TreeValue({'a': 1, 'b': [2, 3], 'x': {'c': raw({'x': 1, 'y': 2}), 'd': "this is a string"}})

    print("Tree t:")
    print(t)

    print("Json data of t:")
    print(json.dumps(jsonify(t), indent=4, sort_keys=True))

The result should be

Tree t:
<TreeValue 0x7f5a992050d0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> [2, 3]
└── 'x' --> <TreeValue 0x7f5a9a0a9100 keys: ['c', 'd']>
    ├── 'c' --> {'x': 1, 'y': 2}
    └── 'd' --> 'this is a string'

Json data of t:
{
    "a": 1,
    "b": [
        2,
        3
    ],
    "x": {
        "c": {
            "x": 1,
            "y": 2
        },
        "d": "this is a string"
    }
}

Note

The function raw in the example code above is a wrapper for dictionary object. It can be used to pass dict object as simple value in TreeValue instead of being treated as sub tree.

For further information of function raw, take a look at raw.

For further informaon of function jsonify, take a look at jsonify.

View

Another magic function named view is also provided, to process logic viewing cases.

from treevalue import TreeValue, view

if __name__ == '__main__':
    t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'y': {'e': 5, 'f': 6}}})

    print("Tree t:")
    print(t)

    t_x_y = t.x.y
    vt_x_y = view(t, ['x', 'y'])

    print("t.x.y:")
    print(t_x_y)
    print("View of t, ['x', 'y']:")
    print(vt_x_y)

    t.x = TreeValue({'cc': 33, 'dd': 44, 'y': {'ee': 55, 'ff': 66}})

    print("t_x_y after replacement:")
    print(t_x_y)
    print("View of t, ['x', 'y'] after replacement:")
    print(vt_x_y)

The result will be like below, t_x_y and vt_x_y’s value is different after the replacement of the sub tree named t.x, for vt_x_y’s value’s equality to the current t.x.y.

Tree t:
<TreeValue 0x7f2ada219ca0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f2adb154e20 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> <TreeValue 0x7f2adb0da8e0 keys: ['e', 'f']>
        ├── 'e' --> 5
        └── 'f' --> 6

t.x.y:
<TreeValue 0x7f2adb0da8e0 keys: ['e', 'f']>
├── 'e' --> 5
└── 'f' --> 6

View of t, ['x', 'y']:
<TreeValue 0x7f2adb0da8e0 keys: ['e', 'f']>
├── 'e' --> 5
└── 'f' --> 6

t_x_y after replacement:
<TreeValue 0x7f2adb0da8e0 keys: ['e', 'f']>
├── 'e' --> 5
└── 'f' --> 6

View of t, ['x', 'y'] after replacement:
<TreeValue 0x7f2ad9fcc910 keys: ['ee', 'ff']>
├── 'ee' --> 55
└── 'ff' --> 66

Note

Attention that the view operation is different from sub node getting operation. In viewed tree, it is based on a logic link based on the tree be viewed, and the actual operations are performed in the actual tree node. The main difference between view and getting sub node is that the view tree will be affected by the replacement of sub nodes in viewed tree.

Like the code example above, in the view tree vt_x_y, before do tree operations, the viewed tree will be approached from root tree t by keys of x and y, so after the replacement of the whole subtree t.x, vt_x_y is still the latest value of t.x.y, while t_x_y is still the old sub tree of tree t, before replacement.

For further informaon of function view, take a look at view.

Clone

The TreeValue objects can be cloned deeply by clone function.

from treevalue import TreeValue, clone, raw

if __name__ == '__main__':
    t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'y': raw({'e': 5, 'f': 6})}})

    print("Tree t:")
    print(t)
    print("Id of t.x.y: %x" % id(t.x.y))
    print()
    print()

    print("clone(t):")
    print(clone(t))
    print("Id of clone(t).x.y: %x" % id(clone(t).x.y))
    print()
    print()

    print('clone(t, copy_value=True):')
    print(clone(t, copy_value=True))
    print("Id of clone(t, copy_value=True).x.y: %x" % id(clone(t, copy_value=True)))
    print()
    print()

The result will be like below, all the memory address of the tree nodes in cloned tree are different from those in the original tree t when copy_value argument is not set. But when copy_value is assigned as True, the address of values will be changed because of the deep copy of value.

Tree t:
<TreeValue 0x7fa101d310a0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fa10256ddf0 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> {'e': 5, 'f': 6}

Id of t.x.y: 7fa1029f3cc0


clone(t):
<TreeValue 0x7fa101d19d30 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fa101d19ee0 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> {'e': 5, 'f': 6}

Id of clone(t).x.y: 7fa1029f3cc0


clone(t, copy_value=True):
<TreeValue 0x7fa101ac5cd0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fa101ac5d60 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> {'e': 5, 'f': 6}

Id of clone(t, copy_value=True).x.y: 7fa101ac5fd0

Note

Attention that in function clone, the values will not be deeply copied together with the tree nodes by default. The newly cloned tree’s values have the same memory address with those in original tree.

If deep copy of values is required when using clone, copy_value argument need to be assigned as True. And then a copy.deepcopy will be performed in clone function in order to do the deep copy. Also, you can define your own copy function in copy_value argument by just assign it as a lambda expression like lambda x: pickle.loads(pickle.dumps(x)).

For further informaon of function clone, take a look at clone.

Typetrans

You can use function typetrans to change the type of tree value object, just like the example below.

from treevalue import TreeValue, method_treelize, FastTreeValue, typetrans


class MyTreeValue(TreeValue):
    @method_treelize()
    def pw(self):
        return (self + 1) * (self + 2)


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    print('t1:')
    print(t1)
    print('t1 ** 2:')
    print(t1 ** 2)
    print()

    # Transform t1 to MyTreeValue,
    # __pow__ operator will be disabled and pw method will be enabled.
    t2 = typetrans(t1, MyTreeValue)
    print('t2:')
    print(t2)
    print('t2.pw():')
    print(t2.pw())
    print()

The result will be like below. After the type transformation, ** operator can not be used in t2 but method pw which is implemented in MyTreeValue will be enabled.

t1:
<FastTreeValue 0x7f03985d9d90 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <FastTreeValue 0x7f0398c8cdf0 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

t1 ** 2:
<FastTreeValue 0x7f039834fa60 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 4
└── 'x' --> <FastTreeValue 0x7f039834fbe0 keys: ['c', 'd']>
    ├── 'c' --> 9
    └── 'd' --> 16


t2:
<MyTreeValue 0x7f03985d9d90 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <MyTreeValue 0x7f0398c8cdf0 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

t2.pw():
<MyTreeValue 0x7f0398375460 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 12
└── 'x' --> <MyTreeValue 0x7f03983755e0 keys: ['c', 'd']>
    ├── 'c' --> 20
    └── 'd' --> 30

Note

In treevalue library, there can be more classes based on TreeValue class by your definition.

Different tree value class will have the different operators, methods and class methods for service, and the __eq__ operator’s result between different type of tree values will always be False because of the difference of their types.

For further information of TreeValue class and user’s definition practice, just take a look at:

• API documentation of TreeValue.

• API documentation of FastTreeValue.

• DIY TreeValue Class.

For further informaon of function typetrans, take a look at typetrans.

Object Oriented Usage

In FastTreeValue class, plenty of object-oriented operators, methods and classmethods are supported in order to simplify the actual usage. Here is a simple code example.

from treevalue import FastTreeValue

if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = FastTreeValue({'a': 5, 'b': 6, 'x': {'c': 7, 'd': 8}})

    # operator support
    print('t1 + t2:')
    print(t1 + t2)
    print('t2 ** t1:')
    print(t2 ** t1)
    print()

    # utilities support
    print('t1.map(lambda x: (x + 1) * (x + 2)):')
    print(t1.map(lambda x: (x + 1) * (x + 2)))
    print('t1.reduce(lambda **kwargs: sum(kwargs.values())):',
          t1.reduce(lambda **kwargs: sum(kwargs.values())))
    print()
    print()

    # linking usage
    print('t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0):')
    print(t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0))
    print()

    # structural support
    print("Union result:")
    print(FastTreeValue.union(
        t1.map(lambda x: (x + 1) * (x + 2)),
        t2.map(lambda x: (x - 2) ** (x - 1)),
    ).map(lambda x: 'first: %d, second: %d, which sum is %d' % (x[0], x[1], sum(x))))
    print()

The result should be like below.

t1 + t2:
<FastTreeValue 0x7f52992c9bb0 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 8
└── 'x' --> <FastTreeValue 0x7f52992c9c70 keys: ['c', 'd']>
    ├── 'c' --> 10
    └── 'd' --> 12

t2 ** t1:
<FastTreeValue 0x7f52992cc430 keys: ['a', 'b', 'x']>
├── 'a' --> 5
├── 'b' --> 36
└── 'x' --> <FastTreeValue 0x7f52992cc4f0 keys: ['c', 'd']>
    ├── 'c' --> 343
    └── 'd' --> 4096


t1.map(lambda x: (x + 1) * (x + 2)):
<FastTreeValue 0x7f52992cc760 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 12
└── 'x' --> <FastTreeValue 0x7f52992cc6d0 keys: ['c', 'd']>
    ├── 'c' --> 20
    └── 'd' --> 30

t1.reduce(lambda **kwargs: sum(kwargs.values())): 10


t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0):
<FastTreeValue 0x7f52992dac70 keys: ['b', 'x']>
├── 'b' --> 12
└── 'x' --> <FastTreeValue 0x7f52992daa30 keys: ['c']>
    └── 'c' --> 20


Union result:
<FastTreeValue 0x7f52992df400 keys: ['a', 'b', 'x']>
├── 'a' --> 'first: 6, second: 81, which sum is 87'
├── 'b' --> 'first: 12, second: 1024, which sum is 1036'
└── 'x' --> <FastTreeValue 0x7f52992df550 keys: ['c', 'd']>
    ├── 'c' --> 'first: 20, second: 15625, which sum is 15645'
    └── 'd' --> 'first: 30, second: 279936, which sum is 279966'

For further information of FastTreeValue class, take a look at FastTreeValue, all the supported methods and operators are listed here.

DIY TreeValue Class

You can define your own TreeValue class with your own method and class methods. Like the example code below.

from treevalue import classmethod_treelize, TreeValue, method_treelize


class MyTreeValue(TreeValue):
    # return type will be automatically detected as `MyTreeValue`
    @method_treelize()
    def append(self, b):
        print("Append arguments:", self, b)
        return self + b

    # return type will be automatically detected as `MyTreeValue`
    @classmethod
    @classmethod_treelize()
    def sum(cls, *args):
        print("Sum arguments:", cls, *args)
        return sum(args)


if __name__ == '__main__':
    t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})

    print('t1.append(t2).append(t3):')
    print(t1.append(t2).append(t3))

    print('MyTreeValue.sum(t1, t2, t3):')
    print(MyTreeValue.sum(t1, t2, t3))

The result should be like below.

t1.append(t2).append(t3):
Append arguments: 1 -14
Append arguments: 2 9
Append arguments: 3 3
Append arguments: 4 8
Append arguments: -13 6
Append arguments: 11 0
Append arguments: 6 -5
Append arguments: 12 17
<MyTreeValue 0x7f9749ca2850 keys: ['a', 'b', 'x']>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7f9749ca2910 keys: ['c', 'd']>
    ├── 'c' --> 1
    └── 'd' --> 29

MyTreeValue.sum(t1, t2, t3):
Sum arguments: <class '__main__.MyTreeValue'> 1 -14 6
Sum arguments: <class '__main__.MyTreeValue'> 2 9 0
Sum arguments: <class '__main__.MyTreeValue'> 3 3 -5
Sum arguments: <class '__main__.MyTreeValue'> 4 8 17
<MyTreeValue 0x7f9749ca2bb0 keys: ['a', 'b', 'x']>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7f974938d2b0 keys: ['c', 'd']>
    ├── 'c' --> 1
    └── 'd' --> 29

For further information of function method_treelize and classmethod_treelize, just take a look at:

• method_treelize

• classmethod_treelize

DIY TreeValue Utility Class

You can define a pure utility class in with function utils_class and classmethod_treelize. Like the following example code.

from abc import ABCMeta
from functools import reduce
from operator import __mul__

from treevalue import utils_class, classmethod_treelize, TreeValue


class MyTreeValue(TreeValue):
    pass


# this is the utils class
@utils_class(return_type=MyTreeValue)
class CalcUtils(metaclass=ABCMeta):
    # return type will be detected as `MyTreeValue` as for the utils_class
    @classmethod
    @classmethod_treelize()
    def sum(cls, *args):
        print("Sum arguments:", cls, *args)
        return sum(args)

    # return type will be detected as `MyTreeValue` as for the utils_class
    @classmethod
    @classmethod_treelize()
    def mul(cls, *args):
        print("Mul arguments:", cls, *args)
        return reduce(__mul__, args, 1)


if __name__ == '__main__':
    t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})

    print('CalcUtils.sum(t1, t2, t3):')
    print(CalcUtils.sum(t1, t2, t3))

    print('CalcUtils.mul(t1, t2, t3):')
    print(CalcUtils.mul(t1, t2, t3))

The result should be like below.

CalcUtils.sum(t1, t2, t3):
Sum arguments: <class '__main__.CalcUtils'> 1 -14 6
Sum arguments: <class '__main__.CalcUtils'> 2 9 0
Sum arguments: <class '__main__.CalcUtils'> 3 3 -5
Sum arguments: <class '__main__.CalcUtils'> 4 8 17
<MyTreeValue 0x7fb7a4513400 keys: ['a', 'b', 'x']>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7fb7a45134f0 keys: ['c', 'd']>
    ├── 'c' --> 1
    └── 'd' --> 29

CalcUtils.mul(t1, t2, t3):
Mul arguments: <class '__main__.CalcUtils'> 1 -14 6
Mul arguments: <class '__main__.CalcUtils'> 2 9 0
Mul arguments: <class '__main__.CalcUtils'> 3 3 -5
Mul arguments: <class '__main__.CalcUtils'> 4 8 17
<MyTreeValue 0x7fb7a4513d30 keys: ['a', 'b', 'x']>
├── 'a' --> -84
├── 'b' --> 0
└── 'x' --> <MyTreeValue 0x7fb7a4513e20 keys: ['c', 'd']>
    ├── 'c' --> -45
    └── 'd' --> 544

For further information of function utils_class and classmethod_treelize, just take a look at:

• utils_class

• classmethod_treelize