Advanced Usage

In this part, Some of the advanced usages of TreeValue will be introduced one by one with sample code and graph to explain them.

For further information or usage of TreeValue and FastTreeValue, you may take a look at the following pages:

Function Modes

In the basic usage description in Make function tree supported, we can see that a common function which only support the calculation of the common values can be wrapped to support the tree-based calculation painlessly.

Here is the documentation of function treevalue.tree.func.func_treelize.

treevalue.tree.func.func_treelize(mode: Union[str, treevalue.tree.func.func.TreeMode] = 'strict', return_type: Optional[Type[TreeClassType_]] = <class 'treevalue.tree.tree.tree.TreeValue'>, inherit: bool = True, missing: Union[Any, Callable] = <SingletonMark 'missing_not_allow'>, subside: Optional[Union[Mapping, bool]] = None, rise: Optional[Union[Mapping, bool]] = None)[source]
Overview:

Wrap a common function to tree-supported function.

Arguments:

  • mode (Union[str, TreeMode]): Mode of the wrapping (string or TreeMode both okay), default is strict.

  • return_type (Optional[Type[TreeClassType_]]): Return type of the wrapped function, default is TreeValue.

  • inherit (bool): Allow inherit in wrapped function, default is True.

  • missing (Union[Any, Callable]): Missing value or lambda generator of when missing, default is MISSING_NOT_ALLOW, which means raise KeyError when missing detected.

  • subside (Union[Mapping, bool, None]): Subside enabled to function’s arguments or not, and subside configuration, default is None which means do not use subside. When subside is True, it will use all the default arguments in subside function.

  • rise (Union[Mapping, bool, None]): Rise enabled to function’s return value or not, and rise configuration, default is None which means do not use rise. When rise is True, it will use all the default arguments in rise function. (Not recommend to use auto mode when your return structure is not so strict.)

Returns:

  • decorator (Callable): Wrapper for tree-supported function.

Example:
>>> @func_treelize()
>>> def ssum(a, b):
>>>     return a + b  # the a and b will be integers, not TreeValue
>>>
>>> t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
>>> t2 = TreeValue({'a': 11, 'b': 22, 'x': {'c': 33, 'd': 5}})
>>> ssum(1, 2)    # 3
>>> ssum(t1, t2)  # TreeValue({'a': 12, 'b': 24, 'x': {'c': 36, 'd': 9}})

In the arguments listed above, 3 of them are key arguments:

  • mode, Tree process mode of the calculation. Will be introduced in this section.

  • inherit, Inheriting mode of the calculation on the tree. Will be introduced in Inheriting on Trees.

  • missing, Missing processor of the calculation on the tree. Will be introduced in Process Missing Values.

The mode argument is the most important argument in function func_treelize. For it depends the basic logic of the graph calculation.

The type of mode argument is TreeMode, which documentation is like this.

enum treevalue.tree.func.TreeMode(value)[source]
Overview:

Four mode of the tree calculation

Member Type:

int

Valid values are as follows:

STRICT = <TreeMode.STRICT: 1>

Strict mode, which means the keys should be one to one in every trees.

LEFT = <TreeMode.LEFT: 2>

Left mode, the keys of the result is relied on the left value.

INNER = <TreeMode.INNER: 3>

Inner mode, the keys of the result is relied on the intersection of the trees’ key set.

OUTER = <TreeMode.OUTER: 4>

Outer mode, the keys of the result is relied on the union of the trees’ key set.

In this part, all of the 4 modes will be introduced with details and samples one by one.

Strict Mode

Strict mode is the most frequently-used mode in real cases. It is also the default value of the mode argument in the treelize functions.

To be simple, strict mode need the keys of a node in the tree be strictly mapped one by one, missing or overage of the keys are both forbidden in strict mode, which means KeyError will be raised.

For example, in the trees in strict_demo_1, if we need to added t1 and t2 together, t3 will be the result. Because all the keys (a, b, x of the root nodes, and c, d, e of the x node) can be mapped one by one.

../../_images/strict_demo_1.gv.svg

In another situation, when the keys are not so regular like that in strict_demo_1, like the trees in strict_demo_2. If we try to add t1 and t2 in strict mode, KeyError will be raised due to the missing of key a in t1 and key f in t2.x.

../../_images/strict_demo_2.gv.svg

Here is a real code example of strict mode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='strict')
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t4, t2):', plus(t4, t2))

The stdout and stderr should like below, a KeyError is raised at the second print statement.

1
2
3
4
5
6
7
plus(t1, t2): <TreeValue 0x7eff7562b430 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7eff7569c550 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/docs/source/tutorials/advanced_usage/strict_demo.demox.py", line 15, in <module>
    print('plus(t4, t2):', plus(t4, t2))
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 152, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 142, in _recursion
    )) for key in sorted(_MODE_PROCESSORS[mode].get_key_set(*args, **kwargs))
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/strict.py", line 12, in get_key_set
    raise KeyError(
KeyError: "Argument keys not match in strict mode, key set of argument 1 is ('b', 'x') but 2 in ('a', 'b', 'x')."

Note

How does the treelized function work?

Here is another example which show the actual calculation process of the wrapped function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
import numpy as np

from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='strict')
def plus(a, b):
    print("Current a and b:", type(a), a, type(b), b)
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t2 = FastTreeValue({
        'a': np.array([1, 3, 5, 7]),
        'b': np.array([2, 5, 8, 11]),
        'x': {
            'c': 3,
            'd': 4.8,
            'e': np.array([[2, 3], [5, 6]]),
        }
    })
    t3 = FastTreeValue({
        'a': np.array([1, 3, 5, 7]),
        'b': [2, 5, 8, 11],
        'x': {
            'c': 3,
            'd': 4.8,
            'e': np.array([[2, 3], [5, 6]]),
        }
    })

    print('plus(t1, t2):', plus(t1, t2))
    print()
    print('plus(t1, t3):', plus(t1, t3))
    print()

In this code, once the original function plus is called, the actual value of argument a and b will be printed to stdout.

In the plus(t1, t2), all the calculation can be carried on because add operator between primitive int and np.ndarray is supported, so no error occurs and the result of + between int and np.ndarray will be one of the values of the result tree.

But in plus(t1, t3), when get the result of plus(t1, t3).b, it will try to add t1.b and t2.b together, but there is no implement of operator + between primitive int and list objects, so TypeError will be raised when do this calculation.

The complete stdout and stderr should be like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7]
Current a and b: <class 'int'> 22 <class 'numpy.ndarray'> [ 2  5  8 11]
Current a and b: <class 'int'> 30 <class 'int'> 3
Current a and b: <class 'int'> 48 <class 'float'> 4.8
Current a and b: <class 'int'> 54 <class 'numpy.ndarray'> [[2 3]
 [5 6]]
plus(t1, t2): <TreeValue 0x7f4f5e93dfd0 keys: ['a', 'b', 'x']>
├── 'a' --> array([12, 14, 16, 18])
├── 'b' --> array([24, 27, 30, 33])
└── 'x' --> <TreeValue 0x7f4f5e91f9d0 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52.8
    └── 'e' --> array([[56, 57],
       [59, 60]])


Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7]
Current a and b: <class 'int'> 22 <class 'list'> [2, 5, 8, 11]

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 35, in <module>
    print('plus(t1, t3):', plus(t1, t3))
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 152, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 138, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 139, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 133, in _recursion
    return func(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 9, in plus
    return a + b
TypeError: unsupported operand type(s) for +: 'int' and 'list'

In strict mode, missing or overage of keys are not tolerated at all. So in some of the cases, especially when you can not suppose that all the trees involved in calculation has exactly the same structure. Based on this demand, left mode, inner mode and outer mode is designed, and they will be introduced with details and examples in the next 3 sections.

Left Mode

In left mode, the result tree’s key set will be aligned to the first tree from the left.

In the trees in left_demo_1, t3 is the plus result of t1 and t2. We can see that t2.a is ignored because of t1.a ‘s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of t1.

../../_images/left_demo_1.gv.svg

But in the left_demo_2 (actually it is exactly the same as strict_demo_2), KeyError will be raised because t2.x.f` not found in tree t2`, so the result \ can not be aligned to tree ``t1, calculation failed.

../../_images/left_demo_2.gv.svg

When you decorate a function as left mode, the decorated function will try to find the first tree from left. If there is no less than 1 positional arguments detected, the first positional argument will be used as the left tree. Otherwise, the tree with the smallest lexicographic key will be assigned as the left tree. Obviously, As least 1 positional or key-word-based argument should be in the arguments.

Here is a real code example of left mode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='left')
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t4, t2):', plus(t4, t2))

The stdout and stderr should like below, a KeyError is raised at the second print statement.

1
2
3
4
5
6
plus(t1, t2): <TreeValue 0x7fe79ade6430 keys: ['b', 'x']>
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7fe79ae57550 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/docs/source/tutorials/advanced_usage/left_demo.demox.py", line 15, in <module>
    print('plus(t4, t2):', plus(t4, t2))
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 152, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 138, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 139, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 138, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 139, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 140, in <genexpr>
    *(item(key) for item in pargs),
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 117, in _get_from_key
    raise KeyError("Missing is off, key {key} not found in {item}.".format(
KeyError: "Missing is off, key 'f' not found in <FastTreeValue 0x7fe79adf0b50 keys: ['c', 'd', 'e']>."

In the code example above, the result is aligned to tree a in wrapped function plus. So the first plus calculation will get the proper result, but the second one will fail.

Inner Mode

In inner mode, the result tree’s key set will be aligned to the intersection set of all the tree’s key set in arguments.

In the trees in inner_demo_1, t3 is the plus result of t1 and t2. We can see that t2.a and t1.x.f is ignored because of t1.a and t2.x.f’s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of the intersection between tree t1 and t2.

../../_images/inner_demo_1.gv.svg

Note

In most cases of inner mode, the calculation will be processed properly because of the intersection operation on the key set and key missing or overage are both avoided completely.

But the shortage of inner mode is the loss of some information, for keys and its value or subtrees will be directly ignored without any warning. So before using inner mode, please confirm that you know its running logic and the missing of some nodes.

Here is a real code example of inner mode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='inner')
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t4, t2):', plus(t4, t2))

The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
plus(t1, t2): <TreeValue 0x7fb87f01f430 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7fb87f08f550 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

plus(t4, t2): <TreeValue 0x7fb87f02d340 keys: ['b', 'x']>
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7fb87f039100 keys: ['c', 'd', 'e']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    └── 'e' --> 59

Outer Mode

In outer mode, the result tree’s key set will be aligned to the union set of the tree’s key set in arguments.

In the trees in outer_demo_1, t3 is the plus result of t1 and t2, with the missing value of 0. We can see that all the values in tree t1 and t2 are involved in this calculation, and because of the missing of t1.a and t2.x.f they will be actually treated as 0 in calculation process. Finally no error will be raised, and sum of the tree nodes’ value will form the result tree t3.

../../_images/outer_demo_1.gv.svg

Note

In outer mode, it is strongly recommended to set the value of argument missing. When missing’s value is not set and there are key missing in some of the argument trees, KeyError will be raised due to this missing and value of missing.

For further information and examples of missing argument, take a look at Process Missing Values.

Here is a real code example of inner mode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from treevalue import FastTreeValue, func_treelize


# missing value is very important when use outer mode
@func_treelize(mode='outer', missing=0)
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})

    print('plus(t1, t2):', plus(t1, t2))

The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.

1
2
3
4
5
6
7
8
plus(t1, t2): <TreeValue 0x7fa9210a2dc0 keys: ['a', 'b', 'x']>
├── 'a' --> 11
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7fa921099e20 keys: ['c', 'd', 'e', 'f']>
    ├── 'c' --> 33
    ├── 'd' --> 52
    ├── 'e' --> 59
    └── 'f' --> 6

Inheriting on Trees

Inheriting is another important feature of treelize functions. It will be introduced in this section with necessary examples.

In some cases, we need to calculation between values with different dimensions, such as in numpy, you can add primitive integer with a ndarray object, like this

1
2
3
4
5
6
7
import os

import numpy as np

if __name__ == '__main__':
    ar1 = np.array([[1, 2], [3, 4]])
    print('ar1 + 9:', ar1 + 9, sep=os.linesep)

The result will be like below, result of ar1 + 9 is actually the same as the result of expression ar1 + np.array([[9, 9], [9, 9]]).

1
2
3
ar1 + 9:
[[10 11]
 [12 13]]

In treelize functions, you can achieve this processing way by use the inherit argument. Considering this situation can be find almost anywhere, so the default value of inherit argument is True, which means the inheriting option is on by default.

We can see the example in inherit_demo_1, the tree t1 and tree t2 have different structures, and cannot be processed by any modes in Function Modes, because t1.x is a primitive integer value but t2.x is a subtree node, their types are structurally different.

../../_images/inherit_demo_1.gv.svg

But when inheriting is enabled, this adding operation can still be carried on, for the value of t1.x (is 9 in inherit_demo_1) will be applied to the whole subtree t2.x, like the result tree t3 shows. The result of tree t3.x can be considered as the sum of primitive integer 9 and subtree t2.x.

Based on the structural properties above, in the example of inherit_demo_2, a primitive value can be directly added with a complete tree, like the result tree t2 = t1 + 5 shows.

../../_images/inherit_demo_2.gv.svg

When inheriting is disabled, all the cases with primitive value and tree node at the same path position will cause TypeError. For example, when inheriting is disabled, inherit_demo_1 and inherit_demo_2 will failed, but strict_demo_1 will still success because no primitive value appears at the same path position of tree node.

../../_images/strict_demo_1.gv.svg

Here is a real code example of inheriting.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from treevalue import FastTreeValue, func_treelize


@func_treelize(mode='strict')
def plus(a, b):
    return a + b


@func_treelize(mode='strict', inherit=False)
def plusx(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': 9})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})

    print('plus(t1, t2):', plus(t1, t2))
    print('plus(t2, 5):', plus(t2, 5))
    print()

    print('plusx(t1, t2):', plusx(t1, t2))
    print()

The stdout and stderr should be like below, the the calculation of plus function and the first calculation of plusx function will be processed properly, but the last one will failed due to the disablement of inheriting.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
plus(t1, t2): <TreeValue 0x7f9b9f82f460 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 24
└── 'x' --> <TreeValue 0x7f9b9f8244f0 keys: ['c', 'd', 'e']>
    ├── 'c' --> 39
    ├── 'd' --> 57
    └── 'e' --> 63

plus(t2, 5): <TreeValue 0x7f9b9f8a34f0 keys: ['a', 'b', 'x']>
├── 'a' --> 16
├── 'b' --> 27
└── 'x' --> <TreeValue 0x7f9b9f839ee0 keys: ['c', 'd', 'e']>
    ├── 'c' --> 35
    ├── 'd' --> 53
    └── 'e' --> 59

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
Traceback (most recent call last):
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/docs/source/tutorials/advanced_usage/inherit_demo.demox.py", line 22, in <module>
    print('plusx(t1, t2):', plusx(t1, t2))
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 152, in _new_func
    _result = _recursion(*args, **kwargs)
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 138, in _recursion
    _data = {
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 139, in <dictcomp>
    key: raw(_recursion(
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 135, in _recursion
    pargs = [_value_wrap(item, index) for index, item in enumerate(args)]
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 135, in <listcomp>
    pargs = [_value_wrap(item, index) for index, item in enumerate(args)]
  File "/tmp/tmpuz3cygjh/d08086bb624e8c20ec1bc07a22e1c784949a7006/treevalue/tree/func/func.py", line 125, in _value_wrap
    raise TypeError("Inherit is off, tree value expected but {type} found in args {index}.".format(
TypeError: Inherit is off, tree value expected but 'int' found in args 0.

Note

In most cases, disablement of inheriting is not recommended because it will cause some errors. Disable inheriting can increase the structural strictness of calculation, but all the cross-dimensional tree operation will be forbidden as well.

So before disabling inheriting, make sure you know well about what you are going to do.

Process Missing Values

In some cases of left mode and outer mode, some of keys in result tree can not find the node nor value at the corresponding path position in some of the argument trees. In order the make these cases be processable, missing value can be used by setting value or lambda expression in missing argument.

Just like the example outer_demo_1 mentioned forwards, when t1.a and t2.x.f not found in trees, the given missing value 0 will be used instead.

../../_images/outer_demo_1.gv.svg

Another case is the missing_demo_1 which is based on left mode. In this case, t2.a is ignore due to the property of left mode, t2.x.f is missing but t1.x.f can be found, so t2.x.f will use the missing value 0. Finally, t3 will be the result of left addition of t1 and t2, with missing value of 0.

../../_images/missing_demo_1.gv.svg

Considering another complex case, when the values of tree are primitive lists, like the missing_demo_2 which is based on outer mode. At this time, if we need to do addition with missing value supported, we can set the missing value to an empty list. The result is like below.

../../_images/missing_demo_2.gv.svg

Note

In missing_demo_2, lambda :[] will be the value of missing argument in actual code.

In missing argument, its value will be automatically executed when it is callable, and its execution’s return value will be the actual missing value.

By passing a lambda expression in, you can construct a new object everytime missing value is required, and the duplication of the missing value’s object will be avoided. So it is not recommended to use missing argument like [], {} or myobject() in python code, but lambda :[], lambda :{} and lambda :myobject() are better practices.

Here is a real code example of missing_value.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import os

from treevalue import func_treelize, FastTreeValue


@func_treelize(mode='outer', missing=lambda: [])
def plus(a, b):
    return a + b


if __name__ == '__main__':
    t1 = FastTreeValue({
        'b': [2, 3],
        'x': {
            'c': [5],
            'd': [7, 11, 13],
            'e': [17, 19],
        }
    })
    t2 = FastTreeValue({
        'a': [23],
        'b': [29, 31],
        'x': {
            'c': [37],
            'd': [41, 43],
        }
    })

    print('plus(t1, t2):', plus(t1, t2), sep=os.linesep)

The stdout (no error occurred) should be like below, the calculation of plus function will be processed properly, like the result in missing_demo_2.

1
2
3
4
5
6
7
8
plus(t1, t2):
<TreeValue 0x7f20da44afd0 keys: ['a', 'b', 'x']>
├── 'a' --> [23]
├── 'b' --> [2, 3, 29, 31]
└── 'x' --> <TreeValue 0x7f20da441c70 keys: ['c', 'd', 'e']>
    ├── 'c' --> [5, 37]
    ├── 'd' --> [7, 11, 13, 41, 43]
    └── 'e' --> [17, 19]

Note

Missing value will be only applied in left mode and outer mode.

In strict mode, missing or overage of keys are absolutely not tolerated, missing value will make no sense and a RuntimeWarning will be logged.

In inner mode, missing value will never be actually in use because all the key sets of final result are intersection set of argument trees. There will never be any key missing cases at all when inner mode is used, a RuntimeWarning will be logged with the usage of missing value in inner mode.

Functional Utilities

In the primitive python, some convenient functional operations are supported, such as

1
2
3
4
5
6
if __name__ == '__main__':
    # map function
    print("Result of map:", list(map(lambda x: x + 1, [2, 3, 5, 7])))

    # filter function
    print("Result of filter:", list(filter(lambda x: x % 4 == 3, [2, 3, 5, 7])))

The result should be

1
2
Result of map: [3, 4, 6, 8]
Result of filter: [3, 7]

Actually, functional operations are supported in tree values as well, and they will be introduced in this section.

Mapping

Mapping function is similar to the primitive map function. The relation between TreeValue and mapping function is like that between list and map function.

Here is a simple real code example

1
2
3
4
5
6
7
from treevalue import mapping, FastTreeValue

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})

    print('mapping(t, lambda x: x ** x + 2):')
    print(mapping(t, lambda x: x ** x + 2))

The mapped tree result should be like below.

1
2
3
4
5
6
7
mapping(t, lambda x: x ** x + 2):
<FastTreeValue 0x7fa70bc97130 keys: ['a', 'b', 'x']>
├── 'a' --> 3
├── 'b' --> 6
└── 'x' --> <FastTreeValue 0x7fa70bc97280 keys: ['c', 'd']>
    ├── 'c' --> 29
    └── 'd' --> 258

For further definition or source code implement of function mapping, take a look at mapping.

Filter

Filter function is similar to the primitive filter function. The relation between TreeValue and filter_ function is like that between list and filter function .

Here is a simple real code example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
from treevalue import FastTreeValue, filter_

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    print('filter_(t, lambda x: x % 2 == 1):')
    print(filter_(t, lambda x: x % 2 == 1))

    print('filter_(t, lambda x: x % 2 == 1, remove_empty=False):')
    print(filter_(t, lambda x: x % 2 == 1, remove_empty=False))

The filtered tree result should be like below. When remove_empty is disabled, the empty tree node will be kept to make sure the structure of the result tree is similar to the unfiltered tree.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
filter_(t, lambda x: x % 2 == 1):
<FastTreeValue 0x7f789b34eca0 keys: ['a', 'x']>
├── 'a' --> 1
└── 'x' --> <FastTreeValue 0x7f789b34e8e0 keys: ['c']>
    └── 'c' --> 3

filter_(t, lambda x: x % 2 == 1, remove_empty=False):
<FastTreeValue 0x7f789b33db80 keys: ['a', 'x', 'y']>
├── 'a' --> 1
├── 'x' --> <FastTreeValue 0x7f789b338610 keys: ['c']>
│   └── 'c' --> 3
└── 'y' --> <FastTreeValue 0x7f789b33ad90 keys: []>

Note

Actually, the code above is equal to the code below. The filter_ function can be seen as a combination of mask function and mapping function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
from treevalue import FastTreeValue, mapping, mask

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    print('mask(t, mapping(t, lambda x: x % 2 == 1)):')
    print(mask(t, mapping(t, lambda x: x % 2 == 1)))

    print('mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False):')
    print(mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False))

The result should be like below, exactly the same as the code above.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
mask(t, mapping(t, lambda x: x % 2 == 1)):
<FastTreeValue 0x7f18abffaca0 keys: ['a', 'x']>
├── 'a' --> 1
└── 'x' --> <FastTreeValue 0x7f18abffa8e0 keys: ['c']>
    └── 'c' --> 3

mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False):
<FastTreeValue 0x7f18abfe8b80 keys: ['a', 'x', 'y']>
├── 'a' --> 1
├── 'x' --> <FastTreeValue 0x7f18abfe3610 keys: ['c']>
│   └── 'c' --> 3
└── 'y' --> <FastTreeValue 0x7f18abfe5d90 keys: []>

For further information about mask function, take a look at Mask.

For further definition or source code implement of function filter_, take a look at filter_.

Mask

Mask function allow your choice in one tree, by another tree of true-or-false flags. A simple code example is

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
from treevalue import FastTreeValue, mask

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})
    m = FastTreeValue({'a': True, 'b': False, 'x': {'c': True, 'd': True}, 'y': {'e': False, 'f': False}})

    print('mask(t, m):')
    print(mask(t, m))

    print('mask(t, m, remove_empty=False):')
    print(mask(t, m, remove_empty=False))

The result will be like below, values mapped with the tree m which mask flag is False are all deleted. Also, remove_empty argument is supported.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
mask(t, m):
<FastTreeValue 0x7f59b7b854c0 keys: ['a', 'x']>
├── 'a' --> 1
└── 'x' --> <FastTreeValue 0x7f59b7b85100 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

mask(t, m, remove_empty=False):
<FastTreeValue 0x7f59b7b8f700 keys: ['a', 'x', 'y']>
├── 'a' --> 1
├── 'x' --> <FastTreeValue 0x7f59b7b8f340 keys: ['c', 'd']>
│   ├── 'c' --> 3
│   └── 'd' --> 4
└── 'y' --> <FastTreeValue 0x7f59b7b8f580 keys: []>

For further definition or source code implement of function mask, take a look at mask.

Reduce

By using reduce_ function, you can get some calculation result based on the tree structure. Its meaning is similar to primitive reduce_ function, but its base structure is TreeValue instead of sequence or iterator. For example, we can get the sum and multiply accumulation of the values in the tree.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
from functools import reduce
from operator import __mul__

from treevalue import reduce_, FastTreeValue


def multi(items):
    return reduce(__mul__, items, 1)


if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    print("Sum of t:", reduce_(t, lambda **kwargs: sum(kwargs.values())))
    print("Multiply of t:", reduce_(t, lambda **kwargs: multi(kwargs.values())))

The result should be like below.

1
2
Sum of t: 24
Multiply of t: 1152

If we use reduce_ function with mapping function, huffman weight sum can also be easily calculated with the code below.

1
2
3
4
5
6
7
8
9
from treevalue import reduce_, FastTreeValue, mapping

if __name__ == '__main__':
    t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}})

    weights = mapping(t, lambda v, p: v * len(p))
    print("Weight tree:", weights)
    print("Huffman weight sum of t:",
          reduce_(weights, lambda **kwargs: sum(kwargs.values())))

The result should be like below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
Weight tree: <FastTreeValue 0x7f3e1b82da30 keys: ['a', 'b', 'x', 'y']>
├── 'a' --> 1
├── 'b' --> 2
├── 'x' --> <FastTreeValue 0x7f3e1b82d8e0 keys: ['c', 'd']>
│   ├── 'c' --> 6
│   └── 'd' --> 8
└── 'y' --> <FastTreeValue 0x7f3e1b82dbb0 keys: ['e', 'f']>
    ├── 'e' --> 12
    └── 'f' --> 16

Huffman weight sum of t: 45

Besides, we can easily calculate sum of the np.ndarray objects’ bytes size, like the following code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import numpy as np

from treevalue import reduce_, FastTreeValue

if __name__ == '__main__':
    t = FastTreeValue({
        'a': np.identity(3),
        'b': np.array([[1, 2], [3, 4]]),
        'x': {
            'c': np.zeros(4),
            'd': np.array([[5, 6, 7], [8, 9, 10]])
        },
    })

    print("Size tree:", t.nbytes)
    print("Total bytes of arrays in t:",
          reduce_(t.nbytes, lambda **kwargs: sum(kwargs.values())))

The result should be like below.

1
2
3
4
5
6
7
8
Size tree: <FastTreeValue 0x7f73fc8d81c0 keys: ['a', 'b', 'x']>
├── 'a' --> 72
├── 'b' --> 32
└── 'x' --> <FastTreeValue 0x7f73fc8d3bb0 keys: ['c', 'd']>
    ├── 'c' --> 32
    └── 'd' --> 48

Total bytes of arrays in t: 184

For further definition or source code implement of function reduce_, take a look at reduce_.

Structural Utilities

In order to process some structured data, especially when you need to process a sort of TreeValue objects which is in the primitive collections or mappings, the structural utilities are designed, and they will be introduced in this section with examples.

Union

The union function is similar to the primitive zip function some ways, it can combine a list of TreeValue objects as one TreeValue object which leaf values are tuples, like the simple example below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
from treevalue import FastTreeValue, union

if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}})
    t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}})
    t3 = FastTreeValue({'a': -13, 'b': -7, 'x': {'c': -5, 'd': -3, 'e': -2}})
    t4 = FastTreeValue({'a': -13, 'b': -7, 'x': 8})

    print("union(t1, t2):")
    print(union(t1, t2))

    print("union(t1, t2, t3):")
    print(union(t1, t2, t3))

    print("union(t1, t2, t3, t4):")
    print(union(t1, t2, t3, t4))

The result should be like below, the leaf values are unionised as tuples.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
union(t1, t2):
<FastTreeValue 0x7fe78d9fdca0 keys: ['a', 'b', 'x']>
├── 'a' --> (1, 11)
├── 'b' --> (2, 22)
└── 'x' --> <FastTreeValue 0x7fe78d9fddf0 keys: ['c', 'd', 'e']>
    ├── 'c' --> (3, 30)
    ├── 'd' --> (4, 48)
    └── 'e' --> (5, 54)

union(t1, t2, t3):
<FastTreeValue 0x7fe78da07670 keys: ['a', 'b', 'x']>
├── 'a' --> (1, 11, -13)
├── 'b' --> (2, 22, -7)
└── 'x' --> <FastTreeValue 0x7fe78da07820 keys: ['c', 'd', 'e']>
    ├── 'c' --> (3, 30, -5)
    ├── 'd' --> (4, 48, -3)
    └── 'e' --> (5, 54, -2)

union(t1, t2, t3, t4):
<FastTreeValue 0x7fe78da07fa0 keys: ['a', 'b', 'x']>
├── 'a' --> (1, 11, -13, -13)
├── 'b' --> (2, 22, -7, -7)
└── 'x' --> <FastTreeValue 0x7fe78da0a1f0 keys: ['c', 'd', 'e']>
    ├── 'c' --> (3, 30, -5, 8)
    ├── 'd' --> (4, 48, -3, 8)
    └── 'e' --> (5, 54, -2, 8)

Note

In union function (actually subside function has the same property), all the trees to be unionised need to have the same structure. You can just consider it as strict mode with inheriting is enabled.

For further information and arguments of function union, take a look at union.

Subside

The subside function can transform the collections or mapping nested TreeValue objects into one TreeValue object which leaf values has the dispatch’s structure. The union function mentioned above is also based on the subside function. Function subside will greatly simplify the code when the structured data need to be calculated. Just like this following example code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
from treevalue import TreeValue, subside

if __name__ == '__main__':
    t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})
    t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}})
    t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}})

    st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]}
    print("subside(st):")
    print(subside(st))

The result should be like below, the leaf values are combined as the dispatch structure of st.

1
2
3
4
5
6
7
subside(st):
<TreeValue 0x7fcdb3c265e0 keys: ['a', 'b', 'x']>
├── 'a' --> {'first': (1, -14), 'second': [6, {'x': 0, 'y': 3}]}
├── 'b' --> {'first': (2, 9), 'second': [0, {'x': -17, 'y': 9}]}
└── 'x' --> <TreeValue 0x7fcdb3c26820 keys: ['c', 'd']>
    ├── 'c' --> {'first': (3, 3), 'second': [-5, {'x': -8, 'y': 11}]}
    └── 'd' --> {'first': (4, 8), 'second': [17, {'x': 15, 'y': -17}]}

Note

In function subside, only primitive list, tuple and dict (or subclasses) objects will be subsided to leaf values.

For further information and arguments of function subside, take a look at subside.

Rise

Function rise can be seen as the inverse operation of function subside, it will try to extract the greatest common structure of the leaf values, and rise them up to the dispatch above the TreeValue objects. Like the following example code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
from treevalue import TreeValue, subside, rise

if __name__ == '__main__':
    # The same demo as the subside docs
    t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})
    t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}})
    t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}})
    st = {'first': (t1, t2), 'second': (t3, {'x': t4, 'y': t5})}
    tx = subside(st)

    # Rising process
    st2 = rise(tx)
    assert st2 == st
    print('st2:', st2)

    print("st2['first'][0]:")
    print(st2['first'][0])

    print("st2['first'][1]:")
    print(st2['first'][1])

    print("st2['second'][0]:")
    print(st2['second'][0])

    print("st2['second'][1]['x']:")
    print(st2['second'][1]['x'])

    print("st2['second'][1]['y']:")
    print(st2['second'][1]['y'])

The result should be like below, the subsided tree st can be extract back to the original structure.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
st2: {'first': (<TreeValue 0x7fc1d722eca0 keys: ['a', 'b', 'x']>, <TreeValue 0x7fc1d72360d0 keys: ['a', 'b', 'x']>), 'second': (<TreeValue 0x7fc1d72363d0 keys: ['a', 'b', 'x']>, {'x': <TreeValue 0x7fc1d72366d0 keys: ['a', 'b', 'x']>, 'y': <TreeValue 0x7fc1d72369d0 keys: ['a', 'b', 'x']>})}
st2['first'][0]:
<TreeValue 0x7fc1d722eca0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7fc1d722eb20 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

st2['first'][1]:
<TreeValue 0x7fc1d72360d0 keys: ['a', 'b', 'x']>
├── 'a' --> -14
├── 'b' --> 9
└── 'x' --> <TreeValue 0x7fc1d722ef40 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 8

st2['second'][0]:
<TreeValue 0x7fc1d72363d0 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 0
└── 'x' --> <TreeValue 0x7fc1d7236280 keys: ['c', 'd']>
    ├── 'c' --> -5
    └── 'd' --> 17

st2['second'][1]['x']:
<TreeValue 0x7fc1d72366d0 keys: ['a', 'b', 'x']>
├── 'a' --> 0
├── 'b' --> -17
└── 'x' --> <TreeValue 0x7fc1d7236580 keys: ['c', 'd']>
    ├── 'c' --> -8
    └── 'd' --> 15

st2['second'][1]['y']:
<TreeValue 0x7fc1d72369d0 keys: ['a', 'b', 'x']>
├── 'a' --> 3
├── 'b' --> 9
└── 'x' --> <TreeValue 0x7fc1d7236880 keys: ['c', 'd']>
    ├── 'c' --> 11
    └── 'd' --> -17

Note

In function rise, only primitive list, tuple and dict (or subclasses) objects will join the rising operation.

Actually, when template argument is not assigned, the rise function will try to find the greatest common structure and rise them to the dispatch. The following rules will be strictly followed when doing this:

  • If the values in current level have the same type ( must all be list, all be tuple or all be dict), the rising function will carry on the find sub structure, otherwise values in current level will be treated as atomic values.

  • If all of them are dicts, and they have exactly the same key sets with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.

  • If all of them are lists, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.

  • If all of them are tuples, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values. (Actually, this rule is the same as the last one which is about lists.)

Considering this automatic structure finding process, if you only want to extract some of the structure (make sure the extraction will not be too deep or too shallow, and make sure the result will have the same structure as your expectation), you can assign value in template arguments. Like the example code below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
from treevalue import TreeValue, subside, rise

if __name__ == '__main__':
    # The same demo as the subside docs
    t11 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t12 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 15, 'd': 41}})
    t21 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t22 = TreeValue({'a': -31, 'b': 82, 'x': {'c': 47, 'd': 32}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})
    t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}})
    t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}})
    st = {'first': [(t11, t21), (t12, t22)], 'second': (t3, {'x': t4, 'y': t5})}
    tx = subside(st)

    # Rising process, with template
    # only the top-leveled dict will be extracted,
    # neither tuple, list nor low-leveled dict will not be extracted
    # because they are not defined in `template` argument
    st2 = rise(tx, template={'first': [None], 'second': (None, None)})
    print('st2:', st2)

    print("st2['first'][0]:")
    print(st2['first'][0])

    print("st2['first'][1]:")
    print(st2['first'][1])

    print("st2['second'][0]:")
    print(st2['second'][0])

    print("st2['second'][1]:")
    print(st2['second'][1])

The result should be like below, the subsided tree st can be extract back to the structure of dict with only first and second keys.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
st2: {'first': [<TreeValue 0x7f21abf2e490 keys: ['a', 'b', 'x']>, <TreeValue 0x7f21abf2e820 keys: ['a', 'b', 'x']>], 'second': (<TreeValue 0x7f21abf2eb20 keys: ['a', 'b', 'x']>, <TreeValue 0x7f21abf28fa0 keys: ['a', 'b', 'x']>)}
st2['first'][0]:
<TreeValue 0x7f21abf2e490 keys: ['a', 'b', 'x']>
├── 'a' --> (1, -14)
├── 'b' --> (2, 9)
└── 'x' --> <TreeValue 0x7f21abf2e340 keys: ['c', 'd']>
    ├── 'c' --> (3, 3)
    └── 'd' --> (4, 8)

st2['first'][1]:
<TreeValue 0x7f21abf2e820 keys: ['a', 'b', 'x']>
├── 'a' --> (3, -31)
├── 'b' --> (9, 82)
└── 'x' --> <TreeValue 0x7f21abf2e6d0 keys: ['c', 'd']>
    ├── 'c' --> (15, 47)
    └── 'd' --> (41, 32)

st2['second'][0]:
<TreeValue 0x7f21abf2eb20 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 0
└── 'x' --> <TreeValue 0x7f21abf2e9d0 keys: ['c', 'd']>
    ├── 'c' --> -5
    └── 'd' --> 17

st2['second'][1]:
<TreeValue 0x7f21abf28fa0 keys: ['a', 'b', 'x']>
├── 'a' --> {'x': 0, 'y': 3}
├── 'b' --> {'x': -17, 'y': 9}
└── 'x' --> <TreeValue 0x7f21abf28850 keys: ['c', 'd']>
    ├── 'c' --> {'x': -8, 'y': 11}
    └── 'd' --> {'x': 15, 'y': -17}

For further information and arguments of function rise, take a look at rise.

Tree Utilities

In this section, utilities with TreeValue class and its objects themselves will be introduced with examples.

Jsonify

With the usage of jsonify function, you can transform a TreeValue object to json-formatted data.

For example, the following real code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import json

from treevalue import TreeValue, raw, jsonify

if __name__ == '__main__':
    t = TreeValue({'a': 1, 'b': [2, 3], 'x': {'c': raw({'x': 1, 'y': 2}), 'd': "this is a string"}})

    print("Tree t:")
    print(t)

    print("Json data of t:")
    print(json.dumps(jsonify(t), indent=4, sort_keys=True))

The result should be

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Tree t:
<TreeValue 0x7f61eb4ad0a0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> [2, 3]
└── 'x' --> <TreeValue 0x7f61ec340160 keys: ['c', 'd']>
    ├── 'c' --> {'x': 1, 'y': 2}
    └── 'd' --> 'this is a string'

Json data of t:
{
    "a": 1,
    "b": [
        2,
        3
    ],
    "x": {
        "c": {
            "x": 1,
            "y": 2
        },
        "d": "this is a string"
    }
}

Note

The function raw in the example code above is a wrapper for dictionary object. It can be used to pass dict object as simple value in TreeValue instead of being treated as sub tree.

For further information of function raw, take a look at raw.

For further informaon of function jsonify, take a look at jsonify.

View

Another magic function named view is also provided, to process logic viewing cases.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
from treevalue import TreeValue, view

if __name__ == '__main__':
    t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'y': {'e': 5, 'f': 6}}})

    print("Tree t:")
    print(t)

    t_x_y = t.x.y
    vt_x_y = view(t, ['x', 'y'])

    print("t.x.y:")
    print(t_x_y)
    print("View of t, ['x', 'y']:")
    print(vt_x_y)

    t.x = TreeValue({'cc': 33, 'dd': 44, 'y': {'ee': 55, 'ff': 66}})

    print("t_x_y after replacement:")
    print(t_x_y)
    print("View of t, ['x', 'y'] after replacement:")
    print(vt_x_y)

The result will be like below, t_x_y and vt_x_y’s value is different after the replacement of the sub tree named t.x, for vt_x_y’s value’s equality to the current t.x.y.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Tree t:
<TreeValue 0x7faab1f9bd00 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7faab2ec7370 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> <TreeValue 0x7faab2b80940 keys: ['e', 'f']>
        ├── 'e' --> 5
        └── 'f' --> 6

t.x.y:
<TreeValue 0x7faab2b80940 keys: ['e', 'f']>
├── 'e' --> 5
└── 'f' --> 6

View of t, ['x', 'y']:
<TreeValue 0x7faab2b80940 keys: ['e', 'f']>
├── 'e' --> 5
└── 'f' --> 6

t_x_y after replacement:
<TreeValue 0x7faab2b80940 keys: ['e', 'f']>
├── 'e' --> 5
└── 'f' --> 6

View of t, ['x', 'y'] after replacement:
<TreeValue 0x7faab176a910 keys: ['ee', 'ff']>
├── 'ee' --> 55
└── 'ff' --> 66

Note

Attention that the view operation is different from sub node getting operation. In viewed tree, it is based on a logic link based on the tree be viewed, and the actual operations are performed in the actual tree node. The main difference between view and getting sub node is that the view tree will be affected by the replacement of sub nodes in viewed tree.

Like the code example above, in the view tree vt_x_y, before do tree operations, the viewed tree will be approached from root tree t by keys of x and y, so after the replacement of the whole subtree t.x, vt_x_y is still the latest value of t.x.y, while t_x_y is still the old sub tree of tree t, before replacement.

For further informaon of function view, take a look at view.

Clone

The TreeValue objects can be cloned deeply by clone function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
from treevalue import TreeValue, clone, raw

if __name__ == '__main__':
    t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'y': raw({'e': 5, 'f': 6})}})

    print("Tree t:")
    print(t)
    print("Id of t.x.y: %x" % id(t.x.y))
    print()
    print()

    print("clone(t):")
    print(clone(t))
    print("Id of clone(t).x.y: %x" % id(clone(t).x.y))
    print()
    print()

    print('clone(t, copy_value=True):')
    print(clone(t, copy_value=True))
    print("Id of clone(t, copy_value=True).x.y: %x" % id(clone(t, copy_value=True)))
    print()
    print()

The result will be like below, all the memory address of the tree nodes in cloned tree are different from those in the original tree t when copy_value argument is not set. But when copy_value is assigned as True, the address of values will be changed because of the deep copy of value.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Tree t:
<TreeValue 0x7ff1f720a100 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7ff1f7d2fe50 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> {'e': 5, 'f': 6}

Id of t.x.y: 7ff1f811af40


clone(t):
<TreeValue 0x7ff1f69a6e20 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7ff1f71f3d30 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> {'e': 5, 'f': 6}

Id of clone(t).x.y: 7ff1f811af40


clone(t, copy_value=True):
<TreeValue 0x7ff1f69ab5e0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7ff1f69ab1c0 keys: ['c', 'd', 'y']>
    ├── 'c' --> 3
    ├── 'd' --> 4
    └── 'y' --> {'e': 5, 'f': 6}

Id of clone(t, copy_value=True).x.y: 7ff1f6a2b400

Note

Attention that in function clone, the values will not be deeply copied together with the tree nodes by default. The newly cloned tree’s values have the same memory address with those in original tree.

If deep copy of values is required when using clone, copy_value argument need to be assigned as True. And then a copy.deepcopy will be performed in clone function in order to do the deep copy. Also, you can define your own copy function in copy_value argument by just assign it as a lambda expression like lambda x: pickle.loads(pickle.dumps(x)).

For further informaon of function clone, take a look at clone.

Typetrans

You can use function typetrans to change the type of tree value object, just like the example below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from treevalue import TreeValue, method_treelize, FastTreeValue, typetrans


class MyTreeValue(TreeValue):
    @method_treelize()
    def pw(self):
        return (self + 1) * (self + 2)


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    print('t1:')
    print(t1)
    print('t1 ** 2:')
    print(t1 ** 2)
    print()

    # Transform t1 to MyTreeValue,
    # __pow__ operator will be disabled and pw method will be enabled.
    t2 = typetrans(t1, MyTreeValue)
    print('t2:')
    print(t2)
    print('t2.pw():')
    print(t2.pw())
    print()

The result will be like below. After the type transformation, ** operator can not be used in t2 but method pw which is implemented in MyTreeValue will be enabled.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
t1:
<FastTreeValue 0x7fb08b85bdf0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <FastTreeValue 0x7fb08b0bde80 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

t1 ** 2:
<FastTreeValue 0x7fb08b051f70 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 4
└── 'x' --> <FastTreeValue 0x7fb08b04d7f0 keys: ['c', 'd']>
    ├── 'c' --> 9
    └── 'd' --> 16


t2:
<MyTreeValue 0x7fb08b85bdf0 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <MyTreeValue 0x7fb08b0bde80 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

t2.pw():
<MyTreeValue 0x7fb08b05bee0 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 12
└── 'x' --> <MyTreeValue 0x7fb08b0670a0 keys: ['c', 'd']>
    ├── 'c' --> 20
    └── 'd' --> 30

Note

In treevalue library, there can be more classes based on TreeValue class by your definition.

Different tree value class will have the different operators, methods and class methods for service, and the __eq__ operator’s result between different type of tree values will always be False because of the difference of their types.

For further information of TreeValue class and user’s definition practice, just take a look at:

For further informaon of function typetrans, take a look at typetrans.

IO Utilities

Simple Serialize and Deserialize

In this version, pickle.dumps and pickle.loads operations are supported, just like the code below.

import os
import pickle

from treevalue import TreeValue

if __name__ == '__main__':
    t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': [3, 4]}})
    binary = pickle.dumps(t)
    print('t:', t, sep=os.linesep)

    tx = pickle.loads(binary)
    assert tx == t
    print('tx:', tx, sep=os.linesep)

The output is

t:
<TreeValue 0x7f437992e370 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f43795c0940 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> [3, 4]

tx:
<TreeValue 0x7f43781e9310 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f43781e9eb0 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> [3, 4]

Advanced Dump and Load

In this project, we implemented our load and dump function which can be used like this

import os
import tempfile

from treevalue import TreeValue, dump, load

if __name__ == '__main__':
    t = TreeValue({'a': 'ab', 'b': 'Cd', 'x': {'c': 'eF', 'd': 'GH'}})
    print('t:', t, sep=os.linesep)

    with tempfile.NamedTemporaryFile() as tf:
        with open(tf.name, 'wb') as wf:  # dump t to file
            dump(t, file=wf)

        with open(tf.name, 'rb') as rf:  # load dt from file
            dt = load(file=rf)

    assert dt == t
    print('dt:', dt, sep=os.linesep)

The output should be

t:
<TreeValue 0x7fb0e89ae370 keys: ['a', 'b', 'x']>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <TreeValue 0x7fb0e889c940 keys: ['c', 'd']>
    ├── 'c' --> 'eF'
    └── 'd' --> 'GH'

dt:
<TreeValue 0x7fb0e725e550 keys: ['a', 'b', 'x']>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <TreeValue 0x7fb0e725e730 keys: ['c', 'd']>
    ├── 'c' --> 'eF'
    └── 'd' --> 'GH'

Note

In this load (loads function is the same), you must assign a TreeValue class by the type_ argument because the dumped data will not save the type of trees. When the type is not assigned, the default class will be simply TreeValue.

Also, dumps function and loads function are provided to deal with binary data.

import os

from treevalue import dumps, FastTreeValue, loads

if __name__ == '__main__':
    t = FastTreeValue({'a': 'ab', 'b': 'Cd', 'x': {'c': 'eF', 'd': 'GH'}})
    print('t:', t, sep=os.linesep)

    binary = dumps(t)
    dt = loads(binary, type_=FastTreeValue)

    assert dt == t
    print('dt:', dt, sep=os.linesep)

The output should be

t:
<FastTreeValue 0x7f2e228d5370 keys: ['a', 'b', 'x']>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <FastTreeValue 0x7f2e22581940 keys: ['c', 'd']>
    ├── 'c' --> 'eF'
    └── 'd' --> 'GH'

dt:
<FastTreeValue 0x7f2e2199bfd0 keys: ['a', 'b', 'x']>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <FastTreeValue 0x7f2e2199bf40 keys: ['c', 'd']>
    ├── 'c' --> 'eF'
    └── 'd' --> 'GH'

Note

The main differences between our dump / load and pickle ‘s dump / load are:

  • Ours are based on dill project, it can serialize and deserialize more types than native pickle, such as functions and classes.

  • Compression and decompression can be defined when dumping, and when the compressed binary data is going to be loaded, decompression function is not necessary, the decompression can be carried on with the function defined in dumping process.

Here is an example:

import gzip
import os

from treevalue import FastTreeValue, dumps, loads

if __name__ == '__main__':
    t = FastTreeValue({'a': 'ab', 'b': 'Cd', 'x': {'c': 'eF', 'd': 'GH'}})
    st = t.upper
    print('st:', st, sep=os.linesep)  # st is a function tree
    print('st():', st(), sep=os.linesep)  # st() if an upper-case-string tree
    print()

    binary = dumps(st, compress=(gzip.compress, gzip.decompress))
    print('Length of binary:', len(binary))

    # compression function is not needed here
    dst = loads(binary, type_=FastTreeValue)
    print('dst:', dst, sep=os.linesep)
    assert st() == dst()  # st() should be equal to dst()
    print('dst():', dst(), sep=os.linesep)

The output should be

st:
<FastTreeValue 0x7f7d840898e0 keys: ['a', 'b', 'x']>
├── 'a' --> <built-in method upper of str object at 0x7f7d8531ce70>
├── 'b' --> <built-in method upper of str object at 0x7f7d857432f0>
└── 'x' --> <FastTreeValue 0x7f7d84081b50 keys: ['c', 'd']>
    ├── 'c' --> <built-in method upper of str object at 0x7f7d85723370>
    └── 'd' --> <built-in method upper of str object at 0x7f7d85723470>

st():
<FastTreeValue 0x7f7d840691f0 keys: ['a', 'b', 'x']>
├── 'a' --> 'AB'
├── 'b' --> 'CD'
└── 'x' --> <FastTreeValue 0x7f7d8408ca90 keys: ['c', 'd']>
    ├── 'c' --> 'EF'
    └── 'd' --> 'GH'


Length of binary: 202
dst:
<FastTreeValue 0x7f7d8408cca0 keys: ['a', 'b', 'x']>
├── 'a' --> <built-in method upper of str object at 0x7f7d8408d570>
├── 'b' --> <built-in method upper of str object at 0x7f7d8408daf0>
└── 'x' --> <FastTreeValue 0x7f7d8408ce20 keys: ['c', 'd']>
    ├── 'c' --> <built-in method upper of str object at 0x7f7d8408d9b0>
    └── 'd' --> <built-in method upper of str object at 0x7f7d8408dbb0>

dst():
<FastTreeValue 0x7f7d840a5100 keys: ['a', 'b', 'x']>
├── 'a' --> 'AB'
├── 'b' --> 'CD'
└── 'x' --> <FastTreeValue 0x7f7d840a5280 keys: ['c', 'd']>
    ├── 'c' --> 'EF'
    └── 'd' --> 'GH'

Object Oriented Usage

In FastTreeValue class, plenty of object-oriented operators, methods and classmethods are supported in order to simplify the actual usage. Here is a simple code example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
from treevalue import FastTreeValue

if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = FastTreeValue({'a': 5, 'b': 6, 'x': {'c': 7, 'd': 8}})

    # operator support
    print('t1 + t2:')
    print(t1 + t2)
    print('t2 ** t1:')
    print(t2 ** t1)
    print()

    # utilities support
    print('t1.map(lambda x: (x + 1) * (x + 2)):')
    print(t1.map(lambda x: (x + 1) * (x + 2)))
    print('t1.reduce(lambda **kwargs: sum(kwargs.values())):',
          t1.reduce(lambda **kwargs: sum(kwargs.values())))
    print()
    print()

    # linking usage
    print('t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0):')
    print(t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0))
    print()

    # structural support
    print("Union result:")
    print(FastTreeValue.union(
        t1.map(lambda x: (x + 1) * (x + 2)),
        t2.map(lambda x: (x - 2) ** (x - 1)),
    ).map(lambda x: 'first: %d, second: %d, which sum is %d' % (x[0], x[1], sum(x))))
    print()

The result should be like below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
t1 + t2:
<FastTreeValue 0x7fcdb5a1de50 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 8
└── 'x' --> <FastTreeValue 0x7fcdb5a13eb0 keys: ['c', 'd']>
    ├── 'c' --> 10
    └── 'd' --> 12

t2 ** t1:
<FastTreeValue 0x7fcdb5a27f10 keys: ['a', 'b', 'x']>
├── 'a' --> 5
├── 'b' --> 36
└── 'x' --> <FastTreeValue 0x7fcdb5a32040 keys: ['c', 'd']>
    ├── 'c' --> 343
    └── 'd' --> 4096


t1.map(lambda x: (x + 1) * (x + 2)):
<FastTreeValue 0x7fcdb5a351f0 keys: ['a', 'b', 'x']>
├── 'a' --> 6
├── 'b' --> 12
└── 'x' --> <FastTreeValue 0x7fcdb5a35340 keys: ['c', 'd']>
    ├── 'c' --> 20
    └── 'd' --> 30

t1.reduce(lambda **kwargs: sum(kwargs.values())): 10


t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0):
<FastTreeValue 0x7fcdb5a35100 keys: ['b', 'x']>
├── 'b' --> 12
└── 'x' --> <FastTreeValue 0x7fcdb5a35310 keys: ['c']>
    └── 'c' --> 20


Union result:
<FastTreeValue 0x7fcdb5a275b0 keys: ['a', 'b', 'x']>
├── 'a' --> 'first: 6, second: 81, which sum is 87'
├── 'b' --> 'first: 12, second: 1024, which sum is 1036'
└── 'x' --> <FastTreeValue 0x7fcdb5a91670 keys: ['c', 'd']>
    ├── 'c' --> 'first: 20, second: 15625, which sum is 15645'
    └── 'd' --> 'first: 30, second: 279936, which sum is 279966'

For further information of FastTreeValue class, take a look at FastTreeValue, all the supported methods and operators are listed here.

DIY TreeValue Class

You can define your own TreeValue class with your own method and class methods. Like the example code below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import os

from treevalue import classmethod_treelize, TreeValue, method_treelize


class MyTreeValue(TreeValue):
    # return type will be automatically detected as `MyTreeValue`
    @method_treelize()
    def append(self, b):
        print("Append arguments:", self, b)
        return self + b

    # return type will be automatically detected as `MyTreeValue`
    @classmethod
    @classmethod_treelize()
    def sum(cls, *args):
        print("Sum arguments:", cls, *args)
        return sum(args)


if __name__ == '__main__':
    t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})

    print('t1.append(t2).append(t3):',
          t1.append(t2).append(t3), sep=os.linesep)

    print('MyTreeValue.sum(t1, t2, t3):',
          MyTreeValue.sum(t1, t2, t3), sep=os.linesep)

The result should be like below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Append arguments: 1 -14
Append arguments: 2 9
Append arguments: 3 3
Append arguments: 4 8
Append arguments: -13 6
Append arguments: 11 0
Append arguments: 6 -5
Append arguments: 12 17
t1.append(t2).append(t3):
<MyTreeValue 0x7fbc2c6b4340 keys: ['a', 'b', 'x']>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7fbc2c6b4400 keys: ['c', 'd']>
    ├── 'c' --> 1
    └── 'd' --> 29

Sum arguments: <class '__main__.MyTreeValue'> 1 -14 6
Sum arguments: <class '__main__.MyTreeValue'> 2 9 0
Sum arguments: <class '__main__.MyTreeValue'> 3 3 -5
Sum arguments: <class '__main__.MyTreeValue'> 4 8 17
MyTreeValue.sum(t1, t2, t3):
<MyTreeValue 0x7fbc2c6b4af0 keys: ['a', 'b', 'x']>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7fbc2c6b4be0 keys: ['c', 'd']>
    ├── 'c' --> 1
    └── 'd' --> 29

Note

Actually, self-calculation method can be defined like the code below

import os

from treevalue import TreeValue, method_treelize


class MyTreeValue(TreeValue):
    # return type will be automatically detected as `MyTreeValue`
    @method_treelize(self_copy=True)
    def append(self, b):
        return self + b


if __name__ == '__main__':
    t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})

    print('t1:', t1, sep=os.linesep)
    _t1_id = id(t1)

    print('t1.append(t2).append(t3):',
          t1.append(t2).append(t3), sep=os.linesep)
    assert id(t1) == _t1_id

The result should be

t1:
<MyTreeValue 0x7fbd4c6daf40 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <MyTreeValue 0x7fbd4bf1bf10 keys: ['c', 'd']>
    ├── 'c' --> 3
    └── 'd' --> 4

t1.append(t2).append(t3):
<MyTreeValue 0x7fbd4c6daf40 keys: ['a', 'b', 'x']>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7fbd4bf1bf10 keys: ['c', 'd']>
    ├── 'c' --> 1
    └── 'd' --> 29

We can see that the calculation result will replace the self object, without changes of the address. Just enable the self_copy option is okay.

For further information of function method_treelize and classmethod_treelize, just take a look at:

More Flexible Ways to DIY Class

When you implement TreeValue class like the section above (take a look at here DIY TreeValue Class), you have to define your own methods and operators one by one, because in class TreeValue, only the minimum necessary methods and operators are provided. But when you use FastTreeValue class, operators like + and methods like map can be used directly. Let’s see how FastTreeValue is defined in source code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
from .general import general_tree_value


class FastTreeValue(general_tree_value()):
    """
    Overview:
        Fast tree value, can do almost anything with this.
    """
    pass

The definition is quite simple, just inherit from the return value of function general_tree_value, and most of the operators and methods can be used.

Like the FastTreeValue class, we can define our TreeValue class like the code below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import os

from treevalue import general_tree_value


class MyTreeValue(general_tree_value()):
    pass


if __name__ == '__main__':
    t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = MyTreeValue({'a': 11, 'b': 24, 'x': {'c': 30, 'd': 47}})

    # __add__ operator can be directly used
    print('t1 + t2:', t1 + t2, sep=os.linesep)

The output should be

1
2
3
4
5
6
7
t1 + t2:
<MyTreeValue 0x7f900ad38a30 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 26
└── 'x' --> <MyTreeValue 0x7f90098e1250 keys: ['c', 'd']>
    ├── 'c' --> 33
    └── 'd' --> 51

In some cases, you can update the behaviours of some operator or methods, like the code below

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import os

from treevalue import general_tree_value, FastTreeValue


class AddZeroTreeValue(general_tree_value(methods=dict(
    __add__=dict(missing=0, mode='outer'),
    __radd__=dict(missing=0, mode='outer'),
    __iadd__=dict(missing=0, mode='outer'),
))):
    pass


class MulOneTreeValue(general_tree_value(methods=dict(
    __mul__=dict(missing=1, mode='outer'),
    __rmul__=dict(missing=1, mode='outer'),
    __imul__=dict(missing=1, mode='outer'),
))):
    pass


if __name__ == '__main__':
    t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3}})
    t2 = FastTreeValue({'a': 11, 'x': {'c': 30, 'd': 47}})

    # __add__ with default value 0
    at1 = t1.type(AddZeroTreeValue)
    at2 = t2.type(AddZeroTreeValue)
    print('at1 + at2:', at1 + at2, sep=os.linesep)

    # __mul__ with default value 1
    mt1 = t1.type(MulOneTreeValue)
    mt2 = t2.type(MulOneTreeValue)
    print('mt1 * mt2:', mt1 * mt2, sep=os.linesep)

The output should be

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
at1 + at2:
<AddZeroTreeValue 0x7fadcd634d00 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 2
└── 'x' --> <AddZeroTreeValue 0x7fadcd634dc0 keys: ['c', 'd']>
    ├── 'c' --> 33
    └── 'd' --> 47

mt1 * mt2:
<MulOneTreeValue 0x7fadcd5bb5e0 keys: ['a', 'b', 'x']>
├── 'a' --> 11
├── 'b' --> 2
└── 'x' --> <MulOneTreeValue 0x7fadcd5bb6a0 keys: ['c', 'd']>
    ├── 'c' --> 90
    └── 'd' --> 47

More than this, you can do the following things in your DIY TreeValue class based on function general_tree_value:

  • change treelize arguments (like the example code above)

  • disable some operators and methods

  • change behaviour of some operators and methods

Here is another complex demo code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
import os

from treevalue import general_tree_value


class DemoTreeValue(general_tree_value(methods=dict(
    # - operator will be disabled
    __sub__=NotImplemented,
    __rsub__=NotImplemented,
    __isub__=NotImplemented,

    # / operator will raise ArithmeticError
    __truediv__=ArithmeticError('True div is not supported'),
    __rtruediv__=ArithmeticError('True div is not supported'),
    __itruediv__=ArithmeticError('True div is not supported'),

    # +t will be changed to t * 2
    __pos__=lambda x: x * 2,
))):
    pass


if __name__ == '__main__':
    t1 = DemoTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = DemoTreeValue({'a': 11, 'b': 24, 'x': {'c': 30, 'd': 47}})

    # __add__ can be used normally
    print('t1 + t2:', t1 + t2, sep=os.linesep)

    # __sub__ will cause TypeError due to the NotImplemented
    try:
        _ = t1 - t2
    except TypeError as err:
        print('t1 - t2:', err, sep=os.linesep)
    else:
        assert False, 'Should not reach here!'

    # __truediv__ will cause ArithmeticError
    try:
        _ = t1 / t2
    except ArithmeticError as err:
        print('t1 / t2:', err, sep=os.linesep)
    else:
        assert False, 'Should not reach here!'

    # __pos__ will be like t1 * 2
    print('+t1:', +t1, sep=os.linesep)

The output should be

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
t1 + t2:
<DemoTreeValue 0x7fe0297a8670 keys: ['a', 'b', 'x']>
├── 'a' --> 12
├── 'b' --> 26
└── 'x' --> <DemoTreeValue 0x7fe0297c67c0 keys: ['c', 'd']>
    ├── 'c' --> 33
    └── 'd' --> 51

t1 - t2:
unsupported operand type(s) for -: 'DemoTreeValue' and 'DemoTreeValue'
t1 / t2:
True div is not supported
+t1:
<DemoTreeValue 0x7fe0297c6d30 keys: ['a', 'b', 'x']>
├── 'a' --> 2
├── 'b' --> 4
└── 'x' --> <DemoTreeValue 0x7fe0297c6dc0 keys: ['c', 'd']>
    ├── 'c' --> 6
    └── 'd' --> 8

For more details about general_tree_value, take a look at general_tree_value and its source code to find out all the implemented operators and methods.

DIY TreeValue Utility Class

You can define a pure utility class in with function utils_class and classmethod_treelize. Like the following example code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
from abc import ABCMeta
from functools import reduce
from operator import __mul__

from treevalue import utils_class, classmethod_treelize, TreeValue


class MyTreeValue(TreeValue):
    pass


# this is the utils class
@utils_class(return_type=MyTreeValue)
class CalcUtils(metaclass=ABCMeta):
    # return type will be detected as `MyTreeValue` as for the utils_class
    @classmethod
    @classmethod_treelize()
    def sum(cls, *args):
        print("Sum arguments:", cls, *args)
        return sum(args)

    # return type will be detected as `MyTreeValue` as for the utils_class
    @classmethod
    @classmethod_treelize()
    def mul(cls, *args):
        print("Mul arguments:", cls, *args)
        return reduce(__mul__, args, 1)


if __name__ == '__main__':
    t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
    t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
    t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})

    print('CalcUtils.sum(t1, t2, t3):')
    print(CalcUtils.sum(t1, t2, t3))

    print('CalcUtils.mul(t1, t2, t3):')
    print(CalcUtils.mul(t1, t2, t3))

The result should be like below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
CalcUtils.sum(t1, t2, t3):
Sum arguments: <class '__main__.CalcUtils'> 1 -14 6
Sum arguments: <class '__main__.CalcUtils'> 2 9 0
Sum arguments: <class '__main__.CalcUtils'> 3 3 -5
Sum arguments: <class '__main__.CalcUtils'> 4 8 17
<MyTreeValue 0x7fcecd242f10 keys: ['a', 'b', 'x']>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7fcecd248040 keys: ['c', 'd']>
    ├── 'c' --> 1
    └── 'd' --> 29

CalcUtils.mul(t1, t2, t3):
Mul arguments: <class '__main__.CalcUtils'> 1 -14 6
Mul arguments: <class '__main__.CalcUtils'> 2 9 0
Mul arguments: <class '__main__.CalcUtils'> 3 3 -5
Mul arguments: <class '__main__.CalcUtils'> 4 8 17
<MyTreeValue 0x7fcecd2486d0 keys: ['a', 'b', 'x']>
├── 'a' --> -84
├── 'b' --> 0
└── 'x' --> <MyTreeValue 0x7fcecd2487c0 keys: ['c', 'd']>
    ├── 'c' --> -45
    └── 'd' --> 544

For further information of function utils_class and classmethod_treelize, just take a look at:

Draw Graph For TreeValue

You can easily draw a graph of the TreeValue objects with function graphics. Like the following example code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import numpy as np

from treevalue import FastTreeValue, graphics, TreeValue


class MyFastTreeValue(FastTreeValue):
    pass


if __name__ == '__main__':
    t = MyFastTreeValue({
        'a': 1,
        'b': np.array([[5, 6], [7, 8]]),
        'x': {
            'c': 3,
            'd': 4,
            'e': np.array([[1, 2], [3, 4]])
        },
    })
    t2 = TreeValue({'ppp': t.x, 'x': {'t': t, 'y': t.x}})

    g = graphics(
        (t, 't'), (t2, 't2'),
        title="This is a demo of 2 trees.",
        cfg={'bgcolor': '#ffffff00'},
    )
    g.render('graphics.dat.gv', format='svg')

The generated code and graph should be like below ( name of the image file should be graphics.dat.gv.svg, name of the graphviz source code file should be graphics.dat.gv ).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// This is a demo of 2 trees.
digraph this_is_a_demo_of_2_trees {
	graph [bgcolor="#ffffff00" label="This is a demo of 2 trees."]
	node_7f7e129e6f10 [label=t color="#3eb24899" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman bold" penwidth=3 shape=diamond style=filled]
	node_7f7e12024250 [label=t2 color="#5b3eb299" fillcolor="#9673ff99" fontcolor="#0d0033ff" fontname="Times-Roman bold" penwidth=3 shape=diamond style=filled]
	value__node_7f7e129e6f10__a [label=1 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e129e6f10 -> value__node_7f7e129e6f10__a [label=a arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	value__node_7f7e129e6f10__b [label="array([[5, 6],
       [7, 8]])" color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e129e6f10 -> value__node_7f7e129e6f10__b [label=b arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	node_7f7e199bf160 [label="t.x" color="#3eb24899" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman bold" penwidth=1.5 shape=ellipse style=filled]
	node_7f7e129e6f10 -> node_7f7e199bf160 [label=x arrowhead=vee arrowsize=1.0 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	node_7f7e12024250 -> node_7f7e199bf160 [label=ppp arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
	node_7f7e12024c70 [label="t2.x" color="#5b3eb299" fillcolor="#9673ff99" fontcolor="#0d0033ff" fontname="Times-Roman bold" penwidth=1.5 shape=ellipse style=filled]
	node_7f7e12024250 -> node_7f7e12024c70 [label=x arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
	value__node_7f7e199bf160__c [label=3 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e199bf160 -> value__node_7f7e199bf160__c [label=c arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	value__node_7f7e199bf160__d [label=4 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e199bf160 -> value__node_7f7e199bf160__d [label=d arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	value__node_7f7e199bf160__e [label="array([[1, 2],
       [3, 4]])" color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled]
	node_7f7e199bf160 -> value__node_7f7e199bf160__e [label=e arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"]
	node_7f7e12024c70 -> node_7f7e129e6f10 [label=t arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
	node_7f7e12024c70 -> node_7f7e199bf160 [label=y arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"]
}
../../_images/graphics.dat.gv1.svg

Note

The return value’s type of function graphics is class graphviz.dot.Digraph, from the opensource library graphviz, for further information of this project and graphviz.dot.Digraph’s usage, take a look at:

For further information of function graphics, just take a look at