Advanced Usage¶
In this part, Some of the advanced usages of TreeValue
will be introduced one by one with sample code and graph to explain them.
For further information or usage of TreeValue
and FastTreeValue
, you may take a look at the following pages:
TreeValue
and its utilities: treevalue.tree.tree.func_treelize
and other wrappers: treevalue.tree.func.FastTreeValue
and its operators, methods: treevalue.tree.general.
Function Modes¶
In the basic usage description in Make function tree supported, we can see that a common function which only support the calculation of the common values can be wrapped to support the tree-based calculation painlessly.
Here is the documentation of function treevalue.tree.func.func_treelize.
-
treevalue.tree.func.
func_treelize
(mode: Union[str, treevalue.tree.func.func.TreeMode] = 'strict', return_type: Optional[Type[_ClassType]] = <class 'treevalue.tree.tree.tree.TreeValue'>, inherit: bool = True, missing: Union[Any, Callable] = <SingletonMark 'missing_not_allow'>, subside: Optional[Union[Mapping, bool]] = None, rise: Optional[Union[Mapping, bool]] = None)[source] - Overview:
Wrap a common function to tree-supported function.
- Arguments:
mode (
Union[str, TreeMode]
): Mode of the wrapping (string or TreeMode both okay), default is strict.return_type (
Optional[Type[_ClassType]]
): Return type of the wrapped function, default is TreeValue.inherit (
bool
): Allow inherit in wrapped function, default is True.missing (
Union[Any, Callable]
): Missing value or lambda generator of when missing, default is MISSING_NOT_ALLOW, which means raise KeyError when missing detected.subside (
Union[Mapping, bool, None]
): Subside enabled to function’s arguments or not, and subside configuration, default is None which means do not use subside. When subside is True, it will use all the default arguments in subside function.rise (
Union[Mapping, bool, None]
): Rise enabled to function’s return value or not, and rise configuration, default is None which means do not use rise. When rise is True, it will use all the default arguments in rise function. (Not recommend to use auto mode when your return structure is not so strict.)
- Returns:
decorator (
Callable
): Wrapper for tree-supported function.
- Example:
>>> @func_treelize() >>> def ssum(a, b): >>> return a + b # the a and b will be integers, not TreeValue >>> >>> t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) >>> t2 = TreeValue({'a': 11, 'b': 22, 'x': {'c': 33, 'd': 5}}) >>> ssum(1, 2) # 3 >>> ssum(t1, t2) # TreeValue({'a': 12, 'b': 24, 'x': {'c': 36, 'd': 9}})
In the arguments listed above, 3 of them are key arguments:
mode, Tree process mode of the calculation. Will be introduced in this section.
inherit, Inheriting mode of the calculation on the tree. Will be introduced in Inheriting on Trees.
missing, Missing processor of the calculation on the tree. Will be introduced in Process Missing Values.
The mode argument is the most important argument in function func_treelize
. For it depends the basic logic of the graph calculation.
The type of mode
argument is TreeMode
, which documentation is like this.
-
enum
treevalue.tree.func.
TreeMode
(value)[source] - Overview:
Four mode of the tree calculation
- Member Type:
int
Valid values are as follows:
-
STRICT
= <TreeMode.STRICT: 1> Strict mode, which means the keys should be one to one in every trees.
-
LEFT
= <TreeMode.LEFT: 2> Left mode, the keys of the result is relied on the left value.
-
INNER
= <TreeMode.INNER: 3> Inner mode, the keys of the result is relied on the intersection of the trees’ key set.
-
OUTER
= <TreeMode.OUTER: 4> Outer mode, the keys of the result is relied on the union of the trees’ key set.
In this part, all of the 4 modes will be introduced with details and samples one by one.
Strict Mode¶
Strict mode is the most frequently-used mode in real cases. It is also the default value of the mode
argument in the treelize functions.
To be simple, strict mode need the keys of a node in the tree be strictly mapped one by one, missing or overage of the keys are both forbidden in strict mode, which means KeyError
will be raised.
For example, in the trees in strict_demo_1
, if we need to added t1
and t2
together, t3
will be the result. Because all the keys (a
, b
, x
of the root nodes, and c
, d
, e
of the x
node) can be mapped one by one.
In another situation, when the keys are not so regular like that in strict_demo_1
, like the trees in strict_demo_2
. If we try to add t1
and t2
in strict mode, KeyError
will be raised due to the missing of key a
in t1
and key f
in t2.x.
Here is a real code example of strict mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='strict') def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t4, t2):', plus(t4, t2)) |
The stdout and stderr should like below, a KeyError
is raised at the second print
statement.
1 2 3 4 5 6 7 | plus(t1, t2): <TreeValue 0x7f59b452d460 keys: ['a', 'b', 'x']> ├── 'a' --> 12 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f59b452d5b0 keys: ['c', 'd', 'e']> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 |
1 2 3 4 5 6 7 8 9 10 | Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/strict_demo.demox.py", line 15, in <module> print('plus(t4, t2):', plus(t4, t2)) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func _result = _recursion(*args, **kwargs) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 138, in _recursion )) for key in sorted(_MODE_PROCESSORS[mode].get_key_set(*args, **kwargs)) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/strict.py", line 12, in get_key_set raise KeyError( KeyError: "Argument keys not match in strict mode, key set of argument 1 is ('b', 'x') but 2 in ('a', 'b', 'x')." |
Note
How does the treelized function work?
Here is another example which show the actual calculation process of the wrapped function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | import numpy as np from treevalue import FastTreeValue, func_treelize @func_treelize(mode='strict') def plus(a, b): print("Current a and b:", type(a), a, type(b), b) return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t2 = FastTreeValue({ 'a': np.array([1, 3, 5, 7]), 'b': np.array([2, 5, 8, 11]), 'x': { 'c': 3, 'd': 4.8, 'e': np.array([[2, 3], [5, 6]]), } }) t3 = FastTreeValue({ 'a': np.array([1, 3, 5, 7]), 'b': [2, 5, 8, 11], 'x': { 'c': 3, 'd': 4.8, 'e': np.array([[2, 3], [5, 6]]), } }) print('plus(t1, t2):', plus(t1, t2)) print() print('plus(t1, t3):', plus(t1, t3)) print() |
In this code, once the original function plus
is called, the actual value of argument a
and b
will be printed to stdout.
In the plus(t1, t2)
, all the calculation can be carried on because add operator between primitive int
and np.ndarray
is supported, so no error occurs and the result of +
between int
and np.ndarray
will be one of the values of the result tree.
But in plus(t1, t3)
, when get the result of plus(t1, t3).b
, it will try to add t1.b
and t2.b
together, but there is no implement of operator +
between primitive int
and list
objects, so TypeError
will be raised when do this calculation.
The complete stdout and stderr should be like this.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7] Current a and b: <class 'int'> 22 <class 'numpy.ndarray'> [ 2 5 8 11] Current a and b: <class 'int'> 30 <class 'int'> 3 Current a and b: <class 'int'> 48 <class 'float'> 4.8 Current a and b: <class 'int'> 54 <class 'numpy.ndarray'> [[2 3] [5 6]] plus(t1, t2): <TreeValue 0x7fbaa108b6a0 keys: ['a', 'b', 'x']> ├── 'a' --> array([12, 14, 16, 18]) ├── 'b' --> array([24, 27, 30, 33]) └── 'x' --> <TreeValue 0x7fbaa108b8b0 keys: ['c', 'd', 'e']> ├── 'c' --> 33 ├── 'd' --> 52.8 └── 'e' --> array([[56, 57], [59, 60]]) Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7] Current a and b: <class 'int'> 22 <class 'list'> [2, 5, 8, 11] |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 35, in <module> print('plus(t1, t3):', plus(t1, t3)) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func _result = _recursion(*args, **kwargs) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion _data = { File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp> key: raw(_recursion( File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 129, in _recursion return func(*args, **kwargs) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 9, in plus return a + b TypeError: unsupported operand type(s) for +: 'int' and 'list' |
In strict mode, missing or overage of keys are not tolerated at all. So in some of the cases, especially when you can not suppose that all the trees involved in calculation has exactly the same structure. Based on this demand, left mode, inner mode and outer mode is designed, and they will be introduced with details and examples in the next 3 sections.
Left Mode¶
In left mode, the result tree’s key set will be aligned to the first tree from the left.
In the trees in left_demo_1
, t3
is the plus result of t1
and t2
. We can see that t2.a is ignored because of t1.a ‘s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of t1
.
But in the left_demo_2
(actually it is exactly the same as strict_demo_2
), KeyError
will be raised because t2.x.f` not found in tree t2`, so the result \
can not be aligned to tree ``t1
, calculation failed.
When you decorate a function as left mode, the decorated function will try to find the first tree from left. If there is no less than 1 positional arguments detected, the first positional argument will be used as the left tree. Otherwise, the tree with the smallest lexicographic key will be assigned as the left tree. Obviously, As least 1 positional or key-word-based argument should be in the arguments.
Here is a real code example of left mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='left') def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t4, t2):', plus(t4, t2)) |
The stdout and stderr should like below, a KeyError
is raised at the second print
statement.
1 2 3 4 5 6 | plus(t1, t2): <TreeValue 0x7f211fd35460 keys: ['b', 'x']> ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f211fd355b0 keys: ['c', 'd', 'e']> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/left_demo.demox.py", line 15, in <module> print('plus(t4, t2):', plus(t4, t2)) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func _result = _recursion(*args, **kwargs) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion _data = { File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp> key: raw(_recursion( File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion _data = { File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp> key: raw(_recursion( File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 136, in <genexpr> *(item(key) for item in pargs), File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 113, in _get_from_key raise KeyError("Missing is off, key {key} not found in {item}.".format( KeyError: "Missing is off, key 'f' not found in <FastTreeValue 0x7f211fd25ee0 keys: ['c', 'd', 'e']>." |
In the code example above, the result is aligned to tree a
in wrapped function plus
. So the first plus calculation will get the proper result, but the second one will fail.
Inner Mode¶
In inner mode, the result tree’s key set will be aligned to the intersection set of all the tree’s key set in arguments.
In the trees in inner_demo_1
, t3
is the plus result of t1
and t2
. We can see that t2.a
and t1.x.f
is ignored because of t1.a
and t2.x.f
’s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of the intersection between tree t1
and t2
.
Note
In most cases of inner mode, the calculation will be processed properly because of the intersection operation on the key set and key missing or overage are both avoided completely.
But the shortage of inner mode is the loss of some information, for keys and its value or subtrees will be directly ignored without any warning. So before using inner mode, please confirm that you know its running logic and the missing of some nodes.
Here is a real code example of inner mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='inner') def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t4, t2):', plus(t4, t2)) |
The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | plus(t1, t2): <TreeValue 0x7f07cb3c7460 keys: ['a', 'b', 'x']> ├── 'a' --> 12 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f07cb3c75b0 keys: ['c', 'd', 'e']> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 plus(t4, t2): <TreeValue 0x7f07cb3c7b50 keys: ['b', 'x']> ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f07cb3c7ca0 keys: ['c', 'd', 'e']> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 |
Outer Mode¶
In outer mode, the result tree’s key set will be aligned to the union set of the tree’s key set in arguments.
In the trees in outer_demo_1
, t3
is the plus result of t1
and t2
, with the missing value of 0
. We can see that all the values in tree t1
and t2
are involved in this calculation, and because of the missing of t1.a
and t2.x.f
they will be actually treated as 0
in calculation process. Finally no error will be raised, and sum of the tree nodes’ value will form the result tree t3
.
Note
In outer mode, it is strongly recommended to set the value of argument missing
. When missing
’s value is not set and there are key missing in some of the argument trees, KeyError
will be raised
due to this missing and value of missing
.
For further information and examples of missing
argument, take a look at
Process Missing Values.
Here is a real code example of inner mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | from treevalue import FastTreeValue, func_treelize # missing value is very important when use outer mode @func_treelize(mode='outer', missing=0) def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) print('plus(t1, t2):', plus(t1, t2)) |
The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.
1 2 3 4 5 6 7 8 | plus(t1, t2): <TreeValue 0x7fcca16681f0 keys: ['a', 'b', 'x']> ├── 'a' --> 11 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7fcca16682b0 keys: ['c', 'd', 'e', 'f']> ├── 'c' --> 33 ├── 'd' --> 52 ├── 'e' --> 59 └── 'f' --> 6 |
Inheriting on Trees¶
Inheriting is another important feature of treelize functions. It will be introduced in this section with necessary examples.
In some cases, we need to calculation between values with different dimensions, such as in numpy
, you can add primitive integer with a ndarray
object, like this
1 2 3 4 5 6 7 | import os import numpy as np if __name__ == '__main__': ar1 = np.array([[1, 2], [3, 4]]) print('ar1 + 9:', ar1 + 9, sep=os.linesep) |
The result will be like below, result of ar1 + 9
is actually the same as the result of expression ar1 + np.array([[9, 9], [9, 9]])
.
1 2 3 | ar1 + 9: [[10 11] [12 13]] |
In treelize functions, you can achieve this processing way by use the inherit
argument. Considering this situation can be find almost anywhere, so the default value of inherit
argument is True
, which means the inheriting option is on by default.
We can see the example in inherit_demo_1
, the tree t1
and tree t2
have different structures, and cannot be processed by any modes in Function Modes, because t1.x
is a primitive integer value but t2.x
is a subtree node, their types are structurally different.
But when inheriting is enabled, this adding operation can still be carried on, for the value of t1.x
(is 9
in inherit_demo_1
) will be applied to the whole subtree t2.x
, like the result tree t3
shows. The result of tree t3.x
can be considered as the sum of primitive integer 9
and subtree t2.x
.
Based on the structural properties above, in the example of inherit_demo_2
, a primitive value can be directly added with a complete tree, like the result tree t2 = t1 + 5
shows.
When inheriting is disabled, all the cases with primitive value and tree node at the same path position will cause TypeError
. For example, when inheriting is disabled, inherit_demo_1
and inherit_demo_2
will failed, but strict_demo_1
will still success because no primitive value appears at the same path position of tree node.
Here is a real code example of inheriting.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='strict') def plus(a, b): return a + b @func_treelize(mode='strict', inherit=False) def plusx(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': 9}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t2, 5):', plus(t2, 5)) print() print('plusx(t1, t2):', plusx(t1, t2)) print() |
The stdout and stderr should be like below, the the calculation of plus
function and the first calculation of plusx
function will be processed properly, but the last one will failed due to the disablement of inheriting.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | plus(t1, t2): <TreeValue 0x7f1887e81280 keys: ['a', 'b', 'x']> ├── 'a' --> 12 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f1887e81430 keys: ['c', 'd', 'e']> ├── 'c' --> 39 ├── 'd' --> 57 └── 'e' --> 63 plus(t2, 5): <TreeValue 0x7f1887e81670 keys: ['a', 'b', 'x']> ├── 'a' --> 16 ├── 'b' --> 27 └── 'x' --> <TreeValue 0x7f1887e81a60 keys: ['c', 'd', 'e']> ├── 'c' --> 35 ├── 'd' --> 53 └── 'e' --> 59 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/docs/source/tutorials/advanced_usage/inherit_demo.demox.py", line 22, in <module> print('plusx(t1, t2):', plusx(t1, t2)) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 148, in _new_func _result = _recursion(*args, **kwargs) File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 134, in _recursion _data = { File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 135, in <dictcomp> key: raw(_recursion( File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 131, in _recursion pargs = [_value_wrap(item, index) for index, item in enumerate(args)] File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 131, in <listcomp> pargs = [_value_wrap(item, index) for index, item in enumerate(args)] File "/tmp/tmp4q2ykbbk/6675a5649d7e854f9c348a59e3535443ae8d8282/treevalue/tree/func/func.py", line 121, in _value_wrap raise TypeError("Inherit is off, tree value expected but {type} found in args {index}.".format( TypeError: Inherit is off, tree value expected but 'int' found in args 0. |
Note
In most cases, disablement of inheriting is not recommended because it will cause some errors. Disable inheriting can increase the structural strictness of calculation, but all the cross-dimensional tree operation will be forbidden as well.
So before disabling inheriting, make sure you know well about what you are going to do.
Process Missing Values¶
In some cases of left mode and outer mode, some of keys in result tree can not find the node nor value at the corresponding path position in some of the argument trees. In order the make these cases be processable, missing value can be used by setting value or lambda expression in missing
argument.
Just like the example outer_demo_1
mentioned forwards, when t1.a
and t2.x.f
not found in trees, the given missing value 0
will be used instead.
Another case is the missing_demo_1
which is based on left mode. In this case, t2.a
is ignore due to the property of left mode, t2.x.f
is missing but t1.x.f
can be found, so t2.x.f
will use the missing value 0
. Finally, t3
will be the result of left addition of t1
and t2
, with missing value of 0
.
Considering another complex case, when the values of tree are primitive lists, like the missing_demo_2
which is based on outer mode. At this time, if we need to do addition with missing value supported, we can set the missing value to an empty list. The result is like below.
Note
In missing_demo_2
, lambda :[]
will be the value of missing
argument in actual code.
In missing
argument, its value will be automatically executed when it is callable, and its execution’s return value will be the actual missing value.
By passing a lambda expression in, you can construct a new object everytime missing value is required, and the duplication of the missing value’s object will be avoided. So it is not recommended to use missing
argument like []
, {}
or myobject()
in python code, but lambda :[]
, lambda :{}
and lambda :myobject()
are better practices.
Here is a real code example of missing_value.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | import os from treevalue import func_treelize, FastTreeValue @func_treelize(mode='outer', missing=lambda: []) def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({ 'b': [2, 3], 'x': { 'c': [5], 'd': [7, 11, 13], 'e': [17, 19], } }) t2 = FastTreeValue({ 'a': [23], 'b': [29, 31], 'x': { 'c': [37], 'd': [41, 43], } }) print('plus(t1, t2):', plus(t1, t2), sep=os.linesep) |
The stdout (no error occurred) should be like below, the calculation of plus
function will be processed properly, like the result in missing_demo_2
.
1 2 3 4 5 6 7 8 | plus(t1, t2): <TreeValue 0x7f0eba0011f0 keys: ['a', 'b', 'x']> ├── 'a' --> [23] ├── 'b' --> [2, 3, 29, 31] └── 'x' --> <TreeValue 0x7f0eba001340 keys: ['c', 'd', 'e']> ├── 'c' --> [5, 37] ├── 'd' --> [7, 11, 13, 41, 43] └── 'e' --> [17, 19] |
Note
Missing value will be only applied in left mode and outer mode.
In strict mode, missing or overage of keys are absolutely not tolerated, missing value will make no sense and a RuntimeWarning
will be logged.
In inner mode, missing value will never be actually in use because all the key sets of final result are intersection set of argument trees. There will never be any key missing cases at all when inner mode is used, a RuntimeWarning
will be logged with the usage of missing value in inner mode.
Functional Utilities¶
In the primitive python, some convenient functional operations are supported, such as
1 2 3 4 5 6 | if __name__ == '__main__': # map function print("Result of map:", list(map(lambda x: x + 1, [2, 3, 5, 7]))) # filter function print("Result of filter:", list(filter(lambda x: x % 4 == 3, [2, 3, 5, 7]))) |
The result should be
1 2 | Result of map: [3, 4, 6, 8] Result of filter: [3, 7] |
Actually, functional operations are supported in tree values as well, and they will be introduced in this section.
Mapping¶
Mapping function is similar to the primitive map
function. The relation between TreeValue
and mapping
function is like that between list
and map
function.
Here is a simple real code example
1 2 3 4 5 6 7 | from treevalue import mapping, FastTreeValue if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) print('mapping(t, lambda x: x ** x + 2):') print(mapping(t, lambda x: x ** x + 2)) |
The mapped tree result should be like below.
1 2 3 4 5 6 7 | mapping(t, lambda x: x ** x + 2): <FastTreeValue 0x7fb884f10d60 keys: ['a', 'b', 'x']> ├── 'a' --> 3 ├── 'b' --> 6 └── 'x' --> <FastTreeValue 0x7fb884f10eb0 keys: ['c', 'd']> ├── 'c' --> 29 └── 'd' --> 258 |
For further definition or source code implement of function mapping
, take a look at mapping.
Filter¶
Filter function is similar to the primitive filter
function. The relation between TreeValue
and filter_
function is like that between list
and filter
function .
Here is a simple real code example
1 2 3 4 5 6 7 8 9 10 | from treevalue import FastTreeValue, filter_ if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) print('filter_(t, lambda x: x % 2 == 1):') print(filter_(t, lambda x: x % 2 == 1)) print('filter_(t, lambda x: x % 2 == 1, remove_empty=False):') print(filter_(t, lambda x: x % 2 == 1, remove_empty=False)) |
The filtered tree result should be like below. When remove_empty
is disabled, the empty tree node will be kept to make sure the structure of the result tree is similar to the unfiltered tree.
1 2 3 4 5 6 7 8 9 10 11 12 | filter_(t, lambda x: x % 2 == 1): <FastTreeValue 0x7f7038d3ae50 keys: ['a', 'x']> ├── 'a' --> 1 └── 'x' --> <FastTreeValue 0x7f7038d3c280 keys: ['c']> └── 'c' --> 3 filter_(t, lambda x: x % 2 == 1, remove_empty=False): <FastTreeValue 0x7f7038ccd220 keys: ['a', 'x', 'y']> ├── 'a' --> 1 ├── 'x' --> <FastTreeValue 0x7f7038ccad30 keys: ['c']> │ └── 'c' --> 3 └── 'y' --> <FastTreeValue 0x7f7038ccafd0 keys: []> |
Note
Actually, the code above is equal to the code below. The filter_
function can be seen as a combination of mask
function and mapping
function.
1 2 3 4 5 6 7 8 9 10 | from treevalue import FastTreeValue, mapping, mask if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) print('mask(t, mapping(t, lambda x: x % 2 == 1)):') print(mask(t, mapping(t, lambda x: x % 2 == 1))) print('mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False):') print(mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False)) |
The result should be like below, exactly the same as the code above.
1 2 3 4 5 6 7 8 9 10 11 12 | mask(t, mapping(t, lambda x: x % 2 == 1)): <FastTreeValue 0x7fbf7d4a5e50 keys: ['a', 'x']> ├── 'a' --> 1 └── 'x' --> <FastTreeValue 0x7fbf7d4a7280 keys: ['c']> └── 'c' --> 3 mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False): <FastTreeValue 0x7fbf7d436220 keys: ['a', 'x', 'y']> ├── 'a' --> 1 ├── 'x' --> <FastTreeValue 0x7fbf7d434d30 keys: ['c']> │ └── 'c' --> 3 └── 'y' --> <FastTreeValue 0x7fbf7d434fd0 keys: []> |
For further information about mask
function, take a look at Mask.
For further definition or source code implement of function filter_
, take a look at filter_.
Mask¶
Mask function allow your choice in one tree, by another tree of true-or-false flags. A simple code example is
1 2 3 4 5 6 7 8 9 10 11 | from treevalue import FastTreeValue, mask if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) m = FastTreeValue({'a': True, 'b': False, 'x': {'c': True, 'd': True}, 'y': {'e': False, 'f': False}}) print('mask(t, m):') print(mask(t, m)) print('mask(t, m, remove_empty=False):') print(mask(t, m, remove_empty=False)) |
The result will be like below, values mapped with the tree m
which mask flag is False
are all deleted. Also, remove_empty
argument is supported.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | mask(t, m): <FastTreeValue 0x7f79479de2b0 keys: ['a', 'x']> ├── 'a' --> 1 └── 'x' --> <FastTreeValue 0x7f79479dad60 keys: ['c', 'd']> ├── 'c' --> 3 └── 'd' --> 4 mask(t, m, remove_empty=False): <FastTreeValue 0x7f79479da0a0 keys: ['a', 'x', 'y']> ├── 'a' --> 1 ├── 'x' --> <FastTreeValue 0x7f79479da700 keys: ['c', 'd']> │ ├── 'c' --> 3 │ └── 'd' --> 4 └── 'y' --> <FastTreeValue 0x7f79479da3a0 keys: []> |
For further definition or source code implement of function mask
, take a look at mask.
Reduce¶
By using reduce_
function, you can get some calculation result based on the tree structure. Its meaning is similar to primitive reduce_
function, but its base structure is TreeValue
instead of sequence or iterator. For example, we can get the sum and multiply accumulation of the values in the tree.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from functools import reduce from operator import __mul__ from treevalue import reduce_, FastTreeValue def multi(items): return reduce(__mul__, items, 1) if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) print("Sum of t:", reduce_(t, lambda **kwargs: sum(kwargs.values()))) print("Multiply of t:", reduce_(t, lambda **kwargs: multi(kwargs.values()))) |
The result should be like below.
1 2 | Sum of t: 24 Multiply of t: 1152 |
If we use reduce_
function with mapping
function, huffman weight sum can also be easily calculated with the code below.
1 2 3 4 5 6 7 8 9 | from treevalue import reduce_, FastTreeValue, mapping if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) weights = mapping(t, lambda v, p: v * len(p)) print("Weight tree:", weights) print("Huffman weight sum of t:", reduce_(weights, lambda **kwargs: sum(kwargs.values()))) |
The result should be like below.
1 2 3 4 5 6 7 8 9 10 11 | Weight tree: <FastTreeValue 0x7fb0fdd1d820 keys: ['a', 'b', 'x', 'y']> ├── 'a' --> 1 ├── 'b' --> 2 ├── 'x' --> <FastTreeValue 0x7fb0fdd1d6d0 keys: ['c', 'd']> │ ├── 'c' --> 6 │ └── 'd' --> 8 └── 'y' --> <FastTreeValue 0x7fb0fdd1d9a0 keys: ['e', 'f']> ├── 'e' --> 12 └── 'f' --> 16 Huffman weight sum of t: 45 |
Besides, we can easily calculate sum of the np.ndarray
objects’ bytes size, like the following code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | import numpy as np from treevalue import reduce_, FastTreeValue if __name__ == '__main__': t = FastTreeValue({ 'a': np.identity(3), 'b': np.array([[1, 2], [3, 4]]), 'x': { 'c': np.zeros(4), 'd': np.array([[5, 6, 7], [8, 9, 10]]) }, }) print("Size tree:", t.nbytes) print("Total bytes of arrays in t:", reduce_(t.nbytes, lambda **kwargs: sum(kwargs.values()))) |
The result should be like below.
1 2 3 4 5 6 7 8 | Size tree: <FastTreeValue 0x7f3fd003d190 keys: ['a', 'b', 'x']> ├── 'a' --> 72 ├── 'b' --> 32 └── 'x' --> <FastTreeValue 0x7f3fd003d220 keys: ['c', 'd']> ├── 'c' --> 32 └── 'd' --> 48 Total bytes of arrays in t: 184 |
For further definition or source code implement of function reduce_
, take a look at reduce_.
Structural Utilities¶
In order to process some structured data, especially when you need to process a sort of TreeValue
objects which is in the primitive collections or mappings, the structural utilities are designed, and they will be introduced in this section with examples.
Union¶
The union
function is similar to the primitive zip
function some ways, it can combine a list of TreeValue
objects as one TreeValue
object which leaf values are tuples, like the simple example below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | from treevalue import FastTreeValue, union if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t3 = FastTreeValue({'a': -13, 'b': -7, 'x': {'c': -5, 'd': -3, 'e': -2}}) t4 = FastTreeValue({'a': -13, 'b': -7, 'x': 8}) print("union(t1, t2):") print(union(t1, t2)) print("union(t1, t2, t3):") print(union(t1, t2, t3)) print("union(t1, t2, t3, t4):") print(union(t1, t2, t3, t4)) |
The result should be like below, the leaf values are unionised as tuples.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | union(t1, t2): <FastTreeValue 0x7fa73a3267f0 keys: ['a', 'b', 'x']> ├── 'a' --> (1, 11) ├── 'b' --> (2, 22) └── 'x' --> <FastTreeValue 0x7fa73a326940 keys: ['c', 'd', 'e']> ├── 'c' --> (3, 30) ├── 'd' --> (4, 48) └── 'e' --> (5, 54) union(t1, t2, t3): <FastTreeValue 0x7fa73a32b340 keys: ['a', 'b', 'x']> ├── 'a' --> (1, 11, -13) ├── 'b' --> (2, 22, -7) └── 'x' --> <FastTreeValue 0x7fa73a32b4f0 keys: ['c', 'd', 'e']> ├── 'c' --> (3, 30, -5) ├── 'd' --> (4, 48, -3) └── 'e' --> (5, 54, -2) union(t1, t2, t3, t4): <FastTreeValue 0x7fa73a326d90 keys: ['a', 'b', 'x']> ├── 'a' --> (1, 11, -13, -13) ├── 'b' --> (2, 22, -7, -7) └── 'x' --> <FastTreeValue 0x7fa73a326ac0 keys: ['c', 'd', 'e']> ├── 'c' --> (3, 30, -5, 8) ├── 'd' --> (4, 48, -3, 8) └── 'e' --> (5, 54, -2, 8) |
Note
In union
function (actually subside
function has the same property), all the trees to be unionised need to have the same structure. You can just consider it as strict mode with inheriting is enabled.
For further information and arguments of function union
, take a look at union.
Subside¶
The subside
function can transform the collections or mapping nested TreeValue
objects into one TreeValue
object which leaf values has the dispatch’s structure. The union
function mentioned above is also based on the subside
function. Function subside
will greatly simplify the code when the structured data need to be calculated. Just like this following example code.
1 2 3 4 5 6 7 8 9 10 11 12 | from treevalue import TreeValue, subside if __name__ == '__main__': t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}}) t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}}) st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]} print("subside(st):") print(subside(st)) |
The result should be like below, the leaf values are combined as the dispatch structure of st
.
1 2 3 4 5 6 7 | subside(st): <TreeValue 0x7fb2ebd68d90 keys: ['a', 'b', 'x']> ├── 'a' --> {'first': (1, -14), 'second': [6, {'x': 0, 'y': 3}]} ├── 'b' --> {'first': (2, 9), 'second': [0, {'x': -17, 'y': 9}]} └── 'x' --> <TreeValue 0x7fb2ebd6a460 keys: ['c', 'd']> ├── 'c' --> {'first': (3, 3), 'second': [-5, {'x': -8, 'y': 11}]} └── 'd' --> {'first': (4, 8), 'second': [17, {'x': 15, 'y': -17}]} |
Note
In function subside
, only primitive list, tuple and dict (or subclasses) objects will be subsided to leaf values.
For further information and arguments of function subside
, take a look at subside.
Rise¶
Function rise
can be seen as the inverse operation of function subside
, it will try to extract the greatest common structure of the leaf values, and rise them up to the dispatch above the TreeValue
objects. Like the following example code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | from treevalue import TreeValue, subside, rise if __name__ == '__main__': # The same demo as the subside docs t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}}) t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}}) st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]} tx = subside(st) # Rising process st2 = rise(tx) assert st2 == st print('st2:', st2) print("st2['first'][0]:") print(st2['first'][0]) print("st2['first'][1]:") print(st2['first'][1]) print("st2['second'][0]:") print(st2['second'][0]) print("st2['second'][1]['x']:") print(st2['second'][1]['x']) print("st2['second'][1]['y']:") print(st2['second'][1]['y']) |
The result should be like below, the subsided tree st
can be extract back to the original structure.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | st2: {'first': (<TreeValue 0x7f3fdd1dceb0 keys: ['a', 'b', 'x']>, <TreeValue 0x7f3fdd1dc940 keys: ['a', 'b', 'x']>), 'second': [<TreeValue 0x7f3fdd1dc400 keys: ['a', 'b', 'x']>, {'x': <TreeValue 0x7f3fdd1d9d30 keys: ['a', 'b', 'x']>, 'y': <TreeValue 0x7f3fde150ac0 keys: ['a', 'b', 'x']>}]} st2['first'][0]: <TreeValue 0x7f3fdd1dceb0 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7f3fdd1dcc70 keys: ['c', 'd']> ├── 'c' --> 3 └── 'd' --> 4 st2['first'][1]: <TreeValue 0x7f3fdd1dc940 keys: ['a', 'b', 'x']> ├── 'a' --> -14 ├── 'b' --> 9 └── 'x' --> <TreeValue 0x7f3fdd1dca00 keys: ['c', 'd']> ├── 'c' --> 3 └── 'd' --> 8 st2['second'][0]: <TreeValue 0x7f3fdd1dc400 keys: ['a', 'b', 'x']> ├── 'a' --> 6 ├── 'b' --> 0 └── 'x' --> <TreeValue 0x7f3fdd1dc8e0 keys: ['c', 'd']> ├── 'c' --> -5 └── 'd' --> 17 st2['second'][1]['x']: <TreeValue 0x7f3fdd1d9d30 keys: ['a', 'b', 'x']> ├── 'a' --> 0 ├── 'b' --> -17 └── 'x' --> <TreeValue 0x7f3fdd1dc190 keys: ['c', 'd']> ├── 'c' --> -8 └── 'd' --> 15 st2['second'][1]['y']: <TreeValue 0x7f3fde150ac0 keys: ['a', 'b', 'x']> ├── 'a' --> 3 ├── 'b' --> 9 └── 'x' --> <TreeValue 0x7f3fdd211820 keys: ['c', 'd']> ├── 'c' --> 11 └── 'd' --> -17 |
Note
In function rise
, only primitive list, tuple and dict (or subclasses) objects will join the rising operation.
Actually, when template
argument is not assigned, the rise
function will try to find the greatest common structure and rise them to the dispatch. The following rules will be strictly followed when doing this:
If the values in current level have the same type ( must all be list, all be tuple or all be dict), the rising function will carry on the find sub structure, otherwise values in current level will be treated as atomic values.
If all of them are dicts, and they have exactly the same key sets with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.
If all of them are lists, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.
If all of them are tuples, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values. (Actually, this rule is the same as the last one which is about lists.)
Considering this automatic structure finding process, if you only want to extract some of the structure (make sure the extraction will not be too deep or too shallow, and make sure the result will have the same structure as your expectation), you can assign value in template
arguments. Like the example code below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | from treevalue import TreeValue, subside, rise if __name__ == '__main__': # The same demo as the subside docs t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}}) t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}}) st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]} tx = subside(st) # Rising process, with template # only the top-leveled dict will be extracted, # neither tuple, list nor low-leveled dict will not be extracted # because they are not defined in `template` argument st2 = rise(tx, template={'first': None, 'second': [None, None]}) print('st2:', st2) print("st2['first']:") print(st2['first']) print("st2['second'][0]:") print(st2['second'][0]) print("st2['second'][1]:") print(st2['second'][1]) |
The result should be like below, the subsided tree st
can be extract back to the structure of dict with only first
and second
keys.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | st2: {'first': <TreeValue 0x7f94b44cbd00 keys: ['a', 'b', 'x']>, 'second': [<TreeValue 0x7f94b44cb700 keys: ['a', 'b', 'x']>, <TreeValue 0x7f94b44cb580 keys: ['a', 'b', 'x']>]} st2['first']: <TreeValue 0x7f94b44cbd00 keys: ['a', 'b', 'x']> ├── 'a' --> (1, -14) ├── 'b' --> (2, 9) └── 'x' --> <TreeValue 0x7f94b44cd910 keys: ['c', 'd']> ├── 'c' --> (3, 3) └── 'd' --> (4, 8) st2['second'][0]: <TreeValue 0x7f94b44cb700 keys: ['a', 'b', 'x']> ├── 'a' --> 6 ├── 'b' --> 0 └── 'x' --> <TreeValue 0x7f94b44cbee0 keys: ['c', 'd']> ├── 'c' --> -5 └── 'd' --> 17 st2['second'][1]: <TreeValue 0x7f94b44cb580 keys: ['a', 'b', 'x']> ├── 'a' --> {'x': 0, 'y': 3} ├── 'b' --> {'x': -17, 'y': 9} └── 'x' --> <TreeValue 0x7f94b44cb790 keys: ['c', 'd']> ├── 'c' --> {'x': -8, 'y': 11} └── 'd' --> {'x': 15, 'y': -17} |
For further information and arguments of function rise
, take a look at rise.
Tree Utilities¶
In this section, utilities with TreeValue
class and its objects themselves will be introduced with examples.
Jsonify¶
With the usage of jsonify
function, you can transform a TreeValue
object to json-formatted data.
For example, the following real code
1 2 3 4 5 6 7 8 9 10 11 12 | import json from treevalue import TreeValue, raw, jsonify if __name__ == '__main__': t = TreeValue({'a': 1, 'b': [2, 3], 'x': {'c': raw({'x': 1, 'y': 2}), 'd': "this is a string"}}) print("Tree t:") print(t) print("Json data of t:") print(json.dumps(jsonify(t), indent=4, sort_keys=True)) |
The result should be
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | Tree t: <TreeValue 0x7f4c3bead100 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> [2, 3] └── 'x' --> <TreeValue 0x7f4c3c9abf10 keys: ['c', 'd']> ├── 'c' --> {'x': 1, 'y': 2} └── 'd' --> 'this is a string' Json data of t: { "a": 1, "b": [ 2, 3 ], "x": { "c": { "x": 1, "y": 2 }, "d": "this is a string" } } |
Note
The function raw
in the example code above is a wrapper for dictionary object. It can be used to pass dict
object as simple value in TreeValue
instead of being treated as sub tree.
For further information of function raw
, take a look at raw.
For further informaon of function jsonify
, take a look at jsonify.
View¶
Another magic function named view
is also provided, to process logic viewing cases.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | from treevalue import TreeValue, view if __name__ == '__main__': t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'y': {'e': 5, 'f': 6}}}) print("Tree t:") print(t) t_x_y = t.x.y vt_x_y = view(t, ['x', 'y']) print("t.x.y:") print(t_x_y) print("View of t, ['x', 'y']:") print(vt_x_y) t.x = TreeValue({'cc': 33, 'dd': 44, 'y': {'ee': 55, 'ff': 66}}) print("t_x_y after replacement:") print(t_x_y) print("View of t, ['x', 'y'] after replacement:") print(vt_x_y) |
The result will be like below, t_x_y
and vt_x_y
’s value is different after the replacement of the sub tree named t.x
, for vt_x_y
’s value’s equality to the current t.x.y
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | Tree t: <TreeValue 0x7f1ada119eb0 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7f1ada119e20 keys: ['c', 'd', 'y']> ├── 'c' --> 3 ├── 'd' --> 4 └── 'y' --> <TreeValue 0x7f1adad7f9d0 keys: ['e', 'f']> ├── 'e' --> 5 └── 'f' --> 6 t.x.y: <TreeValue 0x7f1adad7f9d0 keys: ['e', 'f']> ├── 'e' --> 5 └── 'f' --> 6 View of t, ['x', 'y']: <TreeValue 0x7f1adad7f9d0 keys: ['e', 'f']> ├── 'e' --> 5 └── 'f' --> 6 t_x_y after replacement: <TreeValue 0x7f1adad7f9d0 keys: ['e', 'f']> ├── 'e' --> 5 └── 'f' --> 6 View of t, ['x', 'y'] after replacement: <TreeValue 0x7f1ad9e0df40 keys: ['ee', 'ff']> ├── 'ee' --> 55 └── 'ff' --> 66 |
Note
Attention that the view
operation is different from sub node getting operation. In viewed tree, it is based on a logic link based on the tree be viewed, and the actual operations are performed in the actual tree node. The main difference between view
and getting sub node is that the view tree will be affected by the replacement of sub nodes in viewed tree.
Like the code example above, in the view tree vt_x_y
, before do tree operations, the viewed tree will be approached from root tree t
by keys of x
and y
, so after the replacement of the whole subtree t.x
, vt_x_y
is still the latest value of t.x.y
, while t_x_y
is still the old sub tree of tree t
, before replacement.
For further informaon of function view
, take a look at view.
Clone¶
The TreeValue
objects can be cloned deeply by clone
function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | from treevalue import TreeValue, clone, raw if __name__ == '__main__': t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'y': raw({'e': 5, 'f': 6})}}) print("Tree t:") print(t) print("Id of t.x.y: %x" % id(t.x.y)) print() print() print("clone(t):") print(clone(t)) print("Id of clone(t).x.y: %x" % id(clone(t).x.y)) print() print() print('clone(t, copy_value=True):') print(clone(t, copy_value=True)) print("Id of clone(t, copy_value=True).x.y: %x" % id(clone(t, copy_value=True))) print() print() |
The result will be like below, all the memory address of the tree nodes in cloned tree are different from those in the original tree t
when copy_value
argument is not set. But when copy_value
is assigned as True
, the address of values will be changed because of the deep copy of value.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | Tree t: <TreeValue 0x7f5852999e80 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7f58538d5400 keys: ['c', 'd', 'y']> ├── 'c' --> 3 ├── 'd' --> 4 └── 'y' --> {'e': 5, 'f': 6} Id of t.x.y: 7f58538dbd40 clone(t): <TreeValue 0x7f5852694dc0 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7f5853521e50 keys: ['c', 'd', 'y']> ├── 'c' --> 3 ├── 'd' --> 4 └── 'y' --> {'e': 5, 'f': 6} Id of clone(t).x.y: 7f58538dbd40 clone(t, copy_value=True): <TreeValue 0x7f58526a5370 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7f58526a5400 keys: ['c', 'd', 'y']> ├── 'c' --> 3 ├── 'd' --> 4 └── 'y' --> {'e': 5, 'f': 6} Id of clone(t, copy_value=True).x.y: 7f58526a59a0 |
Note
Attention that in function clone
, the values will not be deeply copied together with the tree nodes by default. The newly cloned tree’s values have the same memory address with those in original tree.
If deep copy of values is required when using clone
, copy_value
argument need to be assigned as True
. And then a copy.deepcopy
will be performed in clone
function in order to do the deep copy. Also, you can define your own copy function in copy_value
argument by just assign it as a lambda expression like lambda x: pickle.loads(pickle.dumps(x))
.
For further informaon of function clone
, take a look at clone.
Typetrans¶
You can use function typetrans
to change the type of tree value object, just like the example below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | from treevalue import TreeValue, method_treelize, FastTreeValue, typetrans class MyTreeValue(TreeValue): @method_treelize() def pw(self): return (self + 1) * (self + 2) if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) print('t1:') print(t1) print('t1 ** 2:') print(t1 ** 2) print() # Transform t1 to MyTreeValue, # __pow__ operator will be disabled and pw method will be enabled. t2 = typetrans(t1, MyTreeValue) print('t2:') print(t2) print('t2.pw():') print(t2.pw()) print() |
The result will be like below. After the type transformation, **
operator can not be used in t2
but method pw
which is implemented in MyTreeValue
will be enabled.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | t1: <FastTreeValue 0x7f2983199f10 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <FastTreeValue 0x7f2983199e50 keys: ['c', 'd']> ├── 'c' --> 3 └── 'd' --> 4 t1 ** 2: <FastTreeValue 0x7f2982ea2d60 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 4 └── 'x' --> <FastTreeValue 0x7f2982ea8280 keys: ['c', 'd']> ├── 'c' --> 9 └── 'd' --> 16 t2: <MyTreeValue 0x7f2983199f10 keys: ['a', 'b', 'x']> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <MyTreeValue 0x7f2983199e50 keys: ['c', 'd']> ├── 'c' --> 3 └── 'd' --> 4 t2.pw(): <MyTreeValue 0x7f2982ea8ac0 keys: ['a', 'b', 'x']> ├── 'a' --> 6 ├── 'b' --> 12 └── 'x' --> <MyTreeValue 0x7f2982ea8c40 keys: ['c', 'd']> ├── 'c' --> 20 └── 'd' --> 30 |
Note
In treevalue library, there can be more classes based on TreeValue
class by your definition.
Different tree value class will have the different operators, methods and class methods for service, and the __eq__
operator’s result between different type of tree values will always be False
because of the difference of their types.
For further information of TreeValue
class and user’s definition practice, just take a look at:
For further informaon of function typetrans
, take a look at typetrans.
Object Oriented Usage¶
In FastTreeValue
class, plenty of object-oriented operators, methods and classmethods are supported in order to simplify the actual usage. Here is a simple code example.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | from treevalue import FastTreeValue if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = FastTreeValue({'a': 5, 'b': 6, 'x': {'c': 7, 'd': 8}}) # operator support print('t1 + t2:') print(t1 + t2) print('t2 ** t1:') print(t2 ** t1) print() # utilities support print('t1.map(lambda x: (x + 1) * (x + 2)):') print(t1.map(lambda x: (x + 1) * (x + 2))) print('t1.reduce(lambda **kwargs: sum(kwargs.values())):', t1.reduce(lambda **kwargs: sum(kwargs.values()))) print() print() # linking usage print('t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0):') print(t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0)) print() # structural support print("Union result:") print(FastTreeValue.union( t1.map(lambda x: (x + 1) * (x + 2)), t2.map(lambda x: (x - 2) ** (x - 1)), ).map(lambda x: 'first: %d, second: %d, which sum is %d' % (x[0], x[1], sum(x)))) print() |
The result should be like below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | t1 + t2: <FastTreeValue 0x7f7b8c801250 keys: ['a', 'b', 'x']> ├── 'a' --> 6 ├── 'b' --> 8 └── 'x' --> <FastTreeValue 0x7f7b8c801310 keys: ['c', 'd']> ├── 'c' --> 10 └── 'd' --> 12 t2 ** t1: <FastTreeValue 0x7f7b8c801a90 keys: ['a', 'b', 'x']> ├── 'a' --> 5 ├── 'b' --> 36 └── 'x' --> <FastTreeValue 0x7f7b8c801b50 keys: ['c', 'd']> ├── 'c' --> 343 └── 'd' --> 4096 t1.map(lambda x: (x + 1) * (x + 2)): <FastTreeValue 0x7f7b8c8018e0 keys: ['a', 'b', 'x']> ├── 'a' --> 6 ├── 'b' --> 12 └── 'x' --> <FastTreeValue 0x7f7b8c801700 keys: ['c', 'd']> ├── 'c' --> 20 └── 'd' --> 30 t1.reduce(lambda **kwargs: sum(kwargs.values())): 10 t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0): <FastTreeValue 0x7f7b8c795310 keys: ['b', 'x']> ├── 'b' --> 12 └── 'x' --> <FastTreeValue 0x7f7b8c7950d0 keys: ['c']> └── 'c' --> 20 Union result: <FastTreeValue 0x7f7b8c795a60 keys: ['a', 'b', 'x']> ├── 'a' --> 'first: 6, second: 81, which sum is 87' ├── 'b' --> 'first: 12, second: 1024, which sum is 1036' └── 'x' --> <FastTreeValue 0x7f7b8c795bb0 keys: ['c', 'd']> ├── 'c' --> 'first: 20, second: 15625, which sum is 15645' └── 'd' --> 'first: 30, second: 279936, which sum is 279966' |
For further information of FastTreeValue
class, take a look at FastTreeValue, all the supported methods and operators are listed here.
DIY TreeValue Class¶
You can define your own TreeValue
class with your own method and class methods. Like the example code below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | from treevalue import classmethod_treelize, TreeValue, method_treelize class MyTreeValue(TreeValue): # return type will be automatically detected as `MyTreeValue` @method_treelize() def append(self, b): print("Append arguments:", self, b) return self + b # return type will be automatically detected as `MyTreeValue` @classmethod @classmethod_treelize() def sum(cls, *args): print("Sum arguments:", cls, *args) return sum(args) if __name__ == '__main__': t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) print('t1.append(t2).append(t3):') print(t1.append(t2).append(t3)) print('MyTreeValue.sum(t1, t2, t3):') print(MyTreeValue.sum(t1, t2, t3)) |
The result should be like below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | t1.append(t2).append(t3): Append arguments: 1 -14 Append arguments: 2 9 Append arguments: 3 3 Append arguments: 4 8 Append arguments: -13 6 Append arguments: 11 0 Append arguments: 6 -5 Append arguments: 12 17 <MyTreeValue 0x7f4fba25ceb0 keys: ['a', 'b', 'x']> ├── 'a' --> -7 ├── 'b' --> 11 └── 'x' --> <MyTreeValue 0x7f4fba25cf70 keys: ['c', 'd']> ├── 'c' --> 1 └── 'd' --> 29 MyTreeValue.sum(t1, t2, t3): Sum arguments: <class '__main__.MyTreeValue'> 1 -14 6 Sum arguments: <class '__main__.MyTreeValue'> 2 9 0 Sum arguments: <class '__main__.MyTreeValue'> 3 3 -5 Sum arguments: <class '__main__.MyTreeValue'> 4 8 17 <MyTreeValue 0x7f4fba25d820 keys: ['a', 'b', 'x']> ├── 'a' --> -7 ├── 'b' --> 11 └── 'x' --> <MyTreeValue 0x7f4fba25d910 keys: ['c', 'd']> ├── 'c' --> 1 └── 'd' --> 29 |
For further information of function method_treelize
and classmethod_treelize
, just take a look at:
DIY TreeValue Utility Class¶
You can define a pure utility class in with function utils_class
and classmethod_treelize
. Like the following example code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | from abc import ABCMeta from functools import reduce from operator import __mul__ from treevalue import utils_class, classmethod_treelize, TreeValue class MyTreeValue(TreeValue): pass # this is the utils class @utils_class(return_type=MyTreeValue) class CalcUtils(metaclass=ABCMeta): # return type will be detected as `MyTreeValue` as for the utils_class @classmethod @classmethod_treelize() def sum(cls, *args): print("Sum arguments:", cls, *args) return sum(args) # return type will be detected as `MyTreeValue` as for the utils_class @classmethod @classmethod_treelize() def mul(cls, *args): print("Mul arguments:", cls, *args) return reduce(__mul__, args, 1) if __name__ == '__main__': t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) print('CalcUtils.sum(t1, t2, t3):') print(CalcUtils.sum(t1, t2, t3)) print('CalcUtils.mul(t1, t2, t3):') print(CalcUtils.mul(t1, t2, t3)) |
The result should be like below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | CalcUtils.sum(t1, t2, t3): Sum arguments: <class '__main__.CalcUtils'> 1 -14 6 Sum arguments: <class '__main__.CalcUtils'> 2 9 0 Sum arguments: <class '__main__.CalcUtils'> 3 3 -5 Sum arguments: <class '__main__.CalcUtils'> 4 8 17 <MyTreeValue 0x7f774ae8ba60 keys: ['a', 'b', 'x']> ├── 'a' --> -7 ├── 'b' --> 11 └── 'x' --> <MyTreeValue 0x7f774ae8bb50 keys: ['c', 'd']> ├── 'c' --> 1 └── 'd' --> 29 CalcUtils.mul(t1, t2, t3): Mul arguments: <class '__main__.CalcUtils'> 1 -14 6 Mul arguments: <class '__main__.CalcUtils'> 2 9 0 Mul arguments: <class '__main__.CalcUtils'> 3 3 -5 Mul arguments: <class '__main__.CalcUtils'> 4 8 17 <MyTreeValue 0x7f774ae903d0 keys: ['a', 'b', 'x']> ├── 'a' --> -84 ├── 'b' --> 0 └── 'x' --> <MyTreeValue 0x7f774ae904c0 keys: ['c', 'd']> ├── 'c' --> -45 └── 'd' --> 544 |
For further information of function utils_class
and classmethod_treelize
, just take a look at: