Advanced Usage¶
In this part, Some of the advanced usages of TreeValue
will be introduced one by one with sample code and graph to explain them.
For further information or usage of TreeValue
and FastTreeValue
, you may take a look at the following pages:
TreeValue
and its utilities: treevalue.tree.tree.func_treelize
and other wrappers: treevalue.tree.func.FastTreeValue
and its operators, methods: treevalue.tree.general.
Function Modes¶
In the basic usage description in Make function tree supported, we can see that a common function which only support the calculation of the common values can be wrapped to support the tree-based calculation painlessly.
Here is the documentation of function treevalue.tree.func.func_treelize.
-
treevalue.tree.func.
func_treelize
(mode: str = 'strict', return_type: Optional[Type[TreeClassType_]] = <class 'treevalue.tree.tree.tree.TreeValue'>, inherit: bool = True, missing: Union[Any, Callable] = <SingletonMark 'missing_not_allow'>, delayed: bool = False, subside: Optional[Union[Mapping, bool]] = None, rise: Optional[Union[Mapping, bool]] = None)[source] - Overview:
Wrap a common function to tree-supported function.
- Arguments:
mode (
str
): Mode of the wrapping, default is strict.return_type (
Optional[Type[TreeClassType_]]
): Return type of the wrapped function, default is TreeValue.inherit (
bool
): Allow inheriting in wrapped function, default is True.missing (
Union[Any, Callable]
): Missing value or lambda generator of when missing, default is MISSING_NOT_ALLOW, which means raise KeyError when missing detected.delayed (
bool
): Enable delayed mode or not, the calculation will be delayed when enabled, default isFalse
, which means to all the calculation at once.subside (
Union[Mapping, bool, None]
): Subside enabled to function’s arguments or not, and subside configuration, default is None which means do not use subside. When subside is True, it will use all the default arguments in subside function.rise (
Union[Mapping, bool, None]
): Rise enabled to function’s return value or not, and rise configuration, default is None which means do not use rise. When rise is True, it will use all the default arguments in rise function. (Not recommend to use auto mode when your return structure is not so strict.)
- Returns:
decorator (
Callable
): Wrapper for tree-supported function.
- Example:
>>> @func_treelize() >>> def ssum(a, b): >>> return a + b # the a and b will be integers, not TreeValue >>> >>> t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) >>> t2 = TreeValue({'a': 11, 'b': 22, 'x': {'c': 33, 'd': 5}}) >>> ssum(1, 2) # 3 >>> ssum(t1, t2) # TreeValue({'a': 12, 'b': 24, 'x': {'c': 36, 'd': 9}})
In the arguments listed above, 3 of them are key arguments:
mode, Tree process mode of the calculation. Will be introduced in this section.
inherit, Inheriting mode of the calculation on the tree. Will be introduced in Inheriting on Trees.
missing, Missing processor of the calculation on the tree. Will be introduced in Process Missing Values.
The mode argument is the most important argument in function func_treelize
. For it depends the basic logic of the graph calculation.
There are 4 kinds of the modes, listed below:
strict
mode, which is most usually used, means all the values on the tree should be mapped one by one.inner
mode, means only the keys which is commonly owned by all the trees will be processed.outer
mode, means all the keys involved in any of the trees will be processed.left
mode, means only the keys involved in the leftest tree will be processed.
In this part, all of the 4 modes will be introduced with details and samples one by one.
Strict Mode¶
Strict mode is the most frequently-used mode in real cases. It is also the default value of the mode
argument in the treelize functions.
To be simple, strict mode need the keys of a node in the tree be strictly mapped one by one, missing or overage of the keys are both forbidden in strict mode, which means KeyError
will be raised.
For example, in the trees in strict_demo_1
, if we need to added t1
and t2
together, t3
will be the result. Because all the keys (a
, b
, x
of the root nodes, and c
, d
, e
of the x
node) can be mapped one by one.
In another situation, when the keys are not so regular like that in strict_demo_1
, like the trees in strict_demo_2
. If we try to add t1
and t2
in strict mode, KeyError
will be raised due to the missing of key a
in t1
and key f
in t2.x.
Here is a real code example of strict mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='strict') def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t4, t2):', plus(t4, t2)) |
The stdout and stderr should like below, a KeyError
is raised at the second print
statement.
1 2 3 4 5 6 7 | plus(t1, t2): <TreeValue 0x7f0103f2feb0> ├── 'a' --> 12 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f0103f2ff10> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 |
1 2 3 4 5 6 7 8 9 10 | Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/docs/source/tutorials/advanced_usage/strict_demo.demox.py", line 15, in <module> print('plus(t4, t2):', plus(t4, t2)) File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/treevalue/tree/func/func.py", line 56, in _new_func return _treelized(*args, **kwargs) File "treevalue/tree/func/cfunc.pyx", line 176, in treevalue.tree.func.cfunc._w_func_treelize_run File "treevalue/tree/func/cfunc.pyx", line 84, in treevalue.tree.func.cfunc._c_func_treelize_run File "treevalue/tree/func/modes.pyx", line 132, in treevalue.tree.func.modes._c_keyset File "treevalue/tree/func/modes.pyx", line 54, in treevalue.tree.func.modes._c_strict_keyset KeyError: "Argument keys not match in strict mode, key set of argument 0 is {'b', 'x'} but 1 in {'b', 'x', 'a'}." |
Note
How does the treelized function work?
Here is another example which show the actual calculation process of the wrapped function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | import numpy as np from treevalue import FastTreeValue, func_treelize @func_treelize(mode='strict') def plus(a, b): print("Current a and b:", type(a), a, type(b), b) return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t2 = FastTreeValue({ 'a': np.array([1, 3, 5, 7]), 'b': np.array([2, 5, 8, 11]), 'x': { 'c': 3, 'd': 4.8, 'e': np.array([[2, 3], [5, 6]]), } }) t3 = FastTreeValue({ 'a': np.array([1, 3, 5, 7]), 'b': [2, 5, 8, 11], 'x': { 'c': 3, 'd': 4.8, 'e': np.array([[2, 3], [5, 6]]), } }) print('plus(t1, t2):', plus(t1, t2)) print() print('plus(t1, t3):', plus(t1, t3)) print() |
In this code, once the original function plus
is called, the actual value of argument a
and b
will be printed to stdout.
In the plus(t1, t2)
, all the calculation can be carried on because add operator between primitive int
and np.ndarray
is supported, so no error occurs and the result of +
between int
and np.ndarray
will be one of the values of the result tree.
But in plus(t1, t3)
, when get the result of plus(t1, t3).b
, it will try to add t1.b
and t2.b
together, but there is no implement of operator +
between primitive int
and list
objects, so TypeError
will be raised when do this calculation.
The complete stdout and stderr should be like this.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7] Current a and b: <class 'int'> 54 <class 'numpy.ndarray'> [[2 3] [5 6]] Current a and b: <class 'int'> 30 <class 'int'> 3 Current a and b: <class 'int'> 48 <class 'float'> 4.8 Current a and b: <class 'int'> 22 <class 'numpy.ndarray'> [ 2 5 8 11] plus(t1, t2): <TreeValue 0x7f47d3164e50> ├── 'a' --> array([12, 14, 16, 18]) ├── 'b' --> array([24, 27, 30, 33]) └── 'x' --> <TreeValue 0x7f47dfcefeb0> ├── 'c' --> 33 ├── 'd' --> 52.8 └── 'e' --> array([[56, 57], [59, 60]]) Current a and b: <class 'int'> 11 <class 'numpy.ndarray'> [1 3 5 7] Current a and b: <class 'int'> 54 <class 'numpy.ndarray'> [[2 3] [5 6]] Current a and b: <class 'int'> 30 <class 'int'> 3 Current a and b: <class 'int'> 48 <class 'float'> 4.8 Current a and b: <class 'int'> 22 <class 'list'> [2, 5, 8, 11] |
1 2 3 4 5 6 7 8 9 10 11 | Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 35, in <module> print('plus(t1, t3):', plus(t1, t3)) File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/treevalue/tree/func/func.py", line 56, in _new_func return _treelized(*args, **kwargs) File "treevalue/tree/func/cfunc.pyx", line 176, in treevalue.tree.func.cfunc._w_func_treelize_run File "treevalue/tree/func/cfunc.pyx", line 151, in treevalue.tree.func.cfunc._c_func_treelize_run File "treevalue/tree/func/cfunc.pyx", line 75, in treevalue.tree.func.cfunc._c_func_treelize_run File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/docs/source/tutorials/advanced_usage/strict_demo_show.demox.py", line 9, in plus return a + b TypeError: unsupported operand type(s) for +: 'int' and 'list' |
In strict mode, missing or overage of keys are not tolerated at all. So in some of the cases, especially when you can not suppose that all the trees involved in calculation has exactly the same structure. Based on this demand, left mode, inner mode and outer mode is designed, and they will be introduced with details and examples in the next 3 sections.
Left Mode¶
In left mode, the result tree’s key set will be aligned to the first tree from the left.
In the trees in left_demo_1
, t3
is the plus result of t1
and t2
. We can see that t2.a is ignored because of t1.a ‘s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of t1
.
But in the left_demo_2
(actually it is exactly the same as strict_demo_2
), KeyError
will be raised because t2.x.f` not found in tree t2`, so the result \
can not be aligned to tree ``t1
, calculation failed.
When you decorate a function as left mode, the decorated function will try to find the first tree from left. If there is no less than 1 positional arguments detected, the first positional argument will be used as the left tree. Otherwise, the tree with the smallest lexicographic key will be assigned as the left tree. Obviously, As least 1 positional or key-word-based argument should be in the arguments.
Here is a real code example of left mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='left') def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t4, t2):', plus(t4, t2)) |
The stdout and stderr should like below, a KeyError
is raised at the second print
statement.
1 2 3 4 5 6 | plus(t1, t2): <TreeValue 0x7f886faefeb0> ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f886faeff10> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | Traceback (most recent call last): File "treevalue/tree/func/cfunc.pyx", line 89, in treevalue.tree.func.cfunc._c_func_treelize_run KeyError: 'f' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/docs/source/tutorials/advanced_usage/left_demo.demox.py", line 15, in <module> print('plus(t4, t2):', plus(t4, t2)) File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/treevalue/tree/func/func.py", line 56, in _new_func return _treelized(*args, **kwargs) File "treevalue/tree/func/cfunc.pyx", line 176, in treevalue.tree.func.cfunc._w_func_treelize_run File "treevalue/tree/func/cfunc.pyx", line 151, in treevalue.tree.func.cfunc._c_func_treelize_run File "treevalue/tree/func/cfunc.pyx", line 102, in treevalue.tree.func.cfunc._c_func_treelize_run KeyError: "Missing is off, key 'f' not found in {'c': 30, 'd': 48, 'e': 54}." |
In the code example above, the result is aligned to tree a
in wrapped function plus
. So the first plus calculation will get the proper result, but the second one will fail.
Inner Mode¶
In inner mode, the result tree’s key set will be aligned to the intersection set of all the tree’s key set in arguments.
In the trees in inner_demo_1
, t3
is the plus result of t1
and t2
. We can see that t2.a
and t1.x.f
is ignored because of t1.a
and t2.x.f
’s non-existence. Finally no error will be raised, the calculation will be processed properly with the tree structure of the intersection between tree t1
and t2
.
Note
In most cases of inner mode, the calculation will be processed properly because of the intersection operation on the key set and key missing or overage are both avoided completely.
But the shortage of inner mode is the loss of some information, for keys and its value or subtrees will be directly ignored without any warning. So before using inner mode, please confirm that you know its running logic and the missing of some nodes.
Here is a real code example of inner mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='inner') def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t4 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t4, t2):', plus(t4, t2)) |
The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | plus(t1, t2): <TreeValue 0x7f361ab6eeb0> ├── 'a' --> 12 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f361ab6ef10> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 plus(t4, t2): <TreeValue 0x7f361ab6ef10> ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7f361ab6eeb0> ├── 'c' --> 33 ├── 'd' --> 52 └── 'e' --> 59 |
Outer Mode¶
In outer mode, the result tree’s key set will be aligned to the union set of the tree’s key set in arguments.
In the trees in outer_demo_1
, t3
is the plus result of t1
and t2
, with the missing value of 0
. We can see that all the values in tree t1
and t2
are involved in this calculation, and because of the missing of t1.a
and t2.x.f
they will be actually treated as 0
in calculation process. Finally no error will be raised, and sum of the tree nodes’ value will form the result tree t3
.
Note
In outer mode, it is strongly recommended to set the value of argument missing
. When missing
’s value is not set and there are key missing in some of the argument trees, KeyError
will be raised
due to this missing and value of missing
.
For further information and examples of missing
argument, take a look at
Process Missing Values.
Here is a real code example of inner mode.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | from treevalue import FastTreeValue, func_treelize # missing value is very important when use outer mode @func_treelize(mode='outer', missing=0) def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5, 'f': 6}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) print('plus(t1, t2):', plus(t1, t2)) |
The stdout (no stderr content in this case) should like below, no errors occurred with exit code 0.
1 2 3 4 5 6 7 8 | plus(t1, t2): <TreeValue 0x7fca4bb6eeb0> ├── 'a' --> 11 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7fca4bb6ef70> ├── 'c' --> 33 ├── 'd' --> 52 ├── 'e' --> 59 └── 'f' --> 6 |
Inheriting on Trees¶
Inheriting is another important feature of treelize functions. It will be introduced in this section with necessary examples.
In some cases, we need to calculation between values with different dimensions, such as in numpy
, you can add primitive integer with a ndarray
object, like this
1 2 3 4 5 6 7 | import os import numpy as np if __name__ == '__main__': ar1 = np.array([[1, 2], [3, 4]]) print('ar1 + 9:', ar1 + 9, sep=os.linesep) |
The result will be like below, result of ar1 + 9
is actually the same as the result of expression ar1 + np.array([[9, 9], [9, 9]])
.
1 2 3 | ar1 + 9: [[10 11] [12 13]] |
In treelize functions, you can achieve this processing way by use the inherit
argument. Considering this situation can be find almost anywhere, so the default value of inherit
argument is True
, which means the inheriting option is on by default.
We can see the example in inherit_demo_1
, the tree t1
and tree t2
have different structures, and cannot be processed by any modes in Function Modes, because t1.x
is a primitive integer value but t2.x
is a subtree node, their types are structurally different.
But when inheriting is enabled, this adding operation can still be carried on, for the value of t1.x
(is 9
in inherit_demo_1
) will be applied to the whole subtree t2.x
, like the result tree t3
shows. The result of tree t3.x
can be considered as the sum of primitive integer 9
and subtree t2.x
.
Based on the structural properties above, in the example of inherit_demo_2
, a primitive value can be directly added with a complete tree, like the result tree t2 = t1 + 5
shows.
When inheriting is disabled, all the cases with primitive value and tree node at the same path position will cause TypeError
. For example, when inheriting is disabled, inherit_demo_1
and inherit_demo_2
will failed, but strict_demo_1
will still success because no primitive value appears at the same path position of tree node.
Here is a real code example of inheriting.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | from treevalue import FastTreeValue, func_treelize @func_treelize(mode='strict') def plus(a, b): return a + b @func_treelize(mode='strict', inherit=False) def plusx(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': 9}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) print('plus(t1, t2):', plus(t1, t2)) print('plus(t2, 5):', plus(t2, 5)) print() print('plusx(t1, t2):', plusx(t1, t2)) print() |
The stdout and stderr should be like below, the the calculation of plus
function and the first calculation of plusx
function will be processed properly, but the last one will failed due to the disablement of inheriting.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | plus(t1, t2): <TreeValue 0x7ffb83d6fee0> ├── 'a' --> 12 ├── 'b' --> 24 └── 'x' --> <TreeValue 0x7ffb84180250> ├── 'c' --> 39 ├── 'd' --> 57 └── 'e' --> 63 plus(t2, 5): <TreeValue 0x7ffb83d6fee0> ├── 'a' --> 16 ├── 'b' --> 27 └── 'x' --> <TreeValue 0x7ffb84180250> ├── 'c' --> 35 ├── 'd' --> 53 └── 'e' --> 59 |
1 2 3 4 5 6 7 8 9 | Traceback (most recent call last): File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/docs/source/tutorials/advanced_usage/inherit_demo.demox.py", line 22, in <module> print('plusx(t1, t2):', plusx(t1, t2)) File "/tmp/tmp4q2ykbbk/6c9ce0a07a1f9ecaf5f5492d86f6b5e9f12e2475/treevalue/tree/func/func.py", line 56, in _new_func return _treelized(*args, **kwargs) File "treevalue/tree/func/cfunc.pyx", line 176, in treevalue.tree.func.cfunc._w_func_treelize_run File "treevalue/tree/func/cfunc.pyx", line 151, in treevalue.tree.func.cfunc._c_func_treelize_run File "treevalue/tree/func/cfunc.pyx", line 112, in treevalue.tree.func.cfunc._c_func_treelize_run TypeError: Inherit is off, tree value expected but 'int' found in args 0. |
Note
In most cases, disablement of inheriting is not recommended because it will cause some errors. Disable inheriting can increase the structural strictness of calculation, but all the cross-dimensional tree operation will be forbidden as well.
So before disabling inheriting, make sure you know well about what you are going to do.
Process Missing Values¶
In some cases of left mode and outer mode, some of keys in result tree can not find the node nor value at the corresponding path position in some of the argument trees. In order the make these cases be processable, missing value can be used by setting value or lambda expression in missing
argument.
Just like the example outer_demo_1
mentioned forwards, when t1.a
and t2.x.f
not found in trees, the given missing value 0
will be used instead.
Another case is the missing_demo_1
which is based on left mode. In this case, t2.a
is ignore due to the property of left mode, t2.x.f
is missing but t1.x.f
can be found, so t2.x.f
will use the missing value 0
. Finally, t3
will be the result of left addition of t1
and t2
, with missing value of 0
.
Considering another complex case, when the values of tree are primitive lists, like the missing_demo_2
which is based on outer mode. At this time, if we need to do addition with missing value supported, we can set the missing value to an empty list. The result is like below.
Note
In missing_demo_2
, lambda :[]
will be the value of missing
argument in actual code.
In missing
argument, its value will be automatically executed when it is callable, and its execution’s return value will be the actual missing value.
By passing a lambda expression in, you can construct a new object everytime missing value is required, and the duplication of the missing value’s object will be avoided. So it is not recommended to use missing
argument like []
, {}
or myobject()
in python code, but lambda :[]
, lambda :{}
and lambda :myobject()
are better practices.
Here is a real code example of missing_value.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | import os from treevalue import func_treelize, FastTreeValue @func_treelize(mode='outer', missing=lambda: []) def plus(a, b): return a + b if __name__ == '__main__': t1 = FastTreeValue({ 'b': [2, 3], 'x': { 'c': [5], 'd': [7, 11, 13], 'e': [17, 19], } }) t2 = FastTreeValue({ 'a': [23], 'b': [29, 31], 'x': { 'c': [37], 'd': [41, 43], } }) print('plus(t1, t2):', plus(t1, t2), sep=os.linesep) |
The stdout (no error occurred) should be like below, the calculation of plus
function will be processed properly, like the result in missing_demo_2
.
1 2 3 4 5 6 7 8 | plus(t1, t2): <TreeValue 0x7f5a66f2eee0> ├── 'a' --> [23] ├── 'b' --> [2, 3, 29, 31] └── 'x' --> <TreeValue 0x7f5a66f2eeb0> ├── 'c' --> [5, 37] ├── 'd' --> [7, 11, 13, 41, 43] └── 'e' --> [17, 19] |
Note
Missing value will be only applied in left mode and outer mode.
In strict mode, missing or overage of keys are absolutely not tolerated, missing value will make no sense and a RuntimeWarning
will be logged.
In inner mode, missing value will never be actually in use because all the key sets of final result are intersection set of argument trees. There will never be any key missing cases at all when inner mode is used, a RuntimeWarning
will be logged with the usage of missing value in inner mode.
Functional Utilities¶
In the primitive python, some convenient functional operations are supported, such as
1 2 3 4 5 6 | if __name__ == '__main__': # map function print("Result of map:", list(map(lambda x: x + 1, [2, 3, 5, 7]))) # filter function print("Result of filter:", list(filter(lambda x: x % 4 == 3, [2, 3, 5, 7]))) |
The result should be
1 2 | Result of map: [3, 4, 6, 8] Result of filter: [3, 7] |
Actually, functional operations are supported in tree values as well, and they will be introduced in this section.
Mapping¶
Mapping function is similar to the primitive map
function. The relation between TreeValue
and mapping
function is like that between list
and map
function.
Here is a simple real code example
1 2 3 4 5 6 7 | from treevalue import mapping, FastTreeValue if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) print('mapping(t, lambda x: x ** x + 2):') print(mapping(t, lambda x: x ** x + 2)) |
The mapped tree result should be like below.
1 2 3 4 5 6 7 | mapping(t, lambda x: x ** x + 2): <FastTreeValue 0x7f304f5ff250> ├── 'a' --> 3 ├── 'b' --> 6 └── 'x' --> <FastTreeValue 0x7f304f1c17c0> ├── 'c' --> 29 └── 'd' --> 258 |
For further definition or source code implement of function mapping
, take a look at mapping.
Filter¶
Filter function is similar to the primitive filter
function. The relation between TreeValue
and filter_
function is like that between list
and filter
function .
Here is a simple real code example
1 2 3 4 5 6 7 8 9 10 | from treevalue import FastTreeValue, filter_ if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) print('filter_(t, lambda x: x % 2 == 1):') print(filter_(t, lambda x: x % 2 == 1)) print('filter_(t, lambda x: x % 2 == 1, remove_empty=False):') print(filter_(t, lambda x: x % 2 == 1, remove_empty=False)) |
The filtered tree result should be like below. When remove_empty
is disabled, the empty tree node will be kept to make sure the structure of the result tree is similar to the unfiltered tree.
1 2 3 4 5 6 7 8 9 10 11 12 | filter_(t, lambda x: x % 2 == 1): <FastTreeValue 0x7efd1db6ef70> ├── 'a' --> 1 └── 'x' --> <FastTreeValue 0x7efd1dfed460> └── 'c' --> 3 filter_(t, lambda x: x % 2 == 1, remove_empty=False): <FastTreeValue 0x7efd1db6ef40> ├── 'a' --> 1 ├── 'x' --> <FastTreeValue 0x7efd1dfed460> │ └── 'c' --> 3 └── 'y' --> <FastTreeValue 0x7efd1db6ef70> |
Note
Actually, the code above is equal to the code below. The filter_
function can be seen as a combination of mask
function and mapping
function.
1 2 3 4 5 6 7 8 9 10 | from treevalue import FastTreeValue, mapping, mask if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) print('mask(t, mapping(t, lambda x: x % 2 == 1)):') print(mask(t, mapping(t, lambda x: x % 2 == 1))) print('mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False):') print(mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False)) |
The result should be like below, exactly the same as the code above.
1 2 3 4 5 6 7 8 9 10 11 12 | mask(t, mapping(t, lambda x: x % 2 == 1)): <FastTreeValue 0x7f54f3faeee0> ├── 'a' --> 1 └── 'x' --> <FastTreeValue 0x7f54f3faef40> └── 'c' --> 3 mask(t, mapping(t, lambda x: x % 2 == 1), remove_empty=False): <FastTreeValue 0x7f54f345adf0> ├── 'a' --> 1 ├── 'x' --> <FastTreeValue 0x7f54f3faeeb0> │ └── 'c' --> 3 └── 'y' --> <FastTreeValue 0x7f54f3faef10> |
For further information about mask
function, take a look at Mask.
For further definition or source code implement of function filter_
, take a look at filter_.
Mask¶
Mask function allow your choice in one tree, by another tree of true-or-false flags. A simple code example is
1 2 3 4 5 6 7 8 9 10 11 | from treevalue import FastTreeValue, mask if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) m = FastTreeValue({'a': True, 'b': False, 'x': {'c': True, 'd': True}, 'y': {'e': False, 'f': False}}) print('mask(t, m):') print(mask(t, m)) print('mask(t, m, remove_empty=False):') print(mask(t, m, remove_empty=False)) |
The result will be like below, values mapped with the tree m
which mask flag is False
are all deleted. Also, remove_empty
argument is supported.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | mask(t, m): <FastTreeValue 0x7fe8b35ff250> ├── 'a' --> 1 └── 'x' --> <FastTreeValue 0x7fe8b316eee0> ├── 'c' --> 3 └── 'd' --> 4 mask(t, m, remove_empty=False): <FastTreeValue 0x7fe8b291adc0> ├── 'a' --> 1 ├── 'x' --> <FastTreeValue 0x7fe8b35ff250> │ ├── 'c' --> 3 │ └── 'd' --> 4 └── 'y' --> <FastTreeValue 0x7fe8b316eee0> |
For further definition or source code implement of function mask
, take a look at mask.
Reduce¶
By using reduce_
function, you can get some calculation result based on the tree structure. Its meaning is similar to primitive reduce_
function, but its base structure is TreeValue
instead of sequence or iterator. For example, we can get the sum and multiply accumulation of the values in the tree.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from functools import reduce from operator import __mul__ from treevalue import reduce_, FastTreeValue def multi(items): return reduce(__mul__, items, 1) if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) print("Sum of t:", reduce_(t, lambda **kwargs: sum(kwargs.values()))) print("Multiply of t:", reduce_(t, lambda **kwargs: multi(kwargs.values()))) |
The result should be like below.
1 2 | Sum of t: 24 Multiply of t: 1152 |
If we use reduce_
function with mapping
function, huffman weight sum can also be easily calculated with the code below.
1 2 3 4 5 6 7 8 9 | from treevalue import reduce_, FastTreeValue, mapping if __name__ == '__main__': t = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}, 'y': {'e': 6, 'f': 8}}) weights = mapping(t, lambda v, p: v * len(p)) print("Weight tree:", weights) print("Huffman weight sum of t:", reduce_(weights, lambda **kwargs: sum(kwargs.values()))) |
The result should be like below.
1 2 3 4 5 6 7 8 9 10 11 | Weight tree: <FastTreeValue 0x7fabea7aff70> ├── 'a' --> 1 ├── 'b' --> 2 ├── 'x' --> <FastTreeValue 0x7fabeabad460> │ ├── 'c' --> 6 │ └── 'd' --> 8 └── 'y' --> <FastTreeValue 0x7fabea7afeb0> ├── 'e' --> 12 └── 'f' --> 16 Huffman weight sum of t: 45 |
Besides, we can easily calculate sum of the np.ndarray
objects’ bytes size, like the following code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | import numpy as np from treevalue import reduce_, FastTreeValue if __name__ == '__main__': t = FastTreeValue({ 'a': np.identity(3), 'b': np.array([[1, 2], [3, 4]]), 'x': { 'c': np.zeros(4), 'd': np.array([[5, 6, 7], [8, 9, 10]]) }, }) print("Size tree:", t.nbytes) print("Total bytes of arrays in t:", reduce_(t.nbytes, lambda **kwargs: sum(kwargs.values()))) |
The result should be like below.
1 2 3 4 5 6 7 8 | Size tree: <FastTreeValue 0x7f22fe7817c0> ├── 'a' --> 72 ├── 'b' --> 32 └── 'x' --> <FastTreeValue 0x7f22fe781a30> ├── 'c' --> 32 └── 'd' --> 48 Total bytes of arrays in t: 184 |
For further definition or source code implement of function reduce_
, take a look at reduce_.
Structural Utilities¶
In order to process some structured data, especially when you need to process a sort of TreeValue
objects which is in the primitive collections or mappings, the structural utilities are designed, and they will be introduced in this section with examples.
Union¶
The union
function is similar to the primitive zip
function some ways, it can combine a list of TreeValue
objects as one TreeValue
object which leaf values are tuples, like the simple example below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | from treevalue import FastTreeValue, union if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'e': 5}}) t2 = FastTreeValue({'a': 11, 'b': 22, 'x': {'c': 30, 'd': 48, 'e': 54}}) t3 = FastTreeValue({'a': -13, 'b': -7, 'x': {'c': -5, 'd': -3, 'e': -2}}) t4 = FastTreeValue({'a': -13, 'b': -7, 'x': 8}) print("union(t1, t2):") print(union(t1, t2)) print("union(t1, t2, t3):") print(union(t1, t2, t3)) print("union(t1, t2, t3, t4):") print(union(t1, t2, t3, t4)) |
The result should be like below, the leaf values are unionised as tuples.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | union(t1, t2): <FastTreeValue 0x7fa17852d1c0> ├── 'a' --> (1, 11) ├── 'b' --> (2, 22) └── 'x' --> <FastTreeValue 0x7fa17852d460> ├── 'c' --> (3, 30) ├── 'd' --> (4, 48) └── 'e' --> (5, 54) union(t1, t2, t3): <FastTreeValue 0x7fa17852d460> ├── 'a' --> (1, 11, -13) ├── 'b' --> (2, 22, -7) └── 'x' --> <FastTreeValue 0x7fa17852d1c0> ├── 'c' --> (3, 30, -5) ├── 'd' --> (4, 48, -3) └── 'e' --> (5, 54, -2) union(t1, t2, t3, t4): <FastTreeValue 0x7fa17852d1c0> ├── 'a' --> (1, 11, -13, -13) ├── 'b' --> (2, 22, -7, -7) └── 'x' --> <FastTreeValue 0x7fa17852d460> ├── 'c' --> (3, 30, -5, 8) ├── 'd' --> (4, 48, -3, 8) └── 'e' --> (5, 54, -2, 8) |
Note
In union
function (actually subside
function has the same property), all the trees to be unionised need to have the same structure. You can just consider it as strict mode with inheriting is enabled.
For further information and arguments of function union
, take a look at union.
Subside¶
The subside
function can transform the collections or mapping nested TreeValue
objects into one TreeValue
object which leaf values has the dispatch’s structure. The union
function mentioned above is also based on the subside
function. Function subside
will greatly simplify the code when the structured data need to be calculated. Just like this following example code.
1 2 3 4 5 6 7 8 9 10 11 12 | from treevalue import TreeValue, subside if __name__ == '__main__': t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}}) t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}}) st = {'first': (t1, t2), 'second': [t3, {'x': t4, 'y': t5}]} print("subside(st):") print(subside(st)) |
The result should be like below, the leaf values are combined as the dispatch structure of st
.
1 2 3 4 5 6 7 | subside(st): <TreeValue 0x7fae4b3f2190> ├── 'a' --> {'first': (1, -14), 'second': [6, {'x': 0, 'y': 3}]} ├── 'b' --> {'first': (2, 9), 'second': [0, {'x': -17, 'y': 9}]} └── 'x' --> <TreeValue 0x7fae4b3f2220> ├── 'c' --> {'first': (3, 3), 'second': [-5, {'x': -8, 'y': 11}]} └── 'd' --> {'first': (4, 8), 'second': [17, {'x': 15, 'y': -17}]} |
Note
In function subside
, only primitive list, tuple and dict (or subclasses) objects will be subsided to leaf values.
For further information and arguments of function subside
, take a look at subside.
Rise¶
Function rise
can be seen as the inverse operation of function subside
, it will try to extract the greatest common structure of the leaf values, and rise them up to the dispatch above the TreeValue
objects. Like the following example code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | from treevalue import TreeValue, subside, rise if __name__ == '__main__': # The same demo as the subside docs t1 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}}) t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}}) st = {'first': (t1, t2), 'second': (t3, {'x': t4, 'y': t5})} tx = subside(st) # Rising process st2 = rise(tx) assert st2 == st print('st2:', st2) print("st2['first'][0]:") print(st2['first'][0]) print("st2['first'][1]:") print(st2['first'][1]) print("st2['second'][0]:") print(st2['second'][0]) print("st2['second'][1]['x']:") print(st2['second'][1]['x']) print("st2['second'][1]['y']:") print(st2['second'][1]['y']) |
The result should be like below, the subsided tree st
can be extract back to the original structure.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | st2: {'second': (<TreeValue 0x7f15693b1fd0> ├── 'a' --> 6 ├── 'b' --> 0 └── 'x' --> <TreeValue 0x7f15693b1130> ├── 'c' --> -5 └── 'd' --> 17 , {'x': <TreeValue 0x7f1568b63d00> ├── 'a' --> 0 ├── 'b' --> -17 └── 'x' --> <TreeValue 0x7f15693b1880> ├── 'c' --> -8 └── 'd' --> 15 , 'y': <TreeValue 0x7f1568b63730> ├── 'a' --> 3 ├── 'b' --> 9 └── 'x' --> <TreeValue 0x7f1568b63670> ├── 'c' --> 11 └── 'd' --> -17 }), 'first': (<TreeValue 0x7f1568b63250> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7f1568b63dc0> ├── 'c' --> 3 └── 'd' --> 4 , <TreeValue 0x7f1568b63820> ├── 'a' --> -14 ├── 'b' --> 9 └── 'x' --> <TreeValue 0x7f1568b63e80> ├── 'c' --> 3 └── 'd' --> 8 )} st2['first'][0]: <TreeValue 0x7f1568b63250> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7f1568b63dc0> ├── 'c' --> 3 └── 'd' --> 4 st2['first'][1]: <TreeValue 0x7f1568b63820> ├── 'a' --> -14 ├── 'b' --> 9 └── 'x' --> <TreeValue 0x7f1568b63e80> ├── 'c' --> 3 └── 'd' --> 8 st2['second'][0]: <TreeValue 0x7f15693b1fd0> ├── 'a' --> 6 ├── 'b' --> 0 └── 'x' --> <TreeValue 0x7f15693b1130> ├── 'c' --> -5 └── 'd' --> 17 st2['second'][1]['x']: <TreeValue 0x7f1568b63d00> ├── 'a' --> 0 ├── 'b' --> -17 └── 'x' --> <TreeValue 0x7f15693b1880> ├── 'c' --> -8 └── 'd' --> 15 st2['second'][1]['y']: <TreeValue 0x7f1568b63730> ├── 'a' --> 3 ├── 'b' --> 9 └── 'x' --> <TreeValue 0x7f1568b63670> ├── 'c' --> 11 └── 'd' --> -17 |
Note
In function rise
, only primitive list, tuple and dict (or subclasses) objects will join the rising operation.
Actually, when template
argument is not assigned, the rise
function will try to find the greatest common structure and rise them to the dispatch. The following rules will be strictly followed when doing this:
If the values in current level have the same type ( must all be list, all be tuple or all be dict), the rising function will carry on the find sub structure, otherwise values in current level will be treated as atomic values.
If all of them are dicts, and they have exactly the same key sets with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.
If all of them are lists, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values.
If all of them are tuples, and they have the same length with each other, the finding of sub structures will be carried on, otherwise these dicts will be treated as atomic values. (Actually, this rule is the same as the last one which is about lists.)
Considering this automatic structure finding process, if you only want to extract some of the structure (make sure the extraction will not be too deep or too shallow, and make sure the result will have the same structure as your expectation), you can assign value in template
arguments. Like the example code below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | from treevalue import TreeValue, subside, rise if __name__ == '__main__': # The same demo as the subside docs t11 = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t12 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 15, 'd': 41}}) t21 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t22 = TreeValue({'a': -31, 'b': 82, 'x': {'c': 47, 'd': 32}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) t4 = TreeValue({'a': 0, 'b': -17, 'x': {'c': -8, 'd': 15}}) t5 = TreeValue({'a': 3, 'b': 9, 'x': {'c': 11, 'd': -17}}) st = {'first': [(t11, t21), (t12, t22)], 'second': (t3, {'x': t4, 'y': t5})} tx = subside(st) # Rising process, with template # only the top-leveled dict will be extracted, # neither tuple, list nor low-leveled dict will not be extracted # because they are not defined in `template` argument st2 = rise(tx, template={'first': [object, ...], 'second': (object, ...)}) print('st2:', st2) print("st2['first'][0]:") print(st2['first'][0]) print("st2['first'][1]:") print(st2['first'][1]) print("st2['second'][0]:") print(st2['second'][0]) print("st2['second'][1]:") print(st2['second'][1]) |
The result should be like below, the subsided tree st
can be extract back to the structure of dict with only first
and second
keys.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | st2: {'first': [<TreeValue 0x7f35efaf11f0> ├── 'a' --> (1, -14) ├── 'b' --> (2, 9) └── 'x' --> <TreeValue 0x7f35efaf1220> ├── 'c' --> (3, 3) └── 'd' --> (4, 8) , <TreeValue 0x7f35efaf1880> ├── 'a' --> (3, -31) ├── 'b' --> (9, 82) └── 'x' --> <TreeValue 0x7f35efaf1130> ├── 'c' --> (15, 47) └── 'd' --> (41, 32) ], 'second': (<TreeValue 0x7f35efa579a0> ├── 'a' --> 6 ├── 'b' --> 0 └── 'x' --> <TreeValue 0x7f35efa578e0> ├── 'c' --> -5 └── 'd' --> 17 , <TreeValue 0x7f35ef2a34f0> ├── 'a' --> {'x': 0, 'y': 3} ├── 'b' --> {'x': -17, 'y': 9} └── 'x' --> <TreeValue 0x7f35efa57a00> ├── 'c' --> {'x': -8, 'y': 11} └── 'd' --> {'x': 15, 'y': -17} )} st2['first'][0]: <TreeValue 0x7f35efaf11f0> ├── 'a' --> (1, -14) ├── 'b' --> (2, 9) └── 'x' --> <TreeValue 0x7f35efaf1220> ├── 'c' --> (3, 3) └── 'd' --> (4, 8) st2['first'][1]: <TreeValue 0x7f35efaf1880> ├── 'a' --> (3, -31) ├── 'b' --> (9, 82) └── 'x' --> <TreeValue 0x7f35efaf1130> ├── 'c' --> (15, 47) └── 'd' --> (41, 32) st2['second'][0]: <TreeValue 0x7f35efa579a0> ├── 'a' --> 6 ├── 'b' --> 0 └── 'x' --> <TreeValue 0x7f35efa578e0> ├── 'c' --> -5 └── 'd' --> 17 st2['second'][1]: <TreeValue 0x7f35ef2a34f0> ├── 'a' --> {'x': 0, 'y': 3} ├── 'b' --> {'x': -17, 'y': 9} └── 'x' --> <TreeValue 0x7f35efa57a00> ├── 'c' --> {'x': -8, 'y': 11} └── 'd' --> {'x': 15, 'y': -17} |
For further information and arguments of function rise
, take a look at rise.
Tree Utilities¶
In this section, utilities with TreeValue
class and its objects themselves will be introduced with examples.
Jsonify¶
With the usage of jsonify
function, you can transform a TreeValue
object to json-formatted data.
For example, the following real code
1 2 3 4 5 6 7 8 9 10 11 12 | import json from treevalue import TreeValue, raw, jsonify if __name__ == '__main__': t = TreeValue({'a': 1, 'b': [2, 3], 'x': {'c': raw({'x': 1, 'y': 2}), 'd': "this is a string"}}) print("Tree t:") print(t) print("Json data of t:") print(json.dumps(jsonify(t), indent=4, sort_keys=True)) |
The result should be
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | Tree t: <TreeValue 0x7ff31dd017c0> ├── 'a' --> 1 ├── 'b' --> [2, 3] └── 'x' --> <TreeValue 0x7ff31dd77b20> ├── 'c' --> {'x': 1, 'y': 2} └── 'd' --> 'this is a string' Json data of t: { "a": 1, "b": [ 2, 3 ], "x": { "c": { "x": 1, "y": 2 }, "d": "this is a string" } } |
Note
The function raw
in the example code above is a wrapper for dictionary object. It can be used to pass dict
object as simple value in TreeValue
instead of being treated as sub tree.
For further information of function raw
, take a look at raw.
For further informaon of function jsonify
, take a look at jsonify.
Clone¶
The TreeValue
objects can be cloned deeply by clone
function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | from treevalue import TreeValue, clone, raw if __name__ == '__main__': t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4, 'y': raw({'e': 5, 'f': 6})}}) print("Tree t:") print(t) print("Id of t.x.y: %x" % id(t.x.y)) print() print() print("clone(t):") print(clone(t)) print("Id of clone(t).x.y: %x" % id(clone(t).x.y)) print() print() print('clone(t, copy_value=True):') print(clone(t, copy_value=True)) print("Id of clone(t, copy_value=True).x.y: %x" % id(clone(t, copy_value=True))) print() print() |
The result will be like below, all the memory address of the tree nodes in cloned tree are different from those in the original tree t
when copy_value
argument is not set. But when copy_value
is assigned as True
, the address of values will be changed because of the deep copy of value.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | Tree t: <TreeValue 0x7feb3f85c7c0> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7feb3f8d2b20> ├── 'c' --> 3 ├── 'd' --> 4 └── 'y' --> {'e': 5, 'f': 6} Id of t.x.y: 7feb3f978200 clone(t): <TreeValue 0x7feb3f56ef70> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7feb3f85ca30> ├── 'c' --> 3 ├── 'd' --> 4 └── 'y' --> {'e': 5, 'f': 6} Id of clone(t).x.y: 7feb3f978200 clone(t, copy_value=True): <TreeValue 0x7feb3f56ef70> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <TreeValue 0x7feb3f85ca30> ├── 'c' --> 3 ├── 'd' --> 4 └── 'y' --> {'e': 5, 'f': 6} Id of clone(t, copy_value=True).x.y: 7feb3e3c6b40 |
Note
Attention that in function clone
, the values will not be deeply copied together with the tree nodes by default. The newly cloned tree’s values have the same memory address with those in original tree.
If deep copy of values is required when using clone
, copy_value
argument need to be assigned as True
. And then a copy.deepcopy
will be performed in clone
function in order to do the deep copy. Also, you can define your own copy function in copy_value
argument by just assign it as a lambda expression like lambda x: pickle.loads(pickle.dumps(x))
.
For further informaon of function clone
, take a look at clone.
Typetrans¶
You can use function typetrans
to change the type of tree value object, just like the example below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | from treevalue import TreeValue, method_treelize, FastTreeValue, typetrans class MyTreeValue(TreeValue): @method_treelize() def pw(self): return (self + 1) * (self + 2) if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) print('t1:') print(t1) print('t1 ** 2:') print(t1 ** 2) print() # Transform t1 to MyTreeValue, # __pow__ operator will be disabled and pw method will be enabled. t2 = typetrans(t1, MyTreeValue) print('t2:') print(t2) print('t2.pw():') print(t2.pw()) print() |
The result will be like below. After the type transformation, **
operator can not be used in t2
but method pw
which is implemented in MyTreeValue
will be enabled.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | t1: <FastTreeValue 0x7f9dca8f7b20> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <FastTreeValue 0x7f9dca954ee0> ├── 'c' --> 3 └── 'd' --> 4 t1 ** 2: <FastTreeValue 0x7f9dca8817c0> ├── 'a' --> 1 ├── 'b' --> 4 └── 'x' --> <FastTreeValue 0x7f9dca97f250> ├── 'c' --> 9 └── 'd' --> 16 t2: <MyTreeValue 0x7f9dca8f7b20> ├── 'a' --> 1 ├── 'b' --> 2 └── 'x' --> <MyTreeValue 0x7f9dca954ee0> ├── 'c' --> 3 └── 'd' --> 4 t2.pw(): <MyTreeValue 0x7f9dca8817c0> ├── 'a' --> 6 ├── 'b' --> 12 └── 'x' --> <MyTreeValue 0x7f9dca97f250> ├── 'c' --> 20 └── 'd' --> 30 |
Note
In treevalue library, there can be more classes based on TreeValue
class by your definition.
Different tree value class will have the different operators, methods and class methods for service, and the __eq__
operator’s result between different type of tree values will always be False
because of the difference of their types.
For further information of TreeValue
class and user’s definition practice, just take a look at:
For further informaon of function typetrans
, take a look at typetrans.
Walk¶
You can use function treevalue.tree.tree.walk()
to iterate all the nodes in the tree, like the example below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import TreeValue, raw, walk if __name__ == '__main__': t = TreeValue({ 'a': 1, 'b': 2, 'c': raw({'x': 3, 'y': 4}), 'd': { 'x': 3, 'y': 4 }, }) for path, node in walk(t): print(path, '-->', node) |
The output should be like below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | () --> <TreeValue 0x7fdc593017c0> ├── 'a' --> 1 ├── 'b' --> 2 ├── 'c' --> {'x': 3, 'y': 4} └── 'd' --> <TreeValue 0x7fdc59377b20> ├── 'x' --> 3 └── 'y' --> 4 ('a',) --> 1 ('b',) --> 2 ('c',) --> {'x': 3, 'y': 4} ('d',) --> <TreeValue 0x7fdc59377b20> ├── 'x' --> 3 └── 'y' --> 4 ('d', 'x') --> 3 ('d', 'y') --> 4 |
For further informaon of function walk
, take a look at walk.
Flatten Utilities¶
In order to support the parallel calculation in values of TreeValue
object, treevalue.tree.tree.flatten()
and treevalue.tree.tree.unflatten()
is provided to dump a sequence of nodes’ information and value, and recover it to original tree structure when calculation is completed.
Flatten¶
The values and node structures can be dumped by function treevalue.tree.tree.flatten()
as a list, like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | from treevalue import TreeValue, raw, flatten if __name__ == '__main__': t = TreeValue({ 'a': 1, 'b': 2, 'c': raw({'x': 3, 'y': 4}), 'd': { 'x': 3, 'y': 4 }, }) print('flatten(t):') print(flatten(t)) |
The result should be like below:
1 2 | flatten(t): [(('a',), 1), (('b',), 2), (('c',), {'x': 3, 'y': 4}), (('d', 'x'), 3), (('d', 'y'), 4)] |
Note
Function treevalue.tree.tree.flatten()
is different from treevalue.tree.tree.walk()
because flatten
has fewer features because it is designed entirely for the dumping process before parallel calculation.
So when you need to do parallel calculation, please use treevalue.tree.tree.flatten()
to dump the values to ensure the speed performance.
For further informaon of function flatten
, take a look at flatten.
Unflatten¶
The dumped nodes’ information and value can be recovered to a TreeValue
object by function treevalue.tree.tree.unflatten()
, like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 | from treevalue import unflatten if __name__ == '__main__': flatted = [ (('a',), 1), (('b',), 2), (('c',), {'x': 3, 'y': 4}), (('d', 'x'), 3), (('d', 'y'), 4) ] print('unflatten(flatted):') print(unflatten(flatted)) |
The result should be like below:
1 2 3 4 5 6 7 8 | unflatten(flatted): <TreeValue 0x7f06a1df7b20> ├── 'a' --> 1 ├── 'b' --> 2 ├── 'c' --> {'x': 3, 'y': 4} └── 'd' --> <TreeValue 0x7f06a1d807c0> ├── 'x' --> 3 └── 'y' --> 4 |
Note
It is recommended to pass an ordered iterable object in pairs
, this will improve the speed performance of function treevalue.tree.tree.unflatten()
.
Because of this, it is a good idea to keep the treevalue.tree.tree.flatten()
’s result’s order when executing your own parallel processing logic.
For further informaon of function unflatten
, take a look at unflatten.
IO Utilities¶
Simple Serialize and Deserialize¶
In this version, pickle.dumps
and pickle.loads
operations are supported, just like the code below.
import os
import pickle
from treevalue import TreeValue
if __name__ == '__main__':
t = TreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': [3, 4]}})
binary = pickle.dumps(t)
print('t:', t, sep=os.linesep)
tx = pickle.loads(binary)
assert tx == t
print('tx:', tx, sep=os.linesep)
The output is
t:
<TreeValue 0x7f4a8d0b7b20>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f4a8d114ee0>
├── 'c' --> 3
└── 'd' --> [3, 4]
tx:
<TreeValue 0x7f4a8d040a30>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <TreeValue 0x7f4a8d0407c0>
├── 'c' --> 3
└── 'd' --> [3, 4]
Advanced Dump and Load¶
In this project, we implemented our load
and dump
function which can be used like this
import os
import tempfile
from treevalue import TreeValue, dump, load
if __name__ == '__main__':
t = TreeValue({'a': 'ab', 'b': 'Cd', 'x': {'c': 'eF', 'd': 'GH'}})
print('t:', t, sep=os.linesep)
with tempfile.NamedTemporaryFile() as tf:
with open(tf.name, 'wb') as wf: # dump t to file
dump(t, file=wf)
with open(tf.name, 'rb') as rf: # load dt from file
dt = load(file=rf)
assert dt == t
print('dt:', dt, sep=os.linesep)
The output should be
t:
<TreeValue 0x7f9df1ad2b20>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <TreeValue 0x7f9df1b54ee0>
├── 'c' --> 'eF'
└── 'd' --> 'GH'
dt:
<TreeValue 0x7f9df0b9e9d0>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <TreeValue 0x7f9df0b9e880>
├── 'c' --> 'eF'
└── 'd' --> 'GH'
Note
In this load
(loads
function is the same), you must assign a TreeValue
class by the type_
argument because the dumped data will not save the type of trees. When the type is not assigned, the default class will be simply TreeValue
.
Also, dumps
function and loads
function are provided to deal with binary data.
import os
from treevalue import dumps, FastTreeValue, loads
if __name__ == '__main__':
t = FastTreeValue({'a': 'ab', 'b': 'Cd', 'x': {'c': 'eF', 'd': 'GH'}})
print('t:', t, sep=os.linesep)
binary = dumps(t)
dt = loads(binary, type_=FastTreeValue)
assert dt == t
print('dt:', dt, sep=os.linesep)
The output should be
t:
<FastTreeValue 0x7fdd409f7b20>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <FastTreeValue 0x7fdd40d94ee0>
├── 'c' --> 'eF'
└── 'd' --> 'GH'
dt:
<FastTreeValue 0x7fdd400e4130>
├── 'a' --> 'ab'
├── 'b' --> 'Cd'
└── 'x' --> <FastTreeValue 0x7fdd400e4190>
├── 'c' --> 'eF'
└── 'd' --> 'GH'
Note
The main differences between our dump
/ load
and pickle
‘s dump
/ load
are:
Ours are based on dill project, it can serialize and deserialize more types than native
pickle
, such as functions and classes.Compression and decompression can be defined when dumping, and when the compressed binary data is going to be loaded, decompression function is not necessary, the decompression can be carried on with the function defined in dumping process.
Here is an example:
import gzip
import os
from treevalue import FastTreeValue, dumps, loads
if __name__ == '__main__':
t = FastTreeValue({'a': 'ab', 'b': 'Cd', 'x': {'c': 'eF', 'd': 'GH'}})
st = t.upper
print('st:', st, sep=os.linesep) # st is a function tree
print('st():', st(), sep=os.linesep) # st() if an upper-case-string tree
print()
binary = dumps(st, compress=(gzip.compress, gzip.decompress))
print('Length of binary:', len(binary))
# compression function is not needed here
dst = loads(binary, type_=FastTreeValue)
print('dst:', dst, sep=os.linesep)
assert st() == dst() # st() should be equal to dst()
print('dst():', dst(), sep=os.linesep)
The output should be
st:
<FastTreeValue 0x7f9520540a30>
├── 'a' --> <built-in method upper of str object at 0x7f95200ebe70>
├── 'b' --> <built-in method upper of str object at 0x7f95205c3470>
└── 'x' --> <FastTreeValue 0x7f9520886460>
├── 'c' --> <built-in method upper of str object at 0x7f95205a25f0>
└── 'd' --> <built-in method upper of str object at 0x7f95205a26f0>
st():
<FastTreeValue 0x7f951f97cac0>
├── 'a' --> 'AB'
├── 'b' --> 'CD'
└── 'x' --> <FastTreeValue 0x7f95205407c0>
├── 'c' --> 'EF'
└── 'd' --> 'GH'
Length of binary: 213
dst:
<FastTreeValue 0x7f951f972a90>
├── 'a' --> <built-in method upper of str object at 0x7f951f10a0b0>
├── 'b' --> <built-in method upper of str object at 0x7f951f0f0bf0>
└── 'x' --> <FastTreeValue 0x7f951f972a30>
├── 'c' --> <built-in method upper of str object at 0x7f951f0f0cf0>
└── 'd' --> <built-in method upper of str object at 0x7f951f0fe1f0>
dst():
<FastTreeValue 0x7f951f972700>
├── 'a' --> 'AB'
├── 'b' --> 'CD'
└── 'x' --> <FastTreeValue 0x7f951f972b20>
├── 'c' --> 'EF'
└── 'd' --> 'GH'
Object Oriented Usage¶
In FastTreeValue
class, plenty of object-oriented operators, methods and classmethods are supported in order to simplify the actual usage. Here is a simple code example.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | from treevalue import FastTreeValue if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = FastTreeValue({'a': 5, 'b': 6, 'x': {'c': 7, 'd': 8}}) # operator support print('t1 + t2:') print(t1 + t2) print('t2 ** t1:') print(t2 ** t1) print() # utilities support print('t1.map(lambda x: (x + 1) * (x + 2)):') print(t1.map(lambda x: (x + 1) * (x + 2))) print('t1.reduce(lambda **kwargs: sum(kwargs.values())):', t1.reduce(lambda **kwargs: sum(kwargs.values()))) print() print() # linking usage print('t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0):') print(t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0)) print() # structural support print("Union result:") print(FastTreeValue.union( t1.map(lambda x: (x + 1) * (x + 2)), t2.map(lambda x: (x - 2) ** (x - 1)), ).map(lambda x: 'first: %d, second: %d, which sum is %d' % (x[0], x[1], sum(x)))) print() |
The result should be like below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | t1 + t2: <FastTreeValue 0x7f58d8f6ef70> ├── 'a' --> 6 ├── 'b' --> 8 └── 'x' --> <FastTreeValue 0x7f58d92bfa30> ├── 'c' --> 10 └── 'd' --> 12 t2 ** t1: <FastTreeValue 0x7f58d8f6ef70> ├── 'a' --> 5 ├── 'b' --> 36 └── 'x' --> <FastTreeValue 0x7f58d92bfa30> ├── 'c' --> 343 └── 'd' --> 4096 t1.map(lambda x: (x + 1) * (x + 2)): <FastTreeValue 0x7f58d8f6ef70> ├── 'a' --> 6 ├── 'b' --> 12 └── 'x' --> <FastTreeValue 0x7f58d92bfa30> ├── 'c' --> 20 └── 'd' --> 30 t1.reduce(lambda **kwargs: sum(kwargs.values())): 10 t1.map(lambda x: (x + 1) * (x + 2)).filter(lambda x: x % 4 == 0): <FastTreeValue 0x7f58d8f6eee0> ├── 'b' --> 12 └── 'x' --> <FastTreeValue 0x7f58d8f6ef10> └── 'c' --> 20 Union result: <FastTreeValue 0x7f58d8f6eee0> ├── 'a' --> 'first: 6, second: 81, which sum is 87' ├── 'b' --> 'first: 12, second: 1024, which sum is 1036' └── 'x' --> <FastTreeValue 0x7f58d92bfa30> ├── 'c' --> 'first: 20, second: 15625, which sum is 15645' └── 'd' --> 'first: 30, second: 279936, which sum is 279966' |
For further information of FastTreeValue
class, take a look at FastTreeValue, all the supported methods and operators are listed here.
DIY TreeValue Class¶
You can define your own TreeValue
class with your own method and class methods. Like the example code below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | import os from treevalue import classmethod_treelize, TreeValue, method_treelize class MyTreeValue(TreeValue): # return type will be automatically detected as `MyTreeValue` @method_treelize() def append(self, b): print("Append arguments:", self, b) return self + b # return type will be automatically detected as `MyTreeValue` @classmethod @classmethod_treelize() def sum(cls, *args): print("Sum arguments:", cls, *args) return sum(args) if __name__ == '__main__': t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}}) t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}}) print('t1.append(t2).append(t3):', t1.append(t2).append(t3), sep=os.linesep) print('MyTreeValue.sum(t1, t2, t3):', MyTreeValue.sum(t1, t2, t3), sep=os.linesep) |
The result should be like below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | Append arguments: 1 -14 Append arguments: 2 9 Append arguments: 4 8 Append arguments: 3 3 Append arguments: -13 6 Append arguments: 11 0 Append arguments: 12 17 Append arguments: 6 -5 t1.append(t2).append(t3): <MyTreeValue 0x7ff08129aeb0> ├── 'a' --> -7 ├── 'b' --> 11 └── 'x' --> <MyTreeValue 0x7ff08129afd0> ├── 'c' --> 1 └── 'd' --> 29 Sum arguments: <class '__main__.MyTreeValue'> 1 -14 6 Sum arguments: <class '__main__.MyTreeValue'> 2 9 0 Sum arguments: <class '__main__.MyTreeValue'> 4 8 17 Sum arguments: <class '__main__.MyTreeValue'> 3 3 -5 MyTreeValue.sum(t1, t2, t3): <MyTreeValue 0x7ff08129aeb0> ├── 'a' --> -7 ├── 'b' --> 11 └── 'x' --> <MyTreeValue 0x7ff08129afd0> ├── 'c' --> 1 └── 'd' --> 29 |
Note
Actually, self-calculation method can be defined like the code below
import os
from treevalue import TreeValue, method_treelize
class MyTreeValue(TreeValue):
# return type will be automatically detected as `MyTreeValue`
@method_treelize(self_copy=True)
def append(self, b):
return self + b
if __name__ == '__main__':
t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
t2 = TreeValue({'a': -14, 'b': 9, 'x': {'c': 3, 'd': 8}})
t3 = TreeValue({'a': 6, 'b': 0, 'x': {'c': -5, 'd': 17}})
print('t1:', t1, sep=os.linesep)
_t1_id = id(t1)
print('t1.append(t2).append(t3):',
t1.append(t2).append(t3), sep=os.linesep)
assert id(t1) == _t1_id
The result should be
t1:
<MyTreeValue 0x7f130bbf7b20>
├── 'a' --> 1
├── 'b' --> 2
└── 'x' --> <MyTreeValue 0x7f130bf95ee0>
├── 'c' --> 3
└── 'd' --> 4
t1.append(t2).append(t3):
<MyTreeValue 0x7f130bbf7b20>
├── 'a' --> -7
├── 'b' --> 11
└── 'x' --> <MyTreeValue 0x7f130bf95ee0>
├── 'c' --> 1
└── 'd' --> 29
We can see that the calculation result will replace the self
object, without changes of the address. Just enable the self_copy
option is okay.
For further information of function method_treelize
and classmethod_treelize
, just take a look at:
More Flexible Ways to DIY Class¶
When you implement TreeValue
class like the section above (take a look at here DIY TreeValue Class), you have to define your own methods and operators one by one, because in class TreeValue
, only the minimum necessary methods and operators are provided. But when you use FastTreeValue
class, operators like +
and methods like map
can be used directly. Let’s see how FastTreeValue
is defined in source code:
1 2 3 4 5 6 7 8 9 10 | from .general import general_tree_value class FastTreeValue(general_tree_value()): """ Overview: Fast tree value, can do almost anything with this. """ pass |
The definition is quite simple, just inherit from the return value of function general_tree_value
, and most of the operators and methods can be used.
Like the FastTreeValue
class, we can define our TreeValue
class like the code below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | import os from treevalue import general_tree_value class MyTreeValue(general_tree_value()): pass if __name__ == '__main__': t1 = MyTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = MyTreeValue({'a': 11, 'b': 24, 'x': {'c': 30, 'd': 47}}) # __add__ operator can be directly used print('t1 + t2:', t1 + t2, sep=os.linesep) |
The output should be
1 2 3 4 5 6 7 | t1 + t2: <MyTreeValue 0x7fe95889dfd0> ├── 'a' --> 12 ├── 'b' --> 26 └── 'x' --> <MyTreeValue 0x7fe95889dcd0> ├── 'c' --> 33 └── 'd' --> 51 |
In some cases, you can update the behaviours of some operator or methods, like the code below
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | import os from treevalue import general_tree_value, FastTreeValue class AddZeroTreeValue(general_tree_value(methods=dict( __add__=dict(missing=0, mode='outer'), __radd__=dict(missing=0, mode='outer'), __iadd__=dict(missing=0, mode='outer'), ))): pass class MulOneTreeValue(general_tree_value(methods=dict( __mul__=dict(missing=1, mode='outer'), __rmul__=dict(missing=1, mode='outer'), __imul__=dict(missing=1, mode='outer'), ))): pass if __name__ == '__main__': t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3}}) t2 = FastTreeValue({'a': 11, 'x': {'c': 30, 'd': 47}}) # __add__ with default value 0 at1 = t1.type(AddZeroTreeValue) at2 = t2.type(AddZeroTreeValue) print('at1 + at2:', at1 + at2, sep=os.linesep) # __mul__ with default value 1 mt1 = t1.type(MulOneTreeValue) mt2 = t2.type(MulOneTreeValue) print('mt1 * mt2:', mt1 * mt2, sep=os.linesep) |
The output should be
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | at1 + at2: <AddZeroTreeValue 0x7fe4923ca190> ├── 'a' --> 12 ├── 'b' --> 2 └── 'x' --> <AddZeroTreeValue 0x7fe4923ca340> ├── 'c' --> 33 └── 'd' --> 47 mt1 * mt2: <MulOneTreeValue 0x7fe4923ca190> ├── 'a' --> 11 ├── 'b' --> 2 └── 'x' --> <MulOneTreeValue 0x7fe4923ca340> ├── 'c' --> 90 └── 'd' --> 47 |
More than this, you can do the following things in your DIY TreeValue
class based on function general_tree_value
:
change treelize arguments (like the example code above)
disable some operators and methods
change behaviour of some operators and methods
Here is another complex demo code
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | import os from treevalue import general_tree_value class DemoTreeValue(general_tree_value(methods=dict( # - operator will be disabled __sub__=NotImplemented, __rsub__=NotImplemented, __isub__=NotImplemented, # / operator will raise ArithmeticError __truediv__=ArithmeticError('True div is not supported'), __rtruediv__=ArithmeticError('True div is not supported'), __itruediv__=ArithmeticError('True div is not supported'), # +t will be changed to t * 2 __pos__=lambda x: x * 2, ))): pass if __name__ == '__main__': t1 = DemoTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) t2 = DemoTreeValue({'a': 11, 'b': 24, 'x': {'c': 30, 'd': 47}}) # __add__ can be used normally print('t1 + t2:', t1 + t2, sep=os.linesep) # __sub__ will cause TypeError due to the NotImplemented try: _ = t1 - t2 except TypeError as err: print('t1 - t2:', err, sep=os.linesep) else: assert False, 'Should not reach here!' # __truediv__ will cause ArithmeticError try: _ = t1 / t2 except ArithmeticError as err: print('t1 / t2:', err, sep=os.linesep) else: assert False, 'Should not reach here!' # __pos__ will be like t1 * 2 print('+t1:', +t1, sep=os.linesep) |
The output should be
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | t1 + t2: <DemoTreeValue 0x7fa273eb6fd0> ├── 'a' --> 12 ├── 'b' --> 26 └── 'x' --> <DemoTreeValue 0x7fa273eb6f40> ├── 'c' --> 33 └── 'd' --> 51 t1 - t2: unsupported operand type(s) for -: 'DemoTreeValue' and 'DemoTreeValue' t1 / t2: True div is not supported +t1: <DemoTreeValue 0x7fa273eb6d90> ├── 'a' --> 2 ├── 'b' --> 4 └── 'x' --> <DemoTreeValue 0x7fa273eb6fd0> ├── 'c' --> 6 └── 'd' --> 8 |
For more details about general_tree_value
, take a look at general_tree_value and its source code to find out all the implemented operators and methods.
Draw Graph For TreeValue¶
You can easily draw a graph of the TreeValue
objects with function graphics
. Like the following example code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | import numpy as np from treevalue import FastTreeValue, graphics, TreeValue class MyFastTreeValue(FastTreeValue): pass if __name__ == '__main__': t = MyFastTreeValue({ 'a': 1, 'b': np.array([[5, 6], [7, 8]]), 'x': { 'c': 3, 'd': 4, 'e': np.array([[1, 2], [3, 4]]) }, }) t2 = TreeValue({'ppp': t.x, 'x': {'t': t, 'y': t.x}}) g = graphics( (t, 't'), (t2, 't2'), title="This is a demo of 2 trees.", cfg={'bgcolor': '#ffffff00'}, ) g.render('graphics.dat.gv', format='svg') |
The generated code and graph should be like below ( name of the image file should be graphics.dat.gv.svg
, name of the graphviz source code file should be graphics.dat.gv
).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | // This is a demo of 2 trees. digraph this_is_a_demo_of_2_trees { graph [bgcolor="#ffffff00" label="This is a demo of 2 trees."] node_7f4006479b20 [label=t color="#3eb24899" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman bold" penwidth=3 shape=diamond style=filled] node_7f4006514460 [label=t2 color="#5b3eb299" fillcolor="#9673ff99" fontcolor="#0d0033ff" fontname="Times-Roman bold" penwidth=3 shape=diamond style=filled] value__node_7f4006479b20__a [label=1 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled] node_7f4006479b20 -> value__node_7f4006479b20__a [label=a arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"] value__node_7f4006479b20__b [label="array([[5, 6], [7, 8]])" color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled] node_7f4006479b20 -> value__node_7f4006479b20__b [label=b arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"] node_7f40064fdee0 [label="t.x" color="#3eb24899" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman bold" penwidth=1.5 shape=ellipse style=filled] node_7f4006479b20 -> node_7f40064fdee0 [label=x arrowhead=vee arrowsize=1.0 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"] node_7f4006514460 -> node_7f40064fdee0 [label=ppp arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"] node_7f40064827c0 [label="t2.x" color="#5b3eb299" fillcolor="#9673ff99" fontcolor="#0d0033ff" fontname="Times-Roman bold" penwidth=1.5 shape=ellipse style=filled] node_7f4006514460 -> node_7f40064827c0 [label=x arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"] value__node_7f40064fdee0__c [label=3 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled] node_7f40064fdee0 -> value__node_7f40064fdee0__c [label=c arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"] value__node_7f40064fdee0__d [label=4 color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled] node_7f40064fdee0 -> value__node_7f40064fdee0__d [label=d arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"] value__node_7f40064fdee0__e [label="array([[1, 2], [3, 4]])" color="#3eb24800" fillcolor="#73ff7e99" fontcolor="#003304ff" fontname="Times-Roman" penwidth=1.5 shape=box style=filled] node_7f40064fdee0 -> value__node_7f40064fdee0__e [label=e arrowhead=dot arrowsize=0.5 color="#1f9929cc" fontcolor="#00730aff" fontname="Times-Roman bold"] node_7f40064827c0 -> node_7f4006479b20 [label=t arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"] node_7f40064827c0 -> node_7f40064fdee0 [label=y arrowhead=vee arrowsize=1.0 color="#3d1f99cc" fontcolor="#1d0073ff" fontname="Times-Roman bold"] } |
Note
The return value’s type of function graphics
is class graphviz.dot.Digraph
, from the opensource library graphviz
, for further information of this project and graphviz.dot.Digraph
’s usage, take a look at:
For further information of function graphics
, just take a look at