github/pyan
1
1
Fork 0
mirror of https://github.com/davidfraser/pyan.git synced 2026-01-08 22:07:55 -05:00
Code Issues Packages Projects Releases Wiki Activity

Pyan3: Python 3 support

Browse Source
This commit is contained in:
Juha Jeronen
2017-11-13 23:20:16 +02:00
parent 26faced4ed
commit 864685503a
10 changed files with 2165 additions and 912 deletions
Download Patch File Download Diff File Expand all files Collapse all files

155
README.md
View File

@@ -1,86 +1,107 @@
pyan - Static Analysis of function and method call dependencies
===============================================================
# Pyan3: Offline call graph generator for Python 3
`pyan` is a Python module that performs static analysis of Python code
to determine a call dependency graph between functions and methods.
This is different from running the code and seeing which functions are
called and how often; there are various tools that will generate a call graph
in that way, usually using debugger or profiling trace hooks - for example:
https://pycallgraph.readthedocs.org/
Generate approximate call graphs for Python programs.
This code was originally written by Edmund Horner, and then modified by Juha Jeronen.
See the notes at the end of this file for licensing info, the original blog posts,
and links to their repositories.
Pyan takes one or more Python source files, performs a (rather superficial) static analysis, and constructs a directed graph of the objects in the combined source, and how they define or use each other. The graph can be output for rendering, mainly by GraphViz.
Command-line options
--------------------
*And now it is available for Python 3!*
*Output format* (one of these is required)
[![Example output](graph0.png "Example: GraphViz rendering of Pyan output (click for .svg)")](graph0.svg)
- `--dot` Output to GraphViz
- `--tgf` Output in Trivial Graph Format
**Defines** relations are drawn with *dotted gray arrows*.
*GraphViz only options*
**Uses** relations are drawn with *black solid arrows*.
- Color nodes automatically (`-c` or `--colored`).
A HSL color model is used, picking the hue based on the top-level namespace (effectively, the module).
The colors start out light, and then darken for each level of nesting.
Seven different hues are available, cycled automatically.
- Group nodes in the same namespace (`-g` or `--grouped`, `-e` or `--nested-groups`).
GraphViz clusters are used for this. The namespace name is used as the cluster label.
Groups can be created as standalone (`-g` or `--grouped`, always inside top-level graph)
or nested (`-e` or `--nested-groups`). The nested mode follows the namespace structure of the code.
**Nodes** are always filled, and made translucent to clearly show any arrows passing underneath them. This is especially useful for large graphs with GraphViz's `fdp` filter. If colored output is not enabled, the fill is white.
*Generation options*
In **node coloring**, the [HSL](https://en.wikipedia.org/wiki/HSL_and_HSV) color model is used. The **hue** is determined by the *top-level namespace* the node is in. The **lightness** is determined by *depth of namespace nesting*, with darker meaning more deeply nested. Saturation is constant. The spacing between different hues depends on the number of files analyzed; better results are obtained for fewer files.
- Disable generation of links for “defines” relationships (`-n` or `--no-defines`).
This can make the resulting graph look much clearer, when there are a lot of “uses” relationships.
This is especially useful for layout with `fdp`.
To enable (the default), use `-u` or `--defines`
- Disable generation of links for “uses” relationships (`-N` or `--no-uses`).
Can be useful for visualizing just where functions are defined.
To enable (the default), use `-u` or `--uses`
**Groups** are filled with translucent gray to avoid clashes with any node color.
*General*
The nodes can be **annotated** by *filename and source line number* information.
- `-v` or `--verbose` for verbose output
- `-h` or `--help` for help
## Note
Drawing Style
-------------
The static analysis approach Pyan takes is different from running the code and seeing which functions are called and how often. There are various tools that will generate a call graph that way, usually using a debugger or profiling trace hooks, such as [Python Call Graph](https://pycallgraph.readthedocs.org/).
The “defines” relations are drawn with gray arrows,
so that it’s easier to visually tell them apart from the “uses” relations
when there are a lot of edges of both types in the graph.
Nodes are always filled (white if color disabled), and made translucent to clearly show arrows passing underneath them.
This is useful for large graphs with the fdp filter.
Original blog posts
-------------------
- https://ejrh.wordpress.com/2011/12/23/call-graphs-in-python/
- https://ejrh.wordpress.com/2012/01/31/call-graphs-in-python-part-2/
- https://ejrh.wordpress.com/2012/08/18/coloured-call-graphs/
In Pyan3, the analyzer was ported from `compiler` ([good riddance](https://stackoverflow.com/a/909172)) to a combination of `ast` and `symtable`, and slightly extended.
Original source repositories
----------------------------
# Usage
- Edmund Horner's original code is now best found in his github repository at:
https://github.com/ejrh/ejrh/blob/master/utils/pyan.py.
- Juha Jeronen's repository is at:
https://yousource.it.jyu.fi/jjrandom2/miniprojects/blobs/master/refactoring/
- Daffyd Crosby has also made a repository with both versions, but with two files and no history:
https://github.com/dafyddcrosby/pyan
- Since both original repositories have lots of other software,
I've made this clean version combining their contributions into my own repository just for pyan.
This contains commits filtered out of their original repositories, and reordered into a logical sequence:
https://github.com/davidfraser/pyan
See `pyan --help`.
Licensing
---------
Example:
This code is made available under the GNU GPL, v2. See the LICENSE.md file,
or consult https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html for more information.
`pyan *.py --uses --no-defines --colored --grouped --annotated --dot >myuses.dot`
Then render using your favorite GraphViz filter, mainly `dot` or `fdp`:
`dot -Tsvg myuses.dot >myuses.svg`
#### Troubleshooting
If GraphViz complains about *trouble in init_rank*, try adding `-Gnewrank=true`, as in:
`dot -Gnewrank=true -Tsvg myuses.dot >myuses.svg`
Usually either old or new rank works; this is a long-standing GraphViz issue with complex graphs.
# Features
*Items tagged with ☆ are new in Pyan3.*
**Graph creation**:
- Nodes for functions and classes
- Edges for defines
- Edges for uses
- Grouping to represent defines, with or without nesting
- Coloring of nodes by top-level namespace
- Unlimited number of hues ☆
**Analysis**:
- Name lookup across the given set of files
- Nested function definitions
- Nested class definitions ☆
- Assignment tracking with lexical scoping
- E.g. if `self.a = MyFancyClass()`, the analyzer knows that any references to `self.a` point to `MyFancyClass`
- All binding forms are supported (assign, augassign, for, comprehensions, generator expressions) ☆
- Simple item-by-item tuple assignments like `x,y,z = a,b,c` ☆
- Chained assignments `a = b = c` ☆
- Local scope for lambda, listcomp, setcomp, dictcomp, genexpr ☆
- Keep in mind that list comprehensions gained a local scope (being treated like a function) only in Python 3. Thus, Pyan3, when applied to legacy Python 2 code, will give subtly wrong results if the code uses list comprehensions.
- Source filename and line number annotation ☆
- The annotation is appended to the node label. If grouping is off, namespace is included in the annotation. If grouping is on, only source filename and line number information is included, because the group title already shows the namespace.
## TODO
- This version is currently missing the PRs from [David Fraser's repo](https://github.com/davidfraser/pyan).
The analyzer **does not currently support**:
- Nested attribute accesses like `self.a.b` (will detect as reference to `*.b` of an unknown object `self.a`).
- Tuples/lists as first-class values (will ignore any assignment of a tuple/list to a single name).
- Starred assignment `a,*b,c = d,e,f,g,h` (will detect some item from the RHS).
- Additional unpacking generalizations ([PEP 448](https://www.python.org/dev/peps/pep-0448/), Python 3.5+).
- Type hints ([PEP 484](https://www.python.org/dev/peps/pep-0484/), Python 3.5+).
- Use of `self` is detected by the literal name `self`, not by capturing the name of the first argument of a method definition.
- Async definitions are detected, but passed through to the corresponding non-async analyzers; could be annotated.
- Cython; could strip or comment out Cython-specific code as a preprocess step, then treat as Python (will need to be careful to get line numbers right).
# Authors
Original [pyan.py](https://github.com/ejrh/ejrh/blob/master/utils/pyan.py) by Edmund Horner. [Original post with explanation](http://ejrh.wordpress.com/2012/01/31/call-graphs-in-python-part-2/).
[Coloring and grouping](https://ejrh.wordpress.com/2012/08/18/coloured-call-graphs/) for GraphViz output by Juha Jeronen.
[Git repository cleanup](https://github.com/davidfraser/pyan/) by David Fraser.
This Python 3 port and refactoring to separate modules by Juha Jeronen.
# License
[GPL v2](LICENSE.md), as per [comments here](https://ejrh.wordpress.com/2012/08/18/coloured-call-graphs/).

BIN
graph0.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 106 KiB

747
graph0.svg Normal file
View File

@@ -0,0 +1,747 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 2.36.0 (20140111.2315)
-->
<!-- Title: G Pages: 1 -->
<svg width="2956pt" height="820pt"
viewBox="0.00 0.00 2956.14 820.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 816)">
<title>G</title>
<polygon fill="white" stroke="none" points="-4,4 -4,-816 2952.14,-816 2952.14,4 -4,4"/>
<g id="clust1" class="cluster"><title>cluster_main</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M1338.14,-603C1338.14,-603 1454.14,-603 1454.14,-603 1460.14,-603 1466.14,-609 1466.14,-615 1466.14,-615 1466.14,-684 1466.14,-684 1466.14,-690 1460.14,-696 1454.14,-696 1454.14,-696 1338.14,-696 1338.14,-696 1332.14,-696 1326.14,-690 1326.14,-684 1326.14,-684 1326.14,-615 1326.14,-615 1326.14,-609 1332.14,-603 1338.14,-603"/>
<text text-anchor="middle" x="1396.14" y="-680.8" font-family="Times,serif" font-size="14.00">main</text>
</g>
<g id="clust2" class="cluster"><title>cluster_model</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M225.135,-300C225.135,-300 359.135,-300 359.135,-300 365.135,-300 371.135,-306 371.135,-312 371.135,-312 371.135,-482 371.135,-482 371.135,-488 365.135,-494 359.135,-494 359.135,-494 225.135,-494 225.135,-494 219.135,-494 213.135,-488 213.135,-482 213.135,-482 213.135,-312 213.135,-312 213.135,-306 219.135,-300 225.135,-300"/>
<text text-anchor="middle" x="292.135" y="-478.8" font-family="Times,serif" font-size="14.00">model</text>
</g>
<g id="clust3" class="cluster"><title>cluster_model__Model</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M64.1352,-8C64.1352,-8 673.135,-8 673.135,-8 679.135,-8 685.135,-14 685.135,-20 685.135,-20 685.135,-280 685.135,-280 685.135,-286 679.135,-292 673.135,-292 673.135,-292 64.1352,-292 64.1352,-292 58.1352,-292 52.1352,-286 52.1352,-280 52.1352,-280 52.1352,-20 52.1352,-20 52.1352,-14 58.1352,-8 64.1352,-8"/>
<text text-anchor="middle" x="368.635" y="-276.8" font-family="Times,serif" font-size="14.00">model.Model</text>
</g>
<g id="clust4" class="cluster"><title>cluster_modelbase</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M993.135,-199C993.135,-199 1153.14,-199 1153.14,-199 1159.14,-199 1165.14,-205 1165.14,-211 1165.14,-211 1165.14,-280 1165.14,-280 1165.14,-286 1159.14,-292 1153.14,-292 1153.14,-292 993.135,-292 993.135,-292 987.135,-292 981.135,-286 981.135,-280 981.135,-280 981.135,-211 981.135,-211 981.135,-205 987.135,-199 993.135,-199"/>
<text text-anchor="middle" x="1073.14" y="-276.8" font-family="Times,serif" font-size="14.00">modelbase</text>
</g>
<g id="clust5" class="cluster"><title>cluster_modelbase__ModelBase</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M705.135,-98C705.135,-98 1423.14,-98 1423.14,-98 1429.14,-98 1435.14,-104 1435.14,-110 1435.14,-110 1435.14,-179 1435.14,-179 1435.14,-185 1429.14,-191 1423.14,-191 1423.14,-191 705.135,-191 705.135,-191 699.135,-191 693.135,-185 693.135,-179 693.135,-179 693.135,-110 693.135,-110 693.135,-104 699.135,-98 705.135,-98"/>
<text text-anchor="middle" x="1064.14" y="-175.8" font-family="Times,serif" font-size="14.00">modelbase.ModelBase</text>
</g>
<g id="clust6" class="cluster"><title>cluster_stage1</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M1056.14,-502C1056.14,-502 1190.14,-502 1190.14,-502 1196.14,-502 1202.14,-508 1202.14,-514 1202.14,-514 1202.14,-684 1202.14,-684 1202.14,-690 1196.14,-696 1190.14,-696 1190.14,-696 1056.14,-696 1056.14,-696 1050.14,-696 1044.14,-690 1044.14,-684 1044.14,-684 1044.14,-514 1044.14,-514 1044.14,-508 1050.14,-502 1056.14,-502"/>
<text text-anchor="middle" x="1123.14" y="-680.8" font-family="Times,serif" font-size="14.00">stage1</text>
</g>
<g id="clust7" class="cluster"><title>cluster_stage1__CodeGenerator</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M909.135,-401C909.135,-401 1187.14,-401 1187.14,-401 1193.14,-401 1199.14,-407 1199.14,-413 1199.14,-413 1199.14,-482 1199.14,-482 1199.14,-488 1193.14,-494 1187.14,-494 1187.14,-494 909.135,-494 909.135,-494 903.135,-494 897.135,-488 897.135,-482 897.135,-482 897.135,-413 897.135,-413 897.135,-407 903.135,-401 909.135,-401"/>
<text text-anchor="middle" x="1048.14" y="-478.8" font-family="Times,serif" font-size="14.00">stage1.CodeGenerator</text>
</g>
<g id="clust8" class="cluster"><title>cluster_stage1__CodeGenerator__run</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M908.135,-300C908.135,-300 1043.14,-300 1043.14,-300 1049.14,-300 1055.14,-306 1055.14,-312 1055.14,-312 1055.14,-381 1055.14,-381 1055.14,-387 1049.14,-393 1043.14,-393 1043.14,-393 908.135,-393 908.135,-393 902.135,-393 896.135,-387 896.135,-381 896.135,-381 896.135,-312 896.135,-312 896.135,-306 902.135,-300 908.135,-300"/>
<text text-anchor="middle" x="975.635" y="-377.8" font-family="Times,serif" font-size="14.00">stage1.CodeGenerator.run</text>
</g>
<g id="clust9" class="cluster"><title>cluster_stage2</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M1518.14,-502C1518.14,-502 1652.14,-502 1652.14,-502 1658.14,-502 1664.14,-508 1664.14,-514 1664.14,-514 1664.14,-684 1664.14,-684 1664.14,-690 1658.14,-696 1652.14,-696 1652.14,-696 1518.14,-696 1518.14,-696 1512.14,-696 1506.14,-690 1506.14,-684 1506.14,-684 1506.14,-514 1506.14,-514 1506.14,-508 1512.14,-502 1518.14,-502"/>
<text text-anchor="middle" x="1585.14" y="-680.8" font-family="Times,serif" font-size="14.00">stage2</text>
</g>
<g id="clust10" class="cluster"><title>cluster_stage2__CodeGenerator</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M1455.14,-98C1455.14,-98 2146.14,-98 2146.14,-98 2152.14,-98 2158.14,-104 2158.14,-110 2158.14,-110 2158.14,-482 2158.14,-482 2158.14,-488 2152.14,-494 2146.14,-494 2146.14,-494 1455.14,-494 1455.14,-494 1449.14,-494 1443.14,-488 1443.14,-482 1443.14,-482 1443.14,-110 1443.14,-110 1443.14,-104 1449.14,-98 1455.14,-98"/>
<text text-anchor="middle" x="1800.64" y="-478.8" font-family="Times,serif" font-size="14.00">stage2.CodeGenerator</text>
</g>
<g id="clust11" class="cluster"><title>cluster_stage2__CodeGenerator__analyze_interface</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M2440.14,-199C2440.14,-199 2928.14,-199 2928.14,-199 2934.14,-199 2940.14,-205 2940.14,-211 2940.14,-211 2940.14,-280 2940.14,-280 2940.14,-286 2934.14,-292 2928.14,-292 2928.14,-292 2440.14,-292 2440.14,-292 2434.14,-292 2428.14,-286 2428.14,-280 2428.14,-280 2428.14,-211 2428.14,-211 2428.14,-205 2434.14,-199 2440.14,-199"/>
<text text-anchor="middle" x="2684.14" y="-276.8" font-family="Times,serif" font-size="14.00">stage2.CodeGenerator.analyze_interface</text>
</g>
<g id="clust12" class="cluster"><title>cluster_stage2__CodeGenerator__run</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M2178.14,-300C2178.14,-300 2482.14,-300 2482.14,-300 2488.14,-300 2494.14,-306 2494.14,-312 2494.14,-312 2494.14,-381 2494.14,-381 2494.14,-387 2488.14,-393 2482.14,-393 2482.14,-393 2178.14,-393 2178.14,-393 2172.14,-393 2166.14,-387 2166.14,-381 2166.14,-381 2166.14,-312 2166.14,-312 2166.14,-306 2172.14,-300 2178.14,-300"/>
<text text-anchor="middle" x="2330.14" y="-377.8" font-family="Times,serif" font-size="14.00">stage2.CodeGenerator.run</text>
</g>
<g id="clust13" class="cluster"><title>cluster_stage2__CodeGenerator__validate_bound_args</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M2178.14,-98C2178.14,-98 2408.14,-98 2408.14,-98 2414.14,-98 2420.14,-104 2420.14,-110 2420.14,-110 2420.14,-280 2420.14,-280 2420.14,-286 2414.14,-292 2408.14,-292 2408.14,-292 2178.14,-292 2178.14,-292 2172.14,-292 2166.14,-286 2166.14,-280 2166.14,-280 2166.14,-110 2166.14,-110 2166.14,-104 2172.14,-98 2178.14,-98"/>
<text text-anchor="middle" x="2293.14" y="-276.8" font-family="Times,serif" font-size="14.00">stage2.CodeGenerator.validate_bound_args</text>
</g>
<g id="clust14" class="cluster"><title>cluster_stage2__main</title>
<path fill="#808080" fill-opacity="0.094118" stroke="black" d="M1752.14,-502C1752.14,-502 2047.14,-502 2047.14,-502 2053.14,-502 2059.14,-508 2059.14,-514 2059.14,-514 2059.14,-583 2059.14,-583 2059.14,-589 2053.14,-595 2047.14,-595 2047.14,-595 1752.14,-595 1752.14,-595 1746.14,-595 1740.14,-589 1740.14,-583 1740.14,-583 1740.14,-514 1740.14,-514 1740.14,-508 1746.14,-502 1752.14,-502"/>
<text text-anchor="middle" x="1899.64" y="-579.8" font-family="Times,serif" font-size="14.00">stage2.main</text>
</g>
<!-- main -->
<g id="node1" class="node"><title>main</title>
<ellipse fill="#ffffff" fill-opacity="0.698039" stroke="black" cx="1353.14" cy="-794" rx="28.3955" ry="18"/>
<text text-anchor="middle" x="1353.14" y="-790.3" font-family="Times,serif" font-size="14.00" fill="#000000">main</text>
</g>
<!-- stage1 -->
<g id="node4" class="node"><title>stage1</title>
<ellipse fill="#ffffff" fill-opacity="0.698039" stroke="black" cx="1283.14" cy="-722" rx="33.2209" ry="18"/>
<text text-anchor="middle" x="1283.14" y="-718.3" font-family="Times,serif" font-size="14.00" fill="#000000">stage1</text>
</g>
<!-- main&#45;&gt;stage1 -->
<g id="edge56" class="edge"><title>main&#45;&gt;stage1</title>
<path fill="none" stroke="black" d="M1338.63,-778.496C1329.13,-768.995 1316.52,-756.383 1305.72,-745.583"/>
<polygon fill="black" stroke="black" points="1308.14,-743.053 1298.59,-738.457 1303.19,-748.003 1308.14,-743.053"/>
</g>
<!-- stage2 -->
<g id="node5" class="node"><title>stage2</title>
<ellipse fill="#ffffff" fill-opacity="0.698039" stroke="black" cx="1423.14" cy="-722" rx="33.2209" ry="18"/>
<text text-anchor="middle" x="1423.14" y="-718.3" font-family="Times,serif" font-size="14.00" fill="#000000">stage2</text>
</g>
<!-- main&#45;&gt;stage2 -->
<g id="edge58" class="edge"><title>main&#45;&gt;stage2</title>
<path fill="none" stroke="black" d="M1367.64,-778.496C1377.14,-768.995 1389.75,-756.383 1400.55,-745.583"/>
<polygon fill="black" stroke="black" points="1403.08,-748.003 1407.68,-738.457 1398.13,-743.053 1403.08,-748.003"/>
</g>
<!-- main__main -->
<g id="node6" class="node"><title>main__main</title>
<ellipse fill="#fef7cc" fill-opacity="0.698039" stroke="black" cx="1396.14" cy="-638" rx="62.3385" ry="26.7407"/>
<text text-anchor="middle" x="1396.14" y="-641.8" font-family="Times,serif" font-size="14.00" fill="#000000">main</text>
<text text-anchor="middle" x="1396.14" y="-626.8" font-family="Times,serif" font-size="14.00" fill="#000000">(main.py:47)</text>
</g>
<!-- main&#45;&gt;main__main -->
<g id="edge30" class="edge"><title>main&#45;&gt;main__main</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1353.47,-775.931C1357.73,-751.552 1369.95,-706.196 1380.47,-674.386"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1383.9,-675.172 1383.82,-664.577 1377.28,-672.912 1383.9,-675.172"/>
</g>
<!-- main&#45;&gt;main__main -->
<g id="edge57" class="edge"><title>main&#45;&gt;main__main</title>
<path fill="none" stroke="black" d="M1361.99,-776.81C1371.32,-752.77 1384.58,-707.016 1391.93,-674.841"/>
<polygon fill="black" stroke="black" points="1395.39,-675.433 1394.1,-664.917 1388.55,-673.94 1395.39,-675.433"/>
</g>
<!-- model -->
<g id="node2" class="node"><title>model</title>
<ellipse fill="#ffffff" fill-opacity="0.698039" stroke="black" cx="1283.14" cy="-638" rx="33.2209" ry="18"/>
<text text-anchor="middle" x="1283.14" y="-634.3" font-family="Times,serif" font-size="14.00" fill="#000000">model</text>
</g>
<!-- modelbase -->
<g id="node3" class="node"><title>modelbase</title>
<ellipse fill="#ffffff" fill-opacity="0.698039" stroke="black" cx="1283.14" cy="-537" rx="48.6179" ry="18"/>
<text text-anchor="middle" x="1283.14" y="-533.3" font-family="Times,serif" font-size="14.00" fill="#000000">modelbase</text>
</g>
<!-- model&#45;&gt;modelbase -->
<g id="edge69" class="edge"><title>model&#45;&gt;modelbase</title>
<path fill="none" stroke="black" d="M1283.14,-619.756C1283.14,-604.708 1283.14,-582.616 1283.14,-565.19"/>
<polygon fill="black" stroke="black" points="1286.64,-565.047 1283.14,-555.047 1279.64,-565.047 1286.64,-565.047"/>
</g>
<!-- model__Model -->
<g id="node7" class="node"><title>model__Model</title>
<ellipse fill="#dafecc" fill-opacity="0.698039" stroke="black" cx="292.135" cy="-335" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="292.135" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">Model</text>
<text text-anchor="middle" x="292.135" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:32)</text>
</g>
<!-- model&#45;&gt;model__Model -->
<g id="edge2" class="edge"><title>model&#45;&gt;model__Model</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1261.04,-624.374C1246.07,-616.553 1225.56,-607.221 1206.14,-603 1135.34,-587.611 625.775,-605.806 554.135,-595 397.418,-571.36 307.016,-620.952 212.135,-494 187.391,-460.892 194.371,-438.321 212.135,-401 219.332,-385.879 231.957,-373.128 245.08,-363.034"/>
<polygon fill="#838b8b" stroke="#838b8b" points="247.23,-365.798 253.271,-357.095 243.121,-360.131 247.23,-365.798"/>
</g>
<!-- model__main -->
<g id="node8" class="node"><title>model__main</title>
<ellipse fill="#dafecc" fill-opacity="0.698039" stroke="black" cx="292.135" cy="-436" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="292.135" y="-439.8" font-family="Times,serif" font-size="14.00" fill="#000000">main</text>
<text text-anchor="middle" x="292.135" y="-424.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:368)</text>
</g>
<!-- model&#45;&gt;model__main -->
<g id="edge1" class="edge"><title>model&#45;&gt;model__main</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1257.22,-626.443C1240.33,-618.416 1218.04,-607.939 1197.14,-603 1125.25,-586.018 1104.39,-604.441 1031.14,-595 778.966,-562.503 485.355,-490.17 357.293,-455.815"/>
<polygon fill="#838b8b" stroke="#838b8b" points="357.823,-452.333 347.257,-453.109 356.001,-459.092 357.823,-452.333"/>
</g>
<!-- model&#45;&gt;model__main -->
<g id="edge68" class="edge"><title>model&#45;&gt;model__main</title>
<path fill="none" stroke="black" d="M1265.2,-622.702C1251.69,-615.404 1232.91,-607.2 1215.14,-603 1143.25,-586.018 1122.39,-604.441 1049.14,-595 792.544,-561.933 493.043,-487.621 360.763,-454.043"/>
<polygon fill="black" stroke="black" points="361.323,-450.574 350.769,-451.503 359.598,-457.359 361.323,-450.574"/>
</g>
<!-- modelbase__ModelBase -->
<g id="node15" class="node"><title>modelbase__ModelBase</title>
<ellipse fill="#ccfee9" fill-opacity="0.698039" stroke="black" cx="1073.14" cy="-234" rx="84.2187" ry="26.7407"/>
<text text-anchor="middle" x="1073.14" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">ModelBase</text>
<text text-anchor="middle" x="1073.14" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(modelbase.py:16)</text>
</g>
<!-- modelbase&#45;&gt;modelbase__ModelBase -->
<g id="edge11" class="edge"><title>modelbase&#45;&gt;modelbase__ModelBase</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1273.96,-518.941C1259.75,-492.897 1231.12,-441.932 1203.14,-401 1170.38,-353.085 1128.33,-300.842 1101.06,-268.014"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1103.56,-265.539 1094.46,-260.101 1098.18,-270.022 1103.56,-265.539"/>
</g>
<!-- stage1&#45;&gt;model -->
<g id="edge59" class="edge"><title>stage1&#45;&gt;model</title>
<path fill="none" stroke="black" d="M1283.14,-703.61C1283.14,-692.774 1283.14,-678.601 1283.14,-666.291"/>
<polygon fill="black" stroke="black" points="1286.64,-666.084 1283.14,-656.084 1279.64,-666.084 1286.64,-666.084"/>
</g>
<!-- stage1__CodeGenerator -->
<g id="node20" class="node"><title>stage1__CodeGenerator</title>
<ellipse fill="#cce9fe" fill-opacity="0.698039" stroke="black" cx="1124.14" cy="-537" rx="70.0054" ry="26.7407"/>
<text text-anchor="middle" x="1124.14" y="-540.8" font-family="Times,serif" font-size="14.00" fill="#000000">CodeGenerator</text>
<text text-anchor="middle" x="1124.14" y="-525.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage1.py:27)</text>
</g>
<!-- stage1&#45;&gt;stage1__CodeGenerator -->
<g id="edge32" class="edge"><title>stage1&#45;&gt;stage1__CodeGenerator</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1258.47,-709.884C1252.2,-706.166 1245.9,-701.53 1241.14,-696 1212,-662.165 1229.62,-638.944 1203.14,-603 1193.1,-589.384 1179.6,-576.903 1166.6,-566.597"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1168.67,-563.775 1158.6,-560.47 1164.41,-569.331 1168.67,-563.775"/>
</g>
<!-- stage1__main -->
<g id="node21" class="node"><title>stage1__main</title>
<ellipse fill="#cce9fe" fill-opacity="0.698039" stroke="black" cx="1123.14" cy="-638" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="1123.14" y="-641.8" font-family="Times,serif" font-size="14.00" fill="#000000">main</text>
<text text-anchor="middle" x="1123.14" y="-626.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage1.py:152)</text>
</g>
<!-- stage1&#45;&gt;stage1__main -->
<g id="edge31" class="edge"><title>stage1&#45;&gt;stage1__main</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1252.54,-715.091C1236.01,-710.765 1215.89,-704.224 1199.14,-696 1183.96,-688.551 1168.32,-678.765 1155.12,-669.407"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1156.97,-666.423 1146.83,-663.349 1152.84,-672.075 1156.97,-666.423"/>
</g>
<!-- stage1&#45;&gt;stage1__main -->
<g id="edge60" class="edge"><title>stage1&#45;&gt;stage1__main</title>
<path fill="none" stroke="black" d="M1256.08,-711.32C1243.88,-707.287 1229.58,-702.107 1217.14,-696 1200.6,-687.886 1183.52,-676.998 1168.45,-666.912"/>
<polygon fill="black" stroke="black" points="1170.08,-663.786 1159.83,-661.093 1166.16,-669.588 1170.08,-663.786"/>
</g>
<!-- stage2__CodeGenerator -->
<g id="node25" class="node"><title>stage2__CodeGenerator</title>
<ellipse fill="#daccfe" fill-opacity="0.698039" stroke="black" cx="1584.14" cy="-537" rx="70.0054" ry="26.7407"/>
<text text-anchor="middle" x="1584.14" y="-540.8" font-family="Times,serif" font-size="14.00" fill="#000000">CodeGenerator</text>
<text text-anchor="middle" x="1584.14" y="-525.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:42)</text>
</g>
<!-- stage2&#45;&gt;stage2__CodeGenerator -->
<g id="edge4" class="edge"><title>stage2&#45;&gt;stage2__CodeGenerator</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1449.87,-710.979C1457.21,-707.193 1464.64,-702.248 1470.14,-696 1499.29,-662.825 1479.43,-638.915 1505.14,-603 1514.89,-589.364 1528.2,-576.946 1541.11,-566.701"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1543.25,-569.471 1549.06,-560.612 1538.99,-563.914 1543.25,-569.471"/>
</g>
<!-- stage2__main -->
<g id="node26" class="node"><title>stage2__main</title>
<ellipse fill="#daccfe" fill-opacity="0.698039" stroke="black" cx="1585.14" cy="-638" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="1585.14" y="-641.8" font-family="Times,serif" font-size="14.00" fill="#000000">main</text>
<text text-anchor="middle" x="1585.14" y="-626.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:499)</text>
</g>
<!-- stage2&#45;&gt;stage2__main -->
<g id="edge3" class="edge"><title>stage2&#45;&gt;stage2__main</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1450.4,-711.664C1463.11,-707.61 1478.11,-702.327 1491.14,-696 1507.7,-687.951 1524.79,-677.074 1539.85,-666.981"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1542.15,-669.655 1548.47,-661.155 1538.23,-663.855 1542.15,-669.655"/>
</g>
<!-- stage2&#45;&gt;stage2__main -->
<g id="edge61" class="edge"><title>stage2&#45;&gt;stage2__main</title>
<path fill="none" stroke="black" d="M1453.98,-715.272C1470.98,-710.984 1491.82,-704.416 1509.14,-696 1524.34,-688.611 1539.99,-678.839 1553.18,-669.478"/>
<polygon fill="black" stroke="black" points="1555.47,-672.143 1561.47,-663.414 1551.33,-666.494 1555.47,-672.143"/>
</g>
<!-- main__main&#45;&gt;stage1__CodeGenerator -->
<g id="edge48" class="edge"><title>main__main&#45;&gt;stage1__CodeGenerator</title>
<path fill="none" stroke="black" d="M1359.17,-616.156C1348.49,-611.011 1336.62,-606.094 1325.14,-603 1282.08,-591.399 1267.77,-608.04 1225.14,-595 1204.31,-588.631 1183.06,-577.471 1165.45,-566.704"/>
<polygon fill="black" stroke="black" points="1166.93,-563.498 1156.6,-561.137 1163.2,-569.423 1166.93,-563.498"/>
</g>
<!-- stage1__CodeGenerator__run -->
<g id="node23" class="node"><title>stage1__CodeGenerator__run</title>
<ellipse fill="#99d3ff" fill-opacity="0.698039" stroke="black" cx="972.135" cy="-436" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="972.135" y="-439.8" font-family="Times,serif" font-size="14.00" fill="#000000">run</text>
<text text-anchor="middle" x="972.135" y="-424.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage1.py:32)</text>
</g>
<!-- main__main&#45;&gt;stage1__CodeGenerator__run -->
<g id="edge46" class="edge"><title>main__main&#45;&gt;stage1__CodeGenerator__run</title>
<path fill="none" stroke="black" d="M1393.62,-610.922C1389.01,-579.172 1376.09,-526.732 1340.14,-502 1313.39,-483.607 1078.78,-504.684 1048.14,-494 1032.17,-488.433 1016.8,-478.181 1004.11,-467.89"/>
<polygon fill="black" stroke="black" points="1006.07,-464.965 996.178,-461.17 1001.54,-470.305 1006.07,-464.965"/>
</g>
<!-- main__main&#45;&gt;stage2__CodeGenerator -->
<g id="edge45" class="edge"><title>main__main&#45;&gt;stage2__CodeGenerator</title>
<path fill="none" stroke="black" d="M1434.63,-616.727C1463.67,-601.435 1503.72,-580.346 1535.06,-563.843"/>
<polygon fill="black" stroke="black" points="1536.85,-566.856 1544.07,-559.1 1533.59,-560.663 1536.85,-566.856"/>
</g>
<!-- stage2__CodeGenerator__run -->
<g id="node32" class="node"><title>stage2__CodeGenerator__run</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="2079.14" cy="-436" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="2079.14" y="-439.8" font-family="Times,serif" font-size="14.00" fill="#000000">run</text>
<text text-anchor="middle" x="2079.14" y="-424.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:322)</text>
</g>
<!-- main__main&#45;&gt;stage2__CodeGenerator__run -->
<g id="edge47" class="edge"><title>main__main&#45;&gt;stage2__CodeGenerator__run</title>
<path fill="none" stroke="black" d="M1441.37,-619.53C1459.92,-613.15 1481.77,-606.617 1502.14,-603 1523.38,-599.227 1678.37,-607.233 1696.14,-595 1733.2,-569.486 1698.94,-527.315 1736.14,-502 1760.21,-485.614 1970.51,-503.213 1998.14,-494 2015.02,-488.369 2031.49,-477.972 2045.13,-467.584"/>
<polygon fill="black" stroke="black" points="2047.57,-470.116 2053.23,-461.16 2043.22,-464.632 2047.57,-470.116"/>
</g>
<!-- model__Model____init__ -->
<g id="node9" class="node"><title>model__Model____init__</title>
<ellipse fill="#b6ff99" fill-opacity="0.698039" stroke="black" cx="610.135" cy="-234" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="610.135" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">__init__</text>
<text text-anchor="middle" x="610.135" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:35)</text>
</g>
<!-- model__Model&#45;&gt;model__Model____init__ -->
<g id="edge10" class="edge"><title>model__Model&#45;&gt;model__Model____init__</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M358.4,-330.705C408.175,-326.297 477.117,-316.008 533.135,-292 548.435,-285.443 563.54,-275.302 576.264,-265.38"/>
<polygon fill="#838b8b" stroke="#838b8b" points="578.673,-267.934 584.26,-258.932 574.279,-262.485 578.673,-267.934"/>
</g>
<!-- model__Model__build_φ -->
<g id="node10" class="node"><title>model__Model__build_φ</title>
<ellipse fill="#b6ff99" fill-opacity="0.698039" stroke="black" cx="129.135" cy="-43" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="129.135" y="-46.8" font-family="Times,serif" font-size="14.00" fill="#000000">build_φ</text>
<text text-anchor="middle" x="129.135" y="-31.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:85)</text>
</g>
<!-- model__Model&#45;&gt;model__Model__build_φ -->
<g id="edge5" class="edge"><title>model__Model&#45;&gt;model__Model__build_φ</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M225.009,-332.738C156.097,-329.855 55.5785,-320.691 31.1352,-292 -25.0769,-226.019 3.60601,-170.486 51.1352,-98 58.2614,-87.132 68.4049,-77.8334 79.0287,-70.1692"/>
<polygon fill="#838b8b" stroke="#838b8b" points="81.2777,-72.8737 87.6033,-64.3744 77.3581,-67.0739 81.2777,-72.8737"/>
</g>
<!-- model__Model__define_api -->
<g id="node11" class="node"><title>model__Model__define_api</title>
<ellipse fill="#b6ff99" fill-opacity="0.698039" stroke="black" cx="292.135" cy="-234" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="292.135" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">define_api</text>
<text text-anchor="middle" x="292.135" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:142)</text>
</g>
<!-- model__Model&#45;&gt;model__Model__define_api -->
<g id="edge6" class="edge"><title>model__Model&#45;&gt;model__Model__define_api</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M292.135,-307.989C292.135,-296.823 292.135,-283.581 292.135,-271.462"/>
<polygon fill="#838b8b" stroke="#838b8b" points="295.635,-271.181 292.135,-261.181 288.635,-271.181 295.635,-271.181"/>
</g>
<!-- model__Model__define_helpers -->
<g id="node12" class="node"><title>model__Model__define_helpers</title>
<ellipse fill="#b6ff99" fill-opacity="0.698039" stroke="black" cx="453.135" cy="-234" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="453.135" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">define_helpers</text>
<text text-anchor="middle" x="453.135" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:258)</text>
</g>
<!-- model__Model&#45;&gt;model__Model__define_helpers -->
<g id="edge7" class="edge"><title>model__Model&#45;&gt;model__Model__define_helpers</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M333.402,-313.636C346.023,-307.063 359.819,-299.523 372.135,-292 385.979,-283.544 400.618,-273.58 413.588,-264.368"/>
<polygon fill="#838b8b" stroke="#838b8b" points="415.759,-267.118 421.845,-258.446 411.679,-261.43 415.759,-267.118"/>
</g>
<!-- model__Model__dφdq -->
<g id="node13" class="node"><title>model__Model__dφdq</title>
<ellipse fill="#b6ff99" fill-opacity="0.698039" stroke="black" cx="131.135" cy="-133" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="131.135" y="-136.8" font-family="Times,serif" font-size="14.00" fill="#000000">dφdq</text>
<text text-anchor="middle" x="131.135" y="-121.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:197)</text>
</g>
<!-- model__Model&#45;&gt;model__Model__dφdq -->
<g id="edge8" class="edge"><title>model__Model&#45;&gt;model__Model__dφdq</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M225.403,-331.7C161.952,-327.942 73.0351,-318.047 51.1352,-292 24.5351,-260.363 33.3713,-236.321 51.1352,-199 58.1382,-184.287 70.2803,-171.817 83.0198,-161.856"/>
<polygon fill="#838b8b" stroke="#838b8b" points="85.4453,-164.417 91.4297,-155.674 81.2993,-158.777 85.4453,-164.417"/>
</g>
<!-- model__Model__simplify -->
<g id="node14" class="node"><title>model__Model__simplify</title>
<ellipse fill="#b6ff99" fill-opacity="0.698039" stroke="black" cx="131.135" cy="-234" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="131.135" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">simplify</text>
<text text-anchor="middle" x="131.135" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(model.py:333)</text>
</g>
<!-- model__Model&#45;&gt;model__Model__simplify -->
<g id="edge9" class="edge"><title>model__Model&#45;&gt;model__Model__simplify</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M250.869,-313.636C238.247,-307.063 224.451,-299.523 212.135,-292 198.292,-283.544 183.652,-273.58 170.682,-264.368"/>
<polygon fill="#838b8b" stroke="#838b8b" points="172.592,-261.43 162.426,-258.446 168.512,-267.118 172.592,-261.43"/>
</g>
<!-- model__Model&#45;&gt;modelbase__ModelBase -->
<g id="edge70" class="edge"><title>model__Model&#45;&gt;modelbase__ModelBase</title>
<path fill="none" stroke="black" d="M357.217,-327.96C435.937,-320.459 572.423,-306.829 689.135,-292 792.634,-278.85 911.372,-260.761 989.076,-248.515"/>
<polygon fill="black" stroke="black" points="989.957,-251.919 999.289,-246.902 988.865,-245.005 989.957,-251.919"/>
</g>
<!-- model__main&#45;&gt;model__Model -->
<g id="edge40" class="edge"><title>model__main&#45;&gt;model__Model</title>
<path fill="none" stroke="black" d="M292.135,-408.989C292.135,-397.823 292.135,-384.581 292.135,-372.462"/>
<polygon fill="black" stroke="black" points="295.635,-372.181 292.135,-362.181 288.635,-372.181 295.635,-372.181"/>
</g>
<!-- modelbase__ModelBase__define_api -->
<g id="node17" class="node"><title>modelbase__ModelBase__define_api</title>
<ellipse fill="#99ffd3" fill-opacity="0.698039" stroke="black" cx="1157.14" cy="-133" rx="84.2187" ry="26.7407"/>
<text text-anchor="middle" x="1157.14" y="-136.8" font-family="Times,serif" font-size="14.00" fill="#000000">define_api</text>
<text text-anchor="middle" x="1157.14" y="-121.8" font-family="Times,serif" font-size="14.00" fill="#000000">(modelbase.py:24)</text>
</g>
<!-- model__main&#45;&gt;modelbase__ModelBase__define_api -->
<g id="edge41" class="edge"><title>model__main&#45;&gt;modelbase__ModelBase__define_api</title>
<path fill="none" stroke="black" d="M361.206,-429.212C416.195,-423.417 494.685,-412.478 561.135,-393 667.881,-361.711 689.42,-339.224 790.135,-292 874.177,-252.594 887.842,-224.349 977.135,-199 1014.49,-188.396 1027.29,-203.243 1064.14,-191 1082.79,-184.803 1101.6,-174.263 1117.38,-163.92"/>
<polygon fill="black" stroke="black" points="1119.48,-166.72 1125.81,-158.222 1115.57,-160.92 1119.48,-166.72"/>
</g>
<!-- modelbase__ModelBase__define_helpers -->
<g id="node18" class="node"><title>modelbase__ModelBase__define_helpers</title>
<ellipse fill="#99ffd3" fill-opacity="0.698039" stroke="black" cx="971.135" cy="-133" rx="84.2187" ry="26.7407"/>
<text text-anchor="middle" x="971.135" y="-136.8" font-family="Times,serif" font-size="14.00" fill="#000000">define_helpers</text>
<text text-anchor="middle" x="971.135" y="-121.8" font-family="Times,serif" font-size="14.00" fill="#000000">(modelbase.py:36)</text>
</g>
<!-- model__main&#45;&gt;modelbase__ModelBase__define_helpers -->
<g id="edge39" class="edge"><title>model__main&#45;&gt;modelbase__ModelBase__define_helpers</title>
<path fill="none" stroke="black" d="M346.08,-418.324C423.363,-393.818 569.474,-345.188 689.135,-292 776.167,-253.316 795.497,-238.35 878.135,-191 893.793,-182.028 910.681,-171.837 925.713,-162.578"/>
<polygon fill="black" stroke="black" points="927.663,-165.488 934.326,-157.25 923.981,-159.534 927.663,-165.488"/>
</g>
<!-- model__Model__define_api&#45;&gt;model__Model__dφdq -->
<g id="edge65" class="edge"><title>model__Model__define_api&#45;&gt;model__Model__dφdq</title>
<path fill="none" stroke="black" d="M256.004,-210.782C232.368,-196.249 201.311,-177.151 176.083,-161.639"/>
<polygon fill="black" stroke="black" points="177.776,-158.571 167.424,-156.314 174.109,-164.534 177.776,-158.571"/>
</g>
<!-- modelbase__ModelBase__simplify -->
<g id="node19" class="node"><title>modelbase__ModelBase__simplify</title>
<ellipse fill="#99ffd3" fill-opacity="0.698039" stroke="black" cx="785.135" cy="-133" rx="84.2187" ry="26.7407"/>
<text text-anchor="middle" x="785.135" y="-136.8" font-family="Times,serif" font-size="14.00" fill="#000000">simplify</text>
<text text-anchor="middle" x="785.135" y="-121.8" font-family="Times,serif" font-size="14.00" fill="#000000">(modelbase.py:53)</text>
</g>
<!-- model__Model__define_helpers&#45;&gt;modelbase__ModelBase__simplify -->
<g id="edge66" class="edge"><title>model__Model__define_helpers&#45;&gt;modelbase__ModelBase__simplify</title>
<path fill="none" stroke="black" d="M495.097,-212.151C507.391,-206.939 521.051,-201.988 534.135,-199 601.385,-183.644 623.108,-210.969 689.135,-191 708.703,-185.082 728.442,-174.365 744.89,-163.809"/>
<polygon fill="black" stroke="black" points="747.264,-166.434 753.666,-157.992 743.397,-160.599 747.264,-166.434"/>
</g>
<!-- model__Model__dφdq&#45;&gt;model__Model__build_φ -->
<g id="edge67" class="edge"><title>model__Model__dφdq&#45;&gt;model__Model__build_φ</title>
<path fill="none" stroke="black" d="M130.546,-106.073C130.362,-97.9993 130.157,-88.9428 129.96,-80.2961"/>
<polygon fill="black" stroke="black" points="133.454,-79.9636 129.727,-70.0457 126.455,-80.1227 133.454,-79.9636"/>
</g>
<!-- modelbase__ModelBase____init__ -->
<g id="node16" class="node"><title>modelbase__ModelBase____init__</title>
<ellipse fill="#99ffd3" fill-opacity="0.698039" stroke="black" cx="1343.14" cy="-133" rx="84.2187" ry="26.7407"/>
<text text-anchor="middle" x="1343.14" y="-136.8" font-family="Times,serif" font-size="14.00" fill="#000000">__init__</text>
<text text-anchor="middle" x="1343.14" y="-121.8" font-family="Times,serif" font-size="14.00" fill="#000000">(modelbase.py:20)</text>
</g>
<!-- modelbase__ModelBase&#45;&gt;modelbase__ModelBase____init__ -->
<g id="edge16" class="edge"><title>modelbase__ModelBase&#45;&gt;modelbase__ModelBase____init__</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1147.11,-220.892C1179.47,-214.22 1217.44,-204.494 1250.14,-191 1267.69,-183.753 1285.77,-173.433 1301.27,-163.562"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1303.6,-166.22 1310.07,-157.828 1299.78,-160.355 1303.6,-166.22"/>
</g>
<!-- modelbase__ModelBase&#45;&gt;modelbase__ModelBase__define_api -->
<g id="edge18" class="edge"><title>modelbase__ModelBase&#45;&gt;modelbase__ModelBase__define_api</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1094.55,-207.762C1105.02,-195.427 1117.75,-180.422 1128.99,-167.174"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1131.96,-169.081 1135.76,-159.192 1126.62,-164.553 1131.96,-169.081"/>
</g>
<!-- modelbase__ModelBase&#45;&gt;modelbase__ModelBase__define_helpers -->
<g id="edge15" class="edge"><title>modelbase__ModelBase&#45;&gt;modelbase__ModelBase__define_helpers</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1047.66,-208.273C1034.39,-195.398 1018.02,-179.51 1003.83,-165.737"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1006.25,-163.208 996.64,-158.754 1001.38,-168.231 1006.25,-163.208"/>
</g>
<!-- modelbase__ModelBase&#45;&gt;modelbase__ModelBase__simplify -->
<g id="edge17" class="edge"><title>modelbase__ModelBase&#45;&gt;modelbase__ModelBase__simplify</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M996.833,-222.708C960.087,-216.22 915.908,-206.131 878.135,-191 860.501,-183.936 842.407,-173.656 826.921,-163.765"/>
<polygon fill="#838b8b" stroke="#838b8b" points="828.413,-160.559 818.128,-158.014 824.581,-166.417 828.413,-160.559"/>
</g>
<!-- stage1__CodeGenerator____init__ -->
<g id="node22" class="node"><title>stage1__CodeGenerator____init__</title>
<ellipse fill="#99d3ff" fill-opacity="0.698039" stroke="black" cx="1124.14" cy="-436" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="1124.14" y="-439.8" font-family="Times,serif" font-size="14.00" fill="#000000">__init__</text>
<text text-anchor="middle" x="1124.14" y="-424.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage1.py:28)</text>
</g>
<!-- stage1__CodeGenerator&#45;&gt;stage1__CodeGenerator____init__ -->
<g id="edge34" class="edge"><title>stage1__CodeGenerator&#45;&gt;stage1__CodeGenerator____init__</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1124.14,-509.989C1124.14,-498.823 1124.14,-485.581 1124.14,-473.462"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1127.64,-473.181 1124.14,-463.181 1120.64,-473.181 1127.64,-473.181"/>
</g>
<!-- stage1__CodeGenerator&#45;&gt;stage1__CodeGenerator__run -->
<g id="edge33" class="edge"><title>stage1__CodeGenerator&#45;&gt;stage1__CodeGenerator__run</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1083.69,-515.003C1072.02,-508.584 1059.4,-501.287 1048.14,-494 1035.2,-485.63 1021.58,-475.776 1009.5,-466.637"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1011.48,-463.746 1001.41,-460.451 1007.23,-469.306 1011.48,-463.746"/>
</g>
<!-- stage1__main&#45;&gt;stage1__CodeGenerator -->
<g id="edge73" class="edge"><title>stage1__main&#45;&gt;stage1__CodeGenerator</title>
<path fill="none" stroke="black" d="M1123.4,-610.989C1123.51,-599.823 1123.64,-586.581 1123.77,-574.462"/>
<polygon fill="black" stroke="black" points="1127.27,-574.216 1123.87,-564.181 1120.27,-574.145 1127.27,-574.216"/>
</g>
<!-- stage1__main&#45;&gt;stage1__CodeGenerator__run -->
<g id="edge71" class="edge"><title>stage1__main&#45;&gt;stage1__CodeGenerator__run</title>
<path fill="none" stroke="black" d="M1075.59,-617.674C1064.61,-611.611 1053.67,-604.074 1045.14,-595 1012.16,-559.944 991.958,-507.347 981.401,-472.608"/>
<polygon fill="black" stroke="black" points="984.721,-471.49 978.554,-462.876 978.003,-473.456 984.721,-471.49"/>
</g>
<!-- stage1__main&#45;&gt;stage2__CodeGenerator__run -->
<g id="edge72" class="edge"><title>stage1__main&#45;&gt;stage2__CodeGenerator__run</title>
<path fill="none" stroke="black" d="M1174.18,-619.13C1194.72,-612.777 1218.79,-606.369 1241.14,-603 1252.87,-601.231 1658.68,-602.166 1668.14,-595 1702.54,-568.927 1661.83,-528.207 1696.14,-502 1709.47,-491.81 1982.2,-499.277 1998.14,-494 2015.03,-488.403 2031.51,-478.013 2045.14,-467.621"/>
<polygon fill="black" stroke="black" points="2047.58,-470.151 2053.24,-461.193 2043.23,-464.668 2047.58,-470.151"/>
</g>
<!-- stage1__CodeGenerator__run&#45;&gt;model__Model -->
<g id="edge78" class="edge"><title>stage1__CodeGenerator__run&#45;&gt;model__Model</title>
<path fill="none" stroke="black" d="M909.684,-425.908C783.608,-407.553 499.28,-366.158 364.992,-346.607"/>
<polygon fill="black" stroke="black" points="365.207,-343.102 354.807,-345.124 364.199,-350.029 365.207,-343.102"/>
</g>
<!-- stage1__CodeGenerator__run&#45;&gt;modelbase__ModelBase__define_api -->
<g id="edge75" class="edge"><title>stage1__CodeGenerator__run&#45;&gt;modelbase__ModelBase__define_api</title>
<path fill="none" stroke="black" d="M920.723,-418.424C909.388,-412.274 898.789,-404.008 892.135,-393 870.754,-357.626 877.97,-338.83 892.135,-300 912.242,-244.883 925.135,-226.168 977.135,-199 1011.55,-181.019 1027.29,-203.243 1064.14,-191 1082.79,-184.803 1101.6,-174.263 1117.38,-163.92"/>
<polygon fill="black" stroke="black" points="1119.48,-166.72 1125.81,-158.222 1115.57,-160.92 1119.48,-166.72"/>
</g>
<!-- stage1__CodeGenerator__run&#45;&gt;modelbase__ModelBase__define_helpers -->
<g id="edge76" class="edge"><title>stage1__CodeGenerator__run&#45;&gt;modelbase__ModelBase__define_helpers</title>
<path fill="none" stroke="black" d="M920.723,-418.424C909.388,-412.274 898.789,-404.008 892.135,-393 870.754,-357.626 883.031,-340.318 892.135,-300 903.092,-251.48 929.986,-200.515 949.353,-168.063"/>
<polygon fill="black" stroke="black" points="952.403,-169.784 954.596,-159.418 946.418,-166.153 952.403,-169.784"/>
</g>
<!-- stage1__CodeGenerator__run&#45;&gt;modelbase__ModelBase__simplify -->
<g id="edge77" class="edge"><title>stage1__CodeGenerator__run&#45;&gt;modelbase__ModelBase__simplify</title>
<path fill="none" stroke="black" d="M917.352,-420.4C902.039,-414.083 886.498,-405.238 875.135,-393 815.953,-329.256 795.404,-224.559 788.489,-169.845"/>
<polygon fill="black" stroke="black" points="791.965,-169.433 787.315,-159.913 785.013,-170.255 791.965,-169.433"/>
</g>
<!-- stage1__CodeGenerator__run__kill_zero -->
<g id="node24" class="node"><title>stage1__CodeGenerator__run__kill_zero</title>
<ellipse fill="#65bdff" fill-opacity="0.698039" stroke="black" cx="972.135" cy="-335" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="972.135" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">kill_zero</text>
<text text-anchor="middle" x="972.135" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage1.py:82)</text>
</g>
<!-- stage1__CodeGenerator__run&#45;&gt;stage1__CodeGenerator__run__kill_zero -->
<g id="edge27" class="edge"><title>stage1__CodeGenerator__run&#45;&gt;stage1__CodeGenerator__run__kill_zero</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M966.112,-408.989C965.359,-397.734 965.191,-384.371 965.607,-372.174"/>
<polygon fill="#838b8b" stroke="#838b8b" points="969.12,-372 966.124,-361.837 962.129,-371.65 969.12,-372"/>
</g>
<!-- stage1__CodeGenerator__run&#45;&gt;stage1__CodeGenerator__run__kill_zero -->
<g id="edge74" class="edge"><title>stage1__CodeGenerator__run&#45;&gt;stage1__CodeGenerator__run__kill_zero</title>
<path fill="none" stroke="black" d="M978.158,-408.989C978.911,-397.734 979.079,-384.371 978.663,-372.174"/>
<polygon fill="black" stroke="black" points="982.141,-371.65 978.147,-361.837 975.15,-372 982.141,-371.65"/>
</g>
<!-- stage2__CodeGenerator____init__ -->
<g id="node27" class="node"><title>stage2__CodeGenerator____init__</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1922.14" cy="-436" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="1922.14" y="-439.8" font-family="Times,serif" font-size="14.00" fill="#000000">__init__</text>
<text text-anchor="middle" x="1922.14" y="-424.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:45)</text>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator____init__ -->
<g id="edge19" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator____init__</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1626.84,-515.461C1639.81,-510.118 1654.3,-505.019 1668.14,-502 1706.6,-493.604 1808.03,-507.175 1845.14,-494 1861.36,-488.239 1877.05,-477.817 1889.99,-467.446"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1892.24,-470.131 1897.67,-461.037 1887.75,-464.756 1892.24,-470.131"/>
</g>
<!-- stage2__CodeGenerator___analyze_args_internal -->
<g id="node28" class="node"><title>stage2__CodeGenerator___analyze_args_internal</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1736.14" cy="-436" rx="100.114" ry="26.7407"/>
<text text-anchor="middle" x="1736.14" y="-439.8" font-family="Times,serif" font-size="14.00" fill="#000000">_analyze_args_internal</text>
<text text-anchor="middle" x="1736.14" y="-424.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:190)</text>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator___analyze_args_internal -->
<g id="edge24" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator___analyze_args_internal</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1618.63,-513.535C1639.91,-499.675 1667.45,-481.734 1690.52,-466.707"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1692.66,-469.496 1699.12,-461.105 1688.84,-463.63 1692.66,-469.496"/>
</g>
<!-- stage2__CodeGenerator__analyze_args -->
<g id="node29" class="node"><title>stage2__CodeGenerator__analyze_args</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1522.14" cy="-335" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="1522.14" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">analyze_args</text>
<text text-anchor="middle" x="1522.14" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:148)</text>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__analyze_args -->
<g id="edge21" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__analyze_args</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1539.16,-516.094C1530.73,-510.232 1523.04,-502.929 1518.14,-494 1497.43,-456.289 1503.59,-405.508 1511.47,-371.81"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1514.91,-372.484 1513.94,-361.934 1508.12,-370.784 1514.91,-372.484"/>
</g>
<!-- stage2__CodeGenerator__analyze_interface -->
<g id="node30" class="node"><title>stage2__CodeGenerator__analyze_interface</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1910.14" cy="-335" rx="79.2942" ry="26.7407"/>
<text text-anchor="middle" x="1910.14" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">analyze_interface</text>
<text text-anchor="middle" x="1910.14" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:56)</text>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__analyze_interface -->
<g id="edge20" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__analyze_interface</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1577.07,-510.178C1570.41,-479.438 1565.26,-428.778 1594.14,-401 1603.27,-392.214 1810.05,-396.805 1822.14,-393 1840.28,-387.29 1858.27,-376.744 1873.21,-366.263"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1875.61,-368.848 1881.64,-360.136 1871.49,-363.187 1875.61,-368.848"/>
</g>
<!-- stage2__CodeGenerator__make_sortkey -->
<g id="node31" class="node"><title>stage2__CodeGenerator__make_sortkey</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="2079.14" cy="-335" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="2079.14" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">make_sortkey</text>
<text text-anchor="middle" x="2079.14" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:205)</text>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__make_sortkey -->
<g id="edge25" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__make_sortkey</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1590.67,-510.053C1600.32,-472.749 1618.2,-407.683 1627.14,-401 1643.65,-388.655 1978.55,-399.434 1998.14,-393 2015.05,-387.445 2031.52,-377.063 2045.16,-366.665"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2047.6,-369.193 2053.25,-360.232 2043.25,-363.712 2047.6,-369.193"/>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__run -->
<g id="edge23" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__run</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1626.37,-515.456C1639.45,-510.043 1654.12,-504.906 1668.14,-502 1704.05,-494.554 1963.3,-505.493 1998.14,-494 2015.04,-488.422 2031.52,-478.036 2045.15,-467.641"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2047.59,-470.17 2053.25,-461.211 2043.24,-464.688 2047.59,-470.17"/>
</g>
<!-- stage2__CodeGenerator__strip_levels -->
<g id="node33" class="node"><title>stage2__CodeGenerator__strip_levels</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1644.14" cy="-133" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="1644.14" y="-136.8" font-family="Times,serif" font-size="14.00" fill="#000000">strip_levels</text>
<text text-anchor="middle" x="1644.14" y="-121.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:225)</text>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__strip_levels -->
<g id="edge22" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__strip_levels</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1535.26,-517.55C1522.86,-511.34 1510.21,-503.513 1500.14,-494 1434.69,-432.225 1403.68,-381.366 1442.14,-300 1472.9,-234.899 1543.1,-186.74 1592.05,-159.523"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1593.82,-162.544 1600.92,-154.686 1590.47,-156.397 1593.82,-162.544"/>
</g>
<!-- stage2__CodeGenerator__validate_bound_args -->
<g id="node34" class="node"><title>stage2__CodeGenerator__validate_bound_args</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1722.14" cy="-335" rx="91.3254" ry="26.7407"/>
<text text-anchor="middle" x="1722.14" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">validate_bound_args</text>
<text text-anchor="middle" x="1722.14" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:230)</text>
</g>
<!-- stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__validate_bound_args -->
<g id="edge26" class="edge"><title>stage2__CodeGenerator&#45;&gt;stage2__CodeGenerator__validate_bound_args</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1564.09,-511.002C1560.81,-505.672 1557.91,-499.871 1556.14,-494 1544.16,-454.44 1529.53,-432.632 1556.14,-401 1569.49,-385.119 1582.36,-399.287 1602.14,-393 1625.78,-385.482 1650.75,-374.217 1671.74,-363.681"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1673.47,-366.729 1680.79,-359.067 1670.29,-360.494 1673.47,-366.729"/>
</g>
<!-- stage2__main&#45;&gt;stage1__CodeGenerator__run -->
<g id="edge82" class="edge"><title>stage2__main&#45;&gt;stage1__CodeGenerator__run</title>
<path fill="none" stroke="black" d="M1542.51,-616.19C1530.22,-609.776 1516.95,-602.434 1505.14,-595 1446.52,-558.127 1445.6,-524.561 1380.14,-502 1345.25,-489.978 1083,-506.089 1048.14,-494 1032.16,-488.459 1016.79,-478.212 1004.1,-467.918"/>
<polygon fill="black" stroke="black" points="1006.06,-464.993 996.169,-461.196 1001.53,-470.332 1006.06,-464.993"/>
</g>
<!-- stage2__main&#45;&gt;stage2__CodeGenerator -->
<g id="edge81" class="edge"><title>stage2__main&#45;&gt;stage2__CodeGenerator</title>
<path fill="none" stroke="black" d="M1584.87,-610.989C1584.76,-599.823 1584.63,-586.581 1584.5,-574.462"/>
<polygon fill="black" stroke="black" points="1588,-574.145 1584.4,-564.181 1581,-574.216 1588,-574.145"/>
</g>
<!-- stage2__main&#45;&gt;stage2__CodeGenerator__run -->
<g id="edge83" class="edge"><title>stage2__main&#45;&gt;stage2__CodeGenerator__run</title>
<path fill="none" stroke="black" d="M1656.46,-635.656C1782.68,-632.568 2034.82,-622.94 2063.14,-595 2094.75,-563.802 2092.91,-509.301 2087.2,-473.179"/>
<polygon fill="black" stroke="black" points="2090.6,-472.305 2085.43,-463.06 2083.71,-473.514 2090.6,-472.305"/>
</g>
<!-- stage2__main__npar -->
<g id="node42" class="node"><title>stage2__main__npar</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1980.14" cy="-537" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="1980.14" y="-540.8" font-family="Times,serif" font-size="14.00" fill="#000000">npar</text>
<text text-anchor="middle" x="1980.14" y="-525.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:509)</text>
</g>
<!-- stage2__main&#45;&gt;stage2__main__npar -->
<g id="edge36" class="edge"><title>stage2__main&#45;&gt;stage2__main__npar</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1656.31,-636.789C1719.66,-634.229 1813.7,-624.92 1890.14,-595 1907.13,-588.346 1923.96,-577.408 1938.48,-566.884"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1940.79,-569.531 1946.75,-560.77 1936.63,-563.903 1940.79,-569.531"/>
</g>
<!-- stage2__main&#45;&gt;stage2__main__npar -->
<g id="edge80" class="edge"><title>stage2__main&#45;&gt;stage2__main__npar</title>
<path fill="none" stroke="black" d="M1656.4,-637.251C1723.7,-635.489 1826.06,-627.125 1908.14,-595 1923.9,-588.83 1939.51,-578.978 1952.26,-569.186"/>
<polygon fill="black" stroke="black" points="1954.59,-571.799 1960.19,-562.8 1950.2,-566.347 1954.59,-571.799"/>
</g>
<!-- stage2__main__relevant -->
<g id="node43" class="node"><title>stage2__main__relevant</title>
<ellipse fill="#b699ff" fill-opacity="0.698039" stroke="black" cx="1819.14" cy="-537" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="1819.14" y="-540.8" font-family="Times,serif" font-size="14.00" fill="#000000">relevant</text>
<text text-anchor="middle" x="1819.14" y="-525.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:507)</text>
</g>
<!-- stage2__main&#45;&gt;stage2__main__relevant -->
<g id="edge35" class="edge"><title>stage2__main&#45;&gt;stage2__main__relevant</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1644.02,-622.848C1669.56,-616.114 1699.39,-606.89 1725.14,-595 1741.99,-587.214 1759.25,-576.288 1774.38,-566.082"/>
<polygon fill="#838b8b" stroke="#838b8b" points="1776.74,-568.711 1783.03,-560.185 1772.79,-562.928 1776.74,-568.711"/>
</g>
<!-- stage2__main&#45;&gt;stage2__main__relevant -->
<g id="edge79" class="edge"><title>stage2__main&#45;&gt;stage2__main__relevant</title>
<path fill="none" stroke="black" d="M1648.79,-625.742C1678.42,-618.85 1713.49,-608.694 1743.14,-595 1758.62,-587.849 1774.44,-578.048 1787.69,-568.589"/>
<polygon fill="black" stroke="black" points="1790.04,-571.209 1796,-562.455 1785.88,-565.578 1790.04,-571.209"/>
</g>
<!-- stage2__CodeGenerator___analyze_args_internal&#45;&gt;stage2__CodeGenerator__analyze_args -->
<g id="edge64" class="edge"><title>stage2__CodeGenerator___analyze_args_internal&#45;&gt;stage2__CodeGenerator__analyze_args</title>
<path fill="none" stroke="black" d="M1662.58,-417.672C1641.21,-411.095 1618.26,-402.788 1598.14,-393 1582.93,-385.604 1567.29,-375.831 1554.09,-366.47"/>
<polygon fill="black" stroke="black" points="1555.94,-363.486 1545.8,-360.407 1551.81,-369.136 1555.94,-363.486"/>
</g>
<!-- stage2__CodeGenerator__analyze_args&#45;&gt;stage2__CodeGenerator___analyze_args_internal -->
<g id="edge44" class="edge"><title>stage2__CodeGenerator__analyze_args&#45;&gt;stage2__CodeGenerator___analyze_args_internal</title>
<path fill="none" stroke="black" d="M1558.8,-358.149C1575.76,-369.688 1596.32,-383.363 1616.14,-393 1630.78,-400.124 1646.93,-406.464 1662.52,-411.923"/>
<polygon fill="black" stroke="black" points="1661.54,-415.288 1672.13,-415.208 1663.8,-408.664 1661.54,-415.288"/>
</g>
<!-- stage2__CodeGenerator__analyze_interface__ReaderState -->
<g id="node35" class="node"><title>stage2__CodeGenerator__analyze_interface__ReaderState</title>
<ellipse fill="#9165ff" fill-opacity="0.698039" stroke="black" cx="2865.14" cy="-234" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="2865.14" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">ReaderState</text>
<text text-anchor="middle" x="2865.14" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:81)</text>
</g>
<!-- stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__ReaderState -->
<g id="edge14" class="edge"><title>stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__ReaderState</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1955.88,-312.974C1969.51,-307.69 1984.67,-302.744 1999.14,-300 2042.26,-291.818 2747.57,-306.09 2789.14,-292 2805.15,-286.571 2820.53,-276.346 2833.21,-266.039"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2835.79,-268.446 2841.14,-259.303 2831.25,-263.112 2835.79,-268.446"/>
</g>
<!-- stage2__CodeGenerator__analyze_interface__commit -->
<g id="node36" class="node"><title>stage2__CodeGenerator__analyze_interface__commit</title>
<ellipse fill="#9165ff" fill-opacity="0.698039" stroke="black" cx="2713.14" cy="-234" rx="67.2629" ry="26.7407"/>
<text text-anchor="middle" x="2713.14" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">commit</text>
<text text-anchor="middle" x="2713.14" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:85)</text>
</g>
<!-- stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__commit -->
<g id="edge13" class="edge"><title>stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__commit</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1950.42,-311.757C1962.89,-306.947 1976.81,-302.545 1990.14,-300 2024.95,-293.35 2594.58,-303.42 2628.14,-292 2644.97,-286.27 2661.11,-275.253 2674.82,-264.421"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2677.04,-267.125 2682.6,-258.105 2672.63,-261.69 2677.04,-267.125"/>
</g>
<!-- stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__commit -->
<g id="edge43" class="edge"><title>stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__commit</title>
<path fill="none" stroke="black" d="M1961.22,-314.249C1976.12,-308.505 1992.54,-302.979 2008.14,-300 2042.95,-293.35 2612.58,-303.42 2646.14,-292 2661.61,-286.733 2676.5,-276.999 2688.41,-267.047"/>
<polygon fill="black" stroke="black" points="2690.99,-269.431 2696.15,-260.174 2686.35,-264.196 2690.99,-269.431"/>
</g>
<!-- stage2__CodeGenerator__analyze_interface__function_header_ends -->
<g id="node37" class="node"><title>stage2__CodeGenerator__analyze_interface__function_header_ends</title>
<ellipse fill="#9165ff" fill-opacity="0.698039" stroke="black" cx="2532.14" cy="-234" rx="96.2499" ry="26.7407"/>
<text text-anchor="middle" x="2532.14" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">function_header_ends</text>
<text text-anchor="middle" x="2532.14" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:88)</text>
</g>
<!-- stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__function_header_ends -->
<g id="edge12" class="edge"><title>stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__function_header_ends</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1950.43,-311.815C1962.9,-307.004 1976.82,-302.586 1990.14,-300 2036.5,-290.993 2369.4,-303.787 2415.14,-292 2438.01,-286.104 2461.44,-274.541 2481.34,-263.361"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2483.28,-266.281 2490.22,-258.274 2479.8,-260.207 2483.28,-266.281"/>
</g>
<!-- stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__function_header_ends -->
<g id="edge42" class="edge"><title>stage2__CodeGenerator__analyze_interface&#45;&gt;stage2__CodeGenerator__analyze_interface__function_header_ends</title>
<path fill="none" stroke="black" d="M1961.23,-314.306C1976.14,-308.566 1992.55,-303.028 2008.14,-300 2054.5,-290.993 2387.4,-303.787 2433.14,-292 2454.64,-286.458 2476.63,-275.909 2494.48,-265.378"/>
<polygon fill="black" stroke="black" points="2496.73,-268.11 2503.42,-259.899 2493.07,-262.142 2496.73,-268.11"/>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__analyze_args -->
<g id="edge51" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__analyze_args</title>
<path fill="none" stroke="black" d="M2037.65,-414.065C2025.25,-408.783 2011.41,-403.815 1998.14,-401 1955.63,-391.987 1648.58,-406.073 1607.14,-393 1589.29,-387.37 1571.74,-376.723 1557.26,-366.142"/>
<polygon fill="black" stroke="black" points="1559.2,-363.216 1549.11,-359.957 1554.96,-368.791 1559.2,-363.216"/>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__analyze_interface -->
<g id="edge50" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__analyze_interface</title>
<path fill="none" stroke="black" d="M2037.66,-414.088C2025.15,-407.554 2011.48,-400.174 1999.14,-393 1984.03,-384.218 1967.84,-374.082 1953.47,-364.815"/>
<polygon fill="black" stroke="black" points="1955.08,-361.685 1944.79,-359.176 1951.27,-367.557 1955.08,-361.685"/>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__make_sortkey -->
<g id="edge53" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__make_sortkey</title>
<path fill="none" stroke="black" d="M2079.14,-408.989C2079.14,-397.823 2079.14,-384.581 2079.14,-372.462"/>
<polygon fill="black" stroke="black" points="2082.64,-372.181 2079.14,-362.181 2075.64,-372.181 2082.64,-372.181"/>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__strip_levels -->
<g id="edge55" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__strip_levels</title>
<path fill="none" stroke="black" d="M2037.65,-414.072C2025.25,-408.79 2011.41,-403.821 1998.14,-401 1987.92,-398.829 1629.44,-400.465 1622.14,-393 1592.01,-362.224 1619.04,-233.358 1634.74,-169.782"/>
<polygon fill="black" stroke="black" points="1638.2,-170.351 1637.24,-159.8 1631.41,-168.647 1638.2,-170.351"/>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__validate_bound_args -->
<g id="edge49" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__validate_bound_args</title>
<path fill="none" stroke="black" d="M2037.19,-414.099C2024.89,-408.886 2011.23,-403.947 1998.14,-401 1959.94,-392.402 1859.74,-403.885 1822.14,-393 1801.87,-387.133 1781.3,-376.408 1764.14,-365.831"/>
<polygon fill="black" stroke="black" points="1765.81,-362.747 1755.49,-360.34 1762.06,-368.656 1765.81,-362.747"/>
</g>
<!-- stage2__CodeGenerator__run__bind_local -->
<g id="node38" class="node"><title>stage2__CodeGenerator__run__bind_local</title>
<ellipse fill="#9165ff" fill-opacity="0.698039" stroke="black" cx="2415.14" cy="-335" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="2415.14" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">bind_local</text>
<text text-anchor="middle" x="2415.14" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:428)</text>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__bind_local -->
<g id="edge29" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__bind_local</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M2149.26,-431.244C2199.72,-426.585 2268.45,-416.184 2325.14,-393 2341.9,-386.144 2358.58,-375.281 2373.04,-364.868"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2375.28,-367.561 2381.27,-358.823 2371.14,-361.917 2375.28,-367.561"/>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__bind_local -->
<g id="edge54" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__bind_local</title>
<path fill="none" stroke="black" d="M2149.79,-432.425C2204.51,-428.45 2281.05,-418.392 2343.14,-393 2358.67,-386.646 2374.14,-376.85 2386.83,-367.16"/>
<polygon fill="black" stroke="black" points="2389.11,-369.819 2394.74,-360.846 2384.74,-364.348 2389.11,-369.819"/>
</g>
<!-- stage2__CodeGenerator__run__make_sorted_by -->
<g id="node39" class="node"><title>stage2__CodeGenerator__run__make_sorted_by</title>
<ellipse fill="#9165ff" fill-opacity="0.698039" stroke="black" cx="2250.14" cy="-335" rx="75.4903" ry="26.7407"/>
<text text-anchor="middle" x="2250.14" y="-338.8" font-family="Times,serif" font-size="14.00" fill="#000000">make_sorted_by</text>
<text text-anchor="middle" x="2250.14" y="-323.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:325)</text>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__make_sorted_by -->
<g id="edge28" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__make_sorted_by</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M2116.16,-412.972C2128.09,-406.768 2141.23,-399.84 2153.14,-393 2168.97,-383.902 2185.9,-373.182 2201.19,-363.508"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2203.39,-366.254 2209.98,-357.953 2199.65,-360.337 2203.39,-366.254"/>
</g>
<!-- stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__make_sorted_by -->
<g id="edge52" class="edge"><title>stage2__CodeGenerator__run&#45;&gt;stage2__CodeGenerator__run__make_sorted_by</title>
<path fill="none" stroke="black" d="M2127.34,-415.974C2141.72,-409.053 2157.27,-400.967 2171.14,-393 2185.63,-384.667 2201.06,-374.973 2214.47,-365.969"/>
<polygon fill="black" stroke="black" points="2216.67,-368.703 2222.95,-360.173 2212.72,-362.923 2216.67,-368.703"/>
</g>
<!-- stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__strip_levels -->
<g id="edge85" class="edge"><title>stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__strip_levels</title>
<path fill="none" stroke="black" d="M1712.1,-308.263C1698.29,-272.861 1673.39,-209.01 1657.87,-169.207"/>
<polygon fill="black" stroke="black" points="1661.03,-167.683 1654.13,-159.638 1654.51,-170.227 1661.03,-167.683"/>
</g>
<!-- stage2__CodeGenerator__validate_bound_args__process -->
<g id="node40" class="node"><title>stage2__CodeGenerator__validate_bound_args__process</title>
<ellipse fill="#9165ff" fill-opacity="0.698039" stroke="black" cx="2245.14" cy="-234" rx="71.127" ry="26.7407"/>
<text text-anchor="middle" x="2245.14" y="-237.8" font-family="Times,serif" font-size="14.00" fill="#000000">process</text>
<text text-anchor="middle" x="2245.14" y="-222.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:292)</text>
</g>
<!-- stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__validate_bound_args__process -->
<g id="edge37" class="edge"><title>stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__validate_bound_args__process</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1768.7,-311.872C1782.75,-307.06 1798.33,-302.627 1813.14,-300 1850.34,-293.397 2117.18,-303.623 2153.14,-292 2171.24,-286.148 2188.95,-274.968 2204.03,-264.049"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2206.17,-266.816 2212.12,-258.045 2202,-261.195 2206.17,-266.816"/>
</g>
<!-- stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__validate_bound_args__process -->
<g id="edge84" class="edge"><title>stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__validate_bound_args__process</title>
<path fill="none" stroke="black" d="M1779.49,-314.094C1795.98,-308.453 1814.07,-303.028 1831.14,-300 1868.34,-293.397 2135.18,-303.623 2171.14,-292 2187.94,-286.568 2204.41,-276.546 2217.63,-266.402"/>
<polygon fill="black" stroke="black" points="2219.84,-269.117 2225.42,-260.111 2215.45,-263.671 2219.84,-269.117"/>
</g>
<!-- stage2__CodeGenerator__validate_bound_args__update_callers_of -->
<g id="node41" class="node"><title>stage2__CodeGenerator__validate_bound_args__update_callers_of</title>
<ellipse fill="#9165ff" fill-opacity="0.698039" stroke="black" cx="2293.14" cy="-133" rx="79.3553" ry="26.7407"/>
<text text-anchor="middle" x="2293.14" y="-136.8" font-family="Times,serif" font-size="14.00" fill="#000000">update_callers_of</text>
<text text-anchor="middle" x="2293.14" y="-121.8" font-family="Times,serif" font-size="14.00" fill="#000000">(stage2.py:284)</text>
</g>
<!-- stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__validate_bound_args__update_callers_of -->
<g id="edge38" class="edge"><title>stage2__CodeGenerator__validate_bound_args&#45;&gt;stage2__CodeGenerator__validate_bound_args__update_callers_of</title>
<path fill="none" stroke="#838b8b" stroke-dasharray="5,2" d="M1774.27,-312.855C1789.45,-307.639 1806.24,-302.759 1822.14,-300 1835.9,-297.611 2315.33,-301.959 2325.14,-292 2357.26,-259.36 2335.24,-203.893 2315.17,-168.117"/>
<polygon fill="#838b8b" stroke="#838b8b" points="2318.2,-166.357 2310.15,-159.471 2312.15,-169.875 2318.2,-166.357"/>
</g>
<!-- stage2__CodeGenerator__validate_bound_args__process&#45;&gt;stage2__CodeGenerator__strip_levels -->
<g id="edge63" class="edge"><title>stage2__CodeGenerator__validate_bound_args__process&#45;&gt;stage2__CodeGenerator__strip_levels</title>
<path fill="none" stroke="black" d="M2201.51,-212.61C2189.02,-207.47 2175.24,-202.441 2162.14,-199 2010.7,-159.232 1827.77,-143.537 1724.81,-137.534"/>
<polygon fill="black" stroke="black" points="1724.97,-134.038 1714.79,-136.966 1724.58,-141.027 1724.97,-134.038"/>
</g>
<!-- stage2__CodeGenerator__validate_bound_args__process&#45;&gt;stage2__CodeGenerator__validate_bound_args__update_callers_of -->
<g id="edge62" class="edge"><title>stage2__CodeGenerator__validate_bound_args__process&#45;&gt;stage2__CodeGenerator__validate_bound_args__update_callers_of</title>
<path fill="none" stroke="black" d="M2257.5,-207.505C2263.16,-195.818 2269.97,-181.775 2276.11,-169.108"/>
<polygon fill="black" stroke="black" points="2279.44,-170.278 2280.65,-159.753 2273.14,-167.224 2279.44,-170.278"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 67 KiB

851
pyan.py
View File

@@ -1,850 +1,11 @@
#!/usr/bin/env python
#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
pyan.py - Generate approximate call graphs for Python programs.
This program takes one or more Python source files, does a superficial
analysis, and constructs a directed graph of the objects in the combined
source, and how they define or use each other. The graph can be output
for rendering by e.g. GraphViz or yEd.
"""
# Copyright (C) 2011-2014 Edmund Horner and Juha Jeronen
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
import sys
import compiler
from glob import glob
from optparse import OptionParser
import os.path
import re
import math
def verbose_output(msg):
print >>sys.stderr, msg
def hsl2rgb(*args):
"""Convert HSL color tuple to RGB.
Parameters: H,S,L, where
H,S,L = HSL values as double-precision floats, with each component in [0,1].
Return value:
R,G,B tuple
For more information:
https://en.wikipedia.org/wiki/HSL_and_HSV#From_HSL
"""
if len(args) != 3:
raise ValueError("hsl2rgb requires exactly 3 arguments. See docstring.")
H = args[0]
S = args[1]
L = args[2]
if H < 0.0 or H > 1.0:
raise ValueError("H component = %g out of range [0,1]" % H)
if S < 0.0 or S > 1.0:
raise ValueError("S component = %g out of range [0,1]" % S)
if L < 0.0 or L > 1.0:
raise ValueError("L component = %g out of range [0,1]" % L)
# hue chunk
Hpf = H / (60./360.) # "H prime, float" (H', float)
Hp = int(Hpf) # "H prime" (H', int)
if Hp >= 6: # catch special case 360deg = 0deg
Hp = 0
C = (1.0 - math.fabs(2.0*L - 1.0))*S # HSL chroma
X = C * (1.0 - math.fabs( math.modf(Hpf / 2.0)[0] - 1.0 ))
if S == 0.0: # H undefined if S == 0
R1,G1,B1 = (0.0, 0.0, 0.0)
elif Hp == 0:
R1,G1,B1 = (C, X, 0.0)
elif Hp == 1:
R1,G1,B1 = (X, C, 0.0)
elif Hp == 2:
R1,G1,B1 = (0.0, C, X )
elif Hp == 3:
R1,G1,B1 = (0.0, X, C )
elif Hp == 4:
R1,G1,B1 = (X, 0.0, C )
elif Hp == 5:
R1,G1,B1 = (C, 0.0, X )
# match the HSL Lightness
#
m = L - 0.5*C
R,G,B = (R1 + m, G1 + m, B1 + m)
return R,G,B
def htmlize_rgb(*args):
"""HTML-ize an RGB(A) color.
Parameters: R,G,B[,alpha], where
R,G,B = RGB values as double-precision floats, with each component in [0,1].
alpha = optional alpha component for translucency, in [0,1]. (1.0 = opaque)
Example:
htmlize_rgb(1.0, 0.5, 0) => "#FF8000" (RGB)
htmlize_rgb(1.0, 0.5, 0, 0.5) => "#FF800080" (RGBA)
"""
if len(args) < 3:
raise ValueError("htmlize_rgb requires 3 or 4 arguments. See docstring.")
R = args[0]
G = args[1]
B = args[2]
if R < 0.0 or R > 1.0:
raise ValueError("R component = %g out of range [0,1]" % R)
if G < 0.0 or G > 1.0:
raise ValueError("G component = %g out of range [0,1]" % G)
if B < 0.0 or B > 1.0:
raise ValueError("B component = %g out of range [0,1]" % B)
R = int(255.0*R)
G = int(255.0*G)
B = int(255.0*B)
if len(args) > 3:
alp = args[3]
if alp < 0.0 or alp > 1.0:
raise ValueError("alpha component = %g out of range [0,1]" % alp)
alp = int(255.0*alp)
make_RGBA = True
else:
make_RGBA = False
if make_RGBA:
return "#%02X%02X%02X%02X" % (R, G, B, alp)
else:
return "#%02X%02X%02X" % (R, G, B)
class Node(object):
"""A node is an object in the call graph. Nodes have names, and are in
namespaces. The full name of a node is its namespace, a dot, and its name.
If the namespace is Null, it is rendered as *, and considered as an unknown
node. The meaning of this is that a use-edge to an unknown node is created
when the analysis cannot determine which actual node is being used."""
def __init__(self, namespace, name, orig_node):
self.namespace = namespace
self.name = name
self.defined = namespace is None
self.orig_node = orig_node
def get_short_name(self):
"""Return the short name (i.e. excluding the namespace), of this Node.
Names of unknown nodes will include the *. prefix."""
if self.namespace is None:
return '*.' + self.name
else:
return self.name
def get_name(self):
"""Return the full name of this node."""
if self.namespace == '':
return self.name
elif self.namespace is None:
return '*.' + self.name
else:
return self.namespace + '.' + self.name
def get_level(self):
"""Return the level of this node (in terms of nested namespaces).
The level is defined as the number of '.' in the namespace, plus one.
Top level is level 0.
"""
if self.namespace == "":
return 0
else:
return 1 + self.namespace.count('.')
def get_toplevel_namespace(self):
"""Return the name of the top-level namespace of this node, or "" if none."""
if self.namespace == "":
return ""
idx = self.namespace.find('.')
if idx > -1:
return self.namespace[0:idx]
else:
return self.namespace
def get_label(self):
"""Return a label for this node, suitable for use in graph formats.
Unique nodes should have unique labels; and labels should not contain
problematic characters like dots or asterisks."""
return self.get_name().replace('.', '__').replace('*', '')
def __repr__(self):
return '<Node %s>' % self.get_name()
class CallGraphVisitor(object):
"""A visitor that can be walked over a Python AST, and will derive
information about the objects in the AST and how they use each other.
A single CallGraphVisitor object can be run over several ASTs (from a
set of source files). The resulting information is the aggregate from
all files. This way use information between objects in different files
can be gathered."""
def __init__(self):
self.nodes = {}
self.defines_edges = {}
self.uses_edges = {}
self.name_stack = []
self.scope_stack = []
self.last_value = None
self.current_class = None
def visitModule(self, node):
self.name_stack.append(self.module_name)
self.scope_stack.append(self.scopes[node])
self.visit(node.node)
self.scope_stack.pop()
self.name_stack.pop()
self.last_value = None
def visitClass(self, node):
from_node = self.get_current_namespace()
to_node = self.get_node(from_node.get_name(), node.name, node)
if self.add_defines_edge(from_node, to_node):
verbose_output("Def from %s to Class %s" % (from_node, to_node))
self.current_class = to_node
self.name_stack.append(node.name)
self.scope_stack.append(self.scopes[node])
for b in node.bases:
self.visit(b)
self.visit(node.code)
self.scope_stack.pop()
self.name_stack.pop()
def visitFunction(self, node):
if node.name == '__init__':
for d in node.defaults:
self.visit(d)
self.visit(node.code)
return
from_node = self.get_current_namespace()
to_node = self.get_node(from_node.get_name(), node.name, node)
if self.add_defines_edge(from_node, to_node):
verbose_output("Def from %s to Function %s" % (from_node, to_node))
self.name_stack.append(node.name)
self.scope_stack.append(self.scopes[node])
for d in node.defaults:
self.visit(d)
self.visit(node.code)
self.scope_stack.pop()
self.name_stack.pop()
def visitImport(self, node):
for import_item in node.names:
tgt_name = import_item[0].split('.', 1)[0]
from_node = self.get_current_namespace()
to_node = self.get_node('', tgt_name, node)
if self.add_uses_edge(from_node, to_node):
verbose_output("Use from %s to Import %s" % (from_node, to_node))
if tgt_name in self.module_names:
mod_name = self.module_names[tgt_name]
else:
mod_name = tgt_name
tgt_module = self.get_node('', mod_name, node)
self.set_value(tgt_name, tgt_module)
def visitFrom(self, node):
tgt_name = node.modname
from_node = self.get_current_namespace()
to_node = self.get_node(None, tgt_name, node)
if self.add_uses_edge(from_node, to_node):
verbose_output("Use from %s to From %s" % (from_node, to_node))
if tgt_name in self.module_names:
mod_name = self.module_names[tgt_name]
else:
mod_name = tgt_name
for name, new_name in node.names:
if new_name is None:
new_name = name
tgt_module = self.get_node(mod_name, name, node)
self.set_value(new_name, tgt_module)
verbose_output("From setting name %s to %s" % (new_name, tgt_module))
def visitConst(self, node):
t = type(node.value)
tn = t.__name__
self.last_value = self.get_node('', tn, node)
def visitAssAttr(self, node):
save_last_value = self.last_value
self.visit(node.expr)
if isinstance(self.last_value, Node) and self.last_value.orig_node in self.scopes:
sc = self.scopes[self.last_value.orig_node]
sc.defs[node.attrname] = save_last_value
verbose_output('assattr %s on %s to %s' % (node.attrname, self.last_value, save_last_value))
self.last_value = save_last_value
def visitAssName(self, node):
tgt_name = node.name
self.set_value(tgt_name, self.last_value)
def visitAssign(self, node):
self.visit(node.expr)
for ass in node.nodes:
self.visit(ass)
self.last_value = None
def visitCallFunc(self, node):
self.visit(node.node)
for arg in node.args:
self.visit(arg)
if node.star_args is not None:
self.visit(node.star_args)
if node.dstar_args is not None:
self.visit(node.dstar_args)
def visitDiscard(self, node):
self.visit(node.expr)
self.last_value = None
def visitName(self, node):
if node.name == 'self' and self.current_class is not None:
verbose_output('name %s is maps to %s' % (node.name, self.current_class))
self.last_value = self.current_class
return
tgt_name = node.name
from_node = self.get_current_namespace()
to_node = self.get_value(tgt_name)
###TODO if the name is a local variable (i.e. in the top scope), and
###has no known value, then don't try to create a node for it.
if not isinstance(to_node, Node):
to_node = self.get_node(None, tgt_name, node)
if self.add_uses_edge(from_node, to_node):
verbose_output("Use from %s to Name %s" % (from_node, to_node))
self.last_value = to_node
def visitGetattr(self, node):
self.visit(node.expr)
if isinstance(self.last_value, Node) and self.last_value.orig_node in self.scopes and node.attrname in self.scopes[self.last_value.orig_node].defs:
verbose_output('getattr %s from %s returns %s' % (node.attrname, self.last_value, self.scopes[self.last_value.orig_node].defs[node.attrname]))
self.last_value = self.scopes[self.last_value.orig_node].defs[node.attrname]
return
tgt_name = node.attrname
from_node = self.get_current_namespace()
if isinstance(self.last_value, Node) and self.last_value.namespace is not None:
to_node = self.get_node(self.last_value.get_name(), tgt_name, node)
else:
to_node = self.get_node(None, tgt_name, node)
if self.add_uses_edge(from_node, to_node):
verbose_output("Use from %s to Getattr %s" % (from_node, to_node))
self.last_value = to_node
def get_node(self, namespace, name, orig_node=None):
"""Return the unique node matching the namespace and name.
Creates a new node if one doesn't already exist."""
if name in self.nodes:
for n in self.nodes[name]:
if n.namespace == namespace:
return n
n = Node(namespace, name, orig_node)
if name in self.nodes:
self.nodes[name].append(n)
else:
self.nodes[name] = [n]
return n
def get_current_namespace(self):
"""Return a node representing the current namespace."""
namespace = '.'.join(self.name_stack[0:-1])
name = self.name_stack[-1]
return self.get_node(namespace, name, None)
def find_scope(self, name):
"""Search in the scope stack for the top-most scope containing name."""
for sc in reversed(self.scope_stack):
if name in sc.defs:
return sc
return None
def get_value(self, name):
"""Get the value of name in the current scope."""
sc = self.find_scope(name)
if sc is None:
return None
value = sc.defs[name]
if isinstance(value, Node):
return value
return None
def set_value(self, name, value):
"""Set the value of name in the current scope."""
sc = self.find_scope(name)
if sc is not None and isinstance(value, Node):
sc.defs[name] = value
verbose_output('Set %s to %s' % (name, value))
def add_defines_edge(self, from_node, to_node):
"""Add a defines edge in the graph between two nodes.
N.B. This will mark both nodes as defined."""
if from_node not in self.defines_edges:
self.defines_edges[from_node] = set()
if to_node in self.defines_edges[from_node]:
return False
self.defines_edges[from_node].add(to_node)
from_node.defined = True
to_node.defined = True
return True
def add_uses_edge(self, from_node, to_node):
"""Add a uses edge in the graph between two nodes."""
if from_node not in self.uses_edges:
self.uses_edges[from_node] = set()
if to_node in self.uses_edges[from_node]:
return False
self.uses_edges[from_node].add(to_node)
return True
def contract_nonexistents(self):
"""For all use edges to non-existent (i.e. not defined nodes) X.name, replace with edge to *.name."""
new_uses_edges = []
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
if n2.namespace is not None and not n2.defined:
n3 = self.get_node(None, n2.name, n2.orig_node)
new_uses_edges.append((n, n3))
verbose_output("Contracting non-existent from %s to %s" % (n, n2))
for from_node, to_node in new_uses_edges:
self.add_uses_edge(from_node, to_node)
def expand_unknowns(self):
"""For each unknown node *.name, replace all its incoming edges with edges to X.name for all possible Xs."""
new_defines_edges = []
for n in self.defines_edges:
for n2 in self.defines_edges[n]:
if n2.namespace is None:
for n3 in self.nodes[n2.name]:
new_defines_edges.append((n, n3))
for from_node, to_node in new_defines_edges:
self.add_defines_edge(from_node, to_node)
new_uses_edges = []
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
if n2.namespace is None:
for n3 in self.nodes[n2.name]:
new_uses_edges.append((n, n3))
for from_node, to_node in new_uses_edges:
self.add_uses_edge(from_node, to_node)
for name in self.nodes:
for n in self.nodes[name]:
if n.namespace is None:
n.defined = False
def cull_inherited(self):
"""For each use edge from W to X.name, if it also has an edge to W to Y.name where Y is used by X, then remove the first edge."""
removed_uses_edges = []
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
inherited = False
for n3 in self.uses_edges[n]:
if n3.name == n2.name and n2.namespace is not None and n3.namespace is not None and n3.namespace != n2.namespace:
if '.' in n2.namespace:
nsp2,p2 = n2.namespace.rsplit('.', 1)
else:
nsp2,p2 = '',n2.namespace
if '.' in n3.namespace:
nsp3,p3 = n3.namespace.rsplit('.', 1)
else:
nsp3,p3 = '',n3.namespace
pn2 = self.get_node(nsp2, p2, None)
pn3 = self.get_node(nsp3, p3, None)
if pn2 in self.uses_edges and pn3 in self.uses_edges[pn2]:
inherited = True
if inherited and n in self.uses_edges:
removed_uses_edges.append((n, n2))
verbose_output("Removing inherited edge from %s to %s" % (n, n2))
for from_node, to_node in removed_uses_edges:
self.uses_edges[from_node].remove(to_node)
def to_dot(self, **kwargs):
draw_defines = ("draw_defines" in kwargs and kwargs["draw_defines"])
draw_uses = ("draw_uses" in kwargs and kwargs["draw_uses"])
colored = ("colored" in kwargs and kwargs["colored"])
grouped = ("grouped" in kwargs and kwargs["grouped"])
nested_groups = ("nested_groups" in kwargs and kwargs["nested_groups"])
rankdir = kwargs.get("rankdir", "TB")
# Color nodes by top-level namespace. Use HSL: hue = file, lightness = nesting level.
#
# Map top-level namespaces (typically files) to different hues.
#
# The "" namespace (for *.py files) gets the first color.
#
# Since its level is 0, its lightness will be 1.0, i.e. pure white
# regardless of the hue. (See the HSL assignment code below.)
#
# Reference H values (at S=1, L=0.5):
# 0 = pure red
# 60 = pure yellow
# 120 = pure green
# 180 = pure cyan
# 240 = pure blue
# 300 = pure magenta
#
# unused, green (120), orange (50), cyan (190), yellow (90),
# deep blue (240), red (0), magenta (300)
# See https://en.wikipedia.org/wiki/File:HSV-RGB-comparison.svg
# (although this is HSL, the hue should match)
#
hues = map( lambda d: d/360., [ 0, 120, 50, 190, 90, 240, 0, 300 ] )
top_ns_to_hue_idx = {}
global cidx # WTF? Python 2.6 won't pass cidx to the inner function without global...
cidx = 0 # first free hue index
def get_hue_idx(node):
global cidx
ns = node.get_toplevel_namespace()
verbose_output("Coloring %s (top-level namespace %s)" % (node.get_short_name(), ns))
if ns not in top_ns_to_hue_idx: # not seen yet
top_ns_to_hue_idx[ns] = cidx
cidx += 1
if cidx >= len(hues):
verbose_output("WARNING: too many top-level namespaces; colors wrapped")
cidx = 0 # wrap around
return top_ns_to_hue_idx[ns]
s = """digraph G {\n"""
graph_opts = {'rankdir': rankdir}
# enable clustering
if grouped:
graph_opts['clusterrank'] = 'local'
# Newer versions of GraphViz (e.g. 2.36.0 (20140111.2315) in Ubuntu 14.04) have a stricter parser.
# http://www.graphviz.org/doc/info/attrs.html#a:clusterrank
# s += """ graph [clusterrank local];\n"""
graph_opts = ', '.join(
[key + '="' + value + '"' for key, value in graph_opts.items()]
)
s += """ graph [""" + graph_opts + """];\n"""
vis_node_list = [] # for sorting; will store nodes to be visualized
def nodecmp(n1, n2):
if n1.namespace > n2.namespace:
return +1
elif n1.namespace < n2.namespace:
return -1
else:
return 0
# find out which nodes are defined (can be visualized)
for name in self.nodes:
for n in self.nodes[name]:
if n.defined:
vis_node_list.append(n)
vis_node_list.sort(cmp=nodecmp) # sort by namespace for clustering
# Write nodes and subgraphs
#
prev_namespace = ""
namespace_stack = []
indent = ""
for n in vis_node_list:
# new namespace? (NOTE: nodes sorted by namespace!)
if grouped and n.namespace != prev_namespace:
if nested_groups:
# Pop the stack until the newly found namespace is within one of the
# parent namespaces, or until the stack runs out (i.e. this is a
# sibling).
#
j = len(namespace_stack) - 1
if j >= 0:
m = re.match(namespace_stack[j], n.namespace)
# The '.' check catches siblings in cases like MeshGenerator vs. Mesh.
while m is None or n.namespace[m.end()] != '.':
s += """%s}\n""" % indent # terminate previous subgraph
del namespace_stack[j]
j -= 1
indent = " " * (4*len(namespace_stack)) # 4 spaces per level
if j < 0:
break
m = re.match(namespace_stack[j], n.namespace)
namespace_stack.append( n.namespace )
indent = " " * (4*len(namespace_stack)) # 4 spaces per level
else:
if prev_namespace != "":
s += """%s}\n""" % indent # terminate previous subgraph
else:
# first subgraph begins, start indenting
indent = " " # 4 spaces
prev_namespace = n.namespace
# begin new subgraph for this namespace (TODO: refactor the label generation)
# (name must begin with "cluster" to be recognized as a cluster by GraphViz)
s += """%ssubgraph cluster_%s {\n""" % (indent, n.namespace.replace('.', '__').replace('*', ''))
# translucent gray (no hue to avoid visual confusion with any group of colored nodes)
s += """%s graph [style="filled,rounded", fillcolor="#80808018", label="%s"];\n""" % (indent, n.namespace)
# add the node itself
if colored:
idx = get_hue_idx(n)
H = hues[idx]
S = 1.0
L = max( [1.0 - 0.1*n.get_level(), 0.1] )
A = 0.7 # make nodes translucent (to handle possible overlaps)
fill_RGBA = list(hsl2rgb(H,S,L))
fill_RGBA.append(A)
fill_RGBA = htmlize_rgb( *fill_RGBA )
if L >= 0.3:
text_RGB = htmlize_rgb( 0.0, 0.0, 0.0 ) # black text on light nodes
else:
text_RGB = htmlize_rgb( 1.0, 1.0, 1.0 ) # white text on dark nodes
s += """%s %s [label="%s", style="filled", fillcolor="%s", fontcolor="%s", group="%s"];\n""" % (indent, n.get_label(), n.get_short_name(), fill_RGBA, text_RGB, idx)
else:
fill_RGBA = htmlize_rgb( 1.0, 1.0, 1.0, 0.7 )
idx = get_hue_idx(n)
s += """%s %s [label="%s", style="filled", fillcolor="%s", fontcolor="#000000", group="%s"];\n""" % (indent, n.get_label(), n.get_short_name(), fill_RGBA, idx)
if grouped:
if nested_groups:
j = len(namespace_stack) - 1
while j >= 0:
s += """%s}\n""" % indent # terminate all remaining subgraphs
del namespace_stack[j]
j -= 1
indent = " " * (4*len(namespace_stack)) # 4 spaces per level
else:
s += """%s}\n""" % indent # terminate last subgraph
# Write defines relationships
#
if draw_defines:
for n in self.defines_edges:
for n2 in self.defines_edges[n]:
if n2.defined and n2 != n:
# gray lines (so they won't visually obstruct the "uses" lines)
s += """ %s -> %s [style="dashed", color="azure4"];\n""" % (n.get_label(), n2.get_label())
# Write uses relationships
#
if draw_uses:
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
if n2.defined and n2 != n:
s += """ %s -> %s;\n""" % (n.get_label(), n2.get_label())
s += """}\n""" # terminate "digraph G {"
return s
def to_tgf(self, **kwargs):
draw_defines = ("draw_defines" in kwargs and kwargs["draw_defines"])
draw_uses = ("draw_uses" in kwargs and kwargs["draw_uses"])
s = ''
i = 1
id_map = {}
for name in self.nodes:
for n in self.nodes[name]:
if n.defined:
s += """%d %s\n""" % (i, n.get_short_name())
id_map[n] = i
i += 1
#else:
# print >>sys.stderr, "ignoring %s" % n
s += """#\n"""
if draw_defines:
for n in self.defines_edges:
for n2 in self.defines_edges[n]:
if n2.defined and n2 != n:
i1 = id_map[n]
i2 = id_map[n2]
s += """%d %d D\n""" % (i1, i2)
if draw_uses:
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
if n2.defined and n2 != n:
i1 = id_map[n]
i2 = id_map[n2]
s += """%d %d U\n""" % (i1, i2)
return s
def get_module_name(filename):
"""Try to determine the full module name of a source file, by figuring out
if its directory looks like a package (i.e. has an __init__.py file)."""
if os.path.basename(filename) == '__init__.py':
return get_module_name(os.path.dirname(filename))
init_path = os.path.join(os.path.dirname(filename), '__init__.py')
mod_name = os.path.basename(filename).replace('.py', '')
if not os.path.exists(init_path):
return mod_name
if not os.path.dirname(filename):
return mod_name
return get_module_name(os.path.dirname(filename)) + '.' + mod_name
def main():
usage = """usage: %prog FILENAME... [--dot|--tgf]"""
desc = """Analyse one or more Python source files and generate an approximate call graph of the modules, classes and functions within them."""
parser = OptionParser(usage=usage, description=desc)
parser.add_option("--dot",
action="store_true", default=False,
help="output in GraphViz dot format")
parser.add_option("--tgf",
action="store_true", default=False,
help="output in Trivial Graph Format")
parser.add_option("-v", "--verbose",
action="store_true", default=False, dest="verbose",
help="verbose output")
parser.add_option("-d", "--defines",
action="store_true", default=True, dest="draw_defines",
help="add edges for 'defines' relationships [default]")
parser.add_option("-n", "--no-defines",
action="store_false", default=True, dest="draw_defines",
help="do not add edges for 'defines' relationships")
parser.add_option("-u", "--uses",
action="store_true", default=True, dest="draw_uses",
help="add edges for 'uses' relationships [default]")
parser.add_option("-N", "--no-uses",
action="store_false", default=True, dest="draw_uses",
help="do not add edges for 'uses' relationships")
parser.add_option("-c", "--colored",
action="store_true", default=False, dest="colored",
help="color nodes according to namespace [dot only]")
parser.add_option("-g", "--grouped",
action="store_true", default=False, dest="grouped",
help="group nodes (create subgraphs) according to namespace [dot only]")
parser.add_option("-e", "--nested-groups",
action="store_true", default=False, dest="nested_groups",
help="create nested groups (subgraphs) for nested namespaces (implies -g) [dot only]")
parser.add_option("--dot-rankdir", default="TB", dest="rankdir",
help=(
"specifies the dot graph 'rankdir' property for "
"controlling the direction of the graph. "
"Allowed values: ['TB', 'LR', 'BT', 'RL']. "
"[dot only]"
))
options, args = parser.parse_args()
filenames = [fn2 for fn in args for fn2 in glob(fn)]
if len(args) == 0:
parser.error('Need one or more filenames to process')
if options.nested_groups:
options.grouped = True
if not options.verbose:
global verbose_output
verbose_output = lambda msg: None
v = CallGraphVisitor()
v.module_names = {}
# First find module full names for all files
for filename in filenames:
mod_name = get_module_name(filename)
short_name = mod_name.rsplit('.', 1)[-1]
v.module_names[short_name] = mod_name
# Process the set of files, TWICE: so that forward references are picked up
for filename in filenames + filenames:
ast = compiler.parseFile(filename)
module_name = get_module_name(filename)
v.module_name = module_name
s = compiler.symbols.SymbolVisitor()
compiler.walk(ast, s)
v.scopes = s.scopes
compiler.walk(ast, v)
v.contract_nonexistents()
v.expand_unknowns()
v.cull_inherited()
if options.dot:
print v.to_dot(draw_defines=options.draw_defines,
draw_uses=options.draw_uses,
colored=options.colored,
grouped=options.grouped,
nested_groups=options.nested_groups,
rankdir=options.rankdir)
if options.tgf:
print v.to_tgf(draw_defines=options.draw_defines,
draw_uses=options.draw_uses)
import sys
from pyan import main
if __name__ == '__main__':
main()
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

6
pyan/__init__.py Normal file
View File

@@ -0,0 +1,6 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from .main import main
__version__ = "1.0.0"

772
pyan/analyzer.py Normal file
View File

@@ -0,0 +1,772 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""The AST visitor.
Created on Mon Nov 13 03:33:00 2017
Original code by Edmund Horner.
Python 3 port by Juha Jeronen.
"""
import ast
import symtable
from .common import MsgPrinter, MsgLevel, \
format_alias, get_ast_node_name, sanitize_exprs, get_module_name, \
Node, Scope
# TODO: add Cython support (strip type annotations in a preprocess step, then treat as Python)
# TODO: built-in functions (range(), enumerate(), zip(), iter(), ...):
# add to a special scope "built-in" in analyze_scopes() (or ignore altogether)
# TODO: support Node-ifying ListComp et al, List, Tuple
# TODO: make the analyzer smarter (see individual TODOs below)
# Note the use of the term "node" for two different concepts:
#
# - AST nodes (the "node" argument of CallGraphVisitor.visit_*())
#
# - The Node class that mainly stores auxiliary information about AST nodes,
# for the purposes of generating the call graph.
#
# Namespaces also get a Node (with no associated AST node).
# These tables were useful for porting the visitor to Python 3:
#
# https://docs.python.org/2/library/compiler.html#module-compiler.ast
# https://docs.python.org/3/library/ast.html#abstract-grammar
#
class CallGraphVisitor(ast.NodeVisitor):
"""A visitor that can be walked over a Python AST, and will derive
information about the objects in the AST and how they use each other.
A single CallGraphVisitor object can be run over several ASTs (from a
set of source files). The resulting information is the aggregate from
all files. This way use information between objects in different files
can be gathered."""
def __init__(self, filenames, msgprinter=None):
if msgprinter is None:
msgprinter = MsgPrinter()
self.msgprinter = msgprinter
# full module names for all given files
self.module_names = {}
self.module_to_filename = {} # inverse mapping for recording which file each AST node came from
for filename in filenames:
mod_name = get_module_name(filename)
short_name = mod_name.rsplit('.', 1)[-1]
self.module_names[short_name] = mod_name
self.module_to_filename[mod_name] = filename
self.filenames = filenames
# data gathered from analysis
self.defines_edges = {}
self.uses_edges = {}
self.nodes = {} # Node name: list of Node objects (in possibly different namespaces)
self.scopes = {} # fully qualified name of namespace: Scope object
self.ast_node_to_namespace = {} # AST node: fully qualified name of namespace
# current context for analysis
self.module_name = None
self.filename = None
self.name_stack = [] # for building namespace name, node naming
self.scope_stack = [] # the Scope objects
self.class_stack = [] # for resolving "self"
self.last_value = None
def process(self, filename):
"""Analyze the specified Python source file."""
if filename not in self.filenames:
raise ValueError("Filename '%s' has not been preprocessed (was not given to __init__, which got %s)" % (filename, self.filenames))
with open(filename, "rt", encoding="utf-8") as f:
content = f.read()
self.filename = filename
self.module_name = get_module_name(filename)
self.analyze_scopes(content, filename)
self.visit(ast.parse(content, filename))
self.module_name = None
self.filename = None
def postprocess(self):
"""Finalize the analysis."""
self.contract_nonexistents()
self.expand_unknowns()
self.cull_inherited()
###########################################################################
# visitor methods
# Python docs:
# https://docs.python.org/3/library/ast.html#abstract-grammar
def visit_Module(self, node):
self.msgprinter.message("Module", level=MsgLevel.DEBUG)
# TODO: self.get_node() this too, and associate_node() to get the
# source file information for annotated output?
ns = self.module_name
self.name_stack.append(ns)
self.scope_stack.append(self.scopes[ns])
self.ast_node_to_namespace[node] = ns # must be added manually since we don't self.get_node() here
self.generic_visit(node) # visit the **children** of node
self.scope_stack.pop()
self.name_stack.pop()
self.last_value = None
def visit_ClassDef(self, node):
self.msgprinter.message("ClassDef %s" % (node.name), level=MsgLevel.DEBUG)
from_node = self.get_current_namespace()
ns = from_node.get_name()
to_node = self.get_node(ns, node.name, node)
if self.add_defines_edge(from_node, to_node):
self.msgprinter.message("Def from %s to Class %s" % (from_node, to_node), level=MsgLevel.INFO)
# The graph Node may have been created earlier by a FromImport,
# in which case its AST node points to the site of the import.
#
# Change the AST node association of the relevant graph Node
# to this AST node (the definition site) to get the correct
# source line number information in annotated output.
#
self.associate_node(to_node, node, self.filename)
# Bind the name specified by the AST node to the graph Node
# in the current scope.
#
self.set_value(node.name, to_node)
self.class_stack.append(to_node)
self.name_stack.append(node.name)
inner_ns = self.get_current_namespace().get_name()
self.scope_stack.append(self.scopes[inner_ns])
for b in node.bases:
self.visit(b)
for stmt in node.body:
self.visit(stmt)
self.scope_stack.pop()
self.name_stack.pop()
self.class_stack.pop()
def visit_FunctionDef(self, node):
self.msgprinter.message("FunctionDef %s" % (node.name), level=MsgLevel.DEBUG)
# # Place instance members at class level in the call graph
# # TODO: brittle: breaks analysis if __init__ defines an internal helper class,
# # because then the scope lookup will fail. Disabled this special handling for now.
# if node.name == '__init__':
# for d in node.args.defaults:
# self.visit(d)
# for d in node.args.kw_defaults:
# self.visit(d)
# for stmt in node.body:
# self.visit(stmt)
# return
from_node = self.get_current_namespace()
ns = from_node.get_name()
to_node = self.get_node(ns, node.name, node)
if self.add_defines_edge(from_node, to_node):
self.msgprinter.message("Def from %s to Function %s" % (from_node, to_node), level=MsgLevel.INFO)
self.associate_node(to_node, node, self.filename)
self.set_value(node.name, to_node)
self.name_stack.append(node.name)
inner_ns = self.get_current_namespace().get_name()
self.scope_stack.append(self.scopes[inner_ns])
for d in node.args.defaults:
self.visit(d)
for d in node.args.kw_defaults:
self.visit(d)
for stmt in node.body:
self.visit(stmt)
self.scope_stack.pop()
self.name_stack.pop()
def visit_AsyncFunctionDef(self, node):
self.visit_FunctionDef(node) # TODO: alias for now; tag async functions in output in a future version?
# This gives lambdas their own namespaces in the graph;
# if that is not desired, this method can be simply omitted.
#
# (The default visit() already visits all the children of a generic AST node
# by calling generic_visit(); and visit_Name() captures any uses inside the lambda.)
#
def visit_Lambda(self, node):
self.msgprinter.message("Lambda", level=MsgLevel.DEBUG)
def process():
for d in node.args.defaults:
self.visit(d)
for d in node.args.kw_defaults:
self.visit(d)
self.visit(node.body) # single expr
self.with_scope("lambda", process)
def visit_Import(self, node):
self.msgprinter.message("Import %s" % [format_alias(x) for x in node.names], level=MsgLevel.DEBUG)
# TODO: add support for relative imports (path may be like "....something.something")
# https://www.python.org/dev/peps/pep-0328/#id10
for import_item in node.names:
src_name = import_item.name # what is being imported
tgt_name = import_item.asname if import_item.asname is not None else src_name # under which name
from_node = self.get_current_namespace() # where it is being imported to, i.e. the **user**
to_node = self.get_node('', tgt_name, node) # the thing **being used** (under the asname, if any)
if self.add_uses_edge(from_node, to_node):
self.msgprinter.message("Use from %s to Import %s" % (from_node, to_node), level=MsgLevel.INFO)
# conversion: possible short name -> fully qualified name
# (when analyzing a set of files in the same directory)
if src_name in self.module_names:
mod_name = self.module_names[src_name]
else:
mod_name = src_name
tgt_module = self.get_node('', mod_name, node)
self.set_value(tgt_name, tgt_module)
def visit_ImportFrom(self, node):
self.msgprinter.message("ImportFrom: from %s import %s" % (node.module, [format_alias(x) for x in node.names]), level=MsgLevel.DEBUG)
tgt_name = node.module
from_node = self.get_current_namespace()
to_node = self.get_node('', tgt_name, node) # module, in top-level namespace
if self.add_uses_edge(from_node, to_node):
self.msgprinter.message("Use from %s to From %s" % (from_node, to_node), level=MsgLevel.INFO)
if tgt_name in self.module_names:
mod_name = self.module_names[tgt_name]
else:
mod_name = tgt_name
for import_item in node.names:
name = import_item.name
new_name = import_item.asname if import_item.asname is not None else name
tgt_id = self.get_node(mod_name, name, node) # we imported the identifier name from the module mod_name
self.set_value(new_name, tgt_id)
self.msgprinter.message("From setting name %s to %s" % (new_name, tgt_id), level=MsgLevel.INFO)
# TODO: where are Constants used? (instead of Num, Str, ...)
#
# Edmund Horner's original post has info on what this fixed in Python 2.
# https://ejrh.wordpress.com/2012/01/31/call-graphs-in-python-part-2/
def visit_Constant(self, node):
self.msgprinter.message("Constant %s" % (node.value), level=MsgLevel.DEBUG)
t = type(node.value)
tn = t.__name__
self.last_value = self.get_node('', tn, node)
# attribute access (node.ctx determines whether set (ast.Store) or get (ast.Load))
def visit_Attribute(self, node):
self.msgprinter.message("Attribute %s of %s in context %s" % (node.attr, get_ast_node_name(node.value), type(node.ctx)), level=MsgLevel.DEBUG)
if isinstance(node.ctx, ast.Store):
# this is the value being assigned (set by visit_Assign())
save_last_value = self.last_value
# find the object whose attribute we are accessing
# (this munges self.last_value to point to the Node for that object)
#
self.visit(node.value)
if isinstance(self.last_value, Node):
ns = self.ast_node_to_namespace[self.last_value.ast_node]
if ns in self.scopes:
sc = self.scopes[ns]
sc.defs[node.attr] = save_last_value
self.msgprinter.message('setattr %s on %s to %s' % (node.attr, self.last_value, save_last_value), level=MsgLevel.INFO)
self.last_value = save_last_value
elif isinstance(node.ctx, ast.Load):
# TODO: add proper support for nested attributes
#
# Right now we get, for this code:
#
# class MyClass:
# def __init__(self):
# class InnerClass:
# def __init__(self):
# self.a = 3
#
# self.b = InnerClass()
# self.c = self.b.a
#
# the following incorrect result:
#
# Assign ['self.c'] ['self.b.a']
# Attribute a of self.b in context <class '_ast.Load'>
# Get self.b: no Node value (or name not in scope)
# AST node <_ast.Attribute object at 0x7f201681b320> (a) has namespace 'None'
# Use from <Node testpyan.MyClass.__init__> to Getattr <Node *.a>
# Attribute c of self in context <class '_ast.Store'>
# Name self in context <class '_ast.Load'>
# name self maps to <Node testpyan.MyClass>
# setattr c on <Node testpyan.MyClass> to <Node *.a>
# get our Node object corresponding to node.value in the current ns
value = self.get_value(get_ast_node_name(node.value))
# use the original AST node attached to that Node to look up the object's ns
ns = self.ast_node_to_namespace[value.ast_node] if value is not None else None
if ns in self.scopes and node.attr in self.scopes[ns].defs:
result = self.scopes[ns].defs[node.attr]
self.msgprinter.message('getattr %s on %s returns %s' % (node.attr, value, result), level=MsgLevel.INFO)
self.last_value = result
return
tgt_name = node.attr
from_node = self.get_current_namespace()
if isinstance(self.last_value, Node) and self.last_value.namespace is not None:
to_node = self.get_node(self.last_value.get_name(), tgt_name, node)
else:
to_node = self.get_node(None, tgt_name, node)
if self.add_uses_edge(from_node, to_node):
self.msgprinter.message("Use from %s to Getattr %s" % (from_node, to_node), level=MsgLevel.INFO)
self.last_value = to_node
# name access (node.ctx determines whether set (ast.Store) or get (ast.Load))
def visit_Name(self, node):
self.msgprinter.message("Name %s in context %s" % (node.id, type(node.ctx)), level=MsgLevel.DEBUG)
if isinstance(node.ctx, ast.Store):
# when we get here, self.last_value has been set by visit_Assign()
self.set_value(node.id, self.last_value)
elif isinstance(node.ctx, ast.Load):
# TODO: we handle self by its name, not by being the first argument in a method
current_class = self.get_current_class()
if node.id == 'self' and current_class is not None:
self.msgprinter.message('name %s maps to %s' % (node.id, current_class), level=MsgLevel.INFO)
self.last_value = current_class
return
tgt_name = node.id
from_node = self.get_current_namespace()
to_node = self.get_value(tgt_name)
###TODO if the name is a local variable (i.e. in the innermost scope), and
###has no known value, then don't try to create a Node for it.
if not isinstance(to_node, Node):
to_node = self.get_node(None, tgt_name, node) # namespace=None means we don't know the namespace yet
if self.add_uses_edge(from_node, to_node):
self.msgprinter.message("Use from %s to Name %s" % (from_node, to_node), level=MsgLevel.INFO)
self.last_value = to_node
def analyze_binding(self, targets, values):
"""Generic handler for binding forms. Inputs must be sanitize_exprs()d."""
# Before we begin analyzing the assignment, clean up any leftover self.last_value.
#
# (e.g. from any Name in load context (including function names in a Call)
# that did not assign anything.)
#
self.last_value = None
# TODO: properly support tuple unpacking
#
# - the problem is:
# a,*b,c = [1,2,3,4,5] --> Name,Starred,Name = List
# so a simple analysis of the AST won't get us far here.
#
# To fix this:
#
# - find the index of Starred on the LHS
# - unpack the RHS into a tuple/list (if possible)
# - unpack just one level; the items may be tuples/lists and that's just fine
# - if not possible to unpack directly (e.g. enumerate(foo) is a **call**),
# don't try to be too smart; just do some generic fallback handling (or give up)
# - if RHS unpack successful:
# - map the non-starred items directly (one-to-one)
# - map the remaining sublist of the RHS to the Starred term
# - requires support for tuples/lists of AST nodes as values of Nodes
# - but generally, we need that anyway: consider self.a = (f, g, h)
# --> any use of self.a should detect the possible use of f, g, and h;
# currently this is simply ignored.
#
# TODO: support Additional Unpacking Generalizations (Python 3.6+):
# https://www.python.org/dev/peps/pep-0448/
if len(targets) == len(values): # handle correctly the most common trivial case "a1,a2,... = b1,b2,..."
for tgt,value in zip(targets,values):
self.visit(value) # RHS -> set self.last_value to input for this tgt
self.visit(tgt) # LHS
self.last_value = None
else: # FIXME: for now, do the wrong thing in the non-trivial case
# old code, no tuple unpacking support
for value in values:
self.visit(value) # set self.last_value to **something** on the RHS and hope for the best
for tgt in targets: # LHS
self.visit(tgt)
self.last_value = None
def visit_Assign(self, node):
# - chaining assignments like "a = b = c" produces multiple targets
# - tuple unpacking works as a separate mechanism on top of that (see analyze_binding())
#
if len(node.targets) > 1:
self.msgprinter.message("Assign (chained with %d outputs)" % (len(node.targets)), level=MsgLevel.DEBUG)
values = sanitize_exprs(node.value) # values is the same for each set of targets
for targets in node.targets:
targets = sanitize_exprs(targets)
self.msgprinter.message("Assign %s %s" % ([get_ast_node_name(x) for x in targets],
[get_ast_node_name(x) for x in values]),
level=MsgLevel.DEBUG)
self.analyze_binding(targets, values)
def visit_AnnAssign(self, node):
self.visit_Assign(self, node) # TODO: alias for now; add the annotations to output in a future version?
def visit_AugAssign(self, node):
targets = sanitize_exprs(node.target)
values = sanitize_exprs(node.value) # values is the same for each set of targets
self.msgprinter.message("AugAssign %s %s %s" % ([get_ast_node_name(x) for x in targets],
type(node.op),
[get_ast_node_name(x) for x in values]),
level=MsgLevel.DEBUG)
# TODO: maybe no need to handle tuple unpacking in AugAssign? (but simpler to use the same implementation)
self.analyze_binding(targets, values)
# for() is also a binding form.
#
# (Without analyzing the bindings, we would get an unknown node for any
# use of the loop counter(s) in the loop body. This can have confusing
# consequences in the expand_unknowns() step, if the same name is
# in use elsewhere. Thus, we treat for() properly, as a binding form.)
#
def visit_For(self, node):
self.msgprinter.message("For-loop", level=MsgLevel.DEBUG)
targets = sanitize_exprs(node.target)
values = sanitize_exprs(node.iter)
self.analyze_binding(targets, values)
for stmt in node.body:
self.visit(stmt)
for stmt in node.orelse:
self.visit(stmt)
def visit_AsyncFor(self, node):
self.visit_For(node) # TODO: alias for now; tag async for in output in a future version?
def visit_ListComp(self, node):
self.msgprinter.message("ListComp", level=MsgLevel.DEBUG)
def process():
self.visit(node.elt)
self.analyze_generators(node.generators)
self.with_scope("listcomp", process)
def visit_SetComp(self, node):
self.msgprinter.message("SetComp", level=MsgLevel.DEBUG)
def process():
self.visit(node.elt)
self.analyze_generators(node.generators)
self.with_scope("setcomp", process)
def visit_DictComp(self, node):
self.msgprinter.message("DictComp", level=MsgLevel.DEBUG)
def process():
self.visit(node.key)
self.visit(node.value)
self.analyze_generators(node.generators)
self.with_scope("dictcomp", process)
def visit_GeneratorExp(self, node):
self.msgprinter.message("GeneratorExp", level=MsgLevel.DEBUG)
def process():
self.visit(node.elt)
self.analyze_generators(node.generators)
self.with_scope("genexpr", process)
def analyze_generators(self, generators):
"""Analyze the generators in a comprehension form."""
for gen in generators:
# TODO: there's also an is_async field we might want to use in a future version.
targets = sanitize_exprs(gen.target)
values = sanitize_exprs(gen.iter)
self.analyze_binding(targets, values)
for expr in gen.ifs:
self.visit(expr)
def visit_Call(self, node):
self.msgprinter.message("Call %s" % (get_ast_node_name(node.func)), level=MsgLevel.DEBUG)
for arg in node.args:
self.visit(arg)
for kw in node.keywords:
self.visit(kw.value)
# Visit the function name part last, so that inside a binding form,
# it will be left standing as self.last_value.
self.visit(node.func)
###########################################################################
# Scope analysis
def analyze_scopes(self, code, filename):
"""Gather lexical scope information."""
# Below, ns is the fully qualified ("dotted") name of sc.
#
# Technically, the module scope is anonymous, but we treat it as if
# it was in a namespace named after the module, to support analysis
# of several files as a set (keeping their module-level definitions
# in different scopes, as we should).
#
scopes = {}
def process(parent_ns, table):
sc = Scope(table)
ns = "%s.%s" % (parent_ns, sc.name) if len(sc.name) else parent_ns
scopes[ns] = sc
for t in table.get_children():
process(ns, t)
process(self.module_name, symtable.symtable(code, filename, compile_type="exec"))
self.scopes = scopes
self.msgprinter.message("Scopes: %s" % (scopes), level=MsgLevel.DEBUG)
def with_scope(self, scopename, thunk):
"""Run thunk (0-argument function) with the scope stack augmented with an inner scope.
Used to analyze lambda, listcomp et al. (The scope must still be present in self.scopes.)"""
self.name_stack.append(scopename)
inner_ns = self.get_current_namespace().get_name()
self.scope_stack.append(self.scopes[inner_ns])
thunk()
self.scope_stack.pop()
self.name_stack.pop()
def get_current_class(self):
"""Return the node representing the current class, or None if not inside a class definition."""
return self.class_stack[-1] if len(self.class_stack) else None
def get_current_namespace(self):
"""Return a node representing the current namespace, based on self.name_stack."""
# namespace nodes do not have an AST node associated with them.
if not len(self.name_stack): # the top level is the current module
return self.get_node('', self.module_name, None)
namespace = '.'.join(self.name_stack[0:-1])
name = self.name_stack[-1]
return self.get_node(namespace, name, None)
def get_value(self, name):
"""Get the value of name in the current scope. Return None if name is not set to a value."""
# get the innermost scope that has name **and where name has a value**
def find_scope(name):
for sc in reversed(self.scope_stack):
if name in sc.defs and sc.defs[name] is not None:
return sc
sc = find_scope(name)
if sc is not None:
value = sc.defs[name]
if isinstance(value, Node):
self.msgprinter.message('Get %s in %s, found in %s, value %s' % (name, self.scope_stack[-1], sc, value), level=MsgLevel.INFO)
return value
else:
self.msgprinter.message('Get %s in %s, found in %s: value %s is not a Node' % (name, self.scope_stack[-1], sc, value), level=MsgLevel.DEBUG)
else:
self.msgprinter.message('Get %s in %s: no Node value (or name not in scope)' % (name, self.scope_stack[-1]), level=MsgLevel.DEBUG)
return None
def set_value(self, name, value):
"""Set the value of name in the current scope."""
# get the innermost scope that has name (should be the current scope unless name is a global)
def find_scope(name):
for sc in reversed(self.scope_stack):
if name in sc.defs:
return sc
sc = find_scope(name)
if sc is not None:
if isinstance(value, Node):
sc.defs[name] = value
self.msgprinter.message('Set %s in %s to %s' % (name, sc, value), level=MsgLevel.INFO)
else:
self.msgprinter.message('Set %s in %s: value %s is not a Node' % (name, sc, value), level=MsgLevel.DEBUG)
else:
self.msgprinter.message('Set: name %s not in scope' % (name), level=MsgLevel.DEBUG)
###########################################################################
# Graph creation
def get_node(self, namespace, name, ast_node=None):
"""Return the unique node matching the namespace and name.
Creates a new node if one doesn't already exist.
In CallGraphVisitor, always use get_node() to create nodes, because it
also sets some auxiliary information. Do not call the Node constructor
directly.
"""
if name in self.nodes:
for n in self.nodes[name]:
if n.namespace == namespace:
return n
# Try to figure out which source file this Node belongs to
# (for annotated output).
#
if namespace in self.module_to_filename:
# If the namespace is one of the modules being analyzed,
# the the Node belongs to the correponding file.
filename = self.module_to_filename[namespace]
else: # assume it's defined in the current file
filename = self.filename
n = Node(namespace, name, ast_node, filename)
# Make the scope info accessible for the visit_*() methods
# that only have an AST node.
#
# In Python 3, symtable and ast are completely separate, so symtable
# doesn't see our copy of the AST, and symtable's copy of the AST
# is not accessible from the outside.
#
# The visitor only gets an AST, but must be able to access the scope
# information, so we mediate this by saving the full name of the namespace
# where each AST node came from when it is get_node()d for the first time.
#
if ast_node is not None:
self.ast_node_to_namespace[ast_node] = namespace
self.msgprinter.message("Namespace for AST node %s (%s) recorded as '%s'" % (ast_node, name, namespace), level=MsgLevel.DEBUG)
if name in self.nodes:
self.nodes[name].append(n)
else:
self.nodes[name] = [n]
return n
def associate_node(self, graph_node, ast_node, filename=None):
"""Change the AST node (and optionally filename) mapping of a graph node.
This is useful for generating annotated output with source filename
and line number information.
Sometimes a function in the analyzed code is first seen in a FromImport
before its definition has been analyzed. The namespace can be deduced
correctly already at that point, but the source line number information
has to wait until the actual definition is found. However, a graph Node
associated with an AST node must be created immediately, to track the
uses of that function.
This method re-associates the given graph node with a different
AST node, which allows updating the context when the definition
of a function or class is encountered."""
graph_node.ast_node = ast_node
if ast_node is not None:
# Add also the new AST node to the reverse lookup.
self.ast_node_to_namespace[ast_node] = graph_node.namespace
if filename is not None:
graph_node.filename = filename
def add_defines_edge(self, from_node, to_node):
"""Add a defines edge in the graph between two nodes.
N.B. This will mark both nodes as defined."""
if from_node not in self.defines_edges:
self.defines_edges[from_node] = set()
if to_node in self.defines_edges[from_node]:
return False
self.defines_edges[from_node].add(to_node)
from_node.defined = True
to_node.defined = True
return True
def add_uses_edge(self, from_node, to_node):
"""Add a uses edge in the graph between two nodes."""
if from_node not in self.uses_edges:
self.uses_edges[from_node] = set()
if to_node in self.uses_edges[from_node]:
return False
self.uses_edges[from_node].add(to_node)
return True
###########################################################################
# Postprocessing
def contract_nonexistents(self):
"""For all use edges to non-existent (i.e. not defined nodes) X.name, replace with edge to *.name."""
new_uses_edges = []
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
if n2.namespace is not None and not n2.defined:
n3 = self.get_node(None, n2.name, n2.ast_node)
new_uses_edges.append((n, n3))
self.msgprinter.message("Contracting non-existent from %s to %s" % (n, n2), level=MsgLevel.INFO)
for from_node, to_node in new_uses_edges:
self.add_uses_edge(from_node, to_node)
def expand_unknowns(self):
"""For each unknown node *.name, replace all its incoming edges with edges to X.name for all possible Xs."""
new_defines_edges = []
for n in self.defines_edges:
for n2 in self.defines_edges[n]:
if n2.namespace is None:
for n3 in self.nodes[n2.name]:
new_defines_edges.append((n, n3))
for from_node, to_node in new_defines_edges:
self.add_defines_edge(from_node, to_node)
new_uses_edges = []
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
if n2.namespace is None:
for n3 in self.nodes[n2.name]:
new_uses_edges.append((n, n3))
for from_node, to_node in new_uses_edges:
self.add_uses_edge(from_node, to_node)
for name in self.nodes:
for n in self.nodes[name]:
if n.namespace is None:
n.defined = False
def cull_inherited(self):
"""For each use edge from W to X.name, if it also has an edge to W to Y.name where Y is used by X, then remove the first edge."""
removed_uses_edges = []
for n in self.uses_edges:
for n2 in self.uses_edges[n]:
inherited = False
for n3 in self.uses_edges[n]:
if n3.name == n2.name and n2.namespace is not None and n3.namespace is not None and n3.namespace != n2.namespace:
if '.' in n2.namespace:
nsp2,p2 = n2.namespace.rsplit('.', 1)
else:
nsp2,p2 = '',n2.namespace
if '.' in n3.namespace:
nsp3,p3 = n3.namespace.rsplit('.', 1)
else:
nsp3,p3 = '',n3.namespace
pn2 = self.get_node(nsp2, p2, None)
pn3 = self.get_node(nsp3, p3, None)
if pn2 in self.uses_edges and pn3 in self.uses_edges[pn2]:
inherited = True
if inherited and n in self.uses_edges:
removed_uses_edges.append((n, n2))
self.msgprinter.message("Removing inherited edge from %s to %s" % (n, n2), level=MsgLevel.INFO)
for from_node, to_node in removed_uses_edges:
self.uses_edges[from_node].remove(to_node)

191
pyan/common.py Normal file
View File

@@ -0,0 +1,191 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Parts shared between analyzer and graphgen.
Created on Mon Nov 13 03:33:00 2017
Original code by Edmund Horner.
Further development by Juha Jeronen.
"""
from sys import stderr
import os.path
import ast
class MsgLevel:
ERROR = 0
WARNING = 1
INFO = 2
DEBUG = 3
class MsgPrinter:
def __init__(self, verbosity=MsgLevel.WARNING):
self.verbosity = verbosity
def message(self, msg, level):
if level <= self.verbosity:
print(msg, file=stderr)
def set_verbosity(self, verbosity):
self.verbosity = verbosity
def get_module_name(filename):
"""Try to determine the full module name of a source file, by figuring out
if its directory looks like a package (i.e. has an __init__.py file)."""
if os.path.basename(filename) == '__init__.py':
return get_module_name(os.path.dirname(filename))
init_path = os.path.join(os.path.dirname(filename), '__init__.py')
mod_name = os.path.basename(filename).replace('.py', '')
if not os.path.exists(init_path):
return mod_name
return get_module_name(os.path.dirname(filename)) + '.' + mod_name
def format_alias(x):
"""Return human-readable description of an ast.alias (used in Import and ImportFrom nodes)."""
if not isinstance(x, ast.alias):
raise TypeError("Can only format an ast.alias; got %s" % type(x))
if x.asname is not None:
return "%s as %s" % (x.name, x.asname)
else:
return "%s" % (x.name)
def get_ast_node_name(x):
"""Return human-readable name of ast.Attribute or ast.Name. Pass through anything else."""
if isinstance(x, ast.Attribute):
# x.value might also be an ast.Attribute (think "x.y.z")
return "%s.%s" % (get_ast_node_name(x.value), x.attr)
elif isinstance(x, ast.Name):
return x.id
else:
return x
# Helper for handling binding forms.
def sanitize_exprs(exprs):
"""Convert ast.Tuples in exprs to Python tuples; wrap result in a Python tuple."""
def process(expr):
if isinstance(expr, (ast.Tuple, ast.List)):
return expr.elts # .elts is a Python tuple
else:
return [expr]
if isinstance(exprs, (tuple, list)):
return [process(expr) for expr in exprs]
else:
return process(exprs)
class Node:
"""A node is an object in the call graph. Nodes have names, and are in
namespaces. The full name of a node is its namespace, a dot, and its name.
If the namespace is None, it is rendered as *, and considered as an unknown
node. The meaning of this is that a use-edge to an unknown node is created
when the analysis cannot determine which actual node is being used."""
def __init__(self, namespace, name, ast_node, filename):
self.namespace = namespace
self.name = name
self.ast_node = ast_node
self.filename = filename
self.defined = namespace is None # assume that unknown nodes are defined
def get_short_name(self):
"""Return the short name (i.e. excluding the namespace), of this Node.
Names of unknown nodes will include the *. prefix."""
if self.namespace is None:
return '*.' + self.name
else:
return self.name
def get_annotated_name(self):
"""Return the short name, plus module and line number of definition site, if available.
Names of unknown nodes will include the *. prefix."""
if self.namespace is None:
return '*.' + self.name
else:
if self.get_level() >= 1 and self.ast_node is not None:
return "%s\n(%s:%d)" % (self.name, self.filename, self.ast_node.lineno)
else:
return self.name
def get_long_annotated_name(self):
"""Return the short name, plus namespace, and module and line number of definition site, if available.
Names of unknown nodes will include the *. prefix."""
if self.namespace is None:
return '*.' + self.name
else:
if self.get_level() >= 1:
if self.ast_node is not None:
return "%s\\n\\n(%s:%d,\\nin %s)" % (self.name, self.filename, self.ast_node.lineno, self.namespace)
else:
return "%s\\n\\n(in %s)" % (self.name, self.namespace)
else:
return self.name
def get_name(self):
"""Return the full name of this node."""
if self.namespace == '':
return self.name
elif self.namespace is None:
return '*.' + self.name
else:
return self.namespace + '.' + self.name
def get_level(self):
"""Return the level of this node (in terms of nested namespaces).
The level is defined as the number of '.' in the namespace, plus one.
Top level is level 0.
"""
if self.namespace == "":
return 0
else:
return 1 + self.namespace.count('.')
def get_toplevel_namespace(self):
"""Return the name of the top-level namespace of this node, or "" if none."""
if self.namespace == "":
return ""
if self.namespace is None: # group all unknowns in one namespace, "*"
return "*"
idx = self.namespace.find('.')
if idx > -1:
return self.namespace[0:idx]
else:
return self.namespace
def get_label(self):
"""Return a label for this node, suitable for use in graph formats.
Unique nodes should have unique labels; and labels should not contain
problematic characters like dots or asterisks."""
return self.get_name().replace('.', '__').replace('*', '')
def __repr__(self):
return '<Node %s>' % self.get_name()
class Scope:
"""Adaptor that makes scopes look somewhat like those from the Python 2
compiler module, as far as Pyan's CallGraphVisitor is concerned."""
def __init__(self, table):
"""table: SymTable instance from symtable.symtable()"""
name = table.get_name()
if name == 'top':
name = '' # Pyan defines the top level as anonymous
self.name = name
self.type = table.get_type() # useful for __repr__()
self.defs = {iden:None for iden in table.get_identifiers()} # name:assigned_value
def __repr__(self):
return "<Scope: %s %s>" % (self.type, self.name)

251
pyan/graphgen.py Normal file
View File

@@ -0,0 +1,251 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Graph markup generator.
Created on Mon Nov 13 03:32:12 2017
Original code by Edmund Horner.
Coloring logic and grouping for GraphViz output by Juha Jeronen.
"""
# TODO: namespace mode, just one node per namespace and relations between them
# - useful if the full output is too detailed to be visually readable
# - scan the nodes and edges, basically generate a new graph and to_dot() that
import re
import colorsys
from .common import MsgPrinter, MsgLevel
def htmlize_rgb(R,G,B,A=None):
if A is not None:
R,G,B,A = [int(255.0*x) for x in (R,G,B,A)]
return "#%02X%02X%02X%02X" % (R,G,B,A)
else:
R,G,B = [int(255.0*x) for x in (R,G,B)]
return "#%02X%02X%02X" % (R,G,B)
# Set node color by top-level namespace.
#
# HSL: hue = top-level namespace, lightness = nesting level, saturation constant.
#
# The "" namespace (for *.py files) gets the first color. Since its
# level is 0, its lightness will be 1.0, i.e. pure white regardless
# of the hue.
#
class Colorizer:
def __init__(self, n, msgprinter=None): # n: number of hues
if msgprinter is None:
msgprinter = MsgPrinter()
self.msgprinter = msgprinter
self._hues = [j/n for j in range(n)]
self._idx_of = {} # top-level namespace: hue index
self._idx = 0
def _next_idx(self):
result = self._idx
self._idx += 1
if self._idx >= len(self._hues):
self.msgprinter.message("WARNING: colors wrapped", level=MsgLevel.WARNING)
self._idx = 0
return result
def _node_to_idx(self, node):
ns = node.get_toplevel_namespace()
self.msgprinter.message("Coloring %s (top-level namespace %s)" % (node.get_short_name(), ns), level=MsgLevel.INFO)
if ns not in self._idx_of:
self._idx_of[ns] = self._next_idx()
return self._idx_of[ns]
def get(self, node): # return (group number, hue)
idx = self._node_to_idx(node)
return (idx,self._hues[idx])
class GraphGenerator:
def __init__(self, analyzer, msgprinter=None):
"""analyzer: CallGraphVisitor instance"""
self.analyzer = analyzer
if msgprinter is None:
msgprinter = MsgPrinter()
self.msgprinter = msgprinter
# GraphViz docs:
# http://www.graphviz.org/doc/info/lang.html
# http://www.graphviz.org/doc/info/attrs.html
#
def to_dot(self, draw_defines, draw_uses, colored, grouped, nested_groups, annotated):
"""Return, as a string, a GraphViz .dot representation of the graph."""
analyzer = self.analyzer
# Terminology:
# - what Node calls "label" is a computer-friendly unique identifier
# for use in graphing tools
# - the "label" property of a GraphViz node is a **human-readable** name
#
# The annotation determines the human-readable name.
#
if annotated:
if grouped:
# group label includes namespace already
label_node = lambda n: n.get_annotated_name()
else:
# the node label is the only place to put the namespace info
label_node = lambda n: n.get_long_annotated_name()
else:
label_node = lambda n: n.get_short_name()
# find out which nodes are defined (can be visualized)
vis_node_list = []
for name in analyzer.nodes:
for n in analyzer.nodes[name]:
if n.defined:
vis_node_list.append(n)
# Sort by namespace for easy cluster generation.
# Secondary sort by name to make the output have a deterministic ordering.
vis_node_list.sort(key=lambda x: (x.namespace, x.name))
def find_toplevel_namespaces():
namespaces = set()
for node in vis_node_list:
namespaces.add(node.get_toplevel_namespace())
return namespaces
colorizer = Colorizer(n=len(find_toplevel_namespaces())+1)
s = """digraph G {\n"""
# enable clustering
# http://www.graphviz.org/doc/info/attrs.html#a:clusterrank
if grouped:
s += """ graph [clusterrank="local"];\n"""
# Write nodes and subgraphs
#
prev_namespace = "" # The namespace "" (for .py files) is first in vis_node_list.
namespace_stack = []
indent = ""
def update_indent():
return " " * (4*len(namespace_stack)) # 4 spaces per level
for n in vis_node_list:
# new namespace? (NOTE: nodes sorted by namespace!)
if grouped and n.namespace != prev_namespace:
if nested_groups:
# Pop the stack until the newly found namespace is within one of the
# parent namespaces (i.e. this is a sibling at that level), or until
# the stack runs out.
#
if len(namespace_stack):
m = re.match(namespace_stack[-1], n.namespace)
# The '.' check catches siblings in cases like MeshGenerator vs. Mesh.
while m is None or n.namespace[m.end()] != '.':
s += """%s}\n""" % indent # terminate previous subgraph
namespace_stack.pop()
indent = update_indent()
if not len(namespace_stack):
break
m = re.match(namespace_stack[-1], n.namespace)
namespace_stack.append(n.namespace)
indent = update_indent()
else:
if prev_namespace != "":
s += """%s}\n""" % indent # terminate previous subgraph
else:
indent = " " * 4 # first subgraph begins, start indenting
prev_namespace = n.namespace
# Begin new subgraph for this namespace (TODO: refactor the label generation).
#
# Name must begin with "cluster" to be recognized as a cluster by GraphViz.
s += """%ssubgraph cluster_%s {\n""" % (indent, n.namespace.replace('.', '__').replace('*', ''))
# translucent gray (no hue to avoid visual confusion with any group of colored nodes)
s += """%s graph [style="filled,rounded", fillcolor="#80808018", label="%s"];\n""" % (indent, n.namespace)
# add the node itself
if colored:
idx,H = colorizer.get(n)
L = max( [1.0 - 0.1*n.get_level(), 0.1] )
S = 1.0
A = 0.7 # make nodes translucent (to handle possible overlaps)
fill_RGBA = htmlize_rgb(*colorsys.hls_to_rgb(H,L,S), A=A)
if L >= 0.5:
text_RGB = htmlize_rgb(0.0, 0.0, 0.0) # black text on light nodes
else:
text_RGB = htmlize_rgb(1.0, 1.0, 1.0) # white text on dark nodes
s += """%s %s [label="%s", style="filled", fillcolor="%s", fontcolor="%s", group="%s"];\n""" % (indent, n.get_label(), label_node(n), fill_RGBA, text_RGB, idx)
else:
fill_RGBA = htmlize_rgb(1.0, 1.0, 1.0, 0.7)
idx,_ = colorizer.get(n)
s += """%s %s [label="%s", style="filled", fillcolor="%s", fontcolor="#000000", group="%s"];\n""" % (indent, n.get_label(), label_node(n), fill_RGBA, idx)
if grouped:
if nested_groups:
while len(namespace_stack):
s += """%s}\n""" % indent # terminate all remaining subgraphs
namespace_stack.pop()
indent = update_indent()
else:
s += """%s}\n""" % indent # terminate last subgraph
# Write defines relationships
#
if draw_defines:
for n in analyzer.defines_edges:
if n.defined:
for n2 in analyzer.defines_edges[n]:
if n2.defined and n2 != n:
# gray lines (so they won't visually obstruct the "uses" lines)
s += """ %s -> %s [style="dashed", color="azure4"];\n""" % (n.get_label(), n2.get_label())
# Write uses relationships
#
if draw_uses:
for n in analyzer.uses_edges:
if n.defined:
for n2 in analyzer.uses_edges[n]:
if n2.defined and n2 != n:
s += """ %s -> %s;\n""" % (n.get_label(), n2.get_label())
s += """}\n""" # terminate "digraph G {"
return s
def to_tgf(self, draw_defines, draw_uses):
"""Return, as a string, a Trivial Graph Format representation of the graph. Advanced features not available."""
analyzer = self.analyzer
s = ''
i = 1
id_map = {}
for name in analyzer.nodes:
for n in analyzer.nodes[name]:
if n.defined:
s += """%d %s\n""" % (i, n.get_short_name())
id_map[n] = i
i += 1
#else:
# print("ignoring %s" % n, file=sys.stderr)
s += """#\n"""
if draw_defines:
for n in analyzer.defines_edges:
if n.defined:
for n2 in analyzer.defines_edges[n]:
if n2.defined and n2 != n:
i1 = id_map[n]
i2 = id_map[n2]
s += """%d %d D\n""" % (i1, i2)
if draw_uses:
for n in analyzer.uses_edges:
if n.defined:
for n2 in analyzer.uses_edges[n]:
if n2.defined and n2 != n:
i1 = id_map[n]
i2 = id_map[n2]
s += """%d %d U\n""" % (i1, i2)
return s

98
pyan/main.py Normal file
View File

@@ -0,0 +1,98 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
pyan.py - Generate approximate call graphs for Python programs.
This program takes one or more Python source files, does a superficial
analysis, and constructs a directed graph of the objects in the combined
source, and how they define or use each other. The graph can be output
for rendering by e.g. GraphViz or yEd.
"""
from glob import glob
from optparse import OptionParser # TODO: migrate to argparse
from .common import MsgPrinter, MsgLevel
from .analyzer import CallGraphVisitor
from .graphgen import GraphGenerator
def main():
usage = """usage: %prog FILENAME... [--dot|--tgf]"""
desc = """Analyse one or more Python source files and generate an approximate call graph of the modules, classes and functions within them."""
parser = OptionParser(usage=usage, description=desc)
parser.add_option("--dot",
action="store_true", default=False,
help="output in GraphViz dot format")
parser.add_option("--tgf",
action="store_true", default=False,
help="output in Trivial Graph Format")
parser.add_option("-v", "--verbose",
action="store_true", default=False, dest="verbose",
help="verbose output")
parser.add_option("-V", "--very-verbose",
action="store_true", default=False, dest="very_verbose",
help="even more verbose output (mainly for debug)")
parser.add_option("-d", "--defines",
action="store_true", default=True, dest="draw_defines",
help="add edges for 'defines' relationships [default]")
parser.add_option("-n", "--no-defines",
action="store_false", default=True, dest="draw_defines",
help="do not add edges for 'defines' relationships")
parser.add_option("-u", "--uses",
action="store_true", default=True, dest="draw_uses",
help="add edges for 'uses' relationships [default]")
parser.add_option("-N", "--no-uses",
action="store_false", default=True, dest="draw_uses",
help="do not add edges for 'uses' relationships")
parser.add_option("-c", "--colored",
action="store_true", default=False, dest="colored",
help="color nodes according to namespace [dot only]")
parser.add_option("-g", "--grouped",
action="store_true", default=False, dest="grouped",
help="group nodes (create subgraphs) according to namespace [dot only]")
parser.add_option("-e", "--nested-groups",
action="store_true", default=False, dest="nested_groups",
help="create nested groups (subgraphs) for nested namespaces (implies -g) [dot only]")
parser.add_option("-a", "--annotated",
action="store_true", default=False, dest="annotated",
help="annotate with module and source line number [dot only]")
options, args = parser.parse_args()
filenames = [fn2 for fn in args for fn2 in glob(fn)]
if len(args) == 0:
parser.error('Need one or more filenames to process')
if options.nested_groups:
options.grouped = True
# TODO: use an int argument
verbosity = MsgLevel.WARNING
if options.very_verbose:
verbosity = MsgLevel.DEBUG
elif options.verbose:
verbosity = MsgLevel.INFO
m = MsgPrinter(verbosity)
# Process the set of files, TWICE: so that forward references are picked up
v = CallGraphVisitor(filenames, msgprinter=m)
for pas in range(2):
for filename in filenames:
m.message("========== pass %d for file '%s' ==========" % (pas+1, filename), level=MsgLevel.INFO)
v.process(filename)
v.postprocess()
g = GraphGenerator(v, msgprinter=m)
if options.dot:
print(g.to_dot(draw_defines=options.draw_defines,
draw_uses=options.draw_uses,
colored=options.colored,
grouped=options.grouped,
nested_groups=options.nested_groups,
annotated=options.annotated))
if options.tgf:
print(g.to_tgf(draw_defines=options.draw_defines,
draw_uses=options.draw_uses))
if __name__ == '__main__':
main()

6
visualize_pyan_architecture.sh Executable file
View File

@@ -0,0 +1,6 @@
#!/bin/bash
./pyan.py pyan/*.py --no-uses --defines --grouped --nested-groups --colored --dot --annotated >defines.dot
./pyan.py pyan/*.py --uses --no-defines --grouped --nested-groups --colored --dot --annotated >uses.dot
dot -Tsvg defines.dot >defines.svg
dot -Tsvg uses.dot >uses.svg
echo -ne "Pyan architecture: generated defines.svg and uses.svg\n"