perf(compile): 批次加权分桶消除长尾 + 共享符号表去重复序列化 + typedef 收集走 AST 缓存

- Phase2 批次切分:原按文件数均分(batch_size=ceil(N/n)),大文件集中批次拖长尾、worker 空等;改为按源文件大小 LPT 贪心分桶,各批次总成本均匀(任务颗粒度不均匀)
- 共享符号表序列化:shared_data 同时含 symbol_table 对象与 symbol_table_bytes 双重序列化(_shared_sym.json 4.65MB 近半冗余);统一为仅传 symbol_table_bytes,worker/批次端从字节反序列化(includes 与 App 两阶段同步修改)
- Phase1 _collect_typedef_map / _process_file_pyi:直读+ast.parse 改为 parse_python_file mtime AST 缓存,Phase1 内重复收集与 Phase2 主进程解析共享同一缓存
This commit is contained in:
TermiNexus committed 2026-08-18 05:36:12 +08:00
1 parent 7917be0fa1
commit e3886a244d
2 files changed
+34 -13

No files matched your search

+6 -6
View File
@@ -357,9 +357,9 @@ class Phase1Generator:
continue
pyi_path: str = os.path.join(self.temp_dir, fname)
try:
with open(pyi_path, 'r', encoding='utf-8') as f:
content: str = f.read()
tree: ast.Module = ast.parse(content)
content, tree = parse_python_file(pyi_path)
if tree is None:
continue
for node in ast.iter_child_nodes(tree):
if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name):
var_name: str = node.target.id
@@ -676,9 +676,9 @@ class Phase1Generator:
continue
pyi_path: str = os.path.join(self.temp_dir, fname)
try:
with open(pyi_path, 'r', encoding='utf-8') as f:
content: str = f.read()
tree: ast.Module = ast.parse(content)
content, tree = parse_python_file(pyi_path)
if tree is None:
continue
for node in ast.iter_child_nodes(tree):
if isinstance(node, ast.ClassDef):
is_enum: bool = False