TransPyV

English | 中文

The Viper language compiler, written in Viper itself. TransPyV is the self-hosted (bootstrapped) successor of TransPyC. It compiles Viper sources to LLVM IR, then drives llc + clang++ to produce a native Windows executable (TransPyV.exe).

用 Viper 语言编写的 Viper 编译器。TransPyV 是 TransPyC 的自举（self-hosted）后继版本。它把 Viper 源码编译为 LLVM IR，再调用 llc + clang++ 产出 Windows 原生可执行文件（TransPyV.exe）。

---

中文

这是什么

TransPyV 是 Viper 项目自举（bootstrapping）目标：用 Viper 自身重写编译器。TransPyC 是第一代（用 CPython 写的），TransPyV 是第二代（用 Viper 写，编译后用 LLVM 工具链链接）。

虽然 TransPyV 当前只在 Windows 上能跑通（App/main.py 直接 import w32.*），但它的目标是脱离 Python 运行时：将来一旦 TransPyV 能完整编译自身并产出可执行文件，就完成了 Viper 编译器的完全自举。

目录结构

TransPyV/
├── .gitignore
├── project.json             # 顶层项目配置（输出 TransPyV.exe）
├── App/                     # 编译器源码（Viper）
│   ├── main.py              # CLI 入口
│   └── lib/
│       ├── Projectrans/
│       │   ├── Config.py    # project.vpj / project.json 加载
│       │   └── Utils.py
│       └── core/
│           ├── BuildPipeline.py      # .ll → .obj → .exe
│           ├── IncludesScanner.py    # includes 目录扫描
│           ├── Phase1.py             # stub 分离（生成 .stub.ll + .text.ll）
│           ├── Phase2.py             # 多文件项目翻译
│           ├── StubMerger.py         # stub 合并
│           ├── VLogger.py            # 日志系统
│           └── Handles/              # AST 节点翻译（22 个）
│               ├── HandlesBase.py
│               ├── HandlesBody.py
│               ├── HandlesMain.py
│               ├── HandlesTranslator.py
│               ├── HandlesVar.py / Assign / AnnAssign / AugAssign
│               ├── HandlesIf / While / For
│               ├── HandlesExpr / ExprCall / ExprOps
│               ├── HandlesFunctions
│               ├── HandlesClassDef / Struct
│               ├── HandlesEnum
│               ├── HandlesImports
│               ├── HandlesReturn
│               ├── HandlesType
│               └── HandlesNonlocal
└── Test/                    # 编译器的自测用例
    ├── project.vpj
    ├── App/                 # 33 个测试 .py
    │   ├── simple_test.py
    │   ├── func_test.py / func_vtable_test.py
    │   ├── oop_test.py / vtable_test.py / virtual_dispatch_test.py
    │   ├── struct_test.py / class_test
    │   ├── closure_test.py / namespace_test.py
    │   ├── string_test.py / string_min_test.py
    │   ├── flow_test.py / for_test.py
    │   ├── llvmir_test.py
    │   └── ... (33 个)
    ├── NegativeTest/        # 错误用法负向测试
    └── Sha1Test/            # SHA1 命名空间测试

前提条件

工具	说明
python ≥ 3.10	仅当用 TransPyC 编译 TransPyV 时需要
llc	LLVM 静态编译器
clang++	mingw-w64 工具链链接器
Windows 10/11 x64	TransPyV 当前仅在 Windows 上工作（import w32.*）
至少 1GB 可用内存	TransPyV 启动时 stdlib.malloc(POOL_SIZE=1<<30)
TransPyC	用于把 TransPyV 自身编译为 .exe

目录布局要求（编译 TransPyV 时）：

D:\Users\TermiNexus\Desktop\TransPyC\
├── TransPyC\        # 编译器仓库
└── TransPyV\        # 本仓库

快速开始

# 1. 用 TransPyC 编译 TransPyV（自举的"第一阶段"）
cd D:\Users\TermiNexus\Desktop\TransPyC
python Projectrans.py --project TransPyV\project.json --clean

# 2. 产物在 TransPyV\output\TransPyV.exe
.\TransPyV\output\TransPyV.exe --help

TransPyV CLI 参数（与 TransPyC Projectrans.py 一致）：

参数	含义
--project <path>	project.json 路径（默认查找当前目录）
--src <dir>	源文件目录（覆盖 project.json）
--temp <dir>	声明接口临时目录（覆盖 project.json）
--output <dir>	输出目录（覆盖 project.json）
--phase 1|2|all	阶段：1=生成声明，2=翻译+编译，all=全部
--cc <cmd>	LLVM 编译器命令（覆盖 project.json）
--clean	清理 output/ 和 temp/
--run	编译成功后立即执行生成的可执行文件
--rebuild-includes	删除 includes.binary 预编译缓存
--clear-cache	清除 .transpyc_cache 全局缓存

架构

TransPyV 与 TransPyC 共享两阶段编译模型，但实现细节不同：

源文件 (.py)
    │
    ├─ 阶段一：AST 解析 → 翻译 → stub 分离
    │     Phase1.py / Handles/*  →  .stub.ll + .text.ll
    │
    ├─ 阶段二：stub 合并 → 完整 IR → 编译
    │     StubMerger.BuildCombinedIR
    │       (本地 stub + 依赖 stubs + 本地 text)
    │     BuildPipeline.run_pipeline
    │       (.ll → llc → .obj → clang++ → .exe)
    │
    └─ 输出可执行文件

核心模块分工：

模块	职责
App/main.py	CLI 入口：参数解析、内存池初始化、Phase 调度
lib/Projectrans/Config.py	project.json / project.vpj 加载与路径解析
lib/Projectrans/Utils.py	SHA1 计算、目录清理、杂项工具
lib/core/Phase1.py	阶段一入口：扫描 includes、生成 stub
lib/core/Phase2.py	阶段二入口：多文件项目翻译
lib/core/IncludesScanner.py	扫描 includes/ 目录、决定哪些文件需要重编译
lib/core/StubMerger.py	把多个 .stub.ll 合并为完整 IR
lib/core/BuildPipeline.py	.ll → .obj → .exe（llc + clang++）
lib/core/VLogger.py	日志系统（info/warn/error/banner/success）
lib/core/Handles/HandlesTranslator.py	Translator 类：状态管理 + translate() 入口
lib/core/Handles/HandlesBody.py	AST Body 分派（按节点类型路由到具体 Handler）
lib/core/Handles/Handles*.py (22 个)	每种 AST 节点的 LLVM IR 生成

内存模型

TransPyV 用 memhub.MemBuddy 作为全局内存池：

• POOL_SIZE = 1 GiB（1073741824）
• 注释（App/main.py 第 33 行）说明：Phase1 翻译 87 个 includes + Phase2 翻译 30 个测试文件时，512 MiB 不足
• _mbuddy 通过 sys._mbuddy = mb 之类的全局指针传递（参见 App/main.py 第 61-75 行）

与 TransPyC 的关系

维度	TransPyC	TransPyV
实现语言	CPython	Viper
自举状态	❌ 依赖 Python 运行时	✅ 目标完全自举
入口	Projectrans.py	App/main.py
AST 解析	ast 标准库	ast（includes/ast 自研）
当前状态	主路径，跑通所有测试	自举重写中
平台	Windows / Linux	Windows（依赖 w32.*）

当前 TransPyC 是"主路径"（所有 Test/ 在用），TransPyV 是"目标"——一旦 TransPyV 能编译自身产出可执行文件，Viper 就完成了完全自举。

进一步阅读

• TransPyC 编译器仓库
• Viper 语言概览
• Viper 自举路线图
• ViperOS 操作系统（用 Viper 写的 OS）

---

English

What is this

TransPyV is the Viper compiler, written in Viper itself — the bootstrapping target. TransPyC is the first generation (implemented in CPython); TransPyV is the second generation (implemented in Viper, linked with the LLVM toolchain).

While TransPyV currently only runs on Windows (its App/main.py directly imports w32.*), its goal is to escape the Python runtime: once TransPyV can compile itself end-to-end and produce a self-contained executable, Viper will be fully self-hosted.

Directory layout

TransPyV/
├── .gitignore
├── project.json             # top-level project config (outputs TransPyV.exe)
├── App/                     # compiler source (Viper)
│   ├── main.py              # CLI entry
│   └── lib/
│       ├── Projectrans/
│       │   ├── Config.py    # project.vpj / project.json loader
│       │   └── Utils.py
│       └── core/
│           ├── BuildPipeline.py      # .ll -> .obj -> .exe
│           ├── IncludesScanner.py    # includes directory scanner
│           ├── Phase1.py             # stub split (generates .stub.ll + .text.ll)
│           ├── Phase2.py             # multi-file project translation
│           ├── StubMerger.py         # stub merger
│           ├── VLogger.py            # logging
│           └── Handles/              # AST node translators (22 files)
│               ├── HandlesBase.py
│               ├── HandlesBody.py
│               ├── HandlesMain.py
│               ├── HandlesTranslator.py
│               ├── HandlesVar.py / Assign / AnnAssign / AugAssign
│               ├── HandlesIf / While / For
│               ├── HandlesExpr / ExprCall / ExprOps
│               ├── HandlesFunctions
│               ├── HandlesClassDef / Struct
│               ├── HandlesEnum
│               ├── HandlesImports
│               ├── HandlesReturn
│               ├── HandlesType
│               └── HandlesNonlocal
└── Test/                    # compiler self-test cases
    ├── project.vpj
    ├── App/                 # 33 test .py
    │   ├── simple_test.py
    │   ├── func_test.py / func_vtable_test.py
    │   ├── oop_test.py / vtable_test.py / virtual_dispatch_test.py
    │   ├── struct_test.py
    │   ├── closure_test.py / namespace_test.py
    │   ├── string_test.py / string_min_test.py
    │   ├── flow_test.py / for_test.py
    │   ├── llvmir_test.py
    │   └── ... (33 in total)
    ├── NegativeTest/        # negative tests (error cases)
    └── Sha1Test/            # SHA1 namespace tests

Prerequisites

Tool	Notes
python ≥ 3.10	only needed when compiling TransPyV with TransPyC
llc	LLVM static compiler
clang++	mingw-w64 toolchain linker
Windows 10/11 x64	TransPyV currently Windows-only (imports w32.*)
≥ 1 GiB free RAM	TransPyV allocates POOL_SIZE=1<<30 at startup
TransPyC	needed to compile TransPyV itself into an .exe

Required layout (to compile TransPyV):

D:\Users\TermiNexus\Desktop\TransPyC\
├── TransPyC\        # compiler repo
└── TransPyV\        # this repo

Quick start

# 1. Use TransPyC to compile TransPyV (the "first stage" of bootstrapping)
cd D:\Users\TermiNexus\Desktop\TransPyC
python Projectrans.py --project TransPyV\project.json --clean

# 2. Output is at TransPyV\output\TransPyV.exe
.\TransPyV\output\TransPyV.exe --help

TransPyV CLI flags (mirrors Projectrans.py):

Flag	Meaning
--project <path>	project.json path (default: search current dir)
--src <dir>	source dir (overrides project.json)
--temp <dir>	declaration temp dir (overrides project.json)
--output <dir>	output dir (overrides project.json)
--phase 1|2|all	phase: 1=decl only, 2=translate+compile, all=both
--cc <cmd>	LLVM compiler command (overrides project.json)
--clean	clean output/ and temp/
--run	execute the produced executable on success
--rebuild-includes	delete includes.binary precompiled cache
--clear-cache	clear .transpyc_cache global cache

Architecture

TransPyV shares the two-phase model with TransPyC, with these differences:

Source files (.py)
    │
    ├─ Phase 1: AST parse -> translate -> stub split
    │     Phase1.py / Handles/*  ->  .stub.ll + .text.ll
    │
    ├─ Phase 2: stub merge -> full IR -> compile
    │     StubMerger.BuildCombinedIR
    │       (local stub + dep stubs + local text)
    │     BuildPipeline.run_pipeline
    │       (.ll -> llc -> .obj -> clang++ -> .exe)
    │
    └─ Native executable

Module responsibilities:

Module	Responsibility
App/main.py	CLI entry: arg parse, memory pool init, phase dispatch
lib/Projectrans/Config.py	project.json / project.vpj loader + path resolution
lib/Projectrans/Utils.py	SHA1, dir cleanup, misc utilities
lib/core/Phase1.py	Phase 1 entry: scan includes, generate stubs
lib/core/Phase2.py	Phase 2 entry: multi-file project translation
lib/core/IncludesScanner.py	scan includes/, decide what to rebuild
lib/core/StubMerger.py	merge multiple .stub.ll into a single IR
lib/core/BuildPipeline.py	.ll -> .obj -> .exe (llc + clang++)
lib/core/VLogger.py	logging (info/warn/error/banner/success)
lib/core/Handles/HandlesTranslator.py	Translator class: state + translate()
lib/core/Handles/HandlesBody.py	AST body dispatcher (route by node type)
lib/core/Handles/Handles*.py (22)	one LLVM-IR generator per AST node type

Memory model

TransPyV uses memhub.MemBuddy as a global arena:

• POOL_SIZE = 1 GiB (1073741824)
• The comment on App/main.py line 33 explains: Phase1 translates 87 includes files + Phase2 translates 30 test files, 512 MiB is not enough
• _mbuddy is propagated via globals like sys._mbuddy = mb (see App/main.py lines 61–75)

Relation to TransPyC

Dimension	TransPyC	TransPyV
Implementation	CPython	Viper
Self-hosted?	No (Python runtime required)	Target: yes
Entry	Projectrans.py	App/main.py
AST parsing	standard ast	ast (self-implemented in includes/ast)
Current status	Main path, all tests pass	Bootstrapping rewrite in progress
Platform	Windows / Linux	Windows only (depends on w32.*)

Currently TransPyC is the main path (all Test/ use it); TransPyV is the target. Once TransPyV can compile itself into an executable, Viper will be fully self-hosted.

Further reading

• TransPyC compiler repo
• Viper language overview
• Viper bootstrapping roadmap
• ViperOS (an OS written in Viper)