GVSDS/TransPyV
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add bilingual README

Browse Source
This commit is contained in:
Viper
2026-07-19 13:33:50 +08:00
parent e7eaf3e9a1
commit 3cf67e262f
1 changed files with 349 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

349
README.md Normal file
View File

@@ -0,0 +1,349 @@
# TransPyV
[English](#english) | [中文](#中文)
The **Viper language compiler, written in Viper itself**. TransPyV is the self-hosted (bootstrapped) successor of [TransPyC](https://git.gvsds.com/GVSDS/TransPyC). It compiles Viper sources to LLVM IR, then drives `llc` + `clang++` to produce a native Windows executable (`TransPyV.exe`).
**用 Viper 语言编写的 Viper 编译器**。TransPyV 是 [TransPyC](https://git.gvsds.com/GVSDS/TransPyC) 的自举（self-hosted）后继版本。它把 Viper 源码编译为 LLVM IR，再调用 `llc` + `clang++` 产出 Windows 原生可执行文件（`TransPyV.exe`）。
---
<a id="中文"></a>
## 中文
### 这是什么
`TransPyV` 是 Viper 项目**自举（bootstrapping）目标**：用 Viper 自身重写编译器。TransPyC 是第一代（用 CPython 写的），TransPyV 是第二代（用 Viper 写，编译后用 LLVM 工具链链接）。
虽然 TransPyV 当前只在 Windows 上能跑通（`App/main.py` 直接 import `w32.*`），但它的目标是**脱离 Python 运行时**：将来一旦 TransPyV 能完整编译自身并产出可执行文件，就完成了 Viper 编译器的完全自举。
### 目录结构
```
TransPyV/
├── .gitignore
├── project.json # 顶层项目配置（输出 TransPyV.exe）
├── App/ # 编译器源码（Viper）
│ ├── main.py # CLI 入口
│ └── lib/
│ ├── Projectrans/
│ │ ├── Config.py # project.vpj / project.json 加载
│ │ └── Utils.py
│ └── core/
│ ├── BuildPipeline.py # .ll → .obj → .exe
│ ├── IncludesScanner.py # includes 目录扫描
│ ├── Phase1.py # stub 分离（生成 .stub.ll + .text.ll）
│ ├── Phase2.py # 多文件项目翻译
│ ├── StubMerger.py # stub 合并
│ ├── VLogger.py # 日志系统
│ └── Handles/ # AST 节点翻译（22 个）
│ ├── HandlesBase.py
│ ├── HandlesBody.py
│ ├── HandlesMain.py
│ ├── HandlesTranslator.py
│ ├── HandlesVar.py / Assign / AnnAssign / AugAssign
│ ├── HandlesIf / While / For
│ ├── HandlesExpr / ExprCall / ExprOps
│ ├── HandlesFunctions
│ ├── HandlesClassDef / Struct
│ ├── HandlesEnum
│ ├── HandlesImports
│ ├── HandlesReturn
│ ├── HandlesType
│ └── HandlesNonlocal
└── Test/ # 编译器的自测用例
├── project.vpj
├── App/ # 33 个测试 .py
│ ├── simple_test.py
│ ├── func_test.py / func_vtable_test.py
│ ├── oop_test.py / vtable_test.py / virtual_dispatch_test.py
│ ├── struct_test.py / class_test
│ ├── closure_test.py / namespace_test.py
│ ├── string_test.py / string_min_test.py
│ ├── flow_test.py / for_test.py
│ ├── llvmir_test.py
│ └── ... (33 个)
├── NegativeTest/ # 错误用法负向测试
└── Sha1Test/ # SHA1 命名空间测试
```
### 前提条件
| 工具 | 说明 |
|------|------|
| `python` ≥ 3.10 | **仅当用 TransPyC 编译 TransPyV 时**需要 |
| `llc` | LLVM 静态编译器 |
| `clang++` | mingw-w64 工具链链接器 |
| Windows 10/11 x64 | TransPyV 当前仅在 Windows 上工作（import w32.*） |
| 至少 1GB 可用内存 | TransPyV 启动时 `stdlib.malloc(POOL_SIZE=1<<30)` |
| [TransPyC](https://git.gvsds.com/GVSDS/TransPyC) | 用于把 TransPyV 自身编译为 `.exe` |
目录布局要求（编译 TransPyV 时）：
```
D:\Users\TermiNexus\Desktop\TransPyC\
├── TransPyC\ # 编译器仓库
└── TransPyV\ # 本仓库
```
### 快速开始
```powershell
# 1. 用 TransPyC 编译 TransPyV（自举的"第一阶段"）
cd D:\Users\TermiNexus\Desktop\TransPyC
python Projectrans.py --project TransPyV\project.json --clean
# 2. 产物在 TransPyV\output\TransPyV.exe
.\TransPyV\output\TransPyV.exe --help
```
TransPyV CLI 参数（与 TransPyC `Projectrans.py` 一致）：
| 参数 | 含义 |
|------|------|
| `--project <path>` | `project.json` 路径（默认查找当前目录） |
| `--src <dir>` | 源文件目录（覆盖 `project.json`） |
| `--temp <dir>` | 声明接口临时目录（覆盖 `project.json`） |
| `--output <dir>` | 输出目录（覆盖 `project.json`） |
| `--phase 1\|2\|all` | 阶段：1=生成声明，2=翻译+编译，all=全部 |
| `--cc <cmd>` | LLVM 编译器命令（覆盖 `project.json`） |
| `--clean` | 清理 `output/` 和 `temp/` |
| `--run` | 编译成功后立即执行生成的可执行文件 |
| `--rebuild-includes` | 删除 `includes.binary` 预编译缓存 |
| `--clear-cache` | 清除 `.transpyc_cache` 全局缓存 |
### 架构
TransPyV 与 TransPyC 共享两阶段编译模型，但实现细节不同：
```
源文件 (.py)
│
├─ 阶段一：AST 解析 → 翻译 → stub 分离
│ Phase1.py / Handles/* → .stub.ll + .text.ll
│
├─ 阶段二：stub 合并 → 完整 IR → 编译
│ StubMerger.BuildCombinedIR
│ (本地 stub + 依赖 stubs + 本地 text)
│ BuildPipeline.run_pipeline
│ (.ll → llc → .obj → clang++ → .exe)
│
└─ 输出可执行文件
```
**核心模块分工：**
| 模块 | 职责 |
|------|------|
| `App/main.py` | CLI 入口：参数解析、内存池初始化、Phase 调度 |
| `lib/Projectrans/Config.py` | `project.json` / `project.vpj` 加载与路径解析 |
| `lib/Projectrans/Utils.py` | SHA1 计算、目录清理、杂项工具 |
| `lib/core/Phase1.py` | 阶段一入口：扫描 includes、生成 stub |
| `lib/core/Phase2.py` | 阶段二入口：多文件项目翻译 |
| `lib/core/IncludesScanner.py` | 扫描 `includes/` 目录、决定哪些文件需要重编译 |
| `lib/core/StubMerger.py` | 把多个 `.stub.ll` 合并为完整 IR |
| `lib/core/BuildPipeline.py` | `.ll` → `.obj` → `.exe`（llc + clang++） |
| `lib/core/VLogger.py` | 日志系统（info/warn/error/banner/success） |
| `lib/core/Handles/HandlesTranslator.py` | `Translator` 类：状态管理 + `translate()` 入口 |
| `lib/core/Handles/HandlesBody.py` | AST Body 分派（按节点类型路由到具体 Handler） |
| `lib/core/Handles/Handles*.py` (22 个) | 每种 AST 节点的 LLVM IR 生成 |
### 内存模型
TransPyV 用 `memhub.MemBuddy` 作为全局内存池：
- **POOL_SIZE = 1 GiB**（`1073741824`）
- 注释（`App/main.py` 第 33 行）说明：Phase1 翻译 87 个 includes + Phase2 翻译 30 个测试文件时，512 MiB 不足
- `_mbuddy` 通过 `sys._mbuddy = mb` 之类的全局指针传递（参见 `App/main.py` 第 61-75 行）
### 与 TransPyC 的关系
| 维度 | TransPyC | TransPyV |
|------|----------|----------|
| 实现语言 | CPython | Viper |
| 自举状态 | ❌ 依赖 Python 运行时 | ✅ 目标完全自举 |
| 入口 | `Projectrans.py` | `App/main.py` |
| AST 解析 | `ast` 标准库 | `ast`（includes/ast 自研） |
| 当前状态 | 主路径，跑通所有测试 | 自举重写中 |
| 平台 | Windows / Linux | Windows（依赖 w32.*） |
当前 TransPyC 是"主路径"（所有 Test/ 在用），TransPyV 是"目标"——一旦 TransPyV 能编译自身产出可执行文件，Viper 就完成了完全自举。
### 进一步阅读
- [TransPyC 编译器仓库](https://git.gvsds.com/GVSDS/TransPyC)
- [Viper 语言概览](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/01-overview.md)
- [Viper 自举路线图](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/13-bootstrapping.md)
- [ViperOS 操作系统（用 Viper 写的 OS）](https://git.gvsds.com/GVSDS/ViperOS)
---
<a id="english"></a>
## English
### What is this
`TransPyV` is the **Viper compiler, written in Viper itself** — the bootstrapping target. TransPyC is the first generation (implemented in CPython); TransPyV is the second generation (implemented in Viper, linked with the LLVM toolchain).
While TransPyV currently only runs on Windows (its `App/main.py` directly imports `w32.*`), its goal is to **escape the Python runtime**: once TransPyV can compile itself end-to-end and produce a self-contained executable, Viper will be fully self-hosted.
### Directory layout
```
TransPyV/
├── .gitignore
├── project.json # top-level project config (outputs TransPyV.exe)
├── App/ # compiler source (Viper)
│ ├── main.py # CLI entry
│ └── lib/
│ ├── Projectrans/
│ │ ├── Config.py # project.vpj / project.json loader
│ │ └── Utils.py
│ └── core/
│ ├── BuildPipeline.py # .ll -> .obj -> .exe
│ ├── IncludesScanner.py # includes directory scanner
│ ├── Phase1.py # stub split (generates .stub.ll + .text.ll)
│ ├── Phase2.py # multi-file project translation
│ ├── StubMerger.py # stub merger
│ ├── VLogger.py # logging
│ └── Handles/ # AST node translators (22 files)
│ ├── HandlesBase.py
│ ├── HandlesBody.py
│ ├── HandlesMain.py
│ ├── HandlesTranslator.py
│ ├── HandlesVar.py / Assign / AnnAssign / AugAssign
│ ├── HandlesIf / While / For
│ ├── HandlesExpr / ExprCall / ExprOps
│ ├── HandlesFunctions
│ ├── HandlesClassDef / Struct
│ ├── HandlesEnum
│ ├── HandlesImports
│ ├── HandlesReturn
│ ├── HandlesType
│ └── HandlesNonlocal
└── Test/ # compiler self-test cases
├── project.vpj
├── App/ # 33 test .py
│ ├── simple_test.py
│ ├── func_test.py / func_vtable_test.py
│ ├── oop_test.py / vtable_test.py / virtual_dispatch_test.py
│ ├── struct_test.py
│ ├── closure_test.py / namespace_test.py
│ ├── string_test.py / string_min_test.py
│ ├── flow_test.py / for_test.py
│ ├── llvmir_test.py
│ └── ... (33 in total)
├── NegativeTest/ # negative tests (error cases)
└── Sha1Test/ # SHA1 namespace tests
```
### Prerequisites
| Tool | Notes |
|------|-------|
| `python` ≥ 3.10 | only needed when compiling TransPyV with TransPyC |
| `llc` | LLVM static compiler |
| `clang++` | mingw-w64 toolchain linker |
| Windows 10/11 x64 | TransPyV currently Windows-only (imports `w32.*`) |
| ≥ 1 GiB free RAM | TransPyV allocates `POOL_SIZE=1<<30` at startup |
| [TransPyC](https://git.gvsds.com/GVSDS/TransPyC) | needed to compile TransPyV itself into an `.exe` |
Required layout (to compile TransPyV):
```
D:\Users\TermiNexus\Desktop\TransPyC\
├── TransPyC\ # compiler repo
└── TransPyV\ # this repo
```
### Quick start
```powershell
# 1. Use TransPyC to compile TransPyV (the "first stage" of bootstrapping)
cd D:\Users\TermiNexus\Desktop\TransPyC
python Projectrans.py --project TransPyV\project.json --clean
# 2. Output is at TransPyV\output\TransPyV.exe
.\TransPyV\output\TransPyV.exe --help
```
TransPyV CLI flags (mirrors `Projectrans.py`):
| Flag | Meaning |
|------|---------|
| `--project <path>` | `project.json` path (default: search current dir) |
| `--src <dir>` | source dir (overrides `project.json`) |
| `--temp <dir>` | declaration temp dir (overrides `project.json`) |
| `--output <dir>` | output dir (overrides `project.json`) |
| `--phase 1\|2\|all` | phase: 1=decl only, 2=translate+compile, all=both |
| `--cc <cmd>` | LLVM compiler command (overrides `project.json`) |
| `--clean` | clean `output/` and `temp/` |
| `--run` | execute the produced executable on success |
| `--rebuild-includes` | delete `includes.binary` precompiled cache |
| `--clear-cache` | clear `.transpyc_cache` global cache |
### Architecture
TransPyV shares the two-phase model with TransPyC, with these differences:
```
Source files (.py)
│
├─ Phase 1: AST parse -> translate -> stub split
│ Phase1.py / Handles/* -> .stub.ll + .text.ll
│
├─ Phase 2: stub merge -> full IR -> compile
│ StubMerger.BuildCombinedIR
│ (local stub + dep stubs + local text)
│ BuildPipeline.run_pipeline
│ (.ll -> llc -> .obj -> clang++ -> .exe)
│
└─ Native executable
```
**Module responsibilities:**
| Module | Responsibility |
|--------|---------------|
| `App/main.py` | CLI entry: arg parse, memory pool init, phase dispatch |
| `lib/Projectrans/Config.py` | `project.json` / `project.vpj` loader + path resolution |
| `lib/Projectrans/Utils.py` | SHA1, dir cleanup, misc utilities |
| `lib/core/Phase1.py` | Phase 1 entry: scan includes, generate stubs |
| `lib/core/Phase2.py` | Phase 2 entry: multi-file project translation |
| `lib/core/IncludesScanner.py` | scan `includes/`, decide what to rebuild |
| `lib/core/StubMerger.py` | merge multiple `.stub.ll` into a single IR |
| `lib/core/BuildPipeline.py` | `.ll` -> `.obj` -> `.exe` (llc + clang++) |
| `lib/core/VLogger.py` | logging (info/warn/error/banner/success) |
| `lib/core/Handles/HandlesTranslator.py` | `Translator` class: state + `translate()` |
| `lib/core/Handles/HandlesBody.py` | AST body dispatcher (route by node type) |
| `lib/core/Handles/Handles*.py` (22) | one LLVM-IR generator per AST node type |
### Memory model
TransPyV uses `memhub.MemBuddy` as a global arena:
- **POOL_SIZE = 1 GiB** (`1073741824`)
- The comment on `App/main.py` line 33 explains: Phase1 translates 87 includes files + Phase2 translates 30 test files, 512 MiB is not enough
- `_mbuddy` is propagated via globals like `sys._mbuddy = mb` (see `App/main.py` lines 61–75)
### Relation to TransPyC
| Dimension | TransPyC | TransPyV |
|-----------|----------|----------|
| Implementation | CPython | Viper |
| Self-hosted? | No (Python runtime required) | Target: yes |
| Entry | `Projectrans.py` | `App/main.py` |
| AST parsing | standard `ast` | `ast` (self-implemented in `includes/ast`) |
| Current status | Main path, all tests pass | Bootstrapping rewrite in progress |
| Platform | Windows / Linux | Windows only (depends on `w32.*`) |
Currently TransPyC is the main path (all `Test/` use it); TransPyV is the target. Once TransPyV can compile itself into an executable, Viper will be fully self-hosted.
### Further reading
- [TransPyC compiler repo](https://git.gvsds.com/GVSDS/TransPyC)
- [Viper language overview](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/01-overview.md)
- [Viper bootstrapping roadmap](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/13-bootstrapping.md)
- [ViperOS (an OS written in Viper)](https://git.gvsds.com/GVSDS/ViperOS)