# TransPyV

[English](#english) | [中文](#中文)

The **Viper language compiler, written in Viper itself**. TransPyV is the self-hosted (bootstrapped) successor of [TransPyC](https://git.gvsds.com/GVSDS/TransPyC). It compiles Viper sources to LLVM IR, then drives `llc` + `clang++` to produce a native Windows executable (`TransPyV.exe`).

**用 Viper 语言编写的 Viper 编译器**。TransPyV 是 [TransPyC](https://git.gvsds.com/GVSDS/TransPyC) 的自举（self-hosted）后继版本。它把 Viper 源码编译为 LLVM IR，再调用 `llc` + `clang++` 产出 Windows 原生可执行文件（`TransPyV.exe`）。

---

<a id="中文"></a>

## 中文

### 这是什么

`TransPyV` 是 Viper 项目**自举（bootstrapping）目标**：用 Viper 自身重写编译器。TransPyC 是第一代（用 CPython 写的），TransPyV 是第二代（用 Viper 写，编译后用 LLVM 工具链链接）。

虽然 TransPyV 当前只在 Windows 上能跑通（`App/main.py` 直接 import `w32.*`），但它的目标是**脱离 Python 运行时**：将来一旦 TransPyV 能完整编译自身并产出可执行文件，就完成了 Viper 编译器的完全自举。

### 目录结构

```
TransPyV/
├── .gitignore
├── project.json             # 顶层项目配置（输出 TransPyV.exe）
├── App/                     # 编译器源码（Viper）
│   ├── main.py              # CLI 入口
│   └── lib/
│       ├── Projectrans/
│       │   ├── Config.py    # project.vpj / project.json 加载
│       │   └── Utils.py
│       └── core/
│           ├── BuildPipeline.py      # .ll → .obj → .exe
│           ├── IncludesScanner.py    # includes 目录扫描
│           ├── Phase1.py             # stub 分离（生成 .stub.ll + .text.ll）
│           ├── Phase2.py             # 多文件项目翻译
│           ├── StubMerger.py         # stub 合并
│           ├── VLogger.py            # 日志系统
│           └── Handles/              # AST 节点翻译（22 个）
│               ├── HandlesBase.py
│               ├── HandlesBody.py
│               ├── HandlesMain.py
│               ├── HandlesTranslator.py
│               ├── HandlesVar.py / Assign / AnnAssign / AugAssign
│               ├── HandlesIf / While / For
│               ├── HandlesExpr / ExprCall / ExprOps
│               ├── HandlesFunctions
│               ├── HandlesClassDef / Struct
│               ├── HandlesEnum
│               ├── HandlesImports
│               ├── HandlesReturn
│               ├── HandlesType
│               └── HandlesNonlocal
└── Test/                    # 编译器的自测用例
    ├── project.vpj
    ├── App/                 # 33 个测试 .py
    │   ├── simple_test.py
    │   ├── func_test.py / func_vtable_test.py
    │   ├── oop_test.py / vtable_test.py / virtual_dispatch_test.py
    │   ├── struct_test.py / class_test
    │   ├── closure_test.py / namespace_test.py
    │   ├── string_test.py / string_min_test.py
    │   ├── flow_test.py / for_test.py
    │   ├── llvmir_test.py
    │   └── ... (33 个)
    ├── NegativeTest/        # 错误用法负向测试
    └── Sha1Test/            # SHA1 命名空间测试
```

### 前提条件

| 工具 | 说明 |
|------|------|
| `python` ≥ 3.10 | **仅当用 TransPyC 编译 TransPyV 时**需要 |
| `llc` | LLVM 静态编译器 |
| `clang++` | mingw-w64 工具链链接器 |
| Windows 10/11 x64 | TransPyV 当前仅在 Windows 上工作（import w32.*） |
| 至少 1GB 可用内存 | TransPyV 启动时 `stdlib.malloc(POOL_SIZE=1<<30)` |
| [TransPyC](https://git.gvsds.com/GVSDS/TransPyC) | 用于把 TransPyV 自身编译为 `.exe` |

目录布局要求（编译 TransPyV 时）：

```
D:\Users\TermiNexus\Desktop\TransPyC\
├── TransPyC\        # 编译器仓库
└── TransPyV\        # 本仓库
```

### 快速开始

```powershell
# 1. 用 TransPyC 编译 TransPyV（自举的"第一阶段"）
cd D:\Users\TermiNexus\Desktop\TransPyC
python Projectrans.py --project TransPyV\project.json --clean

# 2. 产物在 TransPyV\output\TransPyV.exe
.\TransPyV\output\TransPyV.exe --help
```

TransPyV CLI 参数（与 TransPyC `Projectrans.py` 一致）：

| 参数 | 含义 |
|------|------|
| `--project <path>` | `project.json` 路径（默认查找当前目录） |
| `--src <dir>` | 源文件目录（覆盖 `project.json`） |
| `--temp <dir>` | 声明接口临时目录（覆盖 `project.json`） |
| `--output <dir>` | 输出目录（覆盖 `project.json`） |
| `--phase 1\|2\|all` | 阶段：1=生成声明，2=翻译+编译，all=全部 |
| `--cc <cmd>` | LLVM 编译器命令（覆盖 `project.json`） |
| `--clean` | 清理 `output/` 和 `temp/` |
| `--run` | 编译成功后立即执行生成的可执行文件 |
| `--rebuild-includes` | 删除 `includes.binary` 预编译缓存 |
| `--clear-cache` | 清除 `.transpyc_cache` 全局缓存 |

### 架构

TransPyV 与 TransPyC 共享两阶段编译模型，但实现细节不同：

```
源文件 (.py)
    │
    ├─ 阶段一：AST 解析 → 翻译 → stub 分离
    │     Phase1.py / Handles/*  →  .stub.ll + .text.ll
    │
    ├─ 阶段二：stub 合并 → 完整 IR → 编译
    │     StubMerger.BuildCombinedIR
    │       (本地 stub + 依赖 stubs + 本地 text)
    │     BuildPipeline.run_pipeline
    │       (.ll → llc → .obj → clang++ → .exe)
    │
    └─ 输出可执行文件
```

**核心模块分工：**

| 模块 | 职责 |
|------|------|
| `App/main.py` | CLI 入口：参数解析、内存池初始化、Phase 调度 |
| `lib/Projectrans/Config.py` | `project.json` / `project.vpj` 加载与路径解析 |
| `lib/Projectrans/Utils.py` | SHA1 计算、目录清理、杂项工具 |
| `lib/core/Phase1.py` | 阶段一入口：扫描 includes、生成 stub |
| `lib/core/Phase2.py` | 阶段二入口：多文件项目翻译 |
| `lib/core/IncludesScanner.py` | 扫描 `includes/` 目录、决定哪些文件需要重编译 |
| `lib/core/StubMerger.py` | 把多个 `.stub.ll` 合并为完整 IR |
| `lib/core/BuildPipeline.py` | `.ll` → `.obj` → `.exe`（llc + clang++） |
| `lib/core/VLogger.py` | 日志系统（info/warn/error/banner/success） |
| `lib/core/Handles/HandlesTranslator.py` | `Translator` 类：状态管理 + `translate()` 入口 |
| `lib/core/Handles/HandlesBody.py` | AST Body 分派（按节点类型路由到具体 Handler） |
| `lib/core/Handles/Handles*.py` (22 个) | 每种 AST 节点的 LLVM IR 生成 |

### 内存模型

TransPyV 用 `memhub.MemBuddy` 作为全局内存池：

- **POOL_SIZE = 1 GiB**（`1073741824`）
- 注释（`App/main.py` 第 33 行）说明：Phase1 翻译 87 个 includes + Phase2 翻译 30 个测试文件时，512 MiB 不足
- `_mbuddy` 通过 `sys._mbuddy = mb` 之类的全局指针传递（参见 `App/main.py` 第 61-75 行）

### 与 TransPyC 的关系

| 维度 | TransPyC | TransPyV |
|------|----------|----------|
| 实现语言 | CPython | Viper |
| 自举状态 | ❌ 依赖 Python 运行时 | ✅ 目标完全自举 |
| 入口 | `Projectrans.py` | `App/main.py` |
| AST 解析 | `ast` 标准库 | `ast`（includes/ast 自研） |
| 当前状态 | 主路径，跑通所有测试 | 自举重写中 |
| 平台 | Windows / Linux | Windows（依赖 w32.*） |

当前 TransPyC 是"主路径"（所有 Test/ 在用），TransPyV 是"目标"——一旦 TransPyV 能编译自身产出可执行文件，Viper 就完成了完全自举。

### 进一步阅读

- [TransPyC 编译器仓库](https://git.gvsds.com/GVSDS/TransPyC)
- [Viper 语言概览](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/01-overview.md)
- [Viper 自举路线图](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/13-bootstrapping.md)
- [ViperOS 操作系统（用 Viper 写的 OS）](https://git.gvsds.com/GVSDS/ViperOS)

---

<a id="english"></a>

## English

### What is this

`TransPyV` is the **Viper compiler, written in Viper itself** — the bootstrapping target. TransPyC is the first generation (implemented in CPython); TransPyV is the second generation (implemented in Viper, linked with the LLVM toolchain).

While TransPyV currently only runs on Windows (its `App/main.py` directly imports `w32.*`), its goal is to **escape the Python runtime**: once TransPyV can compile itself end-to-end and produce a self-contained executable, Viper will be fully self-hosted.

### Directory layout

```
TransPyV/
├── .gitignore
├── project.json             # top-level project config (outputs TransPyV.exe)
├── App/                     # compiler source (Viper)
│   ├── main.py              # CLI entry
│   └── lib/
│       ├── Projectrans/
│       │   ├── Config.py    # project.vpj / project.json loader
│       │   └── Utils.py
│       └── core/
│           ├── BuildPipeline.py      # .ll -> .obj -> .exe
│           ├── IncludesScanner.py    # includes directory scanner
│           ├── Phase1.py             # stub split (generates .stub.ll + .text.ll)
│           ├── Phase2.py             # multi-file project translation
│           ├── StubMerger.py         # stub merger
│           ├── VLogger.py            # logging
│           └── Handles/              # AST node translators (22 files)
│               ├── HandlesBase.py
│               ├── HandlesBody.py
│               ├── HandlesMain.py
│               ├── HandlesTranslator.py
│               ├── HandlesVar.py / Assign / AnnAssign / AugAssign
│               ├── HandlesIf / While / For
│               ├── HandlesExpr / ExprCall / ExprOps
│               ├── HandlesFunctions
│               ├── HandlesClassDef / Struct
│               ├── HandlesEnum
│               ├── HandlesImports
│               ├── HandlesReturn
│               ├── HandlesType
│               └── HandlesNonlocal
└── Test/                    # compiler self-test cases
    ├── project.vpj
    ├── App/                 # 33 test .py
    │   ├── simple_test.py
    │   ├── func_test.py / func_vtable_test.py
    │   ├── oop_test.py / vtable_test.py / virtual_dispatch_test.py
    │   ├── struct_test.py
    │   ├── closure_test.py / namespace_test.py
    │   ├── string_test.py / string_min_test.py
    │   ├── flow_test.py / for_test.py
    │   ├── llvmir_test.py
    │   └── ... (33 in total)
    ├── NegativeTest/        # negative tests (error cases)
    └── Sha1Test/            # SHA1 namespace tests
```

### Prerequisites

| Tool | Notes |
|------|-------|
| `python` ≥ 3.10 | only needed when compiling TransPyV with TransPyC |
| `llc` | LLVM static compiler |
| `clang++` | mingw-w64 toolchain linker |
| Windows 10/11 x64 | TransPyV currently Windows-only (imports `w32.*`) |
| ≥ 1 GiB free RAM | TransPyV allocates `POOL_SIZE=1<<30` at startup |
| [TransPyC](https://git.gvsds.com/GVSDS/TransPyC) | needed to compile TransPyV itself into an `.exe` |

Required layout (to compile TransPyV):

```
D:\Users\TermiNexus\Desktop\TransPyC\
├── TransPyC\        # compiler repo
└── TransPyV\        # this repo
```

### Quick start

```powershell
# 1. Use TransPyC to compile TransPyV (the "first stage" of bootstrapping)
cd D:\Users\TermiNexus\Desktop\TransPyC
python Projectrans.py --project TransPyV\project.json --clean

# 2. Output is at TransPyV\output\TransPyV.exe
.\TransPyV\output\TransPyV.exe --help
```

TransPyV CLI flags (mirrors `Projectrans.py`):

| Flag | Meaning |
|------|---------|
| `--project <path>` | `project.json` path (default: search current dir) |
| `--src <dir>` | source dir (overrides `project.json`) |
| `--temp <dir>` | declaration temp dir (overrides `project.json`) |
| `--output <dir>` | output dir (overrides `project.json`) |
| `--phase 1\|2\|all` | phase: 1=decl only, 2=translate+compile, all=both |
| `--cc <cmd>` | LLVM compiler command (overrides `project.json`) |
| `--clean` | clean `output/` and `temp/` |
| `--run` | execute the produced executable on success |
| `--rebuild-includes` | delete `includes.binary` precompiled cache |
| `--clear-cache` | clear `.transpyc_cache` global cache |

### Architecture

TransPyV shares the two-phase model with TransPyC, with these differences:

```
Source files (.py)
    │
    ├─ Phase 1: AST parse -> translate -> stub split
    │     Phase1.py / Handles/*  ->  .stub.ll + .text.ll
    │
    ├─ Phase 2: stub merge -> full IR -> compile
    │     StubMerger.BuildCombinedIR
    │       (local stub + dep stubs + local text)
    │     BuildPipeline.run_pipeline
    │       (.ll -> llc -> .obj -> clang++ -> .exe)
    │
    └─ Native executable
```

**Module responsibilities:**

| Module | Responsibility |
|--------|---------------|
| `App/main.py` | CLI entry: arg parse, memory pool init, phase dispatch |
| `lib/Projectrans/Config.py` | `project.json` / `project.vpj` loader + path resolution |
| `lib/Projectrans/Utils.py` | SHA1, dir cleanup, misc utilities |
| `lib/core/Phase1.py` | Phase 1 entry: scan includes, generate stubs |
| `lib/core/Phase2.py` | Phase 2 entry: multi-file project translation |
| `lib/core/IncludesScanner.py` | scan `includes/`, decide what to rebuild |
| `lib/core/StubMerger.py` | merge multiple `.stub.ll` into a single IR |
| `lib/core/BuildPipeline.py` | `.ll` -> `.obj` -> `.exe` (llc + clang++) |
| `lib/core/VLogger.py` | logging (info/warn/error/banner/success) |
| `lib/core/Handles/HandlesTranslator.py` | `Translator` class: state + `translate()` |
| `lib/core/Handles/HandlesBody.py` | AST body dispatcher (route by node type) |
| `lib/core/Handles/Handles*.py` (22) | one LLVM-IR generator per AST node type |

### Memory model

TransPyV uses `memhub.MemBuddy` as a global arena:

- **POOL_SIZE = 1 GiB** (`1073741824`)
- The comment on `App/main.py` line 33 explains: Phase1 translates 87 includes files + Phase2 translates 30 test files, 512 MiB is not enough
- `_mbuddy` is propagated via globals like `sys._mbuddy = mb` (see `App/main.py` lines 61–75)

### Relation to TransPyC

| Dimension | TransPyC | TransPyV |
|-----------|----------|----------|
| Implementation | CPython | Viper |
| Self-hosted? | No (Python runtime required) | Target: yes |
| Entry | `Projectrans.py` | `App/main.py` |
| AST parsing | standard `ast` | `ast` (self-implemented in `includes/ast`) |
| Current status | Main path, all tests pass | Bootstrapping rewrite in progress |
| Platform | Windows / Linux | Windows only (depends on `w32.*`) |

Currently TransPyC is the main path (all `Test/` use it); TransPyV is the target. Once TransPyV can compile itself into an executable, Viper will be fully self-hosted.

### Further reading

- [TransPyC compiler repo](https://git.gvsds.com/GVSDS/TransPyC)
- [Viper language overview](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/01-overview.md)
- [Viper bootstrapping roadmap](https://git.gvsds.com/GVSDS/TransPyC/blob/main/wiki/13-bootstrapping.md)
- [ViperOS (an OS written in Viper)](https://git.gvsds.com/GVSDS/ViperOS)