# GLM-4-Voice

📄 Report  • 🤗 HF Repo • 🤖 Demo • 🐦 Twitter

Read this in [English](./README_en.md)

GLM-4-Voice 是智谱 AI 推出的端到端语音模型。GLM-4-Voice 能够直接理解和生成中英文语音，进行实时语音对话，并且能够遵循用户的指令要求改变语音的情感、语调、语速、方言等属性。

## Model Architecture
![Model Architecture](./resources/architecture.jpeg)

GLM-4-Voice 由三个部分组成：
* GLM-4-Voice-Tokenizer: 通过在 [Whisper](https://github.com/openai/whisper) 的 Encoder 部分增加 Vector Quantization 并在 ASR 数据上有监督训练，将连续的语音输入转化为离散的 token。每秒音频平均只需要用 12.5 个离散 token 表示。
* GLM-4-Voice-Decoder: 基于 [CosyVoice](https://github.com/FunAudioLLM/CosyVoice) 的 Flow Matching 模型结构训练的支持流式推理的语音解码器，将离散化的语音 token 转化为连续的语音输出。最少只需要 10 个语音 token 即可开始生成，降低端到端对话延迟。
* GLM-4-Voice-9B: 在 [GLM-4-9B](https://github.com/THUDM/GLM-4) 的基础上进行语音模态的预训练和对齐，从而能够理解和生成离散化的语音 token。

预训练方面，为了攻克模型在语音模态下的智商和合成表现力两个难关，我们将 Speech2Speech 任务解耦合为“根据用户音频做出文本回复”和“根据文本回复和用户语音合成回复语音”两个任务，并设计两种预训练目标，分别基于文本预训练数据和无监督音频数据合成语音-文本交错数据以适配这两种任务形式。GLM-4-Voice-9B 在 GLM-4-9B 的基座模型基础之上，经过了数百万小时音频和数千亿 token 的音频文本交错数据预训练，拥有很强的音频理解和建模能力。

对齐方面，为了支持高质量的语音对话，我们设计了一套流式思考架构：根据用户语音，GLM-4-Voice 可以流式交替输出文本和语音两个模态的内容，其中语音模态以文本作为参照保证回复内容的高质量，并根据用户的语音指令要求做出相应的声音变化，在最大程度保留语言模型智商的情况下仍然具有端到端建模的能力，同时具备低延迟性，最低只需要输出 20 个 token 便可以合成语音。

## Model List

|         Model         |       Type       |                                                                     Download                                                                     |
|:---------------------:|:----------------:|:------------------------------------------------------------------------------------------------------------------------------------------------:|
| GLM-4-Voice-Tokenizer | Speech Tokenizer | [🤗 Huggingface](https://huggingface.co/THUDM/glm-4-voice-tokenizer) [🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/glm-4-voice-tokenizer) |
|    GLM-4-Voice-9B     |    Chat Model    |        [🤗 Huggingface](https://huggingface.co/THUDM/glm-4-voice-9b) [🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/glm-4-voice-9b)        |
|  GLM-4-Voice-Decoder  |  Speech Decoder  |   [🤗 Huggingface](https://huggingface.co/THUDM/glm-4-voice-decoder) [🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/glm-4-voice-decoder)   |

## Usage
我们提供了可以直接启动的 Web Demo。用户可以输入语音或文本，模型会同时给出语音和文字回复。

![](resources/web_demo.png)

### Preparation

首先下载仓库
```shell
git clone --recurse-submodules https://github.com/THUDM/GLM-4-Voice
cd GLM-4-Voice
```
然后安装依赖。也可以使用我们提供的镜像 `zhipuai/glm-4-voice:0.1` 以跳过这一步。
```shell
pip install -r requirements.txt
```
由于 Decoder 模型不支持通过 `transformers` 初始化，因此 checkpoint 需要单独下载。

```shell
# git 模型下载，请确保已安装 git-lfs
git lfs install
git clone https://huggingface.co/THUDM/glm-4-voice-decoder
```

### Launch Web Demo

1. 启动模型服务

```shell
python model_server.py --host localhost --model-path THUDM/glm-4-voice-9b --port 10000 --dtype bfloat16 --device cuda:0
```

如果你需要使用 Int4 精度启动，请运行

```shell
python model_server.py --host localhost --model-path THUDM/glm-4-voice-9b --port 10000 --dtype int4 --device cuda:0
```

此命令会自动下载 `glm-4-voice-9b`。如果网络条件不好，也手动下载之后通过 `--model-path` 指定本地的路径。

2. 启动 web 服务

```shell
python web_demo.py --tokenizer-path  THUDM/glm-4-voice-tokenizer --model-path THUDM/glm-4-voice-9b --flow-path ./glm-4-voice-decoder
```

即可在 http://127.0.0.1:8888 访问 web demo。

此命令会自动下载 `glm-4-voice-tokenizer` 和 `glm-4-voice-9b`。 请注意，`glm-4-voice-decoder` 需要手动下载。

如果网络条件不好，可以手动下载这三个模型之后通过 `--tokenizer-path`, `--flow-path` 和 `--model-path` 指定本地的路径。

### Known Issues

* Gradio 的流式音频播放效果不稳定。在生成完成后点击对话框中的音频质量会更高。

## Cases

我们提供了 GLM-4-Voice 的部分对话案例，包括控制情绪、改变语速、生成方言等。

* 用轻柔的声音引导我放松

https://github.com/user-attachments/assets/4e3d9200-076d-4c28-a641-99df3af38eb0

* 用激动的声音解说足球比赛

https://github.com/user-attachments/assets/0163de2d-e876-4999-b1bc-bbfa364b799b

* 用哀怨的声音讲一个鬼故事

https://github.com/user-attachments/assets/a75b2087-d7bc-49fa-a0c5-e8c99935b39a

* 用东北话介绍一下冬天有多冷

https://github.com/user-attachments/assets/91ba54a1-8f5c-4cfe-8e87-16ed1ecf4037

* 用重庆话念“吃葡萄不吐葡萄皮”

https://github.com/user-attachments/assets/7eb72461-9e84-4d8e-9c58-1809cf6a8a9b

* 用北京话念一句绕口令

https://github.com/user-attachments/assets/a9bb223e-9c0a-440d-8537-0a7f16e31651

  * 加快语速

https://github.com/user-attachments/assets/c98a4604-366b-4304-917f-3c850a82fe9f

  * 再快一点

https://github.com/user-attachments/assets/d5ff0815-74f8-4738-b0f1-477cfc8dcc2d

## Acknowledgements

本项目的部分代码来自：
* [CosyVoice](https://github.com/FunAudioLLM/CosyVoice)
* [transformers](https://github.com/huggingface/transformers)
* [GLM-4](https://github.com/THUDM/GLM-4)

## 协议

+ GLM-4 模型的权重的使用则需要遵循 [模型协议](https://huggingface.co/THUDM/glm-4-voice-9b/blob/main/LICENSE)。

+ 本开源仓库的代码则遵循 [Apache 2.0](LICENSE) 协议。

## 引用

```
@misc{zeng2024glm4,
      title={GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot},
      author={Aohan Zeng and Zhengxiao Du and Mingdao Liu and Kedong Wang and Shengmin Jiang and Lei Zhao and Yuxiao Dong and Jie Tang},
      year={2024},
      eprint={2412.02612},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.02612},
}
```

```
@misc{zeng2024scaling,
      title={Scaling Speech-Text Pre-training with Synthetic Interleaved Data},
      author={Aohan Zeng and Zhengxiao Du and Mingdao Liu and Lei Zhang and Shengmin Jiang and Yuxiao Dong and Jie Tang},
      year={2024},
      eprint={2411.17607},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2411.17607},
}
```

README.md

GLM-4-Voice | 端到端中英语音对话模型


# GLM-4-Voice

📄 Report  • 🤗 HF Repo • 🤖 Demo • 🐦 Twitter

GLM-4-Voice is an end-to-end voice model launched by Zhipu AI. GLM-4-Voice can directly understand and generate Chinese and English speech, engage in real-time voice conversations, and change attributes such as emotion, intonation, speech rate, and dialect based on user instructions.

## Model Architecture

![Model Architecture](./resources/architecture.jpeg)
We provide the three components of GLM-4-Voice:
* GLM-4-Voice-Tokenizer: Trained by adding vector quantization to the encoder part of [Whisper](https://github.com/openai/whisper), converting continuous speech input into discrete tokens. Each second of audio is converted into 12.5 discrete tokens.
* GLM-4-Voice-9B: Pre-trained and aligned on speech modality based on [GLM-4-9B](https://github.com/THUDM/GLM-4), enabling understanding and generation of discretized speech.
* GLM-4-Voice-Decoder: A speech decoder supporting streaming inference, retrained based on [CosyVoice](https://github.com/FunAudioLLM/CosyVoice), converting discrete speech tokens into continuous speech output. Generation can start with as few as 10 audio tokens, reducing conversation latency.

## Model List

|         Model         |       Type       |                               Download                               |
|:---------------------:|:----------------:|:--------------------------------------------------------------------:|
| GLM-4-Voice-Tokenizer | Speech Tokenizer | [🤗 Huggingface](https://huggingface.co/THUDM/glm-4-voice-tokenizer) |
|    GLM-4-Voice-9B     |    Chat Model    |    [🤗 Huggingface](https://huggingface.co/THUDM/glm-4-voice-9b)     |
|  GLM-4-Voice-Decoder  |  Speech Decoder  |  [🤗 Huggingface](https://huggingface.co/THUDM/glm-4-voice-decoder)  |

## Usage
We provide a Web Demo that can be launched directly. Users can input speech or text, and the model will respond with both speech and text.

![](resources/web_demo.png)

### Preparation

First, download the repository
```shell
git clone --recurse-submodules https://github.com/THUDM/GLM-4-Voice
cd GLM-4-Voice
```
Then, install the dependencies. You can also use our pre-built docker image `zhipuai/glm-4-voice:0.1` to skip the step.
```shell
pip install -r requirements.txt
```
Since the Decoder model does not support initialization via `transformers`, the checkpoint needs to be downloaded separately.

```shell
# Git model download, please ensure git-lfs is installed
git clone https://huggingface.co/THUDM/glm-4-voice-decoder
```

### Launch Web Demo

1. Start the model server

```shell
python model_server.py --host localhost --model-path THUDM/glm-4-voice-9b --port 10000 --dtype bfloat16 --device cuda:0
```

If you need to launch with Int4 precision, run

```shell
python model_server.py --host localhost --model-path THUDM/glm-4-voice-9b --port 10000 --dtype int4 --device cuda:0
```

This command will automatically download `glm-4-voice-9b`. If network conditions are poor, you can manually download it and specify the local path using `--model-path`.

2. Start the web service

```shell
python web_demo.py --tokenizer-path  THUDM/glm-4-voice-tokenizer --model-path THUDM/glm-4-voice-9b --flow-path ./glm-4-voice-decoder
```

You can access the web demo at [http://127.0.0.1:8888](http://127.0.0.1:8888).
This command will automatically download `glm-4-voice-tokenizer` and `glm-4-voice-9b`. Please note that `glm-4-voice-decoder` needs to be downloaded manually.
If the network connection is poor, you can manually download these three models and specify the local paths using `--tokenizer-path`, `--flow-path`, and `--model-path`.

### Known Issues
* Gradio’s streaming audio playback can be unstable. The audio quality will be higher when clicking on the audio in the dialogue box after generation is complete.

## Examples
We provide some dialogue cases for GLM-4-Voice, including emotion control, speech rate alteration, dialect generation, etc. (The examples are in Chinese.)

* Use a gentle voice to guide me to relax

https://github.com/user-attachments/assets/4e3d9200-076d-4c28-a641-99df3af38eb0

* Use an excited voice to commentate a football match

https://github.com/user-attachments/assets/0163de2d-e876-4999-b1bc-bbfa364b799b

* Tell a ghost story with a mournful voice

https://github.com/user-attachments/assets/a75b2087-d7bc-49fa-a0c5-e8c99935b39a

* Introduce how cold winter is with a Northeastern dialect

https://github.com/user-attachments/assets/91ba54a1-8f5c-4cfe-8e87-16ed1ecf4037

* Say "Eat grapes without spitting out the skins" in Chongqing dialect

https://github.com/user-attachments/assets/7eb72461-9e84-4d8e-9c58-1809cf6a8a9b

* Recite a tongue twister with a Beijing accent

https://github.com/user-attachments/assets/a9bb223e-9c0a-440d-8537-0a7f16e31651

  * Increase the speech rate

https://github.com/user-attachments/assets/c98a4604-366b-4304-917f-3c850a82fe9f

  * Even faster

https://github.com/user-attachments/assets/d5ff0815-74f8-4738-b0f1-477cfc8dcc2d

## Acknowledgements

Some code in this project is from:
* [CosyVoice](https://github.com/FunAudioLLM/CosyVoice)
* [transformers](https://github.com/huggingface/transformers)
* [GLM-4](https://github.com/THUDM/GLM-4)

## License Agreement

+ The use of GLM-4 model weights must follow the [Model License Agreement](https://huggingface.co/THUDM/glm-4-voice-9b/blob/main/LICENSE).

+ The code in this open-source repository is licensed under the [Apache 2.0](LICENSE) License.

## Citation

```
@misc{zeng2024glm4,
      title={GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot},
      author={Aohan Zeng and Zhengxiao Du and Mingdao Liu and Kedong Wang and Shengmin Jiang and Lei Zhao and Yuxiao Dong and Jie Tang},
      year={2024},
      eprint={2412.02612},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.02612},
}
```

```
@misc{zeng2024scaling,
      title={Scaling Speech-Text Pre-training with Synthetic Interleaved Data},
      author={Aohan Zeng and Zhengxiao Du and Mingdao Liu and Lei Zhang and Shengmin Jiang and Yuxiao Dong and Jie Tang},
      year={2024},
      eprint={2411.17607},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2411.17607},
}
```

README_en.md

GLM-4-Voice | 端到端中英语音对话模型


# Automated-AI-Web-Researcher-Ollama

## Description
Automated-AI-Web-Researcher is an innovative research assistant that leverages locally run large language models through Ollama to conduct thorough, automated online research on any given topic or question. Unlike traditional LLM interactions, this tool actually performs structured research by breaking down queries into focused research areas, systematically investigating each area via web searching and scraping relevant websites, and compiling its findings. The findings are automatically saved into a text document with all the content found and links to the sources. Whenever you want it to stop its research, you can input a command, which will terminate the research. The LLM will then review all of the content it found and provide a comprehensive final summary of your original topic or question. Afterward, you can ask the LLM questions about its research findings.

## Project Demonstration
[![My Project Demo](https://img.youtube.com/vi/hS7Q1B8N1mQ/0.jpg)](https://youtu.be/hS7Q1B8N1mQ "My Project Demo")

Click the image above to watch the demonstration of my project.

## Here's How It Works:
1. You provide a research query (e.g., "What year will the global population begin to decrease rather than increase according to research?").
2. The LLM analyzes your query and generates 5 specific research focus areas, each with assigned priorities based on relevance to the topic or question.
3. Starting with the highest priority area, the LLM:
    - Formulates targeted search queries
    - Performs web searches
    - Analyzes search results, selecting the most relevant web pages
    - Scrapes and extracts relevant information from the selected web pages
    - Documents all content found during the research session into a research text file, including links to the websites that the content was retrieved from
4. After investigating all focus areas, the LLM generates new focus areas based on the information found and repeats its research cycle, often discovering new relevant focus areas based on previous findings, leading to interesting and novel research focuses in some cases.
5. You can let it research as long as you like, with the ability to input a quit command at any time. This will stop the research and cause the LLM to review all the content collected so far in full, generating a comprehensive summary in response to your original query or topic.
6. The LLM will then enter a conversation mode where you can ask specific questions about the research findings if desired.

The key distinction is that this isn't just a chatbot—it's an automated research assistant that methodically investigates topics and maintains a documented research trail, all from a single question or topic of your choosing. Depending on your system and model, it can perform over a hundred searches and content retrievals in a relatively short amount of time. You can leave it running and return to a full text document with over a hundred pieces of content from relevant websites and then have it summarize the findings, after which you can ask it questions about what it found.

## Features
- Automated research planning with prioritized focus areas
- Systematic web searching and content analysis
- All research content and source URLs saved into a detailed text document
- Research summary generation
- Post-research Q&A capability about findings
- Self-improving search mechanism
- Rich console output with status indicators
- Comprehensive answer synthesis using web-sourced information
- Research conversation mode for exploring findings

## Installation
**Note:** To use on Windows, follow the instructions on the [/feature/windows-support](https://github.com/TheBlewish/Automated-AI-Web-Researcher-Ollama/tree/feature/windows-support) branch. For Linux and MacOS, use this main branch and the follow steps below:

1. **Clone the repository:**

    ```sh
    git clone https://github.com/TheBlewish/Automated-AI-Web-Researcher-Ollama
    cd Automated-AI-Web-Researcher-Ollama
    ```

2. **Create and activate a virtual environment:**

    ```sh
    python -m venv venv
    source venv/bin/activate
    ```

3. **Install dependencies:**

    ```sh
    pip install -r requirements.txt
    ```

4. **Install and configure Ollama:**

    Install Ollama following the instructions at [https://ollama.ai](https://ollama.ai).

    Using your selected model, reccommended to pick one with the required context length for lots of searches (`phi3:3.8b-mini-128k-instruct` or `phi3:14b-medium-128k-instruct` are recommended).

5. Go to the llm_config.py file which should have an ollama section that looks like this:

```sh
LLM_CONFIG_OLLAMA = {
    "llm_type": "ollama",
    "base_url": "http://localhost:11434",  # default Ollama server URL
    "model_name": "custom-phi3-32k-Q4_K_M",  # Replace with your Ollama model name
    "temperature": 0.7,
    "top_p": 0.9,
    "n_ctx": 55000,
    "stop": ["User:", "\n\n"]
```

Then change to the left of where it says replace with your Ollama model name, the "model_name" function, to the name of the model you have setup in Ollama to use with the program, you can now also change 'n_ctx' to set the desired context size.

## Usage
1. **Start Ollama:**

    ```sh
    ollama serve
    ```

2. **Run the researcher:**

    ```sh
    python Web-LLM.py
    ```

3. **Start a research session:**
    - Type `@` followed by your research query.
    - Press `CTRL+D` to submit.
    - Example: `@What year is the global population projected to start declining?`

4. **During research, you can use the following commands by typing the associated letter and submitting with `CTRL+D`:**
    - Use `s` to show status.
    - Use `f` to show the current focus.
    - Use `p` to pause and assess research progress, which will give you an assessment from the LLM after reviewing the entire research content to determine whether it can answer your query with the content collected so far. It will then wait for you to input one of two commands: `c` to continue with the research or `q` to terminate it, resulting in a summary as if you had terminated it without using the pause feature.
    - Use `q` to quit research.

5. **After the research completes:**
    - Wait for the summary to be generated and review the LLM's findings.
    - Enter conversation mode to ask specific questions about its findings.
    - Access the detailed research content found, available in a research session text file which will be located in the program's directory. This includes:
        - All retrieved content
        - Source URLs for all of the information
        - Focus areas investigated
        - Generated summary

## Configuration
The LLM settings can be modified in `llm_config.py`. You must specify your model name in the configuration for the researcher to function. The default configuration is optimized for research tasks with the specified Phi-3 model.

## Current Status
This is a prototype that demonstrates functional automated research capabilities. While still in development, it successfully performs structured research tasks. It has been tested and works well with the `phi3:3.8b-mini-128k-instruct` model when the context is set as advised previously.

## Dependencies
- Ollama
- Python packages listed in `requirements.txt`
- Recommended models: `phi3:3.8b-mini-128k-instruct` or `phi3:14b-medium-128k-instruct` (with custom context length as specified)

## Contributing
Contributions are welcome! This is a prototype with room for improvements and new features.

## License
This project is licensed under the MIT License—see the [LICENSE](LICENSE) file for details.

## Acknowledgments
- Ollama team for their local LLM runtime
- DuckDuckGo for their search API

## Personal Note
This tool represents an attempt to bridge the gap between simple LLM interactions and genuine research capabilities. By structuring the research process and maintaining documentation, it aims to provide more thorough and verifiable results than traditional LLM conversations. It also represents an attempt to improve on my previous project, 'Web-LLM-Assistant-Llamacpp-Ollama,' which simply gave LLMs the ability to search and scrape websites to answer questions. Unlike its predecessor, I feel this program takes that capability and uses it in a novel and very useful way. As a very new programmer, with this being my second ever program, I feel very good about the result. I hope that it hits the mark!

Given how much I have been using it myself, unlike the previous program, which felt more like a novelty than an actual tool, this is actually quite useful and unique—but I am quite biased!

Please enjoy! And feel free to submit any suggestions for improvements so that we can make this automated AI researcher even more capable.

## Disclaimer
This project is for educational purposes only. Ensure you comply with the terms of service of all APIs and services used.

README.md

A python program that turns an LLM, running on Ollama, into an automated researcher, which will with a single query determine focus areas to investigate, do websearches and scrape content from various relevant websites and do research for you all on its own! And more, not limited to but including saving the findings for you!


# Microsoft Open Source Code of Conduct

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).

Resources:

- [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/)
- [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
- Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns
- Employees can reach out at [aka.ms/opensource/moderation-support](https://aka.ms/opensource/moderation-support)

CODE_OF_CONDUCT.md

Hyperlight is a lightweight Virtual Machine Manager (VMM) designed to be embedded within applications. It enables safe execution of untrusted code within micro virtual machines with very low latency and minimal overhead.


# Contribution Guidelines

This project welcomes contributions. Most contributions require you to signoff on your commits via
the Developer Certificate of Origin (DCO). When you submit a pull request, a DCO-bot will automatically determine
whether you need to provide signoff for your commit. Please follow the instructions provided by DCO-bot, as pull
requests cannot be merged until the author(s) have provided signoff to fulfill the DCO requirement.
You may find more information on the DCO requirements [below](#developer-certificate-of-origin-signing-your-work).

## Issues

This section describes the guidelines for submitting issues

### Issue Types

There are 2 types of issues:

- Bug: You've found a bug with the code, and want to report it, or create an issue to track the bug.
- Proposal: Used for items that propose a new idea or functionality. This allows feedback from others before code is written.

## Contributing to Hyperlight

This section describes the guidelines for contributing code / docs to Hyperlight.

### Pull Requests

All contributions come through pull requests. To submit a proposed change, we recommend following this workflow:

1. Make sure there's an issue (bug or proposal) raised, which sets the expectations for the contribution you are about to make.
2. Fork the relevant repo and create a new branch
3. Create your change
    - Code changes require tests
    - Make sure to run the linters to check and format the code
4. Update relevant documentation for the change
5. Commit with [DCO sign-off](#developer-certificate-of-origin-signing-your-work) and open a PR
6. Wait for the CI process to finish and make sure all checks are green
7. A maintainer of the project will be assigned, and you can expect a review within a few days

#### Use work-in-progress PRs for early feedback

A good way to communicate before investing too much time is to create a "Work-in-progress" PR and share it with your reviewers. The standard way of doing this is to add a "[WIP]" prefix in your PR's title and open the pull request as a draft.

### Developer Certificate of Origin: Signing your work

#### Every commit needs to be signed

The Developer Certificate of Origin (DCO) is a lightweight way for contributors to certify that they wrote or otherwise have the right to submit the code they are contributing to the project. Here is the full text of the [DCO](https://developercertificate.org/), reformatted for readability:
```
By making a contribution to this project, I certify that:

    (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

    (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

    (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

    (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.
```

Contributors sign-off that they adhere to these requirements by adding a `Signed-off-by` line to commit messages.

```text
This is my commit message

Signed-off-by: Random J Developer <random@developer.example.org>
```

Git even has a `-s` command line option to append this automatically to your commit message:

```sh
git commit -s -m 'This is my commit message'
```

Each Pull Request is checked  whether or not commits in a Pull Request do contain a valid Signed-off-by line.

#### I didn't sign my commit, now what?!

No worries - You can easily replay your changes, sign them and force push them!

```sh
git checkout <branch-name>
git commit --amend --no-edit --signoff
git push --force-with-lease <remote-name> <branch-name>
```

*Credit: This doc was cribbed from Dapr.*

### Rust Analyzer

If you are using the [Rust Analyzer](https://rust-analyzer.github.io/manual.html) then you may need to set the configuration option `rust-analyzer.rustfmt.extraArgs` to `["+nightly"]` to ensure that formatting works correctly as this project has a [`rustfmt.toml`](./rustfmt.toml) file that uses nightly features.

CONTRIBUTING.md


# Current Maintainers

| Name               | GitHub ID                                          |
|--------------------|----------------------------------------------------|
| Danilo Chiarlone   | [@danbugs](https://github.com/danbugs)             |
| David Justice      | [@devigned](https://github.com/devigned)           |
| Doru Blânzeanu     | [@dblnz](https://github.com/dblnz)                 |
| Jorge Prendes      | [@jprendes](https://github.com/jprendes)           |
| Lucy Menon         | [@syntactically](https://github.com/syntactically) |
| Ludvig Liljenberg  | [@ludfjig](https://github.com/ludfjig)             |
| Mark Rosetti       | [@marosset](https://github.com/marosset)           |
| Simon Davies       | [@simongdavies](https://github.com/simongdavies)   |
| Tomasz Andrzejak   | [@andreiltd](https://github.com/andreiltd)         |

<!-- Note: Please maintain alphabetical order when adding new entries to the table. -->

MAINTAINERS.md


Hyperlight

Hyperlight is a lightweight Virtual Machine Manager (VMM) designed to be embedded within applications. It enables safe execution of untrusted code within micro virtual machines with very low latency and minimal overhead.

> Note: Hyperlight is a nascent project with an evolving API and no guaranteed support. Assistance is provided on a
> best-effort basis by the developers.

---

## Overview

Hyperlight is a library for creating _micro virtual machines_  — or _sandboxes_ — specifically optimized for securely
running untrusted code with minimal impact. It supports both Windows and Linux,
utilizing [Windows Hypervisor Platform](https://docs.microsoft.com/en-us/virtualization/api/#windows-hypervisor-platform)
on Windows, and either Microsoft Hypervisor (mshv) or [KVM](https://linux-kvm.org/page/Main_Page) on Linux.

These micro VMs operate without a kernel or operating system, keeping overhead low. Instead, guests are built
specifically for Hyperlight using the Hyperlight Guest library, which provides a controlled set of APIs that facilitate
interaction between host and guest:

- The host can call functions implemented and exposed by the guest (known as _guest functions_).
- Once running, the guest can call functions implemented and exposed by the host (known as _host functions_).

By default, Hyperlight restricts guest access to a minimal API. The only _host function_ available by default allows the
guest to print messages, which are displayed on the host console or redirected to stdout, as configured. Hosts can
choose to expose additional host functions, expanding the guest’s capabilities as needed.

Below is an example demonstrating the use of the Hyperlight host library in Rust to execute a simple guest application
and an example of a simple guest application using the Hyperlight guest library in also written in Rust.

### Host

```rust
use std::{thread, sync::{Arc, Mutex}};

use hyperlight_common::flatbuffer_wrappers::function_types::{ParameterValue, ReturnType};
use hyperlight_host::{UninitializedSandbox, MultiUseSandbox, func::HostFunction0, sandbox_state::transition::Noop, sandbox_state::sandbox::EvolvableSandbox};

fn main() -> hyperlight_host::Result<()> {
    // Create an uninitialized sandbox with a guest binary
    let mut uninitialized_sandbox = UninitializedSandbox::new(
        hyperlight_host::GuestBinary::FilePath(hyperlight_testing::simple_guest_as_string().unwrap()),
        None, // default configuration
        None, // default run options
        None, // default host print function
    )?;

    // Register a host function
    fn sleep_5_secs() -> hyperlight_host::Result<()> {
        thread::sleep(std::time::Duration::from_secs(5));
        Ok(())
    }

    let host_function = Arc::new(Mutex::new(sleep_5_secs));

    // Registering a host function makes it available to be called by the guest
    host_function.register(&mut uninitialized_sandbox, "Sleep5Secs")?;
    // Note: This function is unused by the guest code below, it's just here for demonstration purposes

    // Initialize sandbox to be able to call host functions
    let mut multi_use_sandbox: MultiUseSandbox = uninitialized_sandbox.evolve(Noop::default())?;

    // Call a function in the guest
    let message = "Hello, World! I am executing inside of a VM :)\n".to_string();
    // in order to call a function it first must be defined in the guest and exposed so that
    // the host can call it
    let result = multi_use_sandbox.call_guest_function_by_name(
        "PrintOutput",
        ReturnType::Int,
        Some(vec![ParameterValue::String(message.clone())]),
    );

    assert!(result.is_ok());

    Ok(())
}
```

### Guest

```rust
#![no_std]
#![no_main]
extern crate alloc;

use alloc::string::ToString;
use alloc::vec::Vec;
use hyperlight_common::flatbuffer_wrappers::function_call::FunctionCall;
use hyperlight_common::flatbuffer_wrappers::function_types::{
    ParameterType, ParameterValue, ReturnType,
};
use hyperlight_common::flatbuffer_wrappers::guest_error::ErrorCode;
use hyperlight_common::flatbuffer_wrappers::util::get_flatbuffer_result_from_int;

use hyperlight_guest::error::{HyperlightGuestError, Result};
use hyperlight_guest::guest_function_definition::GuestFunctionDefinition;
use hyperlight_guest::guest_function_register::register_function;
use hyperlight_guest::host_function_call::{
    call_host_function, get_host_value_return_as_int,
};

fn print_output(function_call: &FunctionCall) -> Result<Vec<u8>> {
    if let ParameterValue::String(message) = function_call.parameters.clone().unwrap()[0].clone() {
        call_host_function(
            "HostPrint",
            Some(Vec::from(&[ParameterValue::String(message.to_string())])),
            ReturnType::Int,
        )?;
        let result = get_host_value_return_as_int()?;
        Ok(get_flatbuffer_result_from_int(result))
    } else {
        Err(HyperlightGuestError::new(
            ErrorCode::GuestFunctionParameterTypeMismatch,
            "Invalid parameters passed to simple_print_output".to_string(),
        ))
    }
}

#[no_mangle]
pub extern "C" fn hyperlight_main() {
    let print_output_def = GuestFunctionDefinition::new(
        "PrintOutput".to_string(),
        Vec::from(&[ParameterType::String]),
        ReturnType::Int,
        print_output as i64,
    );
    register_function(print_output_def);
}

#[no_mangle]
pub fn guest_dispatch_function(function_call: FunctionCall) -> Result<Vec<u8>> {
    let function_name = function_call.function_name.clone();
    return Err(HyperlightGuestError::new(
        ErrorCode::GuestFunctionNotFound,
        function_name,
    ));
}
```

For additional examples of using the Hyperlight host Rust library, see
the [./src/hyperlight_host/examples](./src/hyperlight_host/examples) directory.

For examples of guest applications, see the [./src/tests/c_guests](./src/tests/c_guests) directory for C guests and
the [./src/tests/rust_guests](./src/tests/rust_guests) directory for Rust guests.

> Note: Hyperlight guests can be written using the Hyperlight Rust or C Guest libraries.

## Repository Structure

- Hyperlight Host Libraries (i.e., the ones that create and manage the VMs)
    - [src/hyperlight_host](./src/hyperlight_host) - This is the Rust Hyperlight host library.

- Hyperlight Guest Libraries (i.e., the ones to make it easier to create guests that run inside the VMs)
    - [src/hyperlight_guest](./src/hyperlight_guest) - This is the Rust Hyperlight guest library.
    - [src/hyperlight_guest_capi](./src/hyperlight_guest_capi) - This is the C compatible wrapper for the Hyperlight
      guest library.

- Hyperlight Common (functionality used by both the host and the guest)
    - [src/hyperlight_common](./src/hyperlight_common)

- Test Guest Applications:
    - [src/tests/rust_guests](./src/tests/rust_guests) - This directory contains three Hyperlight Guest programs written
      in Rust, which are intended to be launched within partitions as "guests".
    - [src/tests/c_guests](./src/tests/c_guests) - This directory contains two Hyperlight Guest programs written in C,
      which are intended to be launched within partitions as "guests".

- Tests:
    - [src/hyperlight-testing](./src/hyperlight_testing) - Shared testing code for Hyperlight projects built in Rust.

## Try it yourself!

You can run Hyperlight on:

- [Linux with KVM][kvm].
- [Windows with Windows Hypervisor Platform (WHP).][whp] -  Note that you need Windows 11 / Windows Server 2022 or later to use hyperlight, if you are running on earlier versions of Windows then you should consider using our devcontainer on [GitHub codespaces]((https://codespaces.new/hyperlight-dev/hyperlight)) or WSL2.
- Windows Subsystem for Linux 2 (see instructions [here](https://learn.microsoft.com/en-us/windows/wsl/install) for Windows client and [here](https://learn.microsoft.com/en-us/windows/wsl/install-on-server) for Windows Server) with KVM.
- Azure Linux with mshv (note that you need mshv to be installed to use Hyperlight)

After having an environment with a hypervisor setup, running the example has the following pre-requisites:

1. On Linux or WSL, you'll most likely need build essential. For Ubuntu, run `sudo apt install build-essential`. For
   Azure Linux, run `sudo dnf install build-essential`.
2. [Rust](https://www.rust-lang.org/tools/install). Install toolchain v1.81 or later.

   Also, install the `x86_64-pc-windows-msvc` and `x86_64-unknown-none` targets, these are needed to build the test
   guest binaries. (Note: install both targets on either Linux or Windows: Hyperlight can load ELF or PE files on either
   OS, and the tests/examples are built for both):

    ```sh
    rustup target add x86_64-unknown-none
    rustup target add x86_64-pc-windows-msvc
    ```

3. [just](https://github.com/casey/just). `cargo install just` On Windows you also need [pwsh](https://learn.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-windows?view=powershell-7.4).
4. [clang and LLVM](https://clang.llvm.org/get_started.html).
    - On Ubuntu, run:

        ```sh
        wget https://apt.llvm.org/llvm.sh
        chmod +x ./llvm.sh
        sudo ./llvm.sh 17 all
        sudo ln -s /usr/lib/llvm-17/bin/clang-cl /usr/bin/clang-cl
        sudo ln -s /usr/lib/llvm-17/bin/llvm-lib /usr/bin/llvm-lib
        sudo ln -s /usr/lib/llvm-17/bin/lld-link /usr/bin/lld-link
        sudo ln -s /usr/lib/llvm-17/bin/llvm-ml /usr/bin/llvm-ml
        sudo ln -s /usr/lib/llvm-17/bin/ld.lld /usr/bin/ld.lld
        sudo ln -s /usr/lib/llvm-17/bin/clang /usr/bin/clang
        ```

    - On Windows, see [this](https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170).

    - On Azure Linux, run:

        ```sh
        sudo dnf remove clang -y || true
        sudo dnf install clang17 -y
        sudo dnf install clang17-tools-extra -y
        ```

Then, we are ready to build and run the example:

```sh
just build  # build the Hyperlight library
just rg     # build the rust test guest binaries
cargo run --example hello-world
```

If all worked as expected, you should see the following message in your console:

```text
Hello, World! I am executing inside of a VM :)
```

If you get the error `Error: NoHypervisorFound` and KVM or mshv is set up then this may be a permissions issue. In bash,
you can use `ls -l /dev/kvm` or  `ls -l /dev/mshv` to check which group owns that device and then `groups` to make sure
your user is a member of that group.

For more details on how to verify that KVM is correctly installed and permissions are correct, follow the
guide [here](https://help.ubuntu.com/community/KVM/Installation).

### Or you can use a codespace

[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hyperlight-dev/hyperlight)

## Contributing to Hyperlight

If you are interested in contributing to Hyperlight, running the entire test-suite is a good way to get started. To do
so, on your console, run the following commands:

```sh
just guests  # build the c and rust test guests
just build  # build the Hyperlight library
just test # runs the tests
```

Also , please review the [CONTRIBUTING.md](./CONTRIBUTING.md) file for more information on how to contribute to
Hyperlight.

> Note: For general Hyperlight development, you may also need flatc (Flatbuffer compiler): for instructions,
> see [here](https://github.com/google/flatbuffers).

## Join our Community Meetings

This project holds fortnightly community meetings to discuss the project's progress, roadmap, and any other topics of interest. The meetings are open to everyone, and we encourage you to join us.

- **When**: Every other Wednesday 09:00 (PST/PDT) [Convert to your local time](https://dateful.com/convert/pst-pdt-pacific-time?t=09)
- **Where**: Zoom! - Agenda and information on how to join can be found in the [Hyperlight Community Meeting Notes](https://hackmd.io/blCrncfOSEuqSbRVT9KYkg#Agenda). Please log into hackmd to edit!

## More Information

For more information, please refer to our compilation of documents in the [`docs/` directory](./docs/README.md).

## Code of Conduct

See the [Code of Conduct](./CODE_OF_CONDUCT.md).

[wsl2]: https://docs.microsoft.com/en-us/windows/wsl/install

[kvm]: https://help.ubuntu.com/community/KVM/Installation

[whp]: https://devblogs.microsoft.com/visualstudio/hyper-v-android-emulator-support/#1-enable-hyper-v-and-the-windows-hypervisor-platform

README.md


## Reporting Security Issues

**Please do not report security vulnerabilities through public GitHub issues.**

Instead, please report them via the [GitHub's private security vulnerability reporting mechanism on this repo](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability#privately-reporting-a-security-vulnerability).

Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:

  * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
  * Full paths of source file(s) related to the manifestation of the issue
  * The location of the affected source code (tag/branch/commit or direct URL)
  * Any special configuration required to reproduce the issue
  * Step-by-step instructions to reproduce the issue
  * Proof-of-concept or exploit code (if possible)
  * Impact of the issue, including how an attacker might exploit the issue

This information will help us triage your report more quickly.

## Preferred Languages

We prefer all communications to be in English.

SECURITY.md


# Support

## How to file issues and get help

This project uses GitHub Issues to track bugs and feature requests. Please search the existing
issues before filing new issues to avoid duplicates.  For new issues, file your bug or
feature request as a new Issue.

[//]: <> (For help and questions about using this project, please use Slack channel [TODO: add Slack channel])

## Microsoft Support Policy

Support for this **Hyperlight** is limited to the resources listed above.

SUPPORT.md


# Hyperlight Project Documentation

Hyperlight is a library for running hypervisor-isolated workloads without the overhead of booting a full guest operating system inside the virtual machine.

By eliminating this overhead, Hyperlight can execute arbitrary code more efficiently. It's primarily aimed at supporting functions-as-a-service workloads, where a user's code must be loaded into memory and executed very quickly with high density.

## Basics: Hyperlight internals

Hyperlight achieves these efficiencies by removing all operating system functionality from inside the virtual machine, and instead requiring all guest binaries be run directly on the virtual CPU (vCPU). This key requirement means all Hyperlight guest binaries must not only be compiled to run on the vCPU's architecture, but also must be statically linked to specialized libraries to support their functionality (e.g. there are no syscalls whatsoever available). Roughly similar to Unikernel technologies, we provide a guest library (in Rust, and a C compatible wrapper for it) to which guest binaries can be statically linked.

Given a guest, then, Hyperlight takes some simple steps prior to executing it, including the following:

- Provisioning memory
- Configuring specialized regions of memory
- Provisioning a virtual machine (VM) and CPU with the platform-appropriate hypervisor API, and mapping memory into the VM
- Configuring virtual registers for the vCPU
- Executing the vCPU at a specified instruction pointer

## Basics: Hyperlight architecture

This project is composed internally of several internal components, depicted in the below diagram:

![Hyperlight architecture](./assets/hyperlight_arch.png)

## Further reading

* [Glossary](./glossary.md)
* [How code gets executed in a VM](./hyperlight-execution-details.md)
* [How to build a Hyperlight guest binary](./how-to-build-a-hyperlight-guest-binary.md)
* [Security considerations](./security.md)
* [Technical requirements document](./technical-requirements-document.md)

## For developers

* [Security guidance for developers](./security-guidance-for-developers.md)
* [Paging Development Notes](./paging-development-notes.md)
* [How to debug a Hyperlight guest](./how-to-debug-a-hyperlight-guest.md)
* [How to use Flatbuffers in Hyperlight](./how-to-use-flatbuffers.md)
* [How to make a Hyperlight release](./how-to-make-releases.md)
* [Getting Hyperlight Metrics, Logs, and Traces](./hyperlight-metrics-logs-and-traces.md)
* [Benchmarking Hyperlight](./benchmarking-hyperlight.md)
* [Hyperlight Surrogate Development Notes](./hyperlight-surrogate-development-notes.md)
* [Debugging Hyperlight](./debugging-hyperlight.md)
* [Signal Handling in Hyperlight](./signal-handlers-development-notes.md)

docs/README.md


# Benchmark Notes

Hyperlight uses the [Criterion](https://bheisler.github.io/criterion.rs/book/index.html) framework to run and analyze benchmarks. A benefit to this framework is that it doesn't require the nightly toolchain.

## When Benchmarks are ran

1. Every time a branch gets a push
    - Compares the current branch benchmarking results to the "dev-latest" release (which is the most recent push to "main" branch). This is done as part of `dep_rust.yml`, which is invoked by `ValidatePullRequest.yml`. These benchmarks are for the developer to compare their branch to main, and the results can only be seen in the GitHub action logs, and nothing is saved.

    ```
    sandboxes/create_sandbox
                        time:   [33.803 ms 34.740 ms 35.763 ms]
                        change: [+0.7173% +3.7017% +7.1346%] (p = 0.03 < 0.05)
                        Change within noise threshold.*
    ```

2. For each release
    - For each release, benchmarks are ran as part of the release pipeline in `CreateRelease.yml`, which invokes `Benchmarks.yml`. These benchmark results are compared to the previous release, and are uploaded as port of the "Release assets" on the GitHub release page.

Currently, benchmarks are ran on windows, linux-kvm (ubuntu), and linux-hyperv (mariner). Only release builds are benchmarked, not debug.

## Criterion artifacts

When running `cargo bench -- --save-baseline my_baseline`, criterion runs all benchmarks defined in `src/hyperlight_host/benches/`, prints the results to the stdout, as well as produces several artifacts. All artifacts can be found in `target/criterion/`. For each benchmarking group, for each benchmark, a subfolder with the name of the benchmark is created. This folder in turn contains folders `my_baseline`, `new`  and `report`. When running `cargo bench`, criterion always creates `new` and `report`, which always contains the most recent benchmark result and html report, but because we provided the `--save-baseline` flag, we also have a `my_baseline` folder, which is an exact copy of `new`. Moreover, if this `my_baseline` folder already existed before we ran `cargo bench -- --save-baseline my_baseline`, criterion would also compare the benchmark results with the old `my_baseline` folder, and then overwrite the folder.

The first time we run `cargo bench -- --save-baseline my_baseline` (starting with a clean project), we get the following structure.

```
target/criterion/
|-- report
`-- sandboxes
    |-- create_sandbox
    |   |-- my_baseline
    |   |-- new
    |   `-- report
    |-- create_sandbox_and_call_context
    |   |-- my_baseline
    |   |-- new
    |   `-- report
    `-- report
```

If we run the exact same command again, we get

```
target/criterion/
|-- report
`-- sandboxes
    |-- create_sandbox
    |   |-- change
    |   |-- my_baseline
    |   |-- new
    |   `-- report
    |-- create_sandbox_and_call_context
    |   |-- change
    |   |-- my_baseline
    |   |-- new
    |   `-- report
    `-- report
```

Note that it overwrote the previous `my_baseline` with the new result. But notably, there is a new `change` folder, which contains the benchmarking difference between the two runs. In addition, on stdout you'll also find a comparison to our previous `my_baseline` run.

```
                        time:   [40.434 ms 40.777 ms 41.166 ms]
                        change: [+0.0506% +1.1399% +2.2775%] (p = 0.06 > 0.05)
                        No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
```

**Note** that Criterion does not differ between release and debug/dev benchmark results, so it's up to the developer to make sure baselines of the same config are compared.

## Running benchmarks locally

Use `just bench [debug/release]` parameter to run benchmarks. Comparing local benchmarks results to github-saved benchmarks doesn't make much sense, since you'd be using different hardware, but you can use `just bench-download os hypervisor [tag] ` to download and extract the GitHub release benchmarks to the correct place folder. You can then run `just bench-ci main` to compare to (and overwrite) the previous release benchmarks. Note that `main` is the name of the baselines stored in GitHub.

docs/benchmarking-hyperlight.md


# Debugging Hyperlight

Support for debugging Hyperlight is currently very limited and experimental. Despite this we offer some very primitive tools to help.

When creating a Uninitialized sandbox, passing a `SandboxRunOptions::RunInProcess(false)` will make the guest run inside a regular host process, rather than inside a hypervisor partition. This allows you to step through the code of the guest using your IDE's debugger. However, there are no symbols, and breakpoints are not supported, so you'll be stepping through assembly.

However, on Windows platform, passing `SandboxRunOptions::RunInProcess(true)` is supported, and will load the guest binary using the win32 `LoadLibrary` function. This has the advantage of also allowing your IDE to set breakpoints in the guest, and also loading symbols, allowing for easy debugging.

## Notes on running guest in-process

The support for running a guest using in-process mode is experimental, highly unsafe, and has many limitations. It requires
enabling cargo feature `inprocess`, and only works when hyperlight-host is built with debug_assertions. Inprocess currently does not support calling guest functions that returns errors. If a guest panics, it will surface as assertion fault ""ERROR: The guest either panicked or returned an Error. Running inprocess-mode currently does not support error handling."

Running in process is specifically only for testing, and should never be used in production as it offers no security guarantees.

## Logging

Hyperlight guests supports logging using the log crate. Any log records logged inside a hyperlight guest using the various
log macros trace!/info!/warning!, etc., will be logged, given that a logger has been instantiated in the host. This can be
very helpful for debugging as well.

## Getting debug print output of memory configuration, virtual processor register state, and other information

Enabling the feature `print_debug` and running a debug build will result in some debug output being printed to the console. Amongst other things this output will show the memory configuration and virtual processor register state.

To enable this permanently in the rust analyzer for Visual Studio Code so that this output shows when running tests using `Run Test` option add the following to your `settings.json` file:

```json
"rust-analyzer.runnables.extraArgs": [
    "--features=print_debug"
],
```

Alternatively, this can be enabled when running a test from the command line:

```sh
cargo test --package hyperlight-host --test integration_test --features print_debug -- static_stack_allocate --exact --show-output
```

## Dumping the memory configuration, virtual processor register state and memory contents on a crash or unexpected VM Exit

To dump the details of the memory configuration, the virtual processors register state and the contents of the VM memory set the feature `crashdump` and run a debug build. This will result in a dump file being created in the temporary directory. The name and location of the dump file will be printed to the console and logged as an error message.

There are no tools at this time to analyze the dump file, but it can be useful for debugging.

docs/debugging-hyperlight.md


# Glossary

* [Hyperlight](#hyperlight)
* [Host Application](#host-application)
* [Host](#host)
* [Hypervisor](#hypervisor)
* [Driver](#driver)
* [Hyper-V](#hyper-v)
* [KVM](#kvm)
* [Guest](#guest)
* [Micro Virtual Machine](#micro-virtual-machine)
* [Workload](#workload)
* [Sandbox](#sandbox)

## Hyperlight

Hyperlight refers to the Hyperlight Project and not a specific component. Hyperlight is intended to be used as a library to embed hypervisor-isolated execution support inside a [host application](#host-application).

## Host Application

This is an application that consumes the Hyperlight library, in order to execute code in an hypervisor-isolated environment.

## Host

Host is the machine on which the [host application](#host-application) is running. A host could be a bare metal or virtual machine, when the host is a virtual machine, the nested virtualization is required to run Hyperlight.

## Hypervisor

Hypervisor is the software responsible for creating isolated [micro virtual machines](#micro-virtual-machine), as well as executing [guests](#guest) inside of those micro virtual machines. Hyperlight has [drivers](#driver) for the following hypervisors: [Hyper-V](#hyper-v) on Windows, [Hyper-V](#hyper-v) on Linux, and [KVM](#kvm).

## Driver

Hyperlight supports executing workloads on particular [hypervisors](#hypervisor) through drivers. Each supported hypervisor has its own driver to manage interacting with that hypervisor.

## Hyper-V

Hyper-V is a [hypervisor](#hypervisor) capable of creating and executing isolated [micro virtual machines](#micro-virtual-machine) on both Windows and Linux. On Linux, Hyper-V is sometimes referred to as MSHV (Microsoft Hypervisor).

## KVM

Kernel-based Virtual Machine (KVM) is a [hypervisor](#hypervisor) capable of creating and executing isolated [micro virtual machines](#micro-virtual-machine) on Linux.

## MSHV

MSHV stands for Microsoft Hypervisor and is the name commonly used for Hyper-V when the hypervisor is running Linux dom0 (as opposed to Windows dom0).

## Guest

A guest is a standalone executable binary that is executed inside a hypervisor [micro virtual machine](#micro-virtual-machine). By having purpose-fit guests binaries, as opposed to running a full operating system, Hyperlight achieves low-latency startup times of workloads, since it doesn't need to first boot an entire operating system before executing the workload.

The interface that a guest must implement is specific to the associated [host](#host) and the type of workloads that it may be specialized for executing, such as WebAssembly Modules (Wasm), or a specific language.

## Micro Virtual Machine

A micro virtual machine is an execution environment managed by a hypervisor that isolates a [guest](#guest) from the [host](#host). A hypervisor prevents the guest from directly accessing the host's resources, such as memory, filesystem, devices, memory or CPU.

We use the term Micro Virtual Machine as the VMs are very lightweight compared to traditional VMs, they contains no operating system or other unnecessary components. The goal is to provide a minimal environment for executing workloads with low latency and high density. However the isolation provided by the hypervisor is the same as that of a traditional VM.

## Workload

A workload is the code that the [host application](#host-application) wants to execute in an isolated [micro virtual machine](#micro-virtual-machine).

## Sandbox

A Sandbox is the abstraction used in Hyperlight to represent the isolated environment in which a workload is executed. A sandbox is used to create, configure, execute and destroy a [micro virtual machine](#micro-virtual-machine) that runs a [guest](#guest) workload. Sandboxes are created by the [host application](#host-application) using the Hyperlight host library.

docs/glossary.md


# Building a Hyperlight guest binary

This document explains how to build a binary to be used as a Hyperlight guest.

When building a guest, one needs to follow some rules so that the resulting
binary can be used with Hyperlight:
- the binary must not use the standard library
- the expected entrypoint function signature is `void hyperlight_main(void)` or
  `pub fn hyperlight_main()`
- Hyperlight expects
  `hl_Vec* c_guest_dispatch_function(const hl_FunctionCall *functioncall)` or
  `pub fn guest_dispatch_function(function_call: FunctionCall) -> Result<Vec<u8>>`
  to be defined in the binary so that in case the host calls a function that is
  not registered by the guest, this function is called instead.
- to be callable by the host, a function needs to be registered by the guest in
  the `hyperlight_main` function.

## Rust guest binary

In the case of a binary that is written in Rust, one needs to make use of the
Hyperlight crate, `hyperlight_guest` that contains the types and APIs that enable
the guest to:
- register functions that can be called by the host application
- call host functions that have been registered by the host.

## C guest binary

For the binary written in C, the generated C bindings can be downloaded from the
latest release page that contain: the `hyperlight_guest.h` header and the
C API library.
The `hyperlight_guest.h` header contains the corresponding APIs to register
guest functions and call host functions from within the guest.

docs/how-to-build-a-hyperlight-guest-binary.md


# How to debug a Hyperlight **KVM** guest using gdb

Hyperlight supports gdb debugging of a **KVM** guest running inside a Hyperlight sandbox.
When Hyperlight is compiled with the `gdb` feature enabled, a Hyperlight KVM sandbox can be configured
to start listening for a gdb connection.

## Supported features

The Hyperlight `gdb` feature enables **KVM** guest debugging:
   - an entry point breakpoint is automatically set for the guest to stop
   - add and remove HW breakpoints (maximum 4 set breakpoints at a time)
   - add and remove SW breakpoints
   - read and write registers
   - read and write addresses
   - step/continue
   - get code offset from target

## Expected behavior

Below is a list describing some cases of expected behavior from a gdb debug
session of a guest binary running inside a KVM Hyperlight sandbox.

- when the `gdb` feature is enabled and a SandboxConfiguration is provided a
  debug port, the created sandbox will wait for a gdb client to connect on the
  configured port
- when the gdb client attaches, the guest vCPU is expected to be stopped at the
  entry point
- if a gdb client disconnects unexpectedly, the debug session will be closed and
  the guest will continue executing disregarding any prior breakpoints
- if multiple sandbox instances are created, each instance will have its own
  gdb thread listening on the configured port
- if two sandbox instances are created with the same debug port, the second
  instance logs an error and the gdb thread will not be created, but the sandbox
  will continue to run without gdb debugging

## Example

### Sandbox configuration

The `guest-debugging` example in Hyperlight demonstrates how to configure a Hyperlight
sandbox to listen for a gdb client on a specific port.

### CLI Gdb configuration

One can use a gdb config file to provide the symbols and desired configuration.

The below contents of the `.gdbinit` file can be used to provide a basic configuration
to gdb startup.

```gdb
# Path to symbols
file path/to/symbols.elf
# The port on which Hyperlight listens for a connection
target remote :8080
set disassembly-flavor intel
set disassemble-next-line on
enable pretty-printer
layout src
```
One can find more information about the `.gdbinit` file at [gdbinit(5)](https://www.man7.org/linux/man-pages/man5/gdbinit.5.html).

### End to end example

Using the example mentioned at [Sandbox configuration](#sandbox-configuration)
one can run the below commands to debug the guest binary:

```bash
# Terminal 1
$ cargo run --example guest-debugging --features gdb
```

```bash
# Terminal 2
$ cat .gdbinit
file src/tests/rust_guests/bin/debug/simpleguest
target remote :8080
set disassembly-flavor intel
set disassemble-next-line on
enable pretty-printer
layout src

$ gdb
```

### Using VSCode to debug a Hyperlight guest

To replicate the above behavior using VSCode follow the below steps:
- install the `gdb` package on the host machine
- install the `C/C++` extension in VSCode to add debugging capabilities
- create a `.vscode/launch.json` file in the project directory with the below content:
    ```json
    {
        "version": "0.2.0",
        "configurations": [
            {
                "name": "GDB",
                "type": "cppdbg",
                "request": "launch",
                "program": "${workspaceFolder}/src/tests/rust_guests/bin/debug/simpleguest",
                "args": [],
                "stopAtEntry": true,
                "hardwareBreakpoints": {"require": false, "limit": 4},
                "cwd": "${workspaceFolder}",
                "environment": [],
                "externalConsole": false,
                "MIMode": "gdb",
                "miDebuggerPath": "/usr/bin/gdb",
                "miDebuggerServerAddress": "localhost:8080",
                "setupCommands": [
                    {
                        "description": "Enable pretty-printing for gdb",
                        "text": "-enable-pretty-printing",
                        "ignoreFailures": true
                    },
                    {
                        "description": "Set Disassembly Flavor to Intel",
                        "text": "-gdb-set disassembly-flavor intel",
                        "ignoreFailures": true
                    }
                ]
            }
        ]
    }
    ```
- in `Run and Debug` tab, select the `GDB` configuration and click on the `Run`
  button to start the debugging session.
  The gdb client will connect to the Hyperlight sandbox and the guest vCPU will
  stop at the entry point.

## How it works

The gdb feature is designed to work like a Request - Response protocol between
a thread that accepts commands from a gdb client and the hypervisor handler over
a communication channel.

All the functionality is implemented on the hypervisor side so it has access to
the shared memory and the vCPU.

The gdb thread uses the `gdbstub` crate to handle the communication with the gdb client.
When the gdb client requests one of the supported features mentioned above, a request
is sent over the communication channel to the hypervisor handler for the sandbox
to resolve.

Below is a sequence diagram that shows the interaction between the entities
involved in the gdb debugging of a Hyperlight guest running inside a KVM sandbox.

```
                               ┌───────────────────────────────────────────────────────────────────────────────────────────────┐
                               │                                       Hyperlight Sandbox                                      │
     USER                      │                                                                                │
┌────────────┐                 │  ┌──────────────┐                      ┌───────────────────────────┐              ┌────────┐  │
│ gdb client │                 │  │  gdb thread  │                      │ hypervisor handler thread │              │  vCPU  │  │
└────────────┘                 │  └──────────────┘                      └───────────────────────────┘              └────────┘  │
      |                        │          |               create_gdb_thread           |                                 |      │
      |                        │          |◄─────────────────────────────────────────┌─┐         vcpu stopped          ┌─┐     │
      |    attach              │         ┌─┐                                         │ │◄──────────────────────────────┴─┘     │
     ┌─┐───────────────────────┼────────►│ │                                         │ │     entrypoint breakpoint      |      │
     │ │   attach response     │         │ │                                         │ │                                |      │
     │ │◄──────────────────────┼─────────│ │                                         │ │                                |      │
     │ │                       │         │ │                                         │ │                                |      │
     │ │  add_breakpoint       │         │ │                                         │ │                                |      │
     │ │───────────────────────┼────────►│ │          add_breakpoint                 │ │                                |      │
     │ │                       │         │ │────────────────────────────────────────►│ │  add_breakpoint                |      │
     │ │                       │         │ │                                         │ │────┐                           |      │
     │ │                       │         │ │                                         │ │    │                           |      │
     │ │                       │         │ │                                         │ │◄───┘                           |      │
     │ │                       │         │ │          add_breakpoint response        │ │                                |      │
     │ │  add_breakpoint response        │ │◄────────────────────────────────────────│ │                                |      │
     │ │◄──────────────────────┬─────────│ │                                         │ │                                |      │
     │ │   continue            │         │ │                                         │ │                                |      │
     │ │───────────────────────┼────────►│ │            continue                     │ │                                |      │
     │ │                       │         │ │────────────────────────────────────────►│ │         resume vcpu            |      │
     │ │                       │         │ │                                         │ │──────────────────────────────►┌─┐     │
     │ │                       │         │ │                                         │ │                               │ │     │
     │ │                       │         │ │                                         │ │                               │ │     │
     │ │                       │         │ │                                         │ │                               │ │     │
     │ │                       │         │ │                                         │ │                               │ │     │
     │ │                       │         │ │                                         │ │         vcpu stopped          │ │     │
     │ │                       │         │ │        notify vcpu stop reason          │ │◄──────────────────────────────┴─┘     │
     │ │   notify vcpu stop reason       │ │◄────────────────────────────────────────│ │                                |      │
     │ │◄──────────────────────┬─────────│ │                                         │ │                                |      │
     │ │    continue until end │         │ │                                         │ │                                |      │
     │ │───────────────────────┼────────►│ │            continue                     │ │         resume vcpu            |      │
     │ │                       │         │ │────────────────────────────────────────►│ │──────────────────────────────►┌─┐     │
     │ │                       │         │ │                                         │ │                               │ │     │
     │ │                       │         │ │        comm channel disconnected        │ │         vcpu halted           │ │     │
     │ │   target finished exec│         │ │◄────────────────────────────────────────┤ │◄──────────────────────────────┴─┘     │
     │ │◄──────────────────────┼─────────┴─┘          target finished exec           └─┘                                |      │
     │ │                       │          |                                           |                                 |      │
     └─┘                       │          |                                           |                                 |      │
      |                        └───────────────────────────────────────────────────────────────────────────────────────────────┘
```

docs/how-to-debug-a-hyperlight-guest.md


# Releasing a new Hyperlight version to Cargo

This document details the process of releasing a new version of Hyperlight to [crates.io](https://crates.io). It's intended to be used as a checklist for the developer doing the release. The checklist is represented in the below sections.

## Update Cargo.toml Versions

Currently, we need to manually update the workspace `Cargo.toml` version number to match to whatever release we are making. This will affect the version of all the crates in the workspace.

> Note: we'll use `v0.4.0` as the version for the above and all subsequent instructions. You should replace this with the version you're releasing. Make sure your version follows [SemVer](https://semver.org) conventions as closely as possible, and is prefixed with a `v` character. *In particular do not use a patch version unless you are patching an issue in a release branch, releases from main should always be minor or major versions*.

Create a PR with this change and merge it into the main branch.

## Create a tag

When the above PR has merged into `main` branch you should create a tag. ***Make sure you have pulled the recently updated `main` branch***, and do the following on the `main` branch:

```bash
git tag -a v0.4.0 -m "A brief description of the release"
git push origin v0.4.0 # if you've named your git remote for the hyperlight-dev/hyperlight repo differently, change 'origin' to your remote name
```

If you are creating a patch release see the instructions [here](#patching-a-release).

## Create a release branch (no manual steps)

After you push your new tag in the previous section, the ["Create a Release Branch"](https://github.com/hyperlight-dev/hyperlight/actions/workflows/CreateReleaseBranch.yml) CI job will automatically run. When this job completes, a new `release/v0.4.0` branch will be automatically created for you.

## Create a new GitHub release

After the previous CI job runs to create the new release branch, go to the ["Create a Release"](https://github.com/hyperlight-dev/hyperlight/actions/workflows/CreateRelease.yml). GitHub actions workflow and do the following:

1. Click the "Run workflow" button near the top right
2. In the Use workflow from dropdown, select the `release/v0.4.0` branch
3. Click the green **Run workflow** button

> Note: In case you see a "Create a Release" job already running before starting this step, that is because the "Create a Release" workflow also automatically runs on push to `main` branch to create a pre-release. You must still do the steps outlined above.

When this job is done, a new [GitHub release](https://github.com/hyperlight-dev/hyperlight/releases) will be created for you. This job also publishes the following rust packages to the crates.io:
- `hyperlight-common`
- `hyperlight-guest`
- `hyperlight-host`

## Patching a release

If you need to update a previously released version of Hyperlight then you should open a Pull Request against the release branch you want to patch, for example if you wish to patch the release `v0.4.0` then you should open a PR against the `release/v0.4.0` branch.

Once the PR is merged, then you should follow the instructions above. In this instance the version number of the tag should be a patch version, for example if you are patching the `release/v0.4.0` branch and this is the first patch release to that branch then the tag should be `v0.4.1`. If you are patching a patch release then the tag should be `v0.4.2` and the target branch should be `release/v0.4.1` and so on.

docs/how-to-make-releases.md


# How to use FlatBuffers

Flatbuffers is used to serialize and deserialize some data structures.

Schema files are used to define the data structures and are used to generate the code to serialize and deserialize the data structures.

Those files are located in the [`schema`](../src/schema) directory.

Code generated from the schema files is checked in to the repository, therefore you only need to generate the code if you change an existing schema file or add a new one. You can find details on how to update schema files [here](https://google.github.io/flatbuffers/flatbuffers_guide_writing_schema.html).

## Generating code

We use [flatc](https://google.github.io/flatbuffers/flatbuffers_guide_using_schema_compiler.html) to generate rust code.

We recommend building `flatc` from source. To generate rust code, use

```console
just gen-all-fbs-rust-code
```

### Note about generated code

Because we invoke `flatc` multiple times when generating the Rust code, the `mod.rs` generated in `./src/hyperlight_common/src/flatbuffers` is overwritten multiple times and will likely be incorrect. Make sure to manually inspect and if necessary update this file before continuing with your changes as certain modules might be missing. After fixing `mod.rs`, you might need to re-run `just fmt`, since it might not have applied to all generated files if your `mod.rs` was invalid.

>`flatc` does support passing multiple schema files (e.g. it is possible to pass `.\src\schema\*.fbs`), so we could regenerate all the files each time a change was made, however that generates incorrect code (see [here](https://github.com/google/flatbuffers/issues/6800) for details).

docs/how-to-use-flatbuffers.md


# How code is run inside a VM

This document details how VMs are very quickly and efficiently created and configured to run arbitrary code.

## Background

Hyperlight is a library for creating micro virtual machines (VMs) intended for executing small, short-running functions. This use case is different from that of many other VM platforms, which are aimed at longer-running, more complex workloads.

A very rough contrast between Hyperlight's offerings and other platforms is as follows:

| Feature                                                                 | Hyperlight | Other platforms    |
|-------------------------------------------------------------------------|------------|--------------------|
| Hardware isolation (vCPU, virtual memory)                               | Yes        | Yes                |
| Shared memory between host and in-VM process                            | Yes        | Yes [2] |
| Lightweight function calls between host and in-VM process (the "guest") | Yes        | No                 |
| Bootloader/OS kernel                                                    | No         | Yes [1] |
| Virtual networking                                                      | No         | Yes [2] |
| Virtual filesystem                                                      | No         | Yes [1] |

As seen in this table, Hyperlight offers little more than a CPU and memory. We've removed every feature we could, while still providing a machine on which arbitrary code can execute, so we can achieve our various use cases and efficiency targets.

## How code runs

With this background in mind, it's well worth focusing on the "lifecycle" of a VM -- how, exactly, a VM is created, modified, loaded, executed, and ultimately destroyed.

At the highest level, Hyperlight takes roughly the following steps to create and run arbitrary code inside a VM:

1. Loads a specially built, statically linked binary (currently, the [PE](https://en.wikipedia.org/wiki/Portable_Executable) and [ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) executable formats are supported) into memory. This is the code that is executed inside a virtual machine.
2. Allocates additional memory regions, for example stack and heap for the guest, as well as some regions used for communication between the host and the guest.
3. Creates a Virtual Machine and maps shared memory into it
5. Create one virtual CPU (vCPU) within the newly created VM
6. Write appropriate values to the new vCPUs registers.
7. In a loop, tell previously created vCPU to run until we reach a halt message, one of several known error states, or an unsupported message
   1. In the former case, exit successfully
   2. In any of the latter cases, exit with a failure message

---

_[1] nearly universal support_

_[2] varied support_

docs/hyperlight-execution-details.md


# Observability

Hyperlight provides the following observability features:

* [Metrics](#metrics) are provided using Prometheus.
* [Logs](#logs) are provided using the Rust [log crate](https://docs.rs/log/0.4.6/log/), and can be consumed by any Rust logger implementation, including LogTracer which can be used to emit log records as tracing events.
* [Tracing](#tracing) is provided using the Rust [tracing crate](https://docs.rs/tracing/0.1.37/tracing/), and can be consumed by any Rust tracing implementation. In addition, the [log feature](https://docs.rs/tracing/latest/tracing/#crate-feature-flags) is enabled which means that should a hyperlight host application not want to consume tracing events, you can still consume them as logs.

## Metrics

Hyperlight provides metrics using Prometheus. The metrics are registered using either the [default_registry](https://docs.rs/prometheus/latest/prometheus/fn.default_registry.html) or a registry instance provided by the host application.

To provide a registry to Hyperlight, use the `set_metrics_registry` function and pass a reference to a registry with `static` lifetime:

```rust
use hyperlight_host::metrics::set_metrics_registry;
use prometheus::Registry;
use lazy_static::lazy_static;

lazy_static! {
    static ref REGISTRY: Registry = Registry::new();
}

set_metrics_registry(&REGISTRY);
```

The following metrics are provided and are enabled by default:

* `hyperlight_guest_error_count` - a vector of counters that tracks the number of guest errors by code and message.
* `hyperlight_number_of_cancelled_guest_execution` - a counter that tracks the number of guest executions that have been cancelled because the execution time exceeded the time allowed.

The following metrics are provided but are disabled by default and require the feature `function_call_metrics` to be enabled:

* `hyperlight_guest_function_call_duration_microseconds` - a vector of histograms that tracks the execution time of guest functions in microseconds by function name. The histogram also tracks the number of calls to each function.
* `hyperlight_host_function_calls_duration_microseconds` - a vector of histograms that tracks the execution time of host functions in microseconds by function name. The histogram also tracks the number of calls to each function.

The rationale for disabling the function call metrics by default is that:

* A Hyperlight host may wish to provide its own metrics for function calls.
* Enabling a trace subscriber will cause the function call metrics to be emitted as trace events, which may be sufficient for some use cases.

There is an example of how to gather metrics in the [examples/metrics](../src/hyperlight_host/examples/metrics) directory.

The metrics capabilities provided by Hyperlight can also be used by a library or host that is using Hyperlight to provide additional metrics, see the [hypervisor metrics module](../src/hyperlight_host/src/hypervisor/metrics.rs) for an example of how to define metrics.

## Logs

Hyperlight provides logs using the Rust [log crate](https://docs.rs/log/0.4.6/log/), and can be consumed by any Rust logger implementation, including LogTracer which can be used to emit log records as tracing events(see below for more details). To consume logs, the host application must provide a logger implementation either by using the `set_logger` function directly or using a logger implementation that is compatible with the log crate.

For an example that uses the `env_logger` crate, see the [examples/logging](../src/hyperlight_host/examples/logging) directory. By default, the `env_logger` crate will only log messages at the `error` level or higher. To see all log messages, set the `RUST_LOG` environment variable to `debug`.

Hyperlight also provides tracing capabilities (see below for more details), if no trace subscriber is registered, trace records will be emitted as log records, using the `log` feature of the [tracing crate](https://docs.rs/tracing/latest/tracing/#crate-feature-flags).

## Tracing

Tracing spans are created for any call to a public API and the parent span will be set to the current span in the host if one exists, the level of the span is set to `info`. The span will be closed when the call returns. Any Result that contains an error variant will be logged as an error event. In addition to the public APIs, all internal functions are instrumented with trace spans at the `trace` level, therefore in order to see full trace information, the trace level should be enabled.

Hyperlight provides tracing using the Rust [tracing crate](https://docs.rs/tracing/0.1.37/tracing/), and can be consumed by any Rust trace subscriber implementation(see[here](https://docs.rs/tracing/latest/tracing/index.html#related-crates) for some examples). In addition to consuming trace output the log records may also be consumed by a tracing subscriber, using the `tracing-log` crate.

There are two examples that show how to consume both tracing events and log records as tracing events.

### Using tracing_forest

In the [examples/tracing](../src/hyperlight_host/examples/tracing) directory, there is an example that shows how to capture and output trace and log information using the tracing_forest crate. With this example the following commands can be used to set the verbosity of the trace output to `INFO` and run the example:

#### Linux

```bash
RUST_LOG='none,hyperlight-host=info,tracing=info' cargo run --example tracing
```

#### Windows

```powershell
$env:RUST_LOG='none,hyperlight-host=info,tracing=info'; cargo run --example tracing
```

### Using OTLP exporter and Jaeger

In the [examples/otlp_tracing](../src/hyperlight_host/examples/otlp_tracing) directory, there is an example that shows how to capture and send trace and log information to an otlp_collector using the opentelemetry_otlp crate. With this example the following commands can be used to set the verbosity of the trace output to `INFO` and run the example to generate trace data:

#### Linux

```bash
RUST_LOG='none,hyperlight_host=info,tracing=info' cargo run --example otlp_tracing
```

#### Windows

```powershell
$env:RUST_LOG='none,hyperlight_host=info,tracing=info';cargo run --example otlp_tracing
```

The sample will run and generate trace data until any key is pressed.

To view the trace data, leave the example running and use the jaegertracing/all-in-one container image with the following command:

```console
 docker run -d --name jaeger -e COLLECTOR_OTLP_ENABLED=true -p 4317:4317 -p 16686:16686 jaegertracing/all-in-one:1.60
```

NOTE: when running this on windows that this is a linux container, so you will need to ensure that docker is configured to run linux containers using WSL2. Alternatively, you can download the Jaeger binaries from [here](https://www.jaegertracing.io/download/). Extract the archive and run the `jaeger-all-in-one` executable as follows:

```powershell
.\jaeger-all-in-one.exe  --collector.otlp.grpc.host-port=4317
```

Once the container or the exe is running, the trace output can be viewed in the jaeger UI at [http://localhost:16686/search](http://localhost:16686/search).

docs/hyperlight-metrics-logs-and-traces.md


### HyperlightSurrogate

`hyperlight_surrogate.exe` is a tiny Rust application we use to create multiple virtual machine (VM) partitions per process when running on Windows with the Windows Hypervisor Platform (WHP, e-g Hyper-V). This binary has no functionality. Its purpose is to provide a running process into which memory will be mapped via the `WHvMapGpaRange2` Windows API. Hyperlight does this memory mapping to pass parameters into, and fetch return values out of, a given VM partition.

> Note: The use of surrogates is a temporary workaround on Windows until WHP allows us to create more than one partition per running process.

These surrogate processes are managed by the host via the [surrogate_process_manager](./src/hyperlight_host/src/hypervisor/surrogate_process_manager.rs) which will launch several of these surrogates (up to the 512), assign memory to them, then launch partitions from there, and reuse them as necessary.

`hyperlight_surrogate.exe` gets built during `hyperlight-host`'s build script, gets embedded into the `hyperlight-host` Rust library via [rust-embed](https://crates.io/crates/rust-embed), and is extracted at runtime next to the executable when the surrogate process manager is initialized.

docs/hyperlight-surrogate-development-notes.md


# Paging in Hyperlight

Hyperlight uses paging, which means the all addresses inside a Hyperlight VM are treated as virtual addresses by the processor. Specifically, Hyperlight uses (ordinary) 4-level paging. 4-level paging is used because we set the following control registers on logical cores inside a VM: `CR0.PG = 1, CR4.PAE = 1, IA32_EFER.LME = 1, and CR4.LA57 = 0`. A Hyperlight VM is limited to 1GB of addressable memory, see below for more details. These control register settings have the following effects:

- `CR0.PG = 1`: Enables paging
- `CR4.PAE = 1`: Enables Physical Address Extension (PAE) mode (this is required for 4-level paging)
- `IA32_EFER.LME = 1`: Enables Long Mode (64-bit mode)
- `CR4.LA57 = 0`: Makes sure 5-level paging is disabled

## Host-to-Guest memory mapping

Into each Hyperlight VM, memory from the host is mapped into the VM as physical memory. The physical memory inside the VM starts at address `0x200_000` and extends linearly to however much memory was mapped into the VM (depends on various parameters).

## Page table setup

The following page table structs are set up in memory before running a Hyperlight VM (See [Access Flags](#access-flags) for details on access flags that are also set on each entry)

### PML4 (Page Map Level 4) Table

The PML4 table is located at physical address specified in CR3. In Hyperlight we set `CR3=0x200_000`, which means the PML4 table is located at physical address `0x200_000`. The PML4 table comprises 512 64-bit entries.

In Hyperlight, we only initialize the first entry (at address `0x200_000`), with value `0x201_000`, implying that we only have a single PDPT.

### PDPT (Page-directory-pointer Table)

The first and only PDPT is located at physical address `0x201_000`. The PDPT comprises 512 64-bit entries. In Hyperlight, we only initialize the first entry of the PDPT (at address `0x201_000`), with the value `0x202_000`, implying that we only have a single PD.

### PD (Page Directory)

The first and only PD is located at physical address `0x202_000`. The PD comprises 512 64-bit entries, each entry `i` is set to the value `(i * 0x1000) + 0x203_000`. Thus, the first entry is `0x203_000`, the second entry is `0x204_000` and so on.

### PT (Page Table)

The page tables start at physical address `0x203_000`. Each page table has 512 64-bit entries. Each entry is set to the value `p << 21|i << 12` where `p` is the page table number and `i` is the index of the entry in the page table. Thus, the first entry of the first page table is `0x000_000`, the second entry is `0x000_000 + 0x1000`, and so on. The first entry of the second page table is `0x200_000 + 0x1000`, the second entry is `0x200_000 + 0x2000`, and so on. Enough page tables are created to cover the size of memory mapped into the VM.

## Address Translation

Given a 64-bit virtual address X, the corresponding physical address is obtained as follows:

1. PML4 table's physical address is located using CR3 (CR3 is `0x200_000`).
2. Bits 47:39 of X are used to index into PML4, giving us the address of the PDPT.
3. Bits 38:30 of X are used to index into PDPT, giving us the address of the PD.
4. Bits 29:21 of X are used to index into PD, giving us the address of the PT.
5. Bits 20:12 of X are used to index into PT, giving us a base address of a 4K page.
6. Bits 11:0 of X are treated as an offset.
7. The final physical address is the base address + the offset.

However, because we have only one PDPT4E and only one PDPT4E, bits 47:30 must always be zero. Each PDE points to a PT, and because each PTE  with index `p,i` (where p is the page table number of i is the entry within that page) has value `p << 21|i << 12`, the base address received in step 5 above is always just bits 29:12 of X itself. **As bits 11:0 are an offset this means that translating a virtual address to a physical address is essentially a NO-OP**.

A diagram to describe how a linear (virtual) address is translated to physical address inside a Hyperlight VM:

![A diagram to describe how a linear (virtual) address is translated to physical](assets/linear-address-translation.png)

Diagram is taken from "The Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3A: System Programming Guide"

### Limitations

Since we only have 1 PML4E and only 1 PDPTE, bits 47:30 of a linear address must be zero. Thus, we have only 30 bits (bit 29:0) to work with, giving us access to (1 << 30) bytes of memory (1GB).

## Access Flags

In addition to providing addresses, page table entries also contain access flags that describe how memory can be accessed, and whether it is present or not. The following access flags are set on each entry:

PML4E, PDPTE, and PD Entries have the present flag set to 1, and the rest of the flags are not set.

PTE Entries all have the present flag set to 1, apart from those for the address range `0x000_000` to `0x1FF_000` which have the present flag set to 0 as we do not map memory below physical address `0x200_000`.

In addition, the following flags are set according to the type of memory being mapped:

For `Host Function Definitions` and `Host Exception Data` the NX flag is set to 1 meaning that the memory is not executable in the guest and is not accessible to guest code (ring 3) and is also read only even in ring 0.

For `Input/Output Data`, `Page Table Data`, `PEB`, `PanicContext` and `GuestErrorData` the NX flag is set to 1 meaning that the memory is not executable in the guest and the RW flag is set to 1 meaning that the memory is read/write in ring 0, this means that this data is not accessible to guest code unless accessed via the Hyperlight Guest API (which will be in ring 0).

For `Code` the NX flag is not set meaning that the memory is executable in the guest and the RW flag is set to 1 meaning the data is read/write, as the  user/supervisor flag is set then the memory is also read/write accessible to user code. (The code section contains both code and data, so it is marked as read/write. In a future update we will parse the layout of the code and set the access flags accordingly).

For `Stack` the NX flag is set to 1 meaning that the memory is not executable in the guest, the RW flag is set to 1 meaning the data is read/write, as the user/supervisor flag is set then the memory is also read/write accessible to user code.

For `Heap` the RW flag is set to 1 meaning the data is read/write, as the user/supervisor flag is set then the memory is also read/write accessible to user code. The NX flag is not set if the feature `executable_heap` is enabled, otherwise the NX flag is set to 1 meaning that the memory is not executable in the guest. The `executable_heap` feature is disabled by default. It is required to allow data in the heap to be executable to when guests dynamically load or generate code, e.g. `hyperlight-wasm` supports loading of AOT compiled WebAssembly modules, these are loaded dynamically by the Wasm runtime and end up in the heap, therefore for this scenario the `executable_heap` feature must be enabled. In a future update we will implement a mechanism to allow the guest to request memory to be executable at runtime via the Hyperlight Guest API.

For `Guard Pages` the NX flag is set to 1 meaning that the memory is not executable in the guest. The RW flag is set to 1 meaning the data is read/write, as the user/supervisor flag is set then the memory is also read/write accessible to user code. **Note that neither of these flags should really be set as the purpose of the guard pages is to cause a fault if accessed, however, as we deal with this fault in the host not in the guest we need to make the memory accessible to the guest, in a future update we will implement exception and interrupt handling in the guest and then change these flags.**

docs/paging-development-notes.md


# Security Requirements for Hyperlight developers

This document discusses the security requirements and best practices for services building on Hyperlight. These requirements are designed to uphold the security promise around the guest to host boundary:

## Terminology
* _MUST_, _MUST NOT_ -- A security requirement that is mandatory to perform, or not to perform, respectively.
* _SHOULD_, _SHOULD NOT_ -- A security recommendation is encouraged to perform, or not perform, respectively.

## Brief Checklist
* All host functions that receive parameters from a guest, or operate indirectly on guest data _MUST_ be continuously fuzzed
* Host functions _MUST NOT_ call APIs or be used to expose functionality deemed risky in a multi-tenant context
* Guests and host processes _MUST_ use the same version of a FlatBuffer definition

More detailed guidance on the requirements and best practices is detailed below.

## All host functions exposed to the guest _MUST_ be continuously fuzzed
In the case of the host function calling guest functions, there will be the need to mock callees. Thus, be aware it may not be an identical state machine, thus have different bugs.
A host exposed function should be able to execute at least 500 million of fuzz test cases iterations without any crash.

For rust code, Cargo-fuzz is the recommended way to harness and satisfy the fuzzing requirements. An example with a complete example implementation can be found [Fuzzing with cargo-fuzz - Rust Fuzz Book (rust-fuzz.github.io)](https://rust-fuzz.github.io/book/cargo-fuzz.html).

```rust
#![no_main]
#[macro_use] extern crate libfuzzer_sys;
extern crate url;

fuzz_target!(|data: &[u8]| {
    if let Ok(s) = std::str::from_utf8(data) {
        let _ = url::Url::parse(s);
    }
});

```

## Host functions _MUST NOT_ call APIs or expose functionality deemed risky in a multi-tenant context

In a multi-tenant context, the following operations are considered security sensitive:
* File creation and manipulation
* Shared Data store access
* Accessing network resources
* Resource allocation and usage: if not designed properly, one guest may exhaust the resources for other tenants
* Managing encryption keys

If any of these operations is performed in a host process, a security audit _MUST_ occur.

## Flatbuffers - Guests and host processes _MUST_ use compatible versions of a FlatBuffer definitions

The guests and host processes _MUST_ use the exact same versions of a FlatBuffer definition. I.e., the same .fbs file _MUST_ be used for generating the encoders and decoders.

## Flatbuffers - If using the same language for development, the guests and host processes _SHOULD_ use the same version of flatc compilers.

This can be seen in the header files containing FLATBUFFERS_COMMON_READER_H. For instance:  `/* Generated by flatcc 0.6.2 FlatBuffers schema compiler`.
We emit this recommendation because there is a history of compiler bugs, which may affect certain behaviors (encoding, decoding). We emit this recommendation because there is a history of compiler bugs, which may adversely affect certain behaviors (encoding, decoding).

## Flatbuffers – a verifier should always be called before any decoder. In the case of failed verification, the input _MUST NOT_ be processed.

For Rust code, if the return code is InvalidFlatBuffer, the input _MUST_ be rejected.

## Flatbuffers – the host process _MUST NOT_ operate on Flatbuffers from several threads.

Because of the zero-copy approach that FlatBuffers is using, there is a risk of memory safety issues. Flatbuffers are unsafe to be used in a multithreaded environment. This is explicitly indicated in several parts of the Flatbuffer documentation.

Additionally, because Flatbuffers tainted data coming from the guests, this is even more critical in a multi-tenant scenario.

## A rust host process _SHOULD_ handle [panics](https://doc.rust-lang.org/book/ch09-03-to-panic-or-not-to-panic.html) or the service _SHOULD_ restart automatically

If the error is recoverable, the service _SHOULD_ process the next input. Otherwise, the service _SHOULD_ gracefully  restart.

docs/security-guidance-for-developers.md


# Security

A primary goal of Hyperlight is to safely execute untrusted or unsafe code.

## Threat Model

Hyperlight assumes that guest binaries are untrusted, and are running arbitrary, potentially malicious code. Despite this, the host should never be compromised. This document outlines some of the steps Hyperlight takes to uphold this strong security guarantee.

### Hypervisor Isolation

Hyperlight runs all guest code inside a Virtual Machine, Each VM only has access to a very specific, small (by default) pre-allocated memory buffer in the host's process, no dynamic memory allocations are allowed. As a result, any attempt by the guest to read or write to memory anywhere outside of that particular buffer is caught by the hypervisor. Similarly, the guest VM does not have any access to devices since none are provided by the hyperlight host library, therefore there is no file, network, etc. access available to guest code.

### Host-Guest Communication (Serialization and Deserialization)

All communication between the host and the guest is done through a shared memory buffer. Messages are serialized and deserialized using [FlatBuffers](https://flatbuffers.dev/). To minimize attack surface area, we rely on FlatBuffers to formally specify the data structures passed to/from the host and guest, and to generate serialization/deserialization code. Of course, a compromised guest can write arbitrary data to the shared memory buffer, but the host will not accept anything that does not match our strongly typed FlatBuffer [schemas](../src/schema).

### Accessing host functionality from the guest

Hyperlight provides a mechanism for the host to register functions that may be called from the guest. This mechanism is useful to allow developers to provide guests with strictly controlled access to functionality we don't make available by default inside the VM. This mechanism likely represents the largest attack surface area of this project.

To mitigate the risk, only functions that have been explicitly exposed to the guest by the host application, are allowed to be called from the guest. Any attempt to call other host functions will result in an error.

Additionally, we provide an API for using Seccomp filters to further restrict the system calls available to the host-provided functions, to help limit the impact of the un-audited or un-managed functions.

docs/security.md