Setting up an Node (Deploy a Node)

A Node is a machine that hosts LLM and responds to the prompts it receives.

The node typically high-end computers with at least 8GB RAM/VRAM to run inference at a reasonable speed (>10 tokens per second).

Currently, there are two ways to host a Node

Last updated 4 months ago

A Node is a machine that hosts LLM and responds to the prompts it receives.

The node typically high-end computers with at least 8GB RAM/VRAM to run inference at a reasonable speed (>10 tokens per second).

Currently, there are two ways to host a Node