AIS (AI Station): Difference between revisions
Jump to navigation
Jump to search
(Created page with "== Overview == == References == <references />") |
(AIS (AI Station) — brief intro (HMSmartLLM)) |
||
| Line 1: | Line 1: | ||
== Overview == | == Overview == | ||
== | AIS (AI Station) is a '''multi-GPU local LLM serving platform''' that lets a user start, manage, chat with, and [[benchmark]] LLMs on their own GPU server through a web dashboard. It runs inside a [[Docker]] container and handles the full serving lifecycle: model discovery, smart GPU/port allocation, server lifecycle (start/stop/restart), live throughput monitoring, chat, an optional ReAct agent, and real [[performance benchmarking]]. | ||
< | |||
== Purpose == | |||
* Give a server owner a single web UI to run and manage local LLMs without manual [[llama.cpp]] commands | |||
* Automatically pick the right GPU, context size, and port for each model | |||
* Show live server state (health, measured throughput, slot occupancy) at a glance | |||
* [[Support]] multi-GPU, multi-server operation on a single box | |||
== Key Concepts == | |||
{| class="wikitable" | |||
! Concept | |||
! Description | |||
|- | |||
| Web Dashboard | |||
| Browser-based UI (port 8090) for all AIS operations | |||
|- | |||
| Smart Suggestion | |||
| Engine that recommends GPU, ctx, and port based on model size and current VRAM | |||
|- | |||
| Auto-Recovery | |||
| A crashed server is re-spawned automatically with backoff | |||
|- | |||
| Speed Column | |||
| Live measured throughput (tok/s) per running server, updated on every poll | |||
|- | |||
| ReAct Agent | |||
| Optional tool-calling agent (file read/write, web fetch/search, shell) | |||
|} | |||
== Workflow == | |||
# Log in to the AIS dashboard (http://<server>:8090) | |||
# Go to '''Models''' — browse cached models or pull a new one from HuggingFace | |||
# Go to '''Serving''' — select a model, review the smart suggestion, click Start | |||
# Go to '''Chat''' — select the running server and start conversing | |||
# Optionally configure an '''Agent''' profile for tool-calling tasks | |||
== Related Pages == | |||
* [[Manual:AIS]] — full user manual (screenshots + step-by-step) | |||
* [[AI Agents]] | |||
* [[GPU]] | |||
[[Category:AI]] | |||
[[Category:Reference]] | |||
Revision as of 11:00, 10 October 2026
Overview
AIS (AI Station) is a multi-GPU local LLM serving platform that lets a user start, manage, chat with, and benchmark LLMs on their own GPU server through a web dashboard. It runs inside a Docker container and handles the full serving lifecycle: model discovery, smart GPU/port allocation, server lifecycle (start/stop/restart), live throughput monitoring, chat, an optional ReAct agent, and real performance benchmarking.
Purpose
- Give a server owner a single web UI to run and manage local LLMs without manual llama.cpp commands
- Automatically pick the right GPU, context size, and port for each model
- Show live server state (health, measured throughput, slot occupancy) at a glance
- Support multi-GPU, multi-server operation on a single box
Key Concepts
| Concept | Description |
|---|---|
| Web Dashboard | Browser-based UI (port 8090) for all AIS operations |
| Smart Suggestion | Engine that recommends GPU, ctx, and port based on model size and current VRAM |
| Auto-Recovery | A crashed server is re-spawned automatically with backoff |
| Speed Column | Live measured throughput (tok/s) per running server, updated on every poll |
| ReAct Agent | Optional tool-calling agent (file read/write, web fetch/search, shell) |
Workflow
- Log in to the AIS dashboard (http://<server>:8090)
- Go to Models — browse cached models or pull a new one from HuggingFace
- Go to Serving — select a model, review the smart suggestion, click Start
- Go to Chat — select the running server and start conversing
- Optionally configure an Agent profile for tool-calling tasks
Related Pages
- Manual:AIS — full user manual (screenshots + step-by-step)
- AI Agents
- GPU