TechRxiv paper evaluates edge-deployed quantized SLMs for virtual-world conversational agents

[TechRxiv] Louis Nisiotis and Nikita Markov said in a TechRxiv preprint that they built a RAG-based back-end AI system to support virtual agent conversational capabilities in a virtual-world prototype and tested two small language model families at three quantization levels (8-bit, 4-bit and 3-bit), evaluating system performance through end-to-end latency, time-to-first-token, throughput and memory headroom. They also assessed response quality through expert evaluations of accuracy, relevance, safety and human alignment against non-quantized baselines. According to the authors, the results show that moderate quantization enables performant practical edge deployment with good response quality, and that edge-deployed quantized SLMs can support real-time narrative interactions and remain practically useful for virtual-world agents. The paper is a preprint and has not been peer-reviewed.