Does Linguistic Register Affect Faithfulness in Spanish Cache-Augmented Generation? An Exploratory Study with Open-Source LLMs

Abstract

Cache-Augmented Generation (CAG) provides an alternative to retrieval-based knowledge augmentation by preloading external knowledge into a reusable key-value (KV) cache. However, the extent to which CAG remains stable when semantically equivalent queries are expressed through different linguistic registers remains insufficiently understood, particularly in Spanish. This exploratory study investigates whether formal-colloquial register variation affects the Faithfulness of CAG responses. A controlled paired design was constructed from 10 Spanish concepts and two complementary tasks definition generation and concept identification yielding 20 semantic cases and 40 formal-colloquial queries per model. Three open-source instruction-tuned LLMs (Qwen2.5-1.5B-Instruct, SmolLM2-1.7B-Instruct and Ministral-3B-Instruct) were evaluated under deterministic generation conditions using a fixed preloaded knowledge context. Faithfulness was assessed with RAGAS using Granite 3.2 as the LLM judge. Of 120 planned generations, 113 produced valid Faithfulness evaluations, yielding 53 complete formal-colloquial pairs. Neither paired parametric nor non-parametric analyses identified statistically significant register effects for any individual model and the between-model analysis did not detect systematic differences in register sensitivity. Descriptively, however, SmolLM2 tended to favor formal formulations, Ministral showed the opposite tendency and Qwen2.5 exhibited the most stable profile, combining comparatively high Faithfulness, limited formal-colloquial separation and relatively low within-condition variability. These findings provide initial evidence that formal-colloquial variation does not produce a detectable systematic effect on CAG Faithfulness under the conditions examined, while suggesting that descriptive sensitivity to linguistic register may depend on the underlying model and task configuration.