MARQ prototype lifts LLM requirement-quality F1 to 0.80, paper says

[arXiv] A preprint titled "MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge," by Jannatul Shefa, Alejandro Salado, Paul Wach and Taylan G. Topcu, presents MARQ, a modular, LLM-agnostic prototype that supplements large language models with a criterion-specific knowledge stack operationalizing INCOSE requirement quality criteria. The authors say they evaluated it against off-the-shelf baselines on an expert-labeled NASA requirement set using Claude Opus 5, GPT-5.6 Sol and the open-weight Gemma 3 12B, with 100 executions per model. According to the paper, MARQ achieved median F1 scores of 0.80 for Claude Opus 5, 0.62 for GPT-5.6 Sol and 0.53 for Gemma 3 12B, versus off-the-shelf baselines of 0.55, 0.52 and 0.18, respectively. The authors said the nature of the gains is model-dependent: Claude Opus 5 improved defect detection and false-alarm control; GPT-5.6 Sol gained detection and better execution-level consistency at the cost of more false alarms; and Gemma 3 12B reached the performance of off-the-shelf proprietary models while remaining highly variable. Criterion-level analysis showed gains concentrated in Correctness and Conformance, trade-offs between misses and false alarms within individual criteria, and model-dependent responses across several criteria. The findings are from an unreviewed preprint.