The authors of the InfiHall-Bench paper report that their CMHM fine-tuning method lifts the Overall score to 58.13 on Infinity-2B, 64.77 on Infinity-8B-512 and 67.49 on Infinity-8B (arXiv). Against the pretrained versions, that is a gain of 7.30, 8.35 and 1.72 points, according to the paper. The benchmark also led them to a diagnosis: in visual autoregressive (VAR) models, semantic errors can emerge at different autoregressive scales and are rarely corrected by subsequent fine-scale steps (arXiv). The figures are the authors’ own, taken from a paper table rather than an independent test.
A benchmark built from composed skills
InfiHall-Bench is a controlled set of 411 prompts, obtained by composing five skills that are explicitly stated in the prompt and visually verifiable. The five skills are the binding of attributes, spatial relations, action relations, the consistency of parts and the consistency of scenes. The paper uses Qwen3.7-Plus as the multimodal evaluator. Scores therefore depend on that judge model, which a practitioner should keep in mind when comparing them with other benchmarks.
The authors say hallucinations in VAR models show scale-wise persistence: an error that becomes established at one autoregressive scale is rarely repaired by subsequent fine-scale steps. If that holds, correcting the problem at inference by adding refinement steps would help little, and training is the more natural place to intervene.
What CMHM changes in training
According to the paper, CMHM changes only how the training loss is spread over visual tokens and scales. The architecture stays as it is, and inference gets no extra module. The reported settings are 0.3 and 0.7 for the two confidence thresholds and 2.0 for beta, with weighting applied at all autoregressive scales. Because inference is untouched, the method’s cost falls on fine-tuning, not on generation speed. For a team already fine-tuning a VAR model, the change is therefore a different loss weighting rather than a new network to deploy.
Results in Table 3
| Table block | Model | Action Rel. | Scene | Spatial Rel. | Overall | Source page |
|---|---|---|---|---|---|---|
| Diffusion T2I Models | FLUX.1-dev | 66.85 | 92.87 | 54.13 | 67.08 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | Infinity-2B | 48.40 | 87.64 | 39.02 | 50.83 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | + infihall-train | 50.28 | 93.46 | 45.64 | 57.30 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | + CMHM (Ours) | 53.05 | 93.65 | 47.77 | 58.13 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | Infinity-8B-512 | 53.05 | 90.90 | 42.70 | 56.42 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | + infihall-train | 60.53 | 95.18 | 53.77 | 63.80 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | + CMHM (Ours) | 61.08 | 95.06 | 56.38 | 64.77 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | Infinity-8B | 64.53 | 96.08 | 56.56 | 65.77 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | + infihall-train | 63.88 | 95.21 | 58.99 | 66.35 | https://arxiv.org/html/2611.18053v1 |
| Finetuned Visual Autoregressive T2I | + CMHM (Ours) | 66.28 | 95.94 | 60.45 | 67.49 | https://arxiv.org/html/2611.18053v1 |
Among pretrained models, FLUX.1-dev has the highest Overall score at 67.08, while Infinity-8B reaches 65.77, according to the paper. CMHM’s Overall edge over the infihall-train baseline, which uses uniform weighting, is 0.83, 0.97 and 1.14 points, one margin for each Infinity configuration. On Infinity-8B, CMHM raises the Overall score above FLUX.1-dev by 0.41 points. In the table, Infinity-2B rises from 50.83 pretrained to 58.13 with CMHM, while the larger Infinity-8B moves from 65.77 to 67.49. Infinity-8B-512 goes from 56.42 pretrained to 64.77 with CMHM, a gain of 8.35 points, against 7.30 points for Infinity-2B and 1.72 for Infinity-8B. The gains are larger on Infinity-2B (7.30) and Infinity-8B-512 (8.35) than on Infinity-8B (1.72), which the authors read as a bigger absolute benefit when the pretrained configuration has more remaining errors. On Infinity-8B, the infihall-train baseline scores 66.35 Overall, which is already above the pretrained 65.77 but below the 67.49 reached with CMHM.
Plausible scenes, weaker actions and layouts
The paper says that across the three pretrained Infinity configurations the Overall score rises from 50.83 to 65.77, and that stronger configurations improve prompt fidelity without eliminating fine-grained hallucinations. Pretrained Infinity-8B scores only 64.53 on Action Relation and 56.56 on Spatial Relation, against 96.08 on Scene Consistency. The authors conclude from that gap that producing a believable overall scene is much simpler than rendering faithfully the actions and relative positions a prompt specifies. The table shows CMHM only partly closing it: Infinity-8B with CMHM scores 66.28 on Action Relation and 95.94 on Scene Consistency. FLUX.1-dev scores 66.85 on Action Relation, above the 66.28 of Infinity-8B with CMHM, even though the CMHM model leads it on Overall. Scene Consistency for Infinity-8B is slightly lower with CMHM at 95.94 than the pretrained 96.08. Taken together, the table suggests that the remaining weakness lies in relational skills rather than in overall scene plausibility, which is where further work would matter most.
What the sources we read do not state
We check six facts on every story in the full text of each source. The pages listed under Sources do not state a price.
How we built this
Published resultsTable 3 of the paper: Results on InfiHall-Bench
Rows of Table 3 of the paper (arXiv HTML version), copied cell for cell by our code. These are the authors' own figures, not an independent test. The first column is the block of the table the row sits in. The table has 7 columns; 5 shown, with every column that holds a figure this article gives. Not shown: Attribute, Part. Rows: 10 of 16, with every row whose value this article gives; not shown: rows 1, 2, 3, 4, 6, 7; whole blocks only (a row with the rows "+ …" under it). Of its 3 blocks, not shown: Autoregressive T2I (2 rows); shown in part: Diffusion T2I Models (1 of 5 rows).
Sources
The sources this article was written from. Links open the original page.