I'd expect a company like IKEA to have the expertise to create interfaces specific to their workflows so hallucinations aren't an issue.

Imo if you're making an open ended chat interface for a business, you're doing it wrong.

Have you considered instruction-tuning it with text, instead of just pictures?