DOI: https://doi.org/10.1145/3726302.3731960
The quality of image products in second-hand marketplaces is a critical factor as it can significantly impact a buyer`s decision-making process as well as overall user trust. Inspired by the successful application of LLM capabilities as text-based task evaluators, we propose an approach that leverages multi-modal large language models (MLLMs) as evaluators of image quality. In this work, we conduct a systematic comparison of several state-of-the-art MLLMs, evaluating the alignment between the scores generated by these models and the human scores collected from a survey of 929 users in a second-hand marketplace. Our findings demonstrate that some of the evaluated MLLMs can achieve a high level of agreement with human judgments. To further understand the differences between LLM-based and human scores, we also present an analysis of the alignment between explanations generated by LLMs and those provided by humans. Overall, we believe our findings underscore the potential of LLMs for automatic image quality assessment.