Recently, a study conducted by the University at Buffalo in collaboration with the University at Albany and the Chinese University of Hong Kong has shed light on the effectiveness of large language models (LLMs) in detecting deepfakes compared to state-of-the-art algorithms. While LLMs have been found to perform poorly in this area, their natural language processing capabilities may prove to be more effective in the future.
The study presented at the IEEE/CVF Conference on Computer Vision & Pattern Recognition asked LLMs, including OpenAI’s ChatGPT and Google’s Gemini, to identify deepfakes of human faces. Unlike existing detection methods, LLMs can explain their findings in a way that is understandable to humans, such as identifying specific details like shadows or mismatched features.
LLMs were not specifically designed for deepfake detection, but their semantic knowledge makes them well-suited for this purpose. The latest versions of LLMs, like ChatGPT and Gemini, can also analyze images using large databases of photos and captions to understand relationships between words and images. This semantic knowledge allows these models to provide more human-understandable explanations for their conclusions.
The study found that ChatGPT was able to accurately detect synthetic artifacts in images generated by different methods, comparable to previous deepfake detection algorithms. However, LLMs like ChatGPT may struggle in capturing statistical differences at a signal level, limiting their detection capabilities in some cases. Furthermore, other LLMs, like Gemini, may provide nonsensical explanations for their analyses or refuse to analyze images altogether.
To improve performance and make them more efficient tools for users and developers, the researchers suggest fine-tuning LLMs for deepfake detection. While LLMs may not be as advanced as current detection algorithms, their natural language processing capabilities and semantic knowledge offer a unique approach to deepfake detection. By leveraging these strengths, LLMs like ChatGPT could play a valuable role in combating the spread of AI-generated misinformation in the future.