Generative AI chatbots have now become part of everyday life and are increasingly being used to search for political information. This is changing the way political information is sought, processed and categorised. Against this backdrop, ConPolicy, on behalf of the State Media Authority of North Rhine-Westphalia, has investigated how users employ generative AI to search for political information.
The recently published study focuses on the analysis of so-called prompts – that is, the text inputs users employ to formulate their questions to AI systems. To this end, a representative survey was conducted with over 2,000 participants, in which they interacted with an AI chatbot in realistic scenarios. The prompts entered were subsequently analysed using natural language processing.
The results show that practical questions relating to everyday life and household matters (56 per cent), health (48 per cent) or purchasing decisions (36 per cent) have so far dominated the use of AI chatbots, yet just under a quarter of respondents are already using them specifically to obtain political information. Most users interact with AI chatbots using simple, question-based inputs and use them in a similar way to search engines; advanced prompting techniques, such as assigning a specific role, play hardly any role at all, accounting for just 0.4 per cent. At the same time, the study makes it clear that the wording of the prompts influences whether a web search is triggered and the basis on which AI-generated responses are produced: For instance, in response to initial queries, ChatGPT only drew on web sources in around half of the cases (52.3 per cent), even though up-to-date, verifiable sources are particularly important when it comes to political topics.
The study was published here. A factsheet on the study can also be found here.
Further information on the project can be found here.