Generative AI chatbots have now become part of everyday life for many people and are increasingly being used to research political information. Applications such as ChatGPT, Copilot and Gemini enable users to ask complex questions in natural language and access information through dialogue. This is changing the way in which political information is sought, selected and interpreted. At the same time, this development raises new questions for media supervision and regulation, as algorithmic systems are playing an increasingly significant role in the dissemination of information and, consequently, potentially also in the formation of political opinion.
Against this backdrop, ConPolicy was commissioned by the State Media Authority of North Rhine-Westphalia to examine in greater detail the use of generative AI applications for searching for political information. The focus was, in particular, on analysing so-called ‘prompts’ – that is, the text inputs users employ to formulate their information requests to AI systems. The aim of the study was to identify typical prompting strategies and to gain a better understanding of how users actually employ generative AI for political information research.
Methodologically, a quantitative survey was conducted which enabled participants to interact with an AI chatbot in a realistic manner. The prompts formulated were subsequently analysed using natural language processing (NLP) methods. On this basis, different types of prompt formulation were examined, as well as correlations with socio-demographic characteristics, digital skills, usage habits and LLM-based web searches.
The project was implemented across four work packages:
- Work Package 1: Kick-off and development of a research design and questionnaire
- Work Package 2: Programming of the questionnaire and conducting the quantitative survey in a real-world usage context
- Work Package 3: Preparation and evaluation of the data using natural language processing and analysis of correlations
- Work Package 4: Summary of the typology and preparation of a final report
The study provided empirical insights into real-world usage patterns of generative AI in the search for political information and lays an important foundation for the further development of auditing and supervisory procedures in the digital information space.
The study was published here. A factsheet on the study can also be found here.
