Paul Van Eecke (VUB)
Katrien Beuls (UNamur)
Tim Brys (VUB)
Adapted by Folco Soffietti (VIU)

Source: Pexels, modified by Folco Soffietti  

Adapted from the article “Studeren zonder ChatGPT, daarna verstandiger werken met AI” published on the Knack on the 9th of April 2024.

The Integration of AI into curricula cannot neglect a sound domain knowledge, good language skills and thorough knowledge of the scientific method.

In Belgium, the Rector of the University of Gent announced that from next academic year, students will be allowed to use generative AI in their thesis. This sparked a debate within the academic community, questioning the purpose and methods of university evaluation in the age of AI.

Generative AI is supposed to be able to reshape every aspect of the research cycle, but is it so? MUHAI’s researchers can help you dive a bit deeper in the matter.

Generative AI systems such as ChatGPT can produce texts that are sometimes indistinguishable from those written by humans, but still have difficulties in imitating logical thought processes. Even so, let us assume that generative AI can still replace important elements of the research process. The detection of AI use is becoming increasingly accurate, just like plagiarism. And what student will risk losing their degree due to fraud, possibly even years after the date?

Even so, we can still make the exercise of assuming that it is not feasible to ban the use of generative AI. Then two paths open up: allow AI use or discard the thesis. Contrary to the Rector of the University of Gent, some professors consider that understanding and processing of knowledge cannot be outsourced to ChatGPT when studying. The risk is, in fact, to no longer acquire these non-negotiable basic competencies. They are crucial in an academic education. Therefore, if the use of generative AI cannot be restricted in practice, it seems that the master's thesis shall be replaced.

Until recently, the master's thesis was the culmination of a long educational curriculum in which a set of skills and understandings were gradually built up in students. In general, we do teach skills for which there exist digital tools that can perform them better or more efficiently. For example, we still learn spelling in primary school, even though spell-checking programmes exist. Why? Because spell check only makes sense if you can already spell yourself and write an intelligible text. The same with arithmetic: we still learn arithmetic rules because a calculator is only useful if you have the necessary mathematical understanding yourself. These tools can help us avoid mistakes and make our work more efficient. But if we cannot interpret the output of these tools, they are completely useless.

It is no different with generative AI at the master's level. In a master's programme, a student is expected to grasp professional literature, acquire research skills, develop critical thinking, and so on. Only someone who has already acquired these skills to a considerable extent can use generative AI responsibly at the level expected of a master. How else can that person interpret what the AI system produces? An incompetent writer cannot judge the quality of the generated text, let alone improve it.

if we want our students to be able to use generative AI competently and responsibly in the workplace, they will still have to acquire the basic equipment of the master's degree. Shortcuts and distractions of generative AI will have to be ignored for a while with the higher goal in mind: being formed into a competent expert in the chosen field of study.

If the master's thesis won’t be a reliable way to test academic growth, there are fortunately alternatives. Evaluating throughout the year, or final exams. Either way, generative AI should not mean the end of the goal of shaping students broadly and scientifically. And this, regardless of a specific tool.

Alexane Jouglar, University of Namur.

Source: https://dictionary.cambridge.org/dictionary/english/cook

Cooking is an act that we perform in our everyday lives to produce (delicious) dishes ready to eat. Many machines exist to help humans cook: slow cookers, multi-cookers, pressure cookers… All these machines are really helpful, but they need the direct intervention of a human. Currently, It is not possible to give a recipe to a machine, place the machine in the kitchen, and ask it to cook the dish. The reason for this is that recipes include a lot of background and implicit knowledge that humans gradually acquire by practicing and interacting with the world and that cannot be understood from the only lecture of the recipe. Let’s take an example.

Source: https://chocolatecoveredkatie.com/vegan-chocolate-chip-cookies-recipe/

As we can see in the figure above, the recipe instructions start with “Combine all dry ingredients in a bowl”. As humans, we know what dry ingredients are. We also know that by “all”, the writer means “from all the ingredients that are listed above”. For a machine, this is a lot harder. It needs to be equipped with a lot of background knowledge. That's precisely the challenge posed by a groundbreaking new benchmark introduced in a recent paper published at LREC-COLING 2024:

Nevens, J., De Haes, R., Ringe, R., Pomarlan, M., Porzel, R., Beuls, K., & Van Eecke, P. (Accepted/In press). A Benchmark for Recipe Understanding in Artificial Agents. In The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation

This benchmark is the result of a collaboration between the Vrije Universiteit Brussels (VUB), and the University of Bremen (UBremen) of the University of Namur (UNamur). It aims to evaluate whether artificial agents can understand and perform everyday cooking activities. It includes several components:

• Corpus of Recipes: A collection of 30 recipes of varying complexity.
• Procedural Semantic Representation Language: This language helps formalize 38 cooking actions in a way that machines can understand. It breaks down recipes into precise steps and provides a framework for representing cooking processes.
• Kitchen Simulators: Both qualitative and quantitative simulators recreate kitchen environments where agents can practice cooking. These simulators simulate the physical aspects of cooking, such as ingredient interactions and cooking times.
• Evaluation Procedure: A standardized method for evaluating agent performance.

The main task of the benchmark is to translate natural language recipes into a series of cooking actions that can be executed in the simulated kitchen to produce the desired dish. This translation process requires the agent to reason over multiple factors, including the recipe text, the state of the simulated kitchen, common-sense knowledge, and domain-specific cooking knowledge.

Success in this benchmark requires a combination of natural language processing and situated reasoning. By mastering the art of cooking, artificial agents can become more versatile and helpful in various real-world scenarios.

Source: Canva

The introduction of this benchmark represents a significant step forward in the field of artificial intelligence. It challenges researchers to develop systems that can understand natural language and apply that understanding to practical tasks like cooking. As these systems improve, they have the potential to revolutionize various aspects of our lives, from personal assistance in the kitchen to industrial food production.

Discover the full paper in our dedicated section: Jens Nevens, Robin de Haes, Rachel Ringe, Mihai Pomarlan, Robert Porzel, Katrien Beuls and Paul van Eecke. A Benchmark for Recipe Understanding in Artificial Agents. In Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti and Nianwen Xue (eds.). Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 2024, 22–42

Anna Morbiato, Venice International University.

In the dazzling realm where cutting-edge technology meets the world of culinary arts, precision is not just a preference – it's the secret sauce to success. Imagine a futuristic kitchen where robotic chefs whip up gourmet delights with a mere tap on a touchscreen. Now, in this technologically driven culinary landscape, the unsung hero emerges: accurate tracking of coreference. Like a culinary GPS, it navigates all complex recipe transformations undergone by ingredients, ensuring that every ‘it’ and ‘they’ (as in ‘put it in the fridge’ or ‘they are ready when a fork slides in easily’) points to the right flavour-packed entity. Let's explore why this seemingly subtle linguistic skill is the backbone of a seamless, tech-infused gastronomic experience.
Take a simple Italian pasta recipe step: 'Boil a pot of water, add some salt, then add your favourite pasta type. Once it's cooked, drain it and then add it directly to the tomato sauce.' While us readers have no doubt what 'it' refers to, it is not as straightforward for a machine to understand that 'it' refers to the pasta and not to other nouns like 'pot,' 'water,' or 'salt.' Still, it is crucial that the right item is added to the sauce for a successful execution of the recipe.

Accurate tracking of coreference becomes crucial in a technologically driven culinary landscape. But this comprehension comes with its set of challenges. Recipes are full of pronouns and other ways to refer to items, ingredients, and tools. However, these are sometimes far from transparent and often include not only pronouns, but also even zero anaphors (Ø) (namely using no words at all!). The same sentence as above could work also with no 'it': Boil a pot of water, put some salt, then add your favourite pasta type. Once Ø cooked, drain Ø and then add Ø directly to the tomato sauce.’ Not only ingredients, but also intermediate products of each step (resultant objects, like a mixture or a cream) are frequently left unmentioned, resulting in potential ambiguity. This phenomenon is not marginal; in fact, zero anaphors (also referred to as zero pronouns, null elements, or implicit entities) are more frequent in recipes as compared to other genres. This is especially evident in pro-drop languages like Italian and topic-drop languages like Chinese, where zero anaphora is widespread and often requires inference for correct interpretation. Further challenges include partial coreference phenomena (also known as bridging) as well as evolutive anaphors, namely linguistic expressions that refer to an entity while also indicating a change in its properties or attributes over time. Evolutive anaphors are very common in culinary narratives, where they typically refer to an ingredient or component of the recipe which, in the meantime, has undergone some transformation.
Imagine we want to make baked potatoes. The recipe starts by telling us ‘Wask and peel the potatoes.’ Then it goes on with the cooking procedure, which includes garlic, warm milk and butter. The last passage says: “Blend with a potato masher or electric mixer until potatoes are smooth and creamy.” For sure, the entity denoted by the term potatoes in the ingredients and that denoted by the same word in the last passage are very different. Then why do we still use the word potatoes? And why does it sound odd if we do the same in a sentence like ‘Juice the apples, then put them onto a pan.’?

Unravelling the complexities of anaphoric references in recipes is the objective of the paper ‘Pragmatics as the secret ingredient for NLP: a cross-linguistic study of reference tracking and evolutive anaphors’. The paper provides an in-depth exploration of coreference encoding and resolution in the domains of linguistics and Natural Language Processing, with a particular emphasis on zero pronouns, partial coreference phenomena, and evolutive anaphors. Additionally, it presents an analysis of the flow of information in recipe texts, as well as of the type, frequency, and nature of anaphoric devices used in authentic recipes drawn from food blogs. It considers texts in English, Italian, and Chinese, which significantly vary in terms of coreference tracking mechanisms and demonstrate different levels of dependence on zero anaphors and inferential processes. The paper gives particular attention to evolutive anaphors and the role played by inference and world knowledge in coreference disambiguation. Recall the pasta recipes above: the pronoun it involves a change in properties of the pasta, from being hard to being soft after cooking. Crucially, inference plays a significant role in coreference disambiguation, in addition to, or in place of, lexical and grammatical encoding. In fact, it is world knowledge and common sense that enable the reader to understand that the anaphors refer to pasta, and not to pot, water, or salt. Any AI systems aiming to effectively address anaphora resolution and achieve complete textual comprehension must integrate inferential processes at a certain stage.

Robert Porzel, University of Bremen.

One major motivation of explainable artificial intelligence (XAI) is the desire to make the predictions of black-box machine learning (ML) models more transparent. Adadi and Berrada conducted a survey of the XAI literature and created an overview of common XAI methods - all of the methods presented range over the internal mechanisms of a ML model [Adadi and Berrada, 2018], which can be called introspective explanations. More recently, there has been an increasing need to explain AI behavior to non-expert users of AI systems. Therefore, XAI needs to focus more on end-users as the recipients of the explanations and on how XAI methods can be evaluated from a human-centered point of view. This sheds a different light on estimating the quality of an explanation and how it can be tailored adequately to the given user. This view is supported by the argument that explanation should be understood as interactive conversations. The usefulness of explanations can depend on the context in which they are given, including the situation and the user. In domains such as everyday activities, human users interact with AI agents repeatedly over time. When the two disagree, helpful and succinct explanations of the AI’s behavior should be tailored to the user based on its memory of their previous interactions.

To foster a human-centered approach to XAI, individuals and their preferences should be considered by appropriate user models. These can encompass, e.g., a level of education, cultural background, familiarity and interest in technology, perception of the agent, mood, and social settings. To accommodate these factors, personalization of AI explanations might be crucial. This is particularly relevant in domains where users interact with the same artificial agent over a long period, such as smart home devices, robots in household and care domains, and other personal AI applications. If users ask for explanations because they expected different AI behaviour, there must be some disagreement between users and AI in their respective beliefs and knowledge, which is what we call their uncommon ground. Explanations should allow the user to realize where the uncommon ground is. In error cases, this can empower users to correct the AI’s behaviour for future interactions; in other situations, it can enable them to see that they are missing important knowledge themselves. To facilitate this, explanations by the AI should reflect its experience from previous interactions with the same user. Personalization can improve user satisfaction by using user models created with direct or indirect user input, i.e., user entry or automated systems that adapt to user behaviour. Although direct user inputs can have advantages, an agent that is not properly configured may be less acceptable, thus personalization through indirect adaptation might be preferential. We argue that it is thus necessary to consider how to generate extrospective explanations: explanations that are given with respect not only to the system itself but also take into account what the system knows about the user’s expectations and draw on experience from earlier interactions. This extrospective perspective on XAI is at the heart of our work, as we focus on domains where users interact with smart software, intelligent devices, and autonomous agents/robots repeatedly over time, e.g., smart home device and household. This, in a sense, entails that users have their ‘own’ personalized AI. Explanations in the context of repeated interactions have not been studied as much. Therefore, we seek to examine this challenging problem systematically.  There is a growing awareness in the XAI community that explanations ought to be considered and evaluated in the context of the user for whom they are intended. Previous works have pointed out the problem of implied-but-not-stated contrastive/counterfactual cases when humans ask for explanations as well as that there are many situations in which the user’s goal in asking for an explanation relies on understanding why the AI prediction differed from their expectation.

We claim that in the domain of AI in the home (e.g., smart home devices, household robots), extrospective explanations can provide more helpful information for end-users in a more human-centered way, enabling them to understand the reason for unexpected agent behaviour. We aim to achieve this by focusing on explanations for transparent, reasoning-based AI systems and explicitly modelling user and discourse context in the agent’s knowledge. If the AI predicts the same out- come based on the same situational knowledge and ontology in two equivalent situations, the respective explanations could differ based on the user asking for them: For each user, the AI would output only a part of their reasoning. Specifically, the AI should output those facts or inferences that it expects are most likely to surprise the user. This should be whatever fact or logical inference the AI has the least reason to believe that the user agrees with or is aware of. In other domains, however, there might be different reasons why explanations should be purely introspective. There are many areas of life where AI is or will be used, in which it will be important that explanations are consistent across different users or that they are a complete representation of the AI reasoning (e.g., medical or legal problems). Therefore, it will be essential to distinguish between explanations aimed at helping individual users understand local, surprising AI predictions and explanations aimed at increasing ML systems’ transparency.

[Adadi and Berrada, 2018] Amina Adadi and Mohammed Berrada. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6:52138–52160, 2018.

Full Paper in the papers page:

Laura Spillner, Nima Zargham, Mihai Pomarlan, Robert Porzel and Rainer Malaka. Finding Uncommon Ground: A Human-Centered Model for Extrospective Explanations. In Proceedings of the 2023 IJCAI workshop on Explainable Artificial Intelligence (XAI). 2023. BibTeX, pdf 

Luc Steels, Venice International University.

The AI Avant-Garde in the 1990s.

In the 1980s and 1990s a new wind was blowing in many sciences, based on the realization that various complex phenomena in Nature can be understood by viewing them as self-organizing dynamical systems. Typical examples are tornados, meandering rivers, paths formed by ant colonies, chemical oscillations, patterns on a zebra skin, honeycombs in beehives, coordinated activities in neural networks in the brain, crashes in the stock market, and many more. A new science of complex systems was born coinciding with profound mathematical developments in fractal geometry, chaos theory and the study of large-scale networks.

But how to archive, register and share this revolutionary moment?

A pop up exhibition was conceived to shows videos of interviews, talks, and panels with the scientists who developed these original ideas and carried out some of the key experiments. They included Nobel laureates Christian de Duve, Manfred Eigen, Roger Penrose and Ilya Prigogine, as well as the mathematicians that founded chaos theory (Benoit Mandelbrot, Daniel Ruelle, Floris Takens, a.o.) and leaders in the application of self-organisation in physics, chemistry, biology, neuroscience, and economics.

The exhibition also displayed books and experimental objects, as well as art works from the ‘Wunderkammer’ of artist Anne Marie Maes. A second theme of the exhibition was how researchers at the forefront of Artificial Intelligence and Artificial Life were swept up by these ideas, leading to a renaissance of neural network research and highly dynamic, embodied robotic agents inspired by nature, called animats. To illustrate this research line, the exhibition included various robots and a large number of videoclips on experiments carried out in the 90s to explore how intelligence can emerge from situated interaction.

To illustrate some of these investigations, such as on the solar system, the network structures of the brain, or the collective behavior of insects, the exhibition was expanded with top pieces from the collections of the Royal Library, including the first edition of Newton's Principia, the observations of Copernicus, the writings of Laplace and Poincare on the predictability of celestial movements, the precious drawings of plants of 17th century botanists like Rembert Dodoens, Jan-Baptista van Helmont, and more.

Pop-up exhibition at the Royal Library (KBR)

https://www.kbr.be/en/

Kunstberg - Mont des Arts 28, 1000 Brussel

Opening: 18 December 2023 2pm

Closing: 22 December 2024 3 pm

Curator: Luc Steels

Carlo R. M. A. Santagiustina, Venice International University.

-Continues from Part 2

To answer the questions raised in the previous chapter of this article in a synthetic way, here follows a short-list of major implications of the aforementioned changes to data access policies. To grasp the societal impact of these changes, I selected two examples of social media data-based and community-centric projects. Today, with the current social media policies, the collection of user-generated data required by these projects would have been impossible to afford. Both projects are ongoing and are related to -and contributed by- MUHAI. The first one, called AquaGranda: A digital community memory, started in 2020 from the EU project Odycceus, after the extreme tide of 187cm that flooded Venice in November 2019, and progressively grew involving other EU projects and research institutions, such as MUHAI, and recently received an Honorary Mention at ArsElectronica, for the European Union's Citizen-Science Prize. The other one, called MUHAI Inequality Observatory, started in 2021, and is being developed within our consortium by a multidisciplinary team of researchers. Both projects focus on topics that are undoubtedly relevant to a wide audience of citizens and institutions, respectively, the impact of extreme weather events on social conflicts, and the understanding of the public perception of social inequalities and intersectionality.

Democracy wanes – access denied, our cause remains

For projects like the aforementioned ones, free access limitations to user-generated  data from social media platforms, such as Twitter, can produce detrimental effects that, besides damaging scientific research, can directly affect communities and their economies at multiple levels. By limiting the generation of externalities that can produce widespread societal benefits, such as:

• Transparency: Free access to social media data allows researchers working on the perception of topical issues, such as the impact of sea level rising on coastal communities or that of social inequality on life expectancy, to gain insights into the expressed views, expectations, preferences, reactions and feelings of the people being (actually or potentially) affected by phenomena that are shaping our collective present, and will likely determine our future, as well as that of other species. Social media data access is also crucial for understanding the dynamics that push some to express their opinions and others to remain silent on issues that may directly affect them, such as climate change and social justice. Moreover, acknowledging the relevance of a particular issue for a wide spectrum of the population can be an incentive (and pressure) for representatives and governance bodies to activate, taking into consideration people’s expressed needs and concerns in relation to these topical issues. More generally, denying or limiting social media data access to researchers can damage evidence-based policy-making, for example with respect to issues related to climate change and its effects on citizens’ daily life.
• Innovation: Open access to social media data fuels innovation in various sectors that build on top of academic and non-profit research, such as the AI industry. If researchers can’t access social media data they won't be able to develop new tools and models to analyze this data, which are widely employed, not only in academia. Furthermore, charging for data access could disproportionately damage and deter innovative research projects with unconventional or exploratory purposes related to social media and their data, which may not have guaranteed or clearly foreseeable outcomes or applications.
• Accountability of social media platforms and users: Social media data can be used to hold platforms accountable for their impact on society, including issues like mis- and dis-information, trust erosion, and algorithmic biases. Restricting access to researchers could make it harder to observe, assess and address these issues.
• Equal access and equal opportunities: Non-profit organizations and academic researchers with limited resources might struggle to afford social-media data access. This could further exacerbate knowledge and information inequalities, especially in poorer countries.
• Knowledge consistency: If different social media platforms adopt different access policies, data sources might become inconsistent, making it difficult for researchers to compare them, or to analyze their dynamics and evolution across time, by integrating their findings across multiple platforms.
• Civil society and community engagement: Access to social media data is crucial for NGOs, IGOs and other non-profit projects for understanding public discourse and socio-cultural movements. Without social media insight, civil society projects might struggle to engage citizens effectively in participatory deliberation or policy-making processes. Moreover, the ability to see how groups of citizens deliberate through social media is essential for facilitating understanding among groups. Lack of access to data could hinder efforts to bridge divides.
• Accountability of elected bodies and representatives: Social media conversations provide citizen’s feedback on policies and their perceived effects. It is therefore important for holding representatives, governance bodies and institutions accountable and reactive. For example, if data access is limited potential abuses may go more easily unnoticed.

As you may have understood, open and free access to social media data for academic, non-profit, and civil society projects is essential for maintaining transparency and ensuring accountability in democratic countries. For this reason we must protect it, and you should help us in doing so.

The opinions expressed in this article are those of the author. They do not reflect the opinions or views of anyone else.

Carlo R. M. A. Santagiustina, Venice International University.

-Continues from Part 1 

Having worked in the last decade on several EU research projects, like MUHAI, which  aim to study social phenomena, such as inequality perception, through user-generated data from social media, I feel the need to put you on alert. Social media, and the information they collect, are certainly formidable tools for studying and comprehending social phenomena, but, unfortunately, they are also formidable artifacts for attempting to influence and manipulate online crowds. Therefore, they should be considered critical (dual-use) technologies in relation to freedom of expression, but also in relation to social justice, peacekeeping and for the proper functioning of public institutions as well as of markets. This is especially relevant in liberal democracies that,  from this point of view, are particularly vulnerable on both sides.

Social Media platforms can be considered the contemporary and digital equivalent of the agoras in ancient Greece, and just as in ancient agoras some people discussed public affairs while others did private business, the same occurs nowadays in these platforms. Both usages are per-se legitimate. But differently from what occurred in the past, nowadays citizens, representatives,  institutions, and civil society have little, if no, control on how these private online agoras are employed, and on the scopes for which the data generated by citizens is sold to (and used by) third parties, for political, social or commercial exploitation. Social media companies are certainly required to comply with privacy and data protection regulation, which may partially protect users, as single individuals and organizations, but certainly this type of protection does not shield us as individuals belonging to, or identifying with groups or communities, based on life-styles, values, beliefs, preferences or any other form of shared identity or common behavior. Because, once user-data is aggregated and anonymized, it can be used for all sorts of purposes, from profiling, categorization and personalized advertising to political influencing campaigns targeting groups of users that are similar in terms of one or several aspects of their online behavior and identity. What is known about social media users at the aggregate level can hence be used to affect their views, preferences and behaviors. Since these platforms are populated by millions of citizens from dozens of countries, including totalitarian countries and façade democracies, many actors and organizations may have incentives to profit on top of their opacity, by using these platforms and their data for manipulating opinions, either of large and varied user populations, or, of carefully targeted influential audiences.

Virtual clashes roar, divergent views explore, data locked, voices sore.

Some social media platforms, like Twitter, have become for the aforementioned reasons virtual -but no less violent- battlegrounds. Where armies of users and bots driven by divergent interests and views, like pro-Russians and pro-Ukrainians, confront each other in harsh debates for setting the agenda of politicians and governance bodies or for influencing the opinions of consumers and citizens that use them. Other platforms, like Parler, stimulate by design the self-selection of aligned users and contents through homophily.  Shared preferences, expectations and views that are sufficiently widespread among their communities, shield community insiders and their partisan convictions from the risks and wonders of communicating with people with differing opinions. 

Unfortunately, in most cases, the growth of these private platforms and of the techniques for analyzing the rich and big data generated by their userbases, was followed by further restrictions for freely accessing the latter for non-profit research purposes. The recent diffusion and commercial success of some general-scope LLMs, which can also be trained using textual data from social media, has unfortunately exacerbated this process of user-data privatization and monetization. This is especially problematic for social science research and other non-profit projects, like AquaGranda: a Digital Community Memory, which, given their nature, do not (and cannot) aim to generate a profit from the usage of social-media data, and therefore have great difficulties in covering data access costs. This category is rather broad, and, besides researchers from academia and public research programs, it also includes researchers working for IGOs and NGOs, artists and activists, advocacy and volunteering groups and other non-profit organizations, who are also those whose activities and outcomes can possibly generate the greatest collective benefits and positive externalities for the public.

Short-circuiting non-profit research programs that study social, political, economic or socio-natural phenomena through the data generated by users on social media is easier than it may seem, and some platform managers and shareholders may want to do it simply to ask (data-) protection money also to academia, for continuing projects already underway.

Social media data policy changes are not simply due to privacy and data protection regulations and related concerns of the platforms that collect user-generated data. But rather, they often stem from the desire to further privatize and monetise the value of data contributed in exchange for no compensation by online communities, that is: your data. The data that you, your friends and your family, among others, generated by posting contents and commenting on each-others’ posts in the last two decades. Unfortunately, this process is not limited to Twitter, which recently withdrew free access to verified research projects, by deleting academic projects and their credentials from the Twitter developer portal without any notice. For example, also Reddit policies were recently changed, and this obliged the free PushShift archive to close its doors also to research projects. As strange as it may seem, Reddit moderators can still access the PushShift archive and its APIs but researchers from academia cannot. These are only two highly visible instances in a rapidly evolving social media landscape, which seems to want to make itself more and more profit-oriented and opaque.

At this point, a couple of concrete questions may be running through your mind:

• What is lost when social media platforms change their free data access policies to researchers and other non-profit organizations?
• What could occur if, because of the increasing data-access costs and constraints, civil society and citizen-science projects are denied the opportunity to see and comprehend how communities organize, debate and deliberate through social media?

(End of Chapter 2 – Find the answers in the next and last chapter)

Carlo R. M. A. Santagiustina, Venice International University.

From climate change to inequalities: Exploring the societal value of social-media for understanding public concerns and the perceptions of anthropogenic phenomena at a time when user data is increasingly monetized.

Web weaves lives, spheres interlace, online's intimate embrace reshapes human space.

In the last two decades, the Web has become a multilayered architecture for distributed communication that connects more than 5 billion human beings. Online communication is nowadays ubiquitous and permeates all spheres of our daily activities: from family relations to work activities; from hobbies to political propaganda; from volunteering to advocacy. Online communication is not simply answering a functional role related to people's insatiable need for information or gossip, it can trigger, permeate and shape intimate spheres of our life, producing profound and widespread transformations at the individual and collective level, shaping our identities, understandings, beliefs and preferences.

The diffusion of smartphones and of social media apps, has rendered communication increasingly multimodal and multimedial. Through our online posting activity, social, cultural, political and economic phenomena can get easily intertwined and percolate from physical to virtual spaces. For example, they can flow from news platforms to social media, to markets and back again to the physical world, affecting its abundances and famines.

On the Web, what might seem volatile as, for example, an opinion about a natural event, like drought, or about a market phenomenon, like inflation, once published online starts acquiring a socio-cultural texture and physiognomy, by occupying and diffusing in a virtual space and by progressively acquiring a set of roles in relation to the online ecosystem in which it is uploaded. These online ecosystems, despite being virtual, are culturalized, socialized, politicized and commoditized, similarly to other environments in which humans operate. That is, they are endowed with morphologies characterizing the aforementioned dimensions of human systems, in which contents and interactions are embedded. Therefore, online posts on social media go far beyond the bits used to store their digitized information and multimedia contents, and are embedded into complex social, cultural, political and economic contexts, to which they are linked and with which they are entangled.

Opinions, preferences, and expectations, as well as expectations of others’ opinions, preferences and expectations, can heavily impact human systems and their institutions, such as markets or governance bodies, affecting their functioning, stability and outcomes. For example, in a few hours, a short video posted on snapchat capturing a clash between police and activists protesting against the inaction of their government in relation to rising inequalities caused by extreme weather events, will be observed, probably in real time, by multiple journalists and bloggers around the globe, some of which will rapidly publish polarizing or clickbait online articles on the subject, which ‌could give rise to the outrage of a broader section of the population that will read them, which will probably express their reactions, feelings and expectations through another cascade of posts and comments on their preferred platforms, which, if not rapidly addressed by competent and reactive institutions, could produce broader social tensions, both in the digital and in the physical space, potentially creating a conflict spiral.

Private platforms gain, Twitter's X-changes reign, Blue checks cost, trust is lost, shifts in the digital domain.

 Most of the social media platforms, in which phenomena like the aforementioned spark and propagate, being private, aim to expand their profits and acquire new market quotas. Hence they operate for increasing actual and projected revenue streams. Recent changes of Twitter,  now also referred to as X, teach us that, in a few months, online platforms can drastically change in terms of business models, social interaction architectures, transparency, data access policies and search-engine algorithms. These changes may in turn affect the views, incentives to communicate and interaction patterns of their userbases. 

For example, among the many changes implemented by Twitter in recent months, one of the most significant is the decision to monetize the system of verified users, those with the blue checks in their profile, who now have also to pay a subscription for displaying this badge, which was previously a fair signal of the trustworthiness of Twitter sources, independently from users’ willingness and possibility to pay for it.

Among the many negative effects of this change, one, illustrated in this (https://www.washingtonpost.com/technology/2023/02/22/russian-propagandists-said-buy-twitter-blue-check-verifications) Washington Post’s article authored by Joseph Menn, is certainly a noteworthy example of the type of users that could be attracted by, and benefit the most from, these recent changes. The article’s title: “Russian propagandists are buying Twitter blue-check verifications”, speaks for itself.

Will this change, related to a new monetization strategy, destroy or bias the informative value of Twitter's blue check? If so, will the consumption of information generated by verified users change? and will Twitter users’ views be increasingly affected by “blue checked” partisan sources, which may want to afford the price of this subscription because of the benefits they derive from being labeled as verified sources? Is Twitter closing an eye, by allowing opaque profiles to display blue checks, only for a financial gain? Is this conflict of interest, due to Twitter’s incentive to maintain some opaque verified-user catchment areas, reshaping online debates in democratic countries?

Until mid-June 2023, with Twitter data freely accessible for academic research, a team of social scientists could have easily investigated these questions, to understand what is going on and to hence explain their findings to citizens, public institutions and civil society.  

(End of Chapter 1 – What effects had theTwitter/X changes? Discover them in Chapter 2!)

Folco Soffietti, Venice International University.

The recent discussion sparked around art prizes being assigned to artworks generated by AI - whether stated or only revealed afterwards - leaves space for interesting reflections.

Boris Eldagsen, Berlin-based photo & video artist, purposely entered the Sony world photography with an AI-generated work, in order to provoke a debate. Eventually he refused the Sony World Photography Awards 2023 prize considering it a flaw in the organization of the contest, since AI images and photos – he stated – should not compete with each other, and this, despite the jury having inquired beforehand if the photography had been co-created by AI. According to the jurors, it was important to acknowledge the need to address the role of AI in art creation and the broader consequences of this new practice(Grierson, 2023).

And indeed, the fact that the most recent release of Photoshop, the world leading software for photo retouching from Adobe, has embedded an AI generative tool able to modify and fill pictures, makes it urgent the need to understand and address AI in creative processes.

The world of photography is however not the only one affected, in our fast changing society: digital art can also be considered a traditional medium as proved by Jason Allen’s win at the Colorado State Fair’s digital arts competition with his work “Théâtre D’opéra Spatial”. Several digital illustrators were not happy with this result when it emerged that the AI platform ‘Midjourney’ was involved in the process, and this despite the claim that 80 hours were needed to complete the artwork.

The fair’s submission guidelines do not directly mention AI-generated art, but they define digital arts as “artistic practice that uses digital technology as part of the creative or presentation process”, hence AI could enter. Also, the judges said “they awarded the top prize based on the story that Théâtre D’opéra Spatial tells, as well as the spirit it invokes” and this would have been the result in spite of the media involved, as far as they are digital (Kuta, 2022).

The concern of professionals and early career artists are understandable (readers will remember the anti-AI posts that appeared consistently on Instagram accounts at the end of 2022). Contests are one of the first steps to enter the art market and gain recognition. AI implies a democratization of the possibility to create artworks in several styles, virtually allowing everyone to be an artist or, at least, a content creator. This also opens the Pandora's box of copyright infringement, an issue not-so easy to address, in which the law is generally still trying to bind the copyright to a human intellectual creative process, with the notable exception of UK as well explained by Borg et Al., 2023.

It will also be interesting to discover how the future art critics and citizens will look at this early age of AI art. Will the human author be remembered or will the AI process be remembered? Or, as it happened in the past, the AI and the authors will, to the non-expert eye, be fused in an art movement or period?

Coming back to art contests, in more practical and urgent terms what should be addressed is the assessment method of the contest itself. It is likely that AI art will be inscribed into dedicated categories, even though Visual Arts categories are usually extremely broad and could allocate AI artworks. Of course it remains difficult to evaluate the level of AI involvement: is it just for retouching, is it for improved resolution or light management, is it only part of the image, is it the starting point?

Regarding the latter case, one we might call “AI based”, what the assessment may still consider, at least in the current AI tools, is another specificity: the prompt.

The prompt contains the instructions and, as such, reveals the intention of the human creator, hence testifying the creativity and the consistency in the idea-result. The references employed would also be presented, helping verifying that no copyright infringement occurred. Moreover, the writing style can be another indicator of quality, for instance achieving a certain visual result with a poem, a rich lexicon, or an effective sentence.

The full implication of AI to the art work are only at their dawn, and if curiosity on how art history will evolve is high, also regarding what machines will do, there is currently the need to ensure that art contests are transparent, that traditional and AI-related entries have categories and opportunities to be assessed and awarded. Transparency of rules and clarity of indicators will help the current and future generation of artists to adapt and integrate, if they wish, AI in their artistic process, without interfering with traditional processes that will coexist along AI art.

References

Borg, W. P.-J., Podoprikhina, G., & Alexander, L. (2023, February 21). AI-Generated Art: Copyright Implications. Lexology. https://www.lexology.com/library/detail.aspx?g=fff221ab-66ce-4cbb-8f0d-687850888696

Kuta, S. (2022, Sept 06). Art Made With Artificial Intelligence Wins at State Fair. Smithsonian Magazine: https://www.smithsonianmag.com/smart-news/artificial-intelligence-art-wins-colorado-state-fair-180980703/

Grierson, J. (2023, Apr 17). Photographer admits prize-winning image was AI-generated. The Guardian: https://www.theguardian.com/technology/2023/apr/17/photographer-admits-prize-winning-image-was-ai-generated

Robert Porzel, University Bremen.

Trust is an important factor in human-centric artificial intelligence – especially for the success and effectiveness of a collaborative task in which the participants rely on each other to achieve specific sub-goals. For example, in household environments, such as a kitchen, mistakes can be made by either party that could not only lead to failure to complete the task, but even to injury through various hot or sharp appliances. Trust in a new system or technology is critical to its success, since people tend to employ systems that they trust, and reject systems that they do not trust.

In the last few years, artificial agents, such as vacuuming robots, have become more common in household environments, and assistants for more complex tasks as cooking or cleaning are being developed. To ensure that these new systems will be accepted, it is important to explore how much people trust an autonomous system to handle these tasks, how this trust changes during use and what factors lead to an increase or decrease in trust. Toward that goal, it is important to find applicable measures for trust. For this, we propose measuring a user's trust in an artificial collaborator during cooperative cooking tasks by analysing the tasks delegated to the artificial partner during collaborative execution of a recipe.

As the delegation of tasks among humans relies on trust, we propose that the tasks given to the artificial collaborator, e.g. while preparing a meal, can supply information on the level of trust the human has in them. If the human assigns intricate, dangerous or important tasks to the artificial agent, e.g. heating or cutting an ingredient, this would indicate, that they trust this partner to complete the task successfully. Should they only delegate minor tasks to the robot – for example, wiping the counter – it indicates, that the robotic partner is only trusted to fulfil simple tasks where errors could easily mitigated. Toward the goal of measuring trust based on task delegation, three different aspects of a task that could influence a human’s tendency to delegate it were chosen in our approach: difficulty, risk and possibility for error mitigation. In addition, it was deemed relevant if a human would supervise the artificial collaborator during a task or even intervene. In addition discount factors were considered that might convince a human to assign a task to a robot even though they do not completely trust the robot, e.g. tediousness of a task or inability to complete a task themselves. These aspects were combined into a basis for a scale, that can be used to determine the level of trust the human put into the artificial partner when delegating this specific task to them.

To observe humans during cooperative cooking with an artificial partner, a VR application was developed in the Unity game engine for use with an Oculus Quest HMD. In this application the user is placed in a kitchen environment together with a virtual robot. The user can interact with various objects in the kitchen by grabbing them with either their hands or the controllers and then complete various cooking tasks by moving them in appropriate ways -- e.g., moving a whisk in circular motions through a bowl containing the different ingredients to be mixed. In addition, the user can order the robot to fulfil any of the needed cooking tasks for recipe completion -- e.g., portioning a certain amount of an ingredient into a bowl -- or some supporting such as cleaning, tidying or fetching objects for the user. For these orders a delegation-type interface is used, where the user orders the robot to fulfil a task in a declarative manner, but is not required to give details on how the task should be completed.

In the future this metric could be part of a bigger set of measures for trust specific to cooperative tasks, that includes other aspects such as the phrasing of orders given to the robot. Similarly, it could be modified for further household tasks, that could in the future be assigned to household robots. Predictions made by a graphical model based on these metrics could also be used to adjust robot behavior at runtime to calibrate trust to the appropriate level for optimal cooperation. The described test environment and scale could be used in the future to explore different robot appearances and behaviors and how they affect trust, as well as trust development over time when the human can observe the robot complete tasks successfully or make errors.

Interview with Katrien Beuls, UNamur.

We are pleased to announce that MUHAI researcher Katrien Beuls is now Lecturer in Artificial Intelligence at the University of Namur (Belgium) and we are happy to welcome UNamur in the MUHAI consortium! Read her interview for the University of Namur's magazine "Omalius".

Katrien Beuls is undoubtedly a fine example of the growing number of women in STEM (science, technology, engineering, and mathematics) careers. After a rather literary career, guided by her curiosity, she began studying computer science and became interested in computational methods for processing human language with the help of Artificial Intelligence (AI).

After an atypical curriculum, her meeting with Luc Steels (VUB), a pioneer of artificial intelligence in Europe and an expert in evolutionary computational linguistics, and an international experience as a student and post-doctoral researcher, Katrien Beuls has just moved to UNamur.

"A position as 'Lecturer in Artificial Intelligence' was vacant. I applied and here I am! I found it interesting that the job description did not restrict the function to a specific sub-field of AI. I find UNamur extraordinary, because of the human size of the institution, which allows for a lot of interaction quickly, as you get to know everyone quickly! I also had the impression that there was a certain open-mindedness in the Faculty of Computer Science and that they were interested in new methodologies. I really feel that UNamur is an excellent place to create my own interdisciplinary research team.

A passionate researcher
"I am trying to build intelligent systems, so-called cognitive agents that implement the 'perceive', 'reason', 'act' cycle that constitutes the decision process.  These intelligent agents have to solve communication tasks in teams.  The aim is to build a common language to refer to a situation, objects, etc. This is done in the form of a game according to an established protocol.  An agent chooses a topic of conversation and has to ask himself the question: what do I have to say so that the other knows what I am talking about?  They have perceptual abilities but see the world or segments of it without labels and have no language system.

This interview was conducted for the "Expert" section of the University of Namur's Omalius magazine #27 (December 2022).

Photo of Katrien Beuls - Credits: © Christophe Danaux

© UNamur

Anna Morbiato, VIU.

Imagine you're at a house party. You've just met someone new and you want to make a good impression. So, you ask them "How's it going?". To a machine, this phrase might seem like a simple question, but to a human it has a whole range of meanings. It could be a polite greeting, a genuine inquiry about how the person is doing, or even a subtle way of asking them to leave. The machine might not be able to pick up on the nuances of the conversation, but a human would be able to decipher the true meaning behind the words. That's pragmatics in action!

Pragmatics is the study of how we use language in context. It looks at how the meaning of words and expressions may change depending on the situation and the people involved. It also considers how our intentions and our social relationships affect the way we communicate. It is like a party trick - it's the art of knowing how to communicate the right thing at the right time!

Pragmatics is the focus of the paper Pragmatics of Narration with Language, which is part of the MUHAI white paper Foundations for Meaning and Understanding in Human-centric AI. Specifically, the paper looks at how research in pragmatics and cognition over the past 50 years has helped us to better understand natural languages, narratives, and language processing. The paper explores how factors such as the speaker's communicative aims, perceptual experiences, stance/viewpoint, as well as cultural and social aspects of communication, can affect how humans use language.

One of the aspects the paper looks at is coreference disambiguation, which is essential for parsers and machine learning when it comes to understanding natural language. Imagine you're making pizza with a friend; she is kneading the dough and you want to check on it. So, you ask "How's it going?". To a machine, this phrase might seem like a simple question, but if the parser incorrectly disambiguates the pronoun "it" as referring to the friend, instead of the dish, it could lead to confusion. The machine might interpret the question as asking about the friend's wellbeing, instead of the progress of the dish. Coreference disambiguation is the process of determining which words or phrases in a sentence refer to the same entity. This helps the parser to accurately interpret the sentence and understand the intended meaning.

Another both interesting and challenging realm is that of speech acts. Speech acts are the ways in which we use language to express our intentions and convey subtle or implicit meaning, such as orders, suggestions, promises, and requests, as well as a variety of other communicative goals. Getting back to your cooking session with your friend, imagine your pizza is baking in the oven, and your friend wants to check on it. So, she suggests: "Maybe it's time to take it out now?". To a machine, this phrase might seem like a simple suggestion, but you would instead be able to detect whether it is a nice way of hinting that the pizza is burning (and you should have already turned the oven off). Pragmatics is like a chef's special ingredient - it's the art of knowing how to season a dish just right so it tastes delicious!

That's why pragmatics is so important, both for humans and for parsers: because it helps to make sense of the context of conversations and interactions. The goal is the development of deep learning models that can understand natural language also taking into account these nuances of human interaction. This technology is still in its infancy, but it shows great promise for improving the accuracy of machine learning models in interpreting natural language.

Credits
Intro image - photo by Andrea Piacquadio via Pexels