Luc Steels, Venice International University.

The AI Avant-Garde in the 1990s.

In the 1980s and 1990s a new wind was blowing in many sciences, based on the realization that various complex phenomena in Nature can be understood by viewing them as self-organizing dynamical systems. Typical examples are tornados, meandering rivers, paths formed by ant colonies, chemical oscillations, patterns on a zebra skin, honeycombs in beehives, coordinated activities in neural networks in the brain, crashes in the stock market, and many more. A new science of complex systems was born coinciding with profound mathematical developments in fractal geometry, chaos theory and the study of large-scale networks.

But how to archive, register and share this revolutionary moment?

A pop up exhibition was conceived to shows videos of interviews, talks, and panels with the scientists who developed these original ideas and carried out some of the key experiments. They included Nobel laureates Christian de Duve, Manfred Eigen, Roger Penrose and Ilya Prigogine, as well as the mathematicians that founded chaos theory (Benoit Mandelbrot, Daniel Ruelle, Floris Takens, a.o.) and leaders in the application of self-organisation in physics, chemistry, biology, neuroscience, and economics.

The exhibition also displayed books and experimental objects, as well as art works from the ‘Wunderkammer’ of artist Anne Marie Maes. A second theme of the exhibition was how researchers at the forefront of Artificial Intelligence and Artificial Life were swept up by these ideas, leading to a renaissance of neural network research and highly dynamic, embodied robotic agents inspired by nature, called animats. To illustrate this research line, the exhibition included various robots and a large number of videoclips on experiments carried out in the 90s to explore how intelligence can emerge from situated interaction.

To illustrate some of these investigations, such as on the solar system, the network structures of the brain, or the collective behavior of insects, the exhibition was expanded with top pieces from the collections of the Royal Library, including the first edition of Newton's Principia, the observations of Copernicus, the writings of Laplace and Poincare on the predictability of celestial movements, the precious drawings of plants of 17th century botanists like Rembert Dodoens, Jan-Baptista van Helmont, and more.

Pop-up exhibition at the Royal Library (KBR)

https://www.kbr.be/en/

Kunstberg - Mont des Arts 28, 1000 Brussel

Opening: 18 December 2023 2pm

Closing: 22 December 2024 3 pm

Curator: Luc Steels

Folco Soffietti, Venice International University.

The recent discussion sparked around art prizes being assigned to artworks generated by AI - whether stated or only revealed afterwards - leaves space for interesting reflections.

Boris Eldagsen, Berlin-based photo & video artist, purposely entered the Sony world photography with an AI-generated work, in order to provoke a debate. Eventually he refused the Sony World Photography Awards 2023 prize considering it a flaw in the organization of the contest, since AI images and photos – he stated – should not compete with each other, and this, despite the jury having inquired beforehand if the photography had been co-created by AI. According to the jurors, it was important to acknowledge the need to address the role of AI in art creation and the broader consequences of this new practice(Grierson, 2023).

And indeed, the fact that the most recent release of Photoshop, the world leading software for photo retouching from Adobe, has embedded an AI generative tool able to modify and fill pictures, makes it urgent the need to understand and address AI in creative processes.

The world of photography is however not the only one affected, in our fast changing society: digital art can also be considered a traditional medium as proved by Jason Allen’s win at the Colorado State Fair’s digital arts competition with his work “Théâtre D’opéra Spatial”. Several digital illustrators were not happy with this result when it emerged that the AI platform ‘Midjourney’ was involved in the process, and this despite the claim that 80 hours were needed to complete the artwork.

The fair’s submission guidelines do not directly mention AI-generated art, but they define digital arts as “artistic practice that uses digital technology as part of the creative or presentation process”, hence AI could enter. Also, the judges said “they awarded the top prize based on the story that Théâtre D’opéra Spatial tells, as well as the spirit it invokes” and this would have been the result in spite of the media involved, as far as they are digital (Kuta, 2022).

The concern of professionals and early career artists are understandable (readers will remember the anti-AI posts that appeared consistently on Instagram accounts at the end of 2022). Contests are one of the first steps to enter the art market and gain recognition. AI implies a democratization of the possibility to create artworks in several styles, virtually allowing everyone to be an artist or, at least, a content creator. This also opens the Pandora's box of copyright infringement, an issue not-so easy to address, in which the law is generally still trying to bind the copyright to a human intellectual creative process, with the notable exception of UK as well explained by Borg et Al., 2023.

It will also be interesting to discover how the future art critics and citizens will look at this early age of AI art. Will the human author be remembered or will the AI process be remembered? Or, as it happened in the past, the AI and the authors will, to the non-expert eye, be fused in an art movement or period?

Coming back to art contests, in more practical and urgent terms what should be addressed is the assessment method of the contest itself. It is likely that AI art will be inscribed into dedicated categories, even though Visual Arts categories are usually extremely broad and could allocate AI artworks. Of course it remains difficult to evaluate the level of AI involvement: is it just for retouching, is it for improved resolution or light management, is it only part of the image, is it the starting point?

Regarding the latter case, one we might call “AI based”, what the assessment may still consider, at least in the current AI tools, is another specificity: the prompt.

The prompt contains the instructions and, as such, reveals the intention of the human creator, hence testifying the creativity and the consistency in the idea-result. The references employed would also be presented, helping verifying that no copyright infringement occurred. Moreover, the writing style can be another indicator of quality, for instance achieving a certain visual result with a poem, a rich lexicon, or an effective sentence.

The full implication of AI to the art work are only at their dawn, and if curiosity on how art history will evolve is high, also regarding what machines will do, there is currently the need to ensure that art contests are transparent, that traditional and AI-related entries have categories and opportunities to be assessed and awarded. Transparency of rules and clarity of indicators will help the current and future generation of artists to adapt and integrate, if they wish, AI in their artistic process, without interfering with traditional processes that will coexist along AI art.

References

Borg, W. P.-J., Podoprikhina, G., & Alexander, L. (2023, February 21). AI-Generated Art: Copyright Implications. Lexology. https://www.lexology.com/library/detail.aspx?g=fff221ab-66ce-4cbb-8f0d-687850888696

Kuta, S. (2022, Sept 06). Art Made With Artificial Intelligence Wins at State Fair. Smithsonian Magazine: https://www.smithsonianmag.com/smart-news/artificial-intelligence-art-wins-colorado-state-fair-180980703/

Grierson, J. (2023, Apr 17). Photographer admits prize-winning image was AI-generated. The Guardian: https://www.theguardian.com/technology/2023/apr/17/photographer-admits-prize-winning-image-was-ai-generated

Within the framework of "Aqua Granda, a digital community memory" initiative, launched in November 2020 by the EU H2020 ODYCCEUS project and Science Gallery Venice, among the main outputs is a book, available in open access at the following link:  https://bit.ly/3bVUJ8n

The book contains background on the historical roots of digital community memories and on today’s role of social media and describes the meteorological phenomena that give rise to big floods in Venice and their impact on the architecture of the city. It details how a digital community memory has been set up about the Aqua Granda floods in Venice and documents the exhibition Navigating Aqua Granda, a digital community memory in which a number of scientists and artists show how they have explored this digital community memory to help creIate memorials for these devastating events.

MUHAI was cooperating partner in the organization of the workshop "Archives and AI: Coping with Climate Change", event held in occasion of the launch of the exhibition entitled "Navigating Aqua Granda, a digital community memory".

AquaGranda is also virtual exhibition app for iOS and Android  

Luc Steels.

AI (Artificial Intelligence) researchers try to understand the structures and processes that underlie intelligence and use that insight to build practical applications. Much has already been achieved. But much remains to be discovered. The way to make these discoveries is to explore domains where human intelligence reaches its most exceptional manifestations.  The creation and interpretation of artworks is certainly one of them.

Recently I had the opportunity, facilitated by the Brussels cultural institution BOZAR and the Gluon Foundation, to collaborate with the renowned contemporary Flemish painter Luc Tuymans as part of the STARTS program of the EU that promotes interaction between art and science. I took this opportunity to advance the MUHAI research agenda and help my AI colleagues open their eyes to the arts. But I also wanted to give people from the arts and the wider public insight into what AI stands for and what it can and cannot do. This is necessary because, at least in my view, AI is now overhyped, with unrealistic expectations and misunderstanding of the complexity and enormous challenges that we still face to build artificial systems that approach human intelligence. 

Other scientists investigating art, such as chemists or physicists, put a work of art on their dissection table and use the measuring instruments of their field to understand the material aspects of a work and thus solve puzzles about provenance and painting technique. In contrast, AI is more comparable to psychology. It is interested in the cognitive processes that take place when viewing, experiencing and interpreting a work of art on the one hand and the conception and creation of a work of art on the other. Psychologists do this through observations and empirical experiments. AI researchers make models. 

AI models do not simulate exactly how a person does it. They are functional models, models that yield the same result as a specific human cognitive capacity but often in a very different way. For example, they may focus on recognizing that a painting represents a face, or that this face is depicted as being angular, that the color palette used in a painting emphasizes greyish colors, or that the person depicted in the painting is holding his lips tightly together. AI is artificial intelligence the way an airplane is an artificial flying machine. An airplane can really fly in the physical world and lift weight but it doesn't do it by flapping its wings like a bird. The validity of a psychological theory is determined by how well it covers the collected data and makes predictions about new data. The validity of a functional AI model is tested by making a computer implementation, possibly embedded in a robotic body, which demonstrably achieves the function being examined.  

Many people entertain the idea that AI is a monolithic physical force, like electricity or gravity, but that is not the case. AI is about algorithms that must work together to realize even basic cognitive functions. We already know thousands of these algorithms and many more remain to be discovered. An AI algorithm usually needs a knowledge base that it consults to achieve its function. For example, if an algorithm wants to recognize faces, it must of course first know the distinctive features of each face. These knowledge bases have to be created by an AI algorithm as well (this is what we call 'learning' in AI parlance) because it is often impossible to collect, disassemble and formalize all the necessary knowledge by hand. For example, an algorithm learning to recognize the parts of a face will first be presented with millions of faces, with annotations where the parts of each face are located. Based on these data, the algorithm then gets to work creating a knowledge base that can then be used to process new faces. 

Many AI algorithms used today in the context of art are about the form of a work and its style. For example, it is popular to show all the paintings of a particular artist, such as Claude Monet or Vincent van Gogh, to a learning algorithm, so that it can build a statistical style model and to then create a new work in the style of the artist. The results achieved using this approach are astonishing because the results are very similar to an original work by the same artist, particularly when neural network techniques known as Deep Learning algorithms are used. But note that these experiments do not go into the meaning of a work, for example why a particular figure is depicted, why certain colors have been chosen or why the face has been strongly distorted. Deep learning and other machine learning algorithms have no idea what the painting represents. The ability to cope with meaning is currently the glass ceiling of AI, the barrier that distinguishes human from machine intelligence.

And it is precisely the goal of the MUHAI project to break through this barrier. 

In this spirit I tried out a completely different approach on art and AI, focusing on the question how a work of art evokes and expresses meanings. This meant that I had to go in two steps: First find AI algorithms that can discover visual features, for example that a face is depicted as being more angular than the original. And then look for AI algorithms that can figure out the meaning of these features, for example by calling on the immense semantic networks that have been developed in symbolic AI in order to develop search engines or language applications. 

Early on I decided to focus on the solo exhibition 'La Pelle' by Luc Tuymans in the Palazzo Grassi in Venice, which was on view from the summer of 2019 to the winter of 2020. Together with several collaborators (in particular Bjorn Wahle and Sinem Aslan) I started to apply all kinds of AI algorithms to the entire collection of the 80 works in the exhibition. Via Studio Luc Tuymans I had access to the original source material and I was able to have a number of conversations with Luc Tuymans himself. These conversations, as well as Tuymans' clear writings, shed a clearer light on his artistic method. An important element of that method is that Tuymans almost always works with existing source material. So it is relevant, and very enlightening, to compare a painting with its source. In this way the transformations that the artist has made come to the fore and that in turn helps us to investigate why the artist chose these specific transformations, in other words which transformations are signifiers, i.e. carriers of meaning. 

After some thorough preliminary studies, I concentrated on one work, Secrets, and built an exhibition about it that is on view at BOZAR from 3 April to 3 May 2021. The exhibition shows the two stages in the interpretation of a work of art: the visual and the conceptual. The visual processing starts from the image and tries to find areas of focus that the human eye naturally tends to, as well as edges, lines, shapes, geometric figures, colored areas, contrasting zones, and more (see Figure 1). Such an analysis yields points and areas in the image that pattern recognition algorithms can then try to relate to known objects, such as parts of a face, or to their features, such as that the eyebrows are thick or the nose sharp. 

 Figure 1. Some examples of image processing applied to the work Secrets. From left to right: (i) The painting itself, (ii) the focal region according to the MSI algorithm, (iii) the focal region according to the Spectral-residue algorithm, (iv) lines found by the TIN algorithm, (v) comparison of the lines between the painting and the original image. 

The conceptual processing next tries to derive factual information and emotional associations from what is depicted and consults knowledge graphs, dictionaries, thesauri, ontologies and other semantic resources on the internet to put the painting into its context: the historical context, the cultural symbolism and the broader context of the exhibition 'La Pelle 'in the Palazzo Grassi. The conceptual processing also uses as input the title of the work (Secrets) and the text of the catalog, which contains keys for searching the web. As the interpretation process unfolds, all these AI algorithms together build a very large 'narrative network' that acts as a coherent description of the painting from multiple points of view. This network opens our eyes to the richness and semantic complexity of the painting (Figure 2).

Figure 2. Fragment of the narrative network built for Secrets. The network contains associations based on image processing but also historical references, biographical data, references to the exhibition as a whole, etc. 

Has this interaction between art and science been successful? I think so. Most scientists are not used to experiencing art in a profound way and are amazed by the multitude of meanings and layers of processing that dedicated viewers recognize. But also for people from the art sector, including Luc Tuymans himself, this experiment is revealing because it illustrates the nature and complexity of AI and what associations and insights machines can already gather today about an artwork. At the same time, this experiment illustrates that the current state of AI is miles away from how humans perceive and interpret art. Claims, such as by Ray Kurzweil, that AI is going to outdo humans in the coming decades sound ludicrous when you examine the tremendous richness of the human mind as manifested in the arts through the 'cognitive microscope' offered by AI algorithms. 

References
Steels, L. and B. Wahle (2020) Perceiving the Focal Point of a Painting with AI: Case Studies on works of Luc Tuymans. Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, 895-901, Scite Press, Setubal Portugal. 

Sinem, A. and L. Steels (2021) Identifying centres of interest in paintings using alignment and edge detection. Case studies on works by Luc Tuymans. In: International Workshop on Fine Art Pattern Extraction and Recognition (FAPER 2020). Proceedings of the International Conference on Pattern Recognition (ICPR) Part III. LNCS 12663. Springer Verlag, Berlin.

[A Dutch version of this note is published in: Monard, E. (ed) (2021) Kunst en wetenschap in symbiose. Viewpoints of the Royal Flemish Academy for Sciences and Arts of Belgium, Brussels.]