Creating with an AI

What is AI art? Is AI as creative as humans? Will AI replace artists in contemporary societies? Philosopher and researcher Raphaël Millière, Presidential Scholar in Society and Neuroscience at Columbia University, defines the terms of the debate: “AI art is generative, meaning that it is created using, at least in part, autonomous processes that do not centrally involve human input.”  

Most algorithms used by AI artists are text-to-image generators such as Stable Diffusion, DALL-E, or Midjourney: they take text prompts as input and produce images as outputs. They belong to a class of algorithms called “artificial neural networks” (ANNs). An image generation ANN can learn from millions of images paired with their captions to generate images based on text prompts. The choice of the data used to train an ANN is, therefore, of prime importance, as it is reflected in the kind of outputs it will produce explains Millière: “Training an image generation ANN on early Italian Renaissance portraits might enable it to generate images that closely match the specific style of such portraits, but not any other style. Consequently, some AI artists like to train their own algorithms by curating a custom dataset on which to train them. This curatorial activity has much in common with museum curation.”  

“The Crow,” made by AI artist Glenn Marshall using OpenAI’s CLIP, won Cannes Short Film Festival in 2022

Another family of algorithms called “language models” can be used to generate literary works, such as GPT-3. GPT-3 is a Natural Language Processing (NLP) tool created using deep learning. Every open-source text available online has been entered into the GPT-3 algorithm, enabling it to analyze which letters, words, and phrases are the most likely to come after one another.  

For those who use text-to-image generative algorithms or language models algorithms, the choice of text prompts to generate the desired images or texts is crucial and is quickly becoming a new king of artistic craft. “The process of artistic creation is also changing in that many AI algorithms take text prompts as input. This means that visual artists can now craft their art with words, much like poets. This new bridge between linguistic and visual forms of artistic expression has fascinating implications for the future of art” explains Millière. 

AI artist Pierre Zandrowicz, co-founder of Atlas V Studio, uses the text-to-image platforms Midjourney and DALL-E 2 for his own creative practice: “Midjourney allows me to illustrate ideas, test visual concepts and even create a form of intellectual stimulation. It’s like a photographer taking shots in their imagination or dreams. I also use DALL-E 2 – OpenAI to create more unusual or bizarre things. It also creates images from text, but it stylizes much less and is less precise, so you often get unexpected results.”  

Algorithms are unlike any other artistic tools explains Millière: “Prompts are rather vague description that do not specify every characteristic of the image to be produced. Suppose that you prompt an algorithm to generate a picture with the input ‘A cat sitting on a mat,’ the algorithm will have to fill in the details of the color of the cat, the shape of the mat, the angle of the framing, and can do so in surprisingly creative ways. Moreover, running the algorithm several times on the very same prompt will produce different results. Consequently, some AI artists like to think of AI algorithms not just as tools, but as collaborators of sorts.”  

Collaborating with an AI can help think through creative ideas explains Zandrowicz: “I use ChatGPT to converse with an AI system. I often work alone, so it’s an opportunity to exchange ideas and create a dialogue. There is something like a dialectic.” For Ribal Rayess, an interdisciplinary artistic and creative director who merges sound and visual storytelling, AI enables artists to foster artistic production at unprecedent levels: “I have spent hours with Midjourney to see what’s possible. I am very good with my pencil skills, but what you can do with these tools in under an hour would perhaps take me a couple months to illustrate.”  

AI is increasingly becoming a companion of the artist. But is AI creative? “There is a sense in which current AI algorithms are not ‘creative’ at all” explains Millière. “Indeed, they lack intentions, and therefore lack the capacity to intentionally fulfill a particular artistic vision. Human artists, by contrast, generally have a goal or a general direction. When an artist uses such algorithms for artistic purposes, they impose their own intentions onto that process. AI art is, first and foremost, human art – it is made by humans, for humans, using AI as a means.”  

For Zandrowicz, AI democratizes creativity, but not creation: “We can think faster and work faster since we can get results sometimes just from intuition. But you will always need cultural background, history and purpose to invent something with these tools.” Indeed, the relationship AI artists have with algorithms is not that different from other artistic tools, such as cameras or pencils. Using algorithms for artistic purposes requires talent and skills.  

Digital and analogue must work hand-in-hand explains Rayess: “I believe that art is a feedback loop that starts in the physical world. We need all three kinds of people, the purist analogue artists who work with their hands, the digital explorers who bring life to art in the digital world, and the alchemists who constantly merge digital and analogue to create a derivative.” Ultimately, for Rayess, collaboration with AI expands the realm of the machine, and foster creative collaborations with humanity: “I consider AI to be an extension of my mind and the natural world around me, as it was made and trained by sentient beings on humanity’s work.”  

Indeed, because algorithms have been given an exhaustive corpus of human knowledge as an input, they are a mirror unto human conscience and human history. AI could be paradoxically a powerful tool for exploring our humanity. This is how curator Julia Kaganskiy apprehends AI art, as a creative tool as much as a reflexive one: “To me, AI art is not just about making beautiful or impressive visuals, text, or audio. AI is an emerging technology that uses vast troves of data to learn about the world and reflect it back to us, in a way that can often reveal things that we’ve come to take for granted, or obscure things behind the veil of technological neutrality. It also raises fascinating and complex philosophical questions about the nature of cognition, intelligence, consciousness, and agency.”  

For Ana Brzezińska, curator at Tribeca Immersive, considerations of the medium and the message are of equal importance when it comes to curate AI art: “The quality of the dialogue that creators introduce and the type of questions their artworks raise are paramount in the way we approach our programming. What we’re interested in, as curators, is to understand the relationship between the toolkit, the artist’s intention or mission, and the final result.”  

“AI art is, first and foremost, human art – it is made by humans, for humans, using AI as a means.” Raphaël Millière

An increasing number of artists engage critically with their own practice through the medium of AI explains Maurice Jones, curator and producer at MUTEK.JP: “Artists that critically engage with questions surrounding AI, such as Sougwen Chung, Nao Tokui, Hexorcismos, Portrait XO, Chris Salter, Lucas LaRochelle, Takashi Ikegami, and others do so from a variety of perspectives that go far beyond the rise in AI-generated visual arts.” Technological tools are becoming a mode of inquiry, questioning our relationship with machines and beyond.  

Such creative use of AI reflects the role of the artists as agents of change and explorers of the future explains Fabien Siouffi, founder of Fabbula, a studio specialized in the production of virtual experiences: “AI helps understand better the role of artists — they are risky explorers, researchers of the yet unknown, trusted advisors for emancipation.” Collaborations between artists and AI can insert itself into curatorial choices, highlighting creative processes as much as artistic outcomes: “At Fabbula, we are specifically interested in participative forms, where artists allow their creative processes to be played by others, materializing the many versions and emergent qualities of an artwork, as it gets perceived specifically by diverse people. For this reason, artists wish for exhibitions to let visitors play with the creative processes, as one would do in video game worlds.”  

Ultimately, the development of AI into the realm of art blurs the boundaries between curation and creation, as Millière recently argued in an article for Wired: “The creation of custom datasets to train an AI algorithm for artistic purposes is a curatorial gesture: it involves the selection of images that share artistically desirable characteristics. There is also a more familiar curatorial process involved in selecting outputs the artist deems particularly significant. Like photographers often do, AI artist must carefully select among dozens, hundreds, or thousands of outputs the ones they confer artistic status to. Thus, curation is involved at multiple steps of the creative process in AI art.” 

AI is also transforming curation itself explains Kaganskiy: “One could already imagine ChaptGPT writing curatorial statements and artwork descriptions. One could also imagine AI-curated art exhibitions, and I’m sure these already exist. In fact, the 2021 Liverpool Biennial was reportedly curated by AI.”  

“I believe that art is a feedback loop that starts in the physical world. We need all three kinds of people, the purist analogue artists who work with their hands, the digital explorers who bring life to art in the digital world, and the alchemists who constantly merge digital and analogue to create a derivative.” Ribal Rayess

The AI revolution touches all traditional artistic genres, from music, to literature, and filmmaking. AI enables artists to test and create faster, at lower cost. AI lowers the barrier to entry for people who may lack more traditional artistic skills. “To some extent, a similar evolution happened with the invention of photography” explains Millière. “It enabled individuals who were not trained as painters to capture images. AI art could be seen as a further step in this evolution: it is now possible to create images of any kind with AI algorithms.”  

In film, using AI enables creators to test and create images using algorithms, redefining how one can conceive, think, and make a film.  

Zandrowicz is currently producing a film using various AI tools: “I am preparing the shooting of Rhizome, an interactive film distributed on the blockchain, that I co-directed with Ferdinand Dervieux. We will use Stable Diffusion to create visual incidents in the film’s narration. We are using Midjourney to cut the film and create frames, a bit like a storyboard. We are testing these tools not only for the content of the film but also for a different approach to production. AI tools allow for us to work with a small team, be more agile, and work very closely with our subject.”  

Zandrowicz also uses AI to achieve specific aesthetic goals: “Currently, I am working on a short film In Search of Time co-directed with Matthew Tierney, which explores the imagination. It is a mix of traditional filming techniques and then treated those shots with an AI system to achieve an animated and ‘drawn’ style.”  

One may also use AI to create images of movies that don’t exist (yet): @ cinem.ai Instagram account curated by Zandrowicz explores the realm of cinematic imaginaries, posting frames that are yet to have a filmic existence.  

Because AI evolves and transforms rapidly, using AI tools to create enables artists to explore new worlds with an evolving, and sometimes interdisciplinary medium. The malleable nature of technology creates new possibilities to blend the audio and the visual, reflecting new experiences and realms of thought in various possible formats. Sonic Mirage, an audiovisual collaborative artwork by Rayess, in collaboration with touch designer Dalal Aljassim, is a prime example: “Sonic Mirage is a collection of beautiful landscapes dreamed up on Midjourney, accompanied by hypnotic soundscapes. It is an immersive experience focused on the power of meditation, and accessible through a hybrid format: it will be exhibited as a projection accompanied by the soundscapes, and later on transformed to virtual landscapes that can be experienced in a VR environment.”  

Artists also increasingly use algorithms to create audio tracks with similar techniques. For Jones, the intersection of music and AI is a crucible of innovation: “The most exciting field, in my personal opinion, is in the development of creative AI tools for sound and music. When we hosted our first AI Music Lab at MUTEK at the end of 2019, the state of available models was very rudimentary. This changed in the last year with models such as RAVE (Realtime Audio Variational autoEnconder), Semilla (by Hexorcismos) and Riffusion, able to deliver high-quality audio outputs. RAVE and Semilla can be applied in real time in any Digital Audio Workstation (DAW), which is an absolute gamechanger. Artists such as Nao Tokui, Hexorcismos, Debashis Sinha, and others are really pushing the boundaries here.”  

International festivals have a role to play in fostering innovation, interdisciplinary, and cross-cultural practices in the years to come explains Jones: “The main contribution that a global festival, such as MUTEK, can bring to the conversation of AI is the diversification of interdisciplinary and cross-cultural perspectives on the technological development, as well as building transnational networks of curators, artists, and even audiences.” 

“AI is an emerging technology that uses vast troves of data to learn about the world and reflect it back to us” Julia Kaganskiy

What is the future of AI art? Will artists and AI increasingly work side by side? For Millière, AI art will inevitably become mainstream. In fact, one could argue that it already is, at least in some circles: “Large auction houses such as Sotheby’s and Christie’s have been selling AI art for years, at high prices. The recent wave of image generation algorithms accessible to the broader public, such as DALL-E 2 and Stable Diffusion, has also greatly contributed to the democratization of AI art. These days, practically anyone can generate images with AI for free on their computers, without extensive technical skills. I suspect we will see many new artists using these tools within the next decade.”  

Pierre Zandrowicz learned AI tools by watching tutorials on YouTube, talking to people on dedicated Discords, and attending workshops in New York: “This is a niche field, but it’s growing rapidly. There is an investigative side to this at the current stage. To understand, you have to search, research, test, and so on.”  

Making AI more accessible to creators and audiences alike is one of the challenges that the technology poses. For the digital artist and curator Alex Czetwertynski, exhibitions have a role to play to democratize AI art for the public: “Most people are not aware of the existence of AI art, and if they are, their understanding is extremely cursory. Exhibitions would need to be designed in such a way that they make the overall issues related to AI art very clear.” For Kaganskiy, exhibitions should highlight the ethical implications of AI technologies: “For me, a good exhibition will not just show the range of creative applications of AI, but help us consider some of the broader implications of this technology and give people the tools to be able to form more informed opinions about AI when they encounter it out in the world.” 

However, AI tools are not free from biases and pitfalls. AI brings ethical challenges for creation and content production. “One ethical question posed by AI art regards fair use and plagiarism” explains Millière. “Image generation algorithms are generally trained on giant datasets containing millions of images including many artworks made by traditional artists, some of which are living artists. From a legal perspective, this is still a grey area that is generally taken to fall under the category of ‘fair use.’ But many artists are worried about their artistic style being plagiarized by algorithms trained on their works.” 

Another ethical question relates to bias in datasets. While exploring the Web and learning as much information as possible, algorithms have also absorbed more problematic content of misinformation, incitement to violence, xenophobic speech. Millière warns about this pitfall: “A recent study found that one of the main datasets used to train text-to-image generation algorithms contained many racist and sexist stereotypes. These stereotypes are likely to be reproduced in images generated by algorithms trained on this dataset. This is one of the reasons why it is important for artists not to use these tools unreflexively. The nature of AI algorithms can have a darker side, which artists ought to consider when they use these tools.” Algorithms remain an honest mirror of humanity and all that it comprises. 

A final challenge pertains to the status of the artist in the years to come: will AI replace artists in the future? Not for Zandrowicz: “In the future, I see the artist’s role as someone who sees AI as a tool. The camera did not replace painting. However, it revealed a new form of art. AI allows the same thing. It will allow a new generation of artists to emerge, just as YouTube, the iPhone, or more recently, the blockchain did.” And to conclude: “What makes us artists is our point of view, ideas, and the ability to evoke some form of emotion in the viewer — that no one or any program can do for us.”