Discovering first person video games in my very early teenage years was both a thrill and a disappointment. I felt let down by a promise that seemed built-in to the potential of the game, something that was dangled as a carrot but never delivered : true and complete interactivity in a digital realm. In other words, I should be able to create actions and elicit reactions that were not scripted, I should be able to dig a hole in the ground and not just walk straight, left, right and jump.
I knew I had been deceived when I started banging my head where there should have been a path in a forest, or a dark corridor that was just a flat backdrop. But why ? Could I blame the game developers for not having succeeded in creating a replica of the seemingly infinite actions one can accomplish in the real world? If anybody, I should blame the marketers for selling the games as a chance to "fully explore" a new world.
The gnawing feeling was however less about being cheated, it was about being controlled. Playing the game(s), I was placed in a world so heavily scripted that every possibility and impossibility had been created for me by a group of people who then hid this structure under attributes of "freedom" or "total control by the user". I was in someone else's construct, and as much as there was a pleasure in exploring new worlds, there was a disappointment in the engineering of those worlds.
The topic of this chain of thoughts is less about video games than it is about what is today dubbed "interactive art". I do need to diligently define what that represents, but for now let's call it any art that requires the viewer to enter into a relationship with the artwork that is not purely mental (as in a piece of conceptual art that would trigger a series of thoughts) or sensorial (as in music). Physical intervention, movements, tactile influence, the awareness of embodied presence all become key elements to the success and function of the work. This is not so much the duchampian idea that spectators create the work, it is about artwork not existing at all without audience's willingness to participate in it.
This type of art isn't new, kinetic artists had required audiences to trigger their installations as early as the 1950's and 60's. But given the amount of technology available to artists today, you would expect interactive art to offer a flourish of powerful interactions, an engagement that pushes the boundaries of what we can experience in daily life.
But it is often not so.
My disappointment with video game interactivity came back several times when experiencing contemporary artworks that require interaction. The types of interaction I was engaging with were less about diving into virtual worlds and more about controlling moving imagery, using my voice to trigger a reaction on a screen, or seeing objects move in relationship to my presence. The paucity of these types of interaction, in contrast with the excitement I could see them causing in the audience, baffles me. A good number of installations often use the lowest common denominator within the range of human's potential actions and translates them into often very simplistic results.
Why is this? And what can be done about it?
To answer this question we should quickly dive into a working definition of interactivity, then maybe discuss what an ideal type of interactivity would be. If we can identify what a richer, more meaningful type of interactivity can entail, maybe we can set a goal for richer experiences that veer less towards low grade entertainment and more towards layered, deep experiences.
Our experience as human beings is, almost by definition, constantly interactive. We "interact" with our world at all times, we react to it and it reacts to us. The type of interactivity we have with the world and other human beings is infinitely rich, full of constant surprises and unexpected turns. How does art interact with us? In our time one tends to think of interaction as the relationship between a human and some kind of machine, a system, designed by the artist to elicit a specific type of exchange. By definition this type of interactivity is limited - scripted, controlled - but it can also be focused and surprising. We typically don't expect a machine or a system to interact with us outside of the boundaries of its function, and artwork is artwork in part because it escapes any type of function,
In the arts, we can see at least two types of interactivity, rich and poor. Rich interactivity happens when the result of the interaction is endlessly compelling and inspiring, where the outcome surpasses the expectation. This is something one often sees in large scale installation work, as the presence of the body, of the eyes, of the relationship between viewer, artwork and other viewers is constantly shifting the connection one builds with the work. A good example of this is Matthew Schreiber's work, which uses laser beams to create immaterial sculptures that encourage a constant movement and perspective shift to fully be "seen". Proper vision requires interactivity.
One could argue that here there is no real "interactivity", because static works of art do not respond to our presence. Not so, if one is to believe phenomenologists such as Merleau-Ponty, for whom perception is far from being passive. Without going too far in the great thinker's complex description of our interaction with the world, we can simply state that for him perception is necessarily embodied. The opposite of an embodied perception would be one in which the perceiving subject is disembodied, which means immobile, or omnipresent. Since this isn't possible (or at least at the human level), any perception has to be done through a moving body, which means a body that is constantly evaluating its relationship to its environment, and to what it is relating to (i.e interacting with) at any particular moment.
What the artwork "does" in response to this shift is simply respond differently. Take a painting : we all know that seeing it from close is a completely different experience than from afar. It gives differently, we see differently, the interaction is different.(Google Arts & Culture does a beautiful job at expressing this endless disparity in vision in their app, so does the Image Diver blog.) The French art historian Daniel Arasse spent a good part of his life discussing the value of detail and what the artwork "gives" in its ability to shift our position as a viewer.
What good does it do us to agree begrudgingly that interactivity is ever present in our experience of art? It reminds us that interactivity does not need to be limited. What "analog" interactivity provides is the equivalent of our "analog" relationship to the world, a relationship that isn't discrete, that has endless levels and shades. Digital interaction seems to mimic the notches of the digital world, blocks vs gradients.
So what would an ideal interactive experience with a machine, software or system consist of? It turns out people have been trying to figure that out since the 1960's, at a time when dealing with machines and software and systems was relatively new. Gordon Pask, a notable cybernetician, writing mostly in the 60's and 70's, worked hard a defining what a meaningful interactivity could look like. He developed the concept of "underspecified goals", which meant in essence "soft" goals. He would develop systems (software and hardware) where machines learned to develop certain abilities through a process of learning (and this was pre-Machine Learning as we know it today). The machine could be rewarded for its successful reactions, and consequently develop certain of its abilities more than the less successful ones. Sensors in a system could be favored over others because they succeed, and thus "grow" stronger. They could also lose sensitivity to certain types of outputs over time, and conversely develop a sharper attention towards others. The goal was to create an evolving system.
"The reasoning behind Pask's interest in underspecified goals is that if a designer specifies all parts of a design and hence all behaviours that the constituent parts can conceivably have at the beginning, then the eventual identity and functioning of that design will be limited by what the designer can predict. It is therefore closed to novelty and can only respond to preconceptions that were explicitly or implicitly built into it. If, on the other hand, a designed construct can choose what it senses, either by having ill-defined sensors or by dynamically determining its own perceptual categories, then it moves a step closer to true autonomy which would be required in an authentically interactive system. In an environmental sense, the human component of interaction then becomes crucial because a person involved in determining input/output criteria is productively engaging in conversations with his or her environment....
This is a completely different notion of interaction from that used in many of today's so-called interactive systems, which are premised on unproductive an prespecified circular, deterministic reactions. In these systems, the machine contains a finite amount of information and the human simply navigates through an emerging landscape to uncover it all. I do something, the device/object/environment does something back to me; I do something else, the environment does something else back to me. The human is at the mercy of the machine's inherent, preconfigured logical system. There is little of the conversation that a truly interactive environment should have, especially in the sense that nothing novel can emerge because all possible responses are already programmed. .."
(Usman Haque "The architectural relevance of Gordon Pask")
If these types of ideas were already set down in the 60's and 70's, and we know so much more today than we did then, why are we still thinking about interactivity in this highly linear fashion? To be more precise, why do we still see people designing installations where waving your hands in front of screen produces moving colors? Or why are our User Interfaces and User Experiences so simplistic? Sure, we are now exploring new methods of interaction with voice, brain waves, eye movement and such, but we are still dealing with systems that are, from the ground up, designed to have limited responses. And in my opinion the reason for that is that we tend to think of the human brain too much as if it was a highly developed computer that can, strangely, feel pleasure. But the pleasure we can feel is so much greater than the satisfaction of moving some particles around a screen or finding the solution to a game. We can feel pleasure by simply conversing, releasing, inhaling or tasting, receiving suggestions or overcoming hesitation, cracking bones or satisfying urges, being surprised or reassured. Our levels are so much deeper than the kind of basic satisfaction we might get from a system that offers the simplest of outcomes.
To wrap this up, what kind of interactions should artists be creating? How can the potential of computer/human interactivity be enriched? Of course there are already tons of successful pieces out there, with artists such as Rafael Lozanno-Hemmer, Golan Levin, Kyle McDonald, Memo Atken, Theo Watson and many others leading the charge. But what makes those pieces interesting varies quite a bit.
I would categorize meaningful interactivity as having two branches : presence and feedback. Simply put, presence is just that, you, in a space, seeing this happen to you, around you, simply because you are there. Examples of that could be Vincent Houze's "Fluid Structure", or Teamlab's various flower rooms. This is rich interaction because presence is rich. It may sound tautological, but far from it : pure presence is one of the deepest states a human can be in, so installations that make us aware of the influence of our presence on an environment force a re-thinking of its effects on the world (and the world's effect on us, mostly when this world is purely digital).
The feedback branch requires more complex behaviors from the audience. They are based on evolving systems that require attention and response, and provide a varied experience based on variables such as amount of viewers, level of involvement etc... These experiences can be rich because they feel limitless, but also because they show us that our interaction with systems don't mean the system has to be stable, closed or complete. And in this field, the biggest promise resides in developments such as AI and machine learning, where a system can teach itself how to get better, based on the inflexion and seeds it was given by an artist.
If "interactive art" is to grow, and not just be "interactive experience design", the audience has to be touched beyond a moment of joy, because, as my experience with video games showed me in my teens, the limitations you impose by forcing people to interact with works in ways that don't befit their intelligence will turn away whole audiences, or, maybe worse, will make them think that this is the best artists can do.