I call this a poem. Of course it's a poem.
"This is a portrait of Iris Clert if I say so."
Language Time Space and Spacetime
To write a poem, you usually start with a word, any word, and soon find that you have written a whole string of them. After placing your words in a string, left to right or right to left, depending on your cultural habits and artistic inclinations, you put the strings one on top of the other in rows, or side by side in columns. That is how we make a poem or any other page of words. You're thinking, But what about other types of poems? What about vispo, parapoetics, Oulipo and eidetic poetry for starters? What about postmodernism in general? What about crossword puzzles or any other possible combinations of letters, words and other symbols in two and three-dimensional space, not to mention four-dimensional space? What about TV and movies? What about hypertext and the Internet? What about cell phones and video games? And why did you start with words? Why not start with letters, or pictures or sounds? Many of the same rules that apply to words apply to all these other things, especially to letters, so why did you start with words and not with letters? And what about . . .
Because a good way to talk about language is in terms of words as symbol strings. And a good way to talk about words, letters or any other category of symbol strings is in terms of time, beginning with the question, What is time?
What is time? is one of the most significant and difficult questions there is. What is love? and Does anything really matter? just as important, are generally perceived as being specific to the human condition, whereas time is considered universal in nature, a priori to human or even genetic experience.
But is it?
I wrote a poem and in the poem I said,
Time doesn't exist outside of language. That's why you can't find the bottom or the smallest unit of time. There is no smallest unit of time. Time is a relationship that exists between two perceptions, or thoughts or memories. Reality, apart from observation, is a unity.
In another poem I said,
Although I personally think that time is a sensory construction and once time is taken out of the equation there is no equation and since time is a relation on events there are no events either, I do wear a watch.
Before I wrote these things, I wrote:
- The world is also a unity, indivisible. It is only through the act of attention that the characteristic nature of things comes into being.
- Instead of "thing," use the word "object." Instead of "characteristic" use the word "attribute." The remaining keywords are "event," "relation" and "message." These are the treasure words, the canonical forms by which words and the world are made one and the same. These terms define the process by which things enter into being and pass away. Even the knowledge that occurs without words or thoughts, the knowledge that comes solely from breathing and being aware, must find its place in the midst of these.
- An object is a collection of attributes in time. An attribute is the paying of attention to a single characteristic that an individual object maintains over time. An attribute is also the paying of attention to a specific characteristic that two or more objects have in common at any given instant. Attributes include all forms of sensory stimuli: size, weight, color, texture, smell and taste, individually and in combination. Attributes also include number, the idea of how many or how much. And even if we use machines such as cameras and radar as artificial eyes and ears to extend our perception, we do not transcend either our senses or the idea of number.
- A single instance of an attribute in time is an event. An event is as near to reality as we can approach. An event is not the absolute reality of the world, but an artifact of that reality. And it is only as an artifact that an event can exist. And although events precede attributes in order of existence, they in turn are defined by attributes. Events represent change over time in the values of attributes. In other words, a change over time in the value of an attribute associated with an object is an event. An instance of a collection of various attributes in time associated with an object is an event. Change over time in the values of attributes associated with a collection of objects is an event.
- An object in turn is a collection of like events in time. And because events are the measure of objects, all objects in turn are artifacts.
- Attributes begin their life as lists: of words, numbers, images, sounds. Lists become objects. But a list is more than attributes. A list is attributes in a certain order. And the order changes over time as the values of the attributes change and the membership of the attributes in one list or another change. And a list becomes a collection of lists that in turn become collections of lists. And attributes become objects, and lists become objects, and collections of lists become objects.
- After events and attributes come relations. A relation is a type of attribute. Relations begin as the ordering of attributes in a list. It is the order of attributes in a list and the changes in the values of those attributes with respect to each other that we call relations. A relation is a value or set of values associated with an attribute that corresponds with a value or set of values associated with a different attribute. A relation is therefore an attribute of attributes. Relations define the interaction and behavior of attributes in their roles as objects. Relations are the glue that ties attributes together to make objects. Relations are what make the elusive, dynamic and process-oriented nature of things possible. Attributes are structure. Relations are process. If attributes are the elemental stone, earth and wood, then relations are fire, water and wind.
- Attributes and relations can be combined in a number of ways to create higher level abstractions that in turn can be used to make statements about objects, their behavior and their relationships to each other. These higher level abstractions include mathematical functions, propositional grammars (i.e. logic), predicate grammars (i.e. computer and mathematical languages) and natural languages (such as English, Spanish, etc.). In turn, all of these higher level abstractions are reducible to attributes and relations.
- It is the organization of events, attributes and relations that make an object possible.
- An event is the archetypal object. It is attention at a moment in time. It is attention a moment at a time. An event can also be a collection of events. Each event in turn consists of stimuli, observations, measurements. Our birth is an event. Our death is an event. Our life is the series of events in between. Every object can be seen as a series of events. Even something as stable and stationary as stone is an event, a series of events. The stone perceived as an object has mass, size, color, texture, smell. And just as with any other object these attributes and their values change over time, even if that time is measured in millions (or billions) of years. Collectively, these events constitute the lifecycle of the stone as an object.
- Beginning with events as artifacts and the understanding that events are observations and measurements over time we can take the next step. The next step is to communicate the events. An attribute contained within an event is local to the event. We do not perceive a requirement to communicate the attribute or its values to other attributes associated with the event. We think of them as being simultaneous with each other and with the event. It is only when an attribute or its values affect the attributes or values associated with a separate event that we begin to consider the communication of that information between the two separate events. If two events are identical in their attributes (including their location) and differ only in time we say that they collectively represent an object. If their values change we say the object is changing. Specifically, if we are able to predict the change of the values of attributes associated with events and those changes are simultaneous with each other we call the collection of events an object. That is how we define an object: as a series of like events in time. If two collections of events are identical in all respects except location we may be tempted to say that they are the same object, even if the locations are far apart. It is non-simultaneity that creates the requirement for a message. It is when two or more collections of events maintain a close correspondence over time but are not simultaneous with each other that we say a communication has occurred, that a message has passed from one object to another. So a message is the non-simultaneous change of the values of attributes associated with events separated by time and space.
When it is not busy being an object in itself, language is a message between and among objects (all objects, us included) over time. Time is always the last dimension in any n-dimensional description of the universe. Space is geometric in nature. Time is algebraic. The combination of space and time is spacetime. In spacetime, geometry, algebra (and calculus) are translatable, one into the other. In spacetime, geometry, algebra and calculus transcode.
In a two-dimensional spatial universe, the third dimension is time. In a three-dimensional spatial universe (such as our normal everyday universe), time is the fourth dimension. Time is the dimension of change. A three-dimensional universe without time is static. You need four dimensions to see a three-dimensional object move. That's why you can't see a hypersphere, a tesseract or any other four-dimensional object move in three-dimensional space. You can only see a three-dimensional "slice" or "net" or "graph" of it in three-dimensional space. You can, however, see the hypersphere or tesseract in your mind, which is the same place you can go to see any other object in n-dimensional space. That is because your mind does not exist in four dimensions. Your mind has a part of its existence that is not Newtonian, that is not purely deterministic and does not follow time's arrow. What happens in the non-Euclidean part of your mind is translated into four-dimensional spacetime and communicated to others through language.
Does time exist apart from mind? Well, we can measure and place in order events (such as cosmological events) that occurred before there were human minds to observe them. And we can measure, place in order and successfully predict certain events (such as cosmological events) that have yet to occur. Even so, I continue to think that time is a construct of mind.
That's just my opinion. There are numerous others, readily available on the Internet, espoused by the most prominent of scientists. For example, that time does not exist objectively or subjectively, period, or that time is not linear, but exists as a closed loop in which it is possible to bump into your past self even as you lurch headlong into your future, or that in a multiverse in which everything possible exists in actuality in some universe or other, there are an infinite number of universes in which space exists but time does not.
Whether time exists outside of human perception, or whether time is the result of genetic evolution doesn't change the fact that time is good at its job. Time is co-equal with space as a primary unit of measurement in the human universe. Time allows us to construct events from perceptions and to collectively organize those events to create objects and relationships within and between those objects and to exchange and interchange objects and their relationships and to do it all with language. Language and time go hand-in-hand. Without language there can be no time. Without time, there can be no language. Without space, there is no justification for time. Language, time and space are so interconnected in our minds and behaviors that we cannot easily conceive of any other possibility. It's been this way since the Greeks.
Beginning with Euclid, analytic geometry has dominated our understanding of the visual field. Four-hundred years ago, Alberti's "On Painting" placed geometry inside a metaphorical window where it continues to frame, not only painting, but photography, television, movies, the Internet – almost all forms of digital visual media. But geometry has also taken a second path where both space and art are concerned. To those who follow this second path, space and the visual field are not synonymous.
When the French artist and mathematician Girard Desargues created his projective geometry to describe perspective, the fields of geometry and algebra became interchangeable. Whatever you could describe with geometry, you could describe with algebra. And because you could describe geometry with algebra, you could describe space in terms of algebra in ways that were nearly impossible to do using traditional geometry. First algebra and then calculus expanded the manifold of space from three dimensions to four, to six, to ten, and most recently, to eleven. Robert Riemann's concept of a multi-dimensional "metric tensor" gave Einstein and others the ability to use a number to represent a point in multi-dimensional curved space, removing forever the spatial limits of height, width and depth. This idea caught on, so much so that our current scientific model of the universe uses as many as eleven dimensions to describe a multiverse of multiple parallel universes existing in n-dimensional space.
There is an important connection to be made between an n-dimensional universe and the visual field. Language as symbol sets organized in plane space is able to describe higher dimensional space. A two-dimensional symbol string can do what a three dimensional model cannot do. Digital media at its core is two-dimensional. Binary language makes it so. Binary language is the simplest language possible. Binary language can be used to translate "from" any language "to" any language. Because binary language provides a universal "interlingua" between all of the other languages that exist in the world, the ability to translate an object to and from a binary representation means that an object, regardless of where it originates, can be translated, transported and re-represented in any other language or collection of objects. The spatial representation of text, images and sounds can easily be changed from two (i.e., binary) to three-dimensions and back again. The representation of media as text, image or sound can literally be changed back and forth from one form to the next. And by establishing conceptual relationships between digital images and other symbolic forms such as mathematics and written language, the digital image can be made an integrated part of a larger multi-dimensional field.
The traditional way to think of cultural production where time and space is concerned is in terms of classical perspective. Everything designed specifically for a museum, gallery or library is designed to fit this model. Museums are a great place to put Euclidean objects. Written language is no different. The written word tends to occupy plane surfaces and collections of plane surfaces made of stone, clay, metal, parchment, paper, plastic, canvas, analog and digital display devices . . . (Those of you possessing the anti-authoritarian gene/meme will quickly identify numerous counterexamples including your name on a grain of rice, autographed baseballs, basketballs and footballs, the letters "IBM" written in atoms with a Scanning Tunneling Microscope, balloons and other inflatable objects, laser light shows, tattoos, the neon signs at Piccadilly Circus, etc.)
In reality though, language is most flexible when it is represented as pure information in the forms of electrons, photons and magnetic fields organized as bits and bytes. In its binary form language is as fluid as water. In its binary form it can be used to represent any object or collection of objects and it can also be used to translate and transcode those representations of objects into any other representations.
Everything in the digital world originates as a written language translated into strings of binary digits. Everything. People too quickly overlook the simple fact that every type of computer output as it is expressed in the forms of movies, TV and every other kind of digital media and communications almost always originates with writing. Web art originates with HTML, XML, Java, Flash and other 3GL and 4GL languages. Video games do the same. In fact, every form of digital media originates from one or more layers of hardware languages, software languages and higher level software applications, all of them existing in one form or another as written language. Even the films that we see in theaters, in all of their analog beauty, are translated from film coming out of a camera into a digital intermediate format where they are edited using computers before they are converted back into the film that we watch.
Does talking about the physical form that language takes beg the question of what language is? Does visual art – poetry, paintings, sculpture, movies, games, digital images and whatever else we interface with in space and time – have an existence independent of the geometric space in which it is expressed? Do the words in our poems have an existence independent of the forms they take? Can they be transformed into new media, even at the level of their digital substrate, in such a way that their original forms and their original meanings can be completely recovered from the new forms they are given? If so, do the words acquire new meanings as they transform from one form to another, from poems to paintings, to movies, to games? The answer is no, yes, yes, yes and yes.
When it comes to answering questions like these, New Media, Cross Media and Transmedia are the tools for the job.
Even though we began with words in a row, it was never about words in a row. It was about words in spacetime. Words in spacetime give you so much more than just words in a row. Words in spacetime give you spoken poetry, written poetry, visual poetry and every other kind of poetry and art that can be spoken, heard, looked at or touched. By spacetime we don't just mean four-dimensional space. We mean any n-dimensional space with time as its nth dimension. That means everything from binary data in two dimensions, to painting, sculpture, music, video and interactive art in four dimensions, to conceptual quantum art in n-dimensions. Except for the n-dimensional part, this is nothing new. This idea of a poetics of space and time was described by Plato in the distinction he made between mimesis and diegesis.
M
i
me
sis
, Dieg
esis, New
Media, Cross Me
dia, Transmedia and Mind
Mimesis is
the act of copying something. Copying the sound of a bird by using our voice or a musical instrument is a good example. Mimesis captures the appearance of things. Mimesis is art as imitation. Photocopy art. Tape recorder art. Because the world occurs in time, mimesis is the act of expressing events in an identical way, in both form and duration, as the original events. Mimesis does not recognize past, present and future. Mimesis only recognizes the present.
Diegesis is
art as narrative. Someone tells a story, real or imagined, and in so doing retells the story. Diegesis is the world seen through the eyes of another. Where mimesis recognizes the literal present, diegesis recognizes everything - actual, probable and impossible. Nothing is off limits. The narrator not only shows you, but tells you what the characters are thinking and feeling. Diegesis can jump backwards and forwards in time. Diegesis can randomly access any object or event in its narrative universe anytime it chooses. It can place them in any order it chooses. In diegetic space and time, everything is available to the narrator, not only what is on the page or on the screen, but what is not on the page or the screen, not only what is heard, but what is not heard, not only what is felt or thought, but what is not felt or thought. A copy is mimetic. But making the copy or observing the copy is diegetic. A Rubik's cube is mimetic. But once you create a story space by thinking while you are playing with a Rubik's cube or talking while you are playing with a Rubik's cube, or doing anything else while you are playing with a Rubik's cube, you have made the mimetic space of the Rubick's cube a component of a diegetic space. That is the real difference between mimesis and diegesis. Mimesis is iterative. Something is repeated verbatim. Diegesis is recursive. Something is repeated with reference to its prior occurrence.
A good example of the relationship of mimesis to diegesis is language acquisition in children. We know that a child initially learns language through mimesis. We say something as we point to an object. The child sees and makes the connection and copies our speech pattern. This is only the beginning. How is it that a child can so quickly learn to say things correctly when there are so many ways to say them incorrectly? The best explanation to date, based on the work of Noam Chomsky, is that we are born with an innate capacity to generate language and that we use that capacity to acquire language skills. This capacity is species specific and applies to any language or languages, including natural languages and formal languages such as mathematics. At any given moment there are billions of languages in existence, each of them unique.
Recursion makes this possible. Recursion creates our language universe of objects and relationships by using a finite number of self-referential rules to break down complex statements into simpler statements, or, going in the other direction, to generate an unbounded, increasingly complex variety of statements from these same rules. Recursion is a type of self-referentiality that allows us to increase or decrease the relative complexity of objects and their relationships.
This simple idea leads to conclusions that are not so obvious. Even though two people may say the same thing one after the other, "The sky is blue," for example, the second time we hear it is different from the first and has a different meaning from the first because the second time has everything that happened in-between to add to its meaning. Every thought, every word, every action, every second of every day is different from the time before because of this. The idea that narrative has a fixed beginning and end is a fallacy. Recursive narrative is not tautology. Self-referential and recursive narrative is unbounded, as unbounded as the surface of a sphere. It is the strange loop in the daily story of our lives. The recursive nature of narrative lies at the heart of our individual and collective identities. In other words, there are no external languages. There are only the external correlations that exist between our expressed, internal languages. Some people will tell you that diegetic sounds or pictures or words are what are inside the story and non-diegetic sounds or pictures or words are what are outside the story. But you tell me, what is outside the story? Everything made known by the story is in the story. And what is posited as being outside the story is inside a bigger story. Everything is a frame tale. Nothing is not. It's the same way with mimesis. In the sense that it must necessarily represent an exact copy, mimesis does not exist. It cannot exist. The human mind is wholly diegetic.
New Media
is the use of digital language and digital technology to create and to integrate new forms of text, pictures, video, sound and human interaction.
Cross Media
is the use of both traditional media and New Media in any combination to represent the content and meaning of a given work in one form, such as text, in a different form, such as a picture. A poem as a painting. A painting as a poem. A voice as a painting.
(See Paul Klee)
Transmedia
is the literal translation of an object or objects in a given media, such as text, into an object or objects in different media, such as pictures and sounds, using formal languages such as mathematics and formal methods such as machine language translation.
LanguagePerspectiveSpacetimeMimesisRecursionDiegesis
In 1964, the physicist J.S. Bell wrote a paper in which he stated that quantum physics provided a model for the universe that was inherently more accurate than any model containing an assumption of local cause and effect. Essentially, Bell proved mathematically that a single object could exist in two places at once and that any change in the object at one location would instantly be "known" at the other location, regardless of the distance between the two locations. In 1982, the physicist Alain Aspect provided an experimental proof of Bell's theorem, knowledge that is now being applied in areas as diverse as quantum computing, cosmology and theories of mind.
Bell's theorem questions our understanding of the nature of unity and multiplicity. What it says is that things can be intimately connected to each other regardless of how far apart they are from one another in space. There is no such thing as a purely local model of the universe. There are and always will be things that we cannot see that are intimately connected to the things that we do see. Our daily experience tells us that the universe is made of many different things. Science has now provided proof that all things are "entangled" and that the concept of a universal unity is scientifically valid.
In quantum physics, when an object exists in more than one position in space or time and there is a fixed relationship between the values at each position we say that the object is in a coherent superposition, that it actually exists in multiple locations simultaneously. But once something external to the object interacts with it, through the environment or an act of observation, the superposition becomes decoherent and the object "collapses" into a single location.
Until recently, most scientific theories of consciousness assumed that consciousness was the result of electrochemical activity among neurons at the level of the synapse. Roger Penrose and Stuart Hameroff have developed a theory of mind that relies on the concept of coherent superposition to explain consciousness. Penrose, Hameroff and others believe that within the neurons, self-organizing protein structures called microtubules provide an environment where consciousness manifests through quantum processes. Within the microtubles, quantum bits (i.e., qubits), while in coherent superposition, become entangled with qubits in other microtubules, creating a level of informational complexity orders of magnitude greater than that of any computer. When this entanglement reaches a certain threshold, decoherence follows and we experience the collapse of this coherent superpositional state as a single conscious event. Each conscious event is discrete. They have even proposed that these events occur at a rate of between 60 to 120 times a second. Like a film made of individual frames viewed in sequence, our unconscious, preconscious and conscious mind is the result of this quantum process.
If science teaches us anything it teaches us that today's ultimate truth is tomorrow's close but no cigar. In this case, I don't care. I really like this theory.
When applied to the everyday world, this idea of quantum mind is analogous to the coherence and decoherence that occurs when we interact with other people. As is the case with subatomic particles, our acts and observations become entangled with the actions and appearances of the people around us. When we reach a limit or threshold in our understanding, the once coherent complexity of the other person becomes decoherent and the nearly infinite possibilities that were present within that person immediately collapse into a singular category of identity that takes the form of a logo, a brand image, or some other form of economic, racial, religious, political or sexual stereotype. That's on a bad day. On a good day we get art.
In summary:
Mind is
a quantum,
diegetic,
n-dimensional space
New Media,
Cross Media and
Transmedia are art forms
that intersect and recursively transcode
in quantum, diegetic n-dimensional space.
Michael Harold is an artist and technologist. Lots of different kinds of art. Not so many different kinds of technology. His web site is www.michaelharold.com aka www.azazaza.com. It used to be just www.azazaza.com, but his friends always seemed to have a hard time getting it right (Did you say "zaza aza" or "za zazaza" or "azza azza"?) He lives in Shreveport, Louisiana.