Mirror as Neosentient

Click here to see large PDF: _alexandriaJarvis-mirror-as-Neosentient

As technology becomes an extension of daily life the human body develops a new phenomenology through which it senses and interprets the world. Each technologically mediated experience enables the human an altered or enhanced set of rules and functions.
As a technology and it’s scenario of use is imparted or discovered within a community, a new social order develops around it as it becomes accessible and acceptable. In the new social order, this new way of being begins to govern or even subsume daily habits and interactions. The shift ushers in a reliance on technology in which community members experience a fragmented reality. Ironically, this fragmentation occurs both while these individuals are united to and separated from this relationship.

This new phenomenology, created by playing off of our old is thought to be the key to recreating a kind of intelligence which will exceed our own. Whether it is artificial, ambient, near or far one can only imagine it would develop it’s own phenomenology. How will we relate to this new sentient being? How will it “see” and understand the world? What will it do with what it “see’s” and senses? Lastly, when this neo-sentient is no longer just a tool, offering the human a new phenomenology, what would happen when we realize it has it’s own phenomenology?


In a short exploration “Neosentient as a Mirror” I chose the most overt visualizations and alterations I could relate to the notion of an altered sense of the world and self: alterations of the representative body. Also, I was interested in the experience and affordances of communication through technology which led me to consider issues of self awareness through technology. After thinking about our relationship to ourselves, technology, and others I chose my agent: a mirror, to house the neosentient-human relationship. The mirror is a simple technology which we have built up both an expectation and mythical mystery about. The scenario I filmed is an exploration of the moment in which a human stumbles upon a neosentient which has recently discovered self-awareness. It’s alterations to the human form are minimal and subtle at best. However, beyond this scenario one could imagine it’s interventions being both expansive, bold, information rich, and applied to a specific context of use. The motivation of the neosentient in this scene is undetermined, but could be explored in further iterations. The schema of interaction could also be explored but in this case the mirror lends us to consideration of the analogy of the neosentient as mirror. Also, the communication between the neosentient and the human is at best strained in this scenario, but one could you imagine a willing and curious observer? All in all, this study has lead me to come into a critical awareness and imagination of my relationship to technology. And from this exploration, I find I am once again delighted by my “old” phenomenology and perhaps less miffed (or more thankful) of the new one I have gained!


Neosentient: detecting a relaying a visual representation of self and others

Technology as an extension…
Technology becomes an extension by which we experience the world. A plane allows a pilot to experience flight, though he himself is not flying. A telescope gives a sailor a magnified vision of the shore, something which is imperceptible at our current vantage point. A camera allows a person to “see” a loved one through video chat and suddenly they become telepresent. We experience the world through our body, but more and more we rely on technology which transforms it into an extension of our body. In the example of the video chat, the camera physically exists within the space in the place of our eyes. The camera becomes our eyes. The technology detects that experience in some ways but does not have the same level of sentience as we do. The camera technology is detecting particular variables of that reality, such as image and audio, and relaying a representation to us. I am interested in creating an experience in which people come into a critical awareness of this phenomenon of extended experience through technology with by altering the representations which they expect to have.

I am interested in studying each element within this relationship
and each transfer of information which happens (arrows)…

In relation to Neo-sentience….
When thinking about technology as extensions Bill Seaman asked the question, if that technological object had a phenomenology in the way that we do, what would that be like? The neo-sentient is that the technology might have not only aspects of the sentience we have but also new aspects of perception and understanding about the environment.
It is interesting to think about this idea of the phenomenology of a neosentient in reference to the diagram above. The neosentient would not just be detecting and relaying a representation to us but also is aware of their own representation and perception of reality. It could detect one thing and relay something else. (We often feel this way with humans.) It could relay a representation that was endowed with additional pertinent information.

In review of all these things here are some questions:
What if the technology could be sentient of the experience it is relaying to us? If it could have an enhanced way of experiencing that moment more similar to the way we experience how could that change our relationship with both technology and the represented reality? What could be the phenomenology of that camera, telescope, or plane which is the intermediary actually creating the vision, clarity, or flight.

Representations of self // Developing the final project….
A mirror is a simple kind of technology which yields a representation of ourselves and has a high degree of expectation from previous interactions. It lends nicely to a comparison of the way in which we create digital representations of ourselves. It is also an object which represents self-consciousness and self-image.

I would like to create a mirror which replaces or alters the reflection you expect to see.
I am using the mirror as a way to think about issues of how we create and rely on digital representations of ourselves and others through digital technology and the internet.

Below: Simplified diagram of the transfer between the self and the digital representation of self and others. (eg: Could be thought of as a diagram of social media)

Below: We expect a mirror to supply a representation (reflection) of ourselves.

However what if this reflection/representation was partly altered or replaced.

  • watching a mirror means watching yourself, looking at yourself
  • moments of lag of in reflection
  • long periods of lag
  • a mirror gives an image of what is in front of it, but what if it wasn’t in front of it
  • a mirror which gives a reflection of the back of your body not the front
  • glitches in image (like pixels)
  • glitches in image: broken image (mirror)
  • image of someone else
  • replaced body part (technology as extensions of body)
  • enhanced body part (fun house)
  • What is the memory of the mirror? What data does it have?
  • What is the phenomenology of the mirror?
  • Could a mirror also be looking at you? Not just you looking at it
  • parallels self-consciousness (watching self) and self-image
  • People don’t look at mirror but past the mirror right? to see themselves (maybe a mirror as its own object has power in that way, if it had it’s own will, because it could provide a false sense of reality)
  • a mirror which matches your emotion
  • a mirror which presents a version of the current you and the previous digital representations of yourself
  • the mirror as a surface: a mirror which has images just below the surface
  • a mirror which alters the reflection of you and another person, switches them in some way?
  • Think about times that you see a mirror in which there is an illusion which doesn’t exist such as in a store where it looks like the room s extended.
  • what if you were looking in the mirror and whatever you fixated on would morph
  • perhaps the image of yourself shows up somewhere else instead of in front of you, like on the ceiling
  • think about how we rely on the representation in the mirror, ie- seeing what we look like to understand who we are/how our physical appearance is

Stories of process….
Previously I was interested in looking at tangible interactions and embodied experience as a way to contrast with the cognitive and disembodied experiences of technology. I was still interested in experience and digital versions of ourselves, making sense of the world through extensions of technology. I was a little confused as to how to think about this relationship and was having problems approaching it through this way of focusing on the differences between the embodied experience I have in the physical world and disembodied experience I have when I am engaging with technology, such as the computer.

Ranulf Glanville suggested I reflect on situations in which I was having an extended experience through technology but instead of feeling disembodied perhaps I it felt as if it was embodied or as if it was transcendent of embodiment. I identified and actually noticed two particular experiences which happened during skype chats with a loved one. This lead me to think about technology as an extension of the body as well as technology relaying representations of ourselves and another person to us.

Below are personal notations of these experiences…

  1. experience 1: I had a video chat in which I experienced a sense of helplessness in the telepresent moment however I realized it was exactly similar to that which would be happening if I was actually present. In this moment of helplessness all I had was the opportunity to be aware of my feelings and empathetically extending them to my loved one whom I was video chatting with. I caught myself looking at this person and it created a self-awareness of this action. It was an odd experience.
  2. experience 2 video chat: extension of body, I had an experience where I was video chatting with a loved one and they left the room for an extended time. I was busy on the phone while they left the video chat and was not aware, upon returning to the chat screen, that 1.the camera was still on and 2. that the person had truly left. This was an odd experience as I realized there was a hyper awareness in which it felt as if I was in fact in the room. My imagination ran wild and I imagined a situation in which I saw something such as a ghost or an intruder in the apartment. This thought was interesting because it operated off of my expectation to see something in the other side of the frame. Also, it made me aware that in the case of the intruder I would experience vulnerability, though I wouldn’t be physically vulnerable. The technology was an extension by which because of my expectation there was a hyper awareness of the room.
  3. accidental pocket dials This previous experience made me think of when someone pocket dials you and for the first 30 seconds of the message you are listening very closely because you are confused and expecting a voice but get background noise or lots of voices instead. I liked the idea of thinking of the phone as translating voice messages as representations of people/places/times. I also liked how in listening to a pocket dial, the way you listen to the information you are receiving changes as you realize it is not a typical message.
  4. surveillance cameras (connection to 2) I also thought about surveillance cameras and looked at an artists project to surveil her love life. This was through the Exposed exhibit at Tate Modern. She noted that when she went on dates the awareness of the camera created a hyper attention in the beginning but wore off as time went on. I also thought of an experience I had in which I was watching a surveillance output at a grocery store elevator. The elevators were in front of me and I was watching people on other floors.
  5. expectations and representations I thought about a bed which has a form of a person from the blankets and we think it is a person, or a tube which we mistake as a snake. I also like the combination of surveillance and representative physical form in this work I saw by Ryan Roe, called hang in: http://ryanroa.com/hanginb.html

Idea Maps

terms and phrases_ ongoing

folds (Deluze)
information aesthetics
informatic temporality
technically symbiotic transformations
new media artwork
global information culture
digital embodiment
emerging digital bodies
cyber feminism
VR, virtual reality
HCI, human computer interaction
computer: originally a job title, to sum up, to recon
imput, output
multi modal, cross modal
bodily entailment: all causal properties that are at operation in the body
aesthetically potent environments
generative system
iterative system
intelligent agents
computational paradigm
dematerializing the body
responsive media environments
time based electronic art
experimental online artwork
human-machine interfaces
self organizing
self replicating
self regulating
non-contiguous or overlapping ontologies
open system
sentience quotient
induction, deduction, abduction
language of agents
socially oriented reactive environment/community
extensions of self
meaningful community
digital convergence
auto-callousness (Rosen)
feedback loops
symbiotic system
systems theory
objective, subjective
the world as interface (Rossler)
Poly sensing
data derived through introspection
data derived through intervention
the viewing space, the trajectory of forgetting (Ruth West)
data is knowledge is information is data
smart materials vs intelligent materials
information cascades
collaborative consumption
central and peripheral processing
centralized and decentralized networks
cause and response vs. stimulus and response
sequential and spacial way finding
synchronized chaos
memory as a gathering of many reconstructions vs memory as pulling from a drawer in the mind

continuum of faciality

A rough draft of a brief essay on faciliazation in HCI …

I am writing this essay for an audience of designers. I would like to tack on a paragraph or two about our discussions of adding another layer of the continuum which is vertical with human computer at top and neo-sentient computer at the bottom. Any suggestions on the text are welcomed. The paper was meant to distill down a few ideas- some of which I am just being introduced to. I gladly accept criticisms but be ready to be my teacher, there is a lot I do not understand.

Facialization is defined as “a system of codifing bodies according to a centralized concept of subjectivity or agency in which the face, literally or metaphorically is the conduit for signifying expressing and organizing the entire body.” (Munster, 122)

Since the birth of the analog computer the machinic language of the computer dictated the conversation between human and machine. As the computer advanced, the everyday embodied experience of the human remained a condition of neglect. Interaction was restrained by the binary world of code. Today the pendulum swings. The seamless visceral human experience of reality directly informs interaction design. This movement marks a significant shift in the relationship between human and computer. Arranged on a continuum according to subjectivity, any instance of human-computer interaction can be located approximate to three zones. The traditional objective human-computer relationship is located one end of the continuum. The middle of the continuum is interaction dependent on physical aspects of embodiment. Lastly, on the other end of the continuum, a new body of thought emerges: post-humanism. Post-humanism, a complete disruption of traditional objective human-computer interactions, merges body and code. It is within this place that we are finding issues of faciality to be disrupted. In her book: Materializing New Media, Embodiment in Information Aesthetics, Anna Munster seeks to understand the impacts of these movements in relation to their effects on our understanding and relationship with computing. She titles her third chapter: Interfaciality: From the Friendly Face of computing to the Alien Terrain of Informatic Bodies. It is in this paradigm shift that we can find the full continuum of HCI and all the weight in which faciality is causing movement.

Traditional interface design deals with an objective relationship between human and computer. The human is forced to respond and position oneself as a “user” entering into the world of computing. The goal of these interactions become utilitarian, information seeking, and first and foremost aim to complete a task. This interaction speaks to the “users” desire for productiveness. Creating an interface on this premise relies heavily on the foundation of interaction heavyweights such as Donald Norman, Brenda Laurel, and Jakob Neilsen. Siting a few salient characteristics, this zone of interaction is marked by designing elements of intuitive visual communication (Neilsen), focused on studies of users in context (Laurel), and considers the interface as a tool which comes second in place to the direct task at hand (Norman).

The lines of HCI have shifted into a human-centered zone of interaction characterized by a strong subjective relationship between human and computer. This human-favored interaction is marked by multi-modal interactions and immersive or dispersed computing environments. This subjective relationship posits embodied interaction as a basis for human-computer communication. This shift puts emphasis on reclaiming the dematerialized body which was lost in pervious HCI interaction. Much work has been done in this area of HCI including the projects of Rokeby and Diane Ludin. This work displays several elements of this middle ground including the creation of an invisible interface (Rokeby) and immersive environments which create continuity of perceptual experience. It also typically operates off of interaction between environment, human, and computer and uses visceral bodily experience as cues for interaction (Ludin).

In a third and most extreme category of HCI there is an emerging post-human computer-body relationship which is characterized by a merging of the two entities and a continuous feedback loop which creates a completely new state of being. The goal of this interaction lies within the ability or possibility to create an adaptive and responsive learning system. The idea behind this area of HCI extends towards a desire to create an advanced, more intelligent entity. To combine computation and code with direct links to body and biological matter opens up a new world which has only just begun to hint at possible outcomes of these emerging digital bodies. Many of these ideas of post-human interaction can only be explored conceptually through new-media artwork. An example of this category of interaction includes the work of Stelarc. Some elements which characterize this node of HCI include biomimetic computational grafts (Stelarc), wearable computing, adaptive systems, and extreme subjectivity. As extreme concepts of post-humanism are just beginning to be realized, these new media artworks are doing a good job at dismantling our understanding of faciality and disturbing any clear boundaries human-computer interaction.

[This is the conclusion paragraph talking about how designers must recognize where along this continuum they are designing. The justification for this choice and the baggage associated with each.]

very initial thoughts

A conversational writing exercise: to begin defining my interests

I am interested in the way we relate to physical space, how we understand or how we are attentive to our environment (awareness to heightening// intensify. ie: shape – mass, use imagination, reflect on phenom. experience, create relationships, delight)

Tangible and social computing and/or physical objects embedded with the ability to sense, integrate rich media, gather information… offers the opportunity to call people to an increased attentiveness of an environment/experience

(Connections to performance– ie: improv everywhere and other performative, immersive gestures. recently: technology performances  and  performative projections)

Drawing out the phenomenological but also the psychological

In a way this is what artists like Robert Irwin and Christo do. Except I like the idea of these experiences being hidden, embedded in space we already have a relationship with. So new experiences would be revealed where there once was nothing (projectors have a nice way of doing this)

So it would offer a new phenomenological experience (much like the artists) but could also offer a psychological meaning by drawing from the information wells of the digital versions of ourselves.

ie: alexandria in physical world /////// representation of alexandria in digital world

Modern art seems to be doing a good enough job as it is to suggesting a re-looking at the world. Why do we need technology to push us? I dont know. Perhaps also it is suggesting also an awareness to the fact that we are creating a digital version of ourselves. or perhaps it is bringing someone into the conerned awareness of information available– Perhaps this is talking more to Negroponte’s perspective that we can use interface as a way to filter this information “the role of the interface is to filter bombardment”

All of these thoughts go back also to how is digital culture influencing.. our experience of culture//time//ourselves//our environment

What I want to do with this interaction.. or suggest by making the person aware– i dont know………..Will i just be making an updated version of these contemporary installation artists- drawing us into awareness of the phenomenological?