First trial with EPOC headset device.

I was very excited to test the EPOC headset and to see what it is capable of. Firstly, I took it out of the box and read through the manual. Before I could start the headset had to be charged and sensors prepared by hydrating them in a special saline solution.

I charged the headset with a special usb cable provided. Here is the headset on charge (without sensors with red light on). When battery is full the green light will indicate so.

The sensors came packed in a plastic case which is suitable for soaking. In total there are 16 felt tip sensors with golden plates. They must be soaked wet and then mounted in the headset arms. It is important that they are not dry or it will result in insufficient conductivity and therefore weak brainwave detection. They are soaked in standard multipurpose contact lens solution which can be purchased in any local drugstore in case i will run out of it.

After preparing the headset I have to fit it correctly on someone’s head. There are few tips on how to place it on and get a good contact with sensors. It says on the manual that “Good contact of reference sensors is the key for a good signal.

In order to establish the connection between the headset and the computer I used a supplied USB Transceiver Dongle and placed it into one of the USB slots. Then I turned on the headset and checked the signal strength as well as sensor connectivity with the help of provided software tool called EPOC Control Panel which came with the purchase.

In this image I can see that all sensors apart from one are green. Green means good connection and there is a slight issue with one sensor on the forehead. There might be an issue with the wetness of the sensor or the position. The manual says that my “objective is to achieve as many green lights as possible by adjusting the position of the various arms on the headset. Note that the EPOC will still function with some sensor locations showing yellow or orange however the detections will be less reliable in this state. Often the contact quality will gradually improve after a few minutes use.”

I noticed that EPOC must be in field of USB dongle. The PC tower was underneath the table and at the start it was hard to get the connection, but when we placed the EPOC closer to the dongle, the connection established straight away. In order to maintain this connection it was important nothing blocked the signal sending, things like a table or a chair.

To test this head device i used two software tools:
1. EPOC Control Panel
2. Mind Your OSC

Here is a small video preview of Mind Your OSC software responding to the headset in use:

[vimeo.vimeo.com/38561788]

This software has a very good response in Affective mode and all subcategories such as: Engaged/Bored, Excitement, Excitement Long Term, Meditation, and Frustration are working fine. The value amplitude is ranging from 0 to 1. Other modes such as Cognitive and Expressive seems to fail picking up any numeric changes. There are spurts of values coming up inconsistently and therefore not reliable for my project.

What is good about this particular software is that all values can be made accessible through the port which is a main bridging element in case of using additional software to interpret these values. I can select the port number but for some odd reason i can’t change the IP address. That means that i HAVE to run this software on same computer to parse the values through the chosen port. I think this is very disappointing due to the fact that this software is for Windows only and should be used on same machine. From previous experience I have worked on several computers and linked them in order to communicate for a single task. In this instance i will need to install Windows onto my Mac and run all programs in it. It is not an issue for me but a unnecessary hassle and limitation provided by software development kit only compatible with Windows operating system and restricted development of Mind Your OSC. I am aware that for additional cost one can make this EPOC headset more suitable for research grounds and i am only confined by my choice of cheaper version. So i will need to go extra mile in order to make maximum use of it!

I think the reason why all values are not reading properly on Mind Your OSC is due to instability of transferring values via bluetooth set or poor programming performance. I had another chance to test Expressive values using a build in 3D face model within EPOC Control Panel and it seemed to work very good. That let’s me conclude on rather poor Mind Your OSC program performance.

Here is a video showing how actual facial expressions are being translated onto a 3D model. Please mind that the footage for both 3D model and actual person are filmed separately and are not in synch. But it gives a rough idea that software actually picks up different face expressions such as eye blinking, eyebrow frowns and mouth smiling:

[vimeo.vimeo.com/38408362]

I also tried a little app inside EPOC Control Panel which allows me to program different facial expressions to certain key strokes. These key strokes are global functions which can be picked up by any text related software and to show it i used a simple Text Editor:

[vimeo.vimeo.com/38561789]

This opportunity to use different programmable assets allows me to expand on device’s usability. I am intended to use Mind Your OSC as a main contributor for value parsing and detecting mood changes while EPOC Control Panel will allow me to map different facial expressions and assign some commands using software which will detect keystroke input. Both programs are supplementing each other and by using them simultaneously i will get more feedback coming from headset into my chosen software.

Advertisements

TEST PROGRAMME

TEST PROGRAMME is specifically designed by myself in order to conduct multiple tests with EPOC headset in order to achieve understanding about how this device works and how i can succeed in creating the final installation.

TEST PROGRAMME is a set amount of different tests which will be conducted with each individual using Emotiv headset. Each test will be recorded on specifically designed chart. I will use charts to compare the results and withdraw conclusive elements necessary for developing aesthetics of my end product. It will also help me to understand the specifics of the device and how i can utilise certain body exercises to boost brain activity and get most of the feedback.

Firstly, i will look at the software i will be using with EPOC headset. This free app can be obtained from Emotiv website by clicking here. This application is for Windows operating system only. Here is the screenshot of how it looks and what it does:


As one can see from the picture above it does measure different emotional states and simple gestures. It is slightly different from traditional EEG devices which pick up different waves describing different states of the brain. This application is preprogrammed to pick up specific state of emotion. It seems to me very limited in depicting different emotional states, but maybe it because it is reduced to very basic brain patterns. Other half of the readings are purely based on face muscle movements and can obtain only two vales. It is either ticked or not. Those values can be used like a switch ‘on’ and ‘off’.

This application limits me as a user. I have only bought the headset and therefore my possibilities to use this device is reduced purely by amount of applications designed for it. A developers edition was too costly for me so I knew about the consequences buying just a headset on it’s own. There are other softwares developed for this device and i am intended to use as many as possible in order to get maximum data out of this headset and then interpret this data in visualising softwares.

I will start with this app first and see how it proceeds.

The main ideas for my TEST PROGRAMME I will drawn from medical tests performed with traditional EEG devices and from an OSC application I mentioned above.

When EEG is performed at the hospital the procedure may vary according to the technology used and type of diagnosis doctors are after. But most of the time EEG records brain in different emotional states. There are tests when patients are held in dark room for some time cutting out any unnecessary stimulus and recording brain activity at that state and there are times when patient is subjected to some tests with flashing light to investigate the chances of epilepsy, etc. I also found an interesting experiment where ‘people listen to tapes of spoken Danish played forward and backward. When tape is played forward, the listening and language centers are activated along with other relevant centres in order to understand the message in what is being said. But when the tape is played backward, the entire brain is activated!’ (Nørretranders, p.117)
This particular test shows how brain works to decode the information when it is familiar and unfamiliar. It is very fascinating to see how brain reacts when receiving unknown source of information, it does engage completely to solve the task. I can imagine this happens when a child is learning about this world from scratch, when nothing is stored in his memory and it has to comprehend and create a meaning/understanding to what it faces and pack this information so that it can be accessed later. As we grow older we establish rigid comprehension about this world and how it runs. We keep developing meaning in more deeper levels of consciousness through philosophy and religion but in terms of reason it seems perfectly clear to us. When we listen to something being played backwards it will cause immediate reaction of knowing that it is totally impossible to comprehend, but it doesn’t mean that brain is thinking the same. It will engage to find the meaning to what is being played and when it doesn’t happen it will keep working on it.

For me this moment of ‘learning’ new things is the most important aspect.

TEST PROGRAMME so far:

  1. Stay calm in dark, silent room. No stimuli.
  2. Stay calm in lit room.
  3. Let the subject answer these questions:
    -What is your name and surname?
    -How old are you?
    -What year is it now?
    -Where are we?
    -What colour is sugar?
  4. Let the subject answer these questions:
    -What is the taste of the concrete?
    -How doest it feel like to have a tail?
    -Which song is it? (Play a song backwards)
  5. Let the subject imagine these things being silent with eyes closed:
    -Swimming in the ocean at night. (after describe the feelings in words)
    -Falling from the 100th floor.
    -Sitting in the field of flowers.
  6. Let the subject imagine these things being silent with eyes closed:
    -Being a huge colourful giant with long legs and flying
    -Being in the horror story and fighting real zombies
    -Pure bright light, nothing else
  7. Ask the subject to do mathematical calculations on paper:
    :: 2+2=
    :: 32-12=
    :: 40:7=
    :: (1378:3) + (6382:12)=
  8. Ask the subject to sing a favourite song.
  9. Playback some very popular song.
  10. Playback recorded song which subject just sang.
  11. Ask subject to draw:
    -Apple
    -Death
    -Self-portrait
  12. Stand up, close eyes and fall back. (someone is catching the person from back)
  13. Dance to some catchy tune.
  14. Show the subject:
    -Disturbing image
    -Sexually arousing image of opposite sex or same sex depending on orientation
    -Funny image
* This test will be conducted with people aged 18+. I want to exclude younger children due to the characteristics of these questions as well as the necessity of parental permission. This test is designed by myself and the results will be used for my personal project only. I will ask participating subjects to sign a disclaimer that they are comfortable with questions and none of those questions/exercises cause any personal or sexual offence. I expect my chosen subjects will be open-minded and participate in this project based on interest and self-initiation to be part of it.

————

REFERENCES:

Nørretranders, T., 1999. The User Illusion. London: Penguin Books.

Brain activity and how to measure it.

Our brain consists of billions of cells called neurons. They use electric impulses when communicating and therefore emitting continuous electrical activity inside the brain. Here is a single nerve cell from the cerebellum of the brain:


[image online] Available at: <http://www.sandgrains.com/Microscope-Services-3D-Imaging-types.html&gt; [Accessed 28 February 2012].


[image online] Available at: <http://connectomethebook.com/?portfolio=golgi-stained-neurons-2&gt; [Accessed 28 February 2012].

Guyton (1971, pp.511-512) states that  ‘both the intensity and patterns of this electrical activity are determined to a great extent by the overall excitation of the brain resulting from functions in the reticular activation system. … Much of the time, the brain waves are irregular and no general pattern can be discerned in the EEG. However, at other times, distinct patterns do appear. Some of these are characteristic of specific abnormalities of the brain, such as epilepsy. Others occur even in normal persons and can be classified into alpha, beta, theta, and delta waves.‘ Here is an image showing these waves:

[image online] Available at: <http://www.realmagick.com/alpha-wave/&gt; [Accessed 28 February 2012].

Alpha waves are rhythmic waves which are found in the EEG’s of almost all normal persons when they are awake in a quiet, resting state of cerebration. During sleep the alpha waves disappear entirely, and when the awake person’s attention is directed to some specific type of mental activity, the alpha waves are replaced by asynchronous higher frequency but lower voltage waves.’ (Guyton, pp.512)
This image demonstrates it efficiently:


Ibid.


Beta waves usually would appear during activation of the central nervous system. Maximum mind power. All five external senses, logical mind, memory from the five senses & logical thinking.

Theta waves will occur during emotional stress such as disappointment and frustration. Also deep meditation. Deep inward thought. This is associated with life-like imagination. High state of mental concentration. A magical mind. Internal pictures / visualisation. Intuition, inner guidance. Access to unconscious material. Dreaming.

Delta waves will occur mainly in deep sleep, in infancy or in serious organic brain disease. Deep dreamless sleep. Deep relaxation. State of oneness, whole body feeling.

Apart from EEG method of displaying brain activity in form of waves there are several other technologies which allows people to measure and display the activity of human brain. Functional magnetic resonance imaging or functional MRI (fMRI) us another way to visualise activity of neurons inside the brain by detecting the blood changes. It uses a very complex method to measure oxygen levels in blood cells after being treated by noise. These changes are displayed by different colours:


[image online] Available at: <http://web.mit.edu/newsoffice/2011/brain-language-0301.html&gt; [Accessed 28 February 2012].

This technique is used more in research rather than clinical treatments but it can also be used to investigate ill patients. ‘Physicians use fMRI to assess how risky brain surgery or similar invasive treatment is for a patient and to learn how a normal, diseased or injured brain is functioning. They map the brain with fMRI to identify regions linked to critical functions such as speaking, moving, sensing, or planning. This is useful to plan for surgery and radiation therapy of the brain. Clinicians also use fMRI to anatomically map the brain and detect the effects of tumors, stroke, head and brain injury, or diseases such as Alzheimer’s.’ (wikipedia article on fMRI)

Diffusion spectrum imaging (DSI)


[image online] Available at: <http://www.humanconnectome.org/about/project/pulse-sequences.html&gt; [Accessed 28 February 2012].


[image online] Available at: <http://www.nibib.nih.gov/Research/Highlights/Connectome&gt; [Accessed 28 February 2012].
————–

REFERENCES:

Guyton, A. C., 1971. Basic Human Physiology: Normal Function and Mechanisms of Disease. London: W. B. Saunders Company.

Huddleston N., 2008. Brain Wave States & How To Access Them. [online] Available at: <http://synthesislearning.com/article/brwav.htm&gt; [Accessed 28 February 2012].

Wikipedia, 2012. Functional magnetic resonance imaging. [online] Available at: <http://en.wikipedia.org/wiki/Functional_magnetic_resonance_imaging&gt; [Accessed 28 February 2012].

FURTHER READING:

  1. http://www.doctorhugo.org/index.html#3.
  2. http://connectomethebook.com/

Electroencephalography or EEG

Electroencephalography [EEG] or a Brain wave test is a method of recording electrical activity around the skull surface. The brain neurones emit electric charge which can be measured with specific EEG devices. Usually these devices use electrodes which can pick up these electrical currents and with the help of special technology can be interpreted on the screen for monitoring. Some electrodes are soaked in special solution like saline (for better conductivity) or, in case of metal ones, they are smeared with some special paste and stick to the skull surface. Some EEG devices look like a mesh and are placed right on the scalp to secure their locations.

Here are some images of EEG readers i found on the internet:


[image online] Available at: <http://brainfunctionss.com/eeg/&gt; [Accessed 27 February 2012].


[image online] Available at: <http://people.brandeis.edu/~sekuler/eegERP.html&gt; [Accessed 27 February 2012].


[image online] Available at: <http://www.autismspot.com/blog/6-Hour-EEG&gt; [Accessed 27 February 2012].


[image online] Available at: <http://www.thecnnh.org/eeg.html&gt; [Accessed 27 February 2012].

Judging from the images there are wide range of EEG readers available nowadays, mainly equipment found in hospitals and research centres. Some use more, some less, electrodes to perform brain readings but mainly it depends on how accurate and specific diagnosis doctors want to make.
Normally such devices will be custom made with accompanying software developed to work with specific device. They are very expensive to buy due to production costs and are very reliable and precise medical tools for diagnosis.

Most frequently EEG is performed to diagnose various illnesses, such as: epilepsy seizures, Alzheimer’s and demetia, brain tumours and infections. Sometimes they are used to track the sleeping pattern and monitor any abnormalities in brain activity while person is at sleep. There are growing amount of research based around this technology and it allows researchers to understand more about processes of the brain in non-invasive way. Sophisticated monitoring systems allow researchers to conduct extensive analysis.

The EEG device I have obtained is less complex and more suitable for non-medical researchers. It is called EPOC Neuroheadset:


[image online] Available at: <http://www.thehindubusinessline.com/features/smartbuy/other-gadgets/article2106394.ece&gt; [Accessed 27 February 2012].

This is first commercially available EEG reader device at reasonable cost. It is limited to the extent of the amount of applications other people have developed to interpret data streamed from this particular device. Even though it uses a bluetooth protocol to transmit values, it is heavily encrypted to avoid any free use of this device. One must purchase a development kit and licence in order to expand and customise on it’s usability.
It is not possible to conduct clinical tests on people but has a wide array of utilisation in experimental/creative approach. It is possible to record brain patterns and apply them onto some specific tasks which is totally impossible with medical EEG readers.
EPOC allows to interpret brain readings in a creative way.

———–

Further reading:

  1. http://en.wikipedia.org/wiki/Electroencephalography
  2. http://emotiv.com/

Learning Agreement for EMP///

Synopsis of study

For my Extended Major Project I would like to create a Bio-interactive installation art work which explores the activity of the human brain and translates it into abstract visual format defined by my artistic interpretation. I will achieve the transition between thought and visual output by using a specific modern technology which enables access to the electric impulses happening in the brain in process of thinking and encodes these readings in digital values which can be interpreted with the help of other technology softwares. I will be looking into using OSC signal protocol to transit digital values coming out of the brain reading device and feeding them into visual generative software. My main input in this process will be creating visual aesthetics and justifying chosen visual style. Because this sort of project can lead to numerous outcomes and can be interpreted on a wide range of stylistics, I have to identify the personalization of this piece through the implication of subjective matter and experiences. This piece will be representation of how I interpret different feelings in terms of colors and shapes and I will let the user alter the visual scenery based on his/her individual input.

The reason why I have chosen working in such medium is due to the fact that modern technologies tend to infuse contemporary art pieces more and more. This is my attempt to fashion original visual piece using commercially available latest technology in Electroencephalography (EEG). EEG technology records electrical activity of the brain and can be performed by placing a special device on the person’s scalp. With this project I intend to pioneer inventive ways of exploring human brain activity and display this data into visually perceivable product. I also believe that this is a new unexplored realm in visual arts where human body becomes a main contributor. Up till now, apart from computer code generated visuals, human heritage in art has been fused with intention and purpose. With my project I open a new page in the history of creating visual information where creator is a raw, unprocessed by our awareness, data streaming out of our brains. This information will be translated into practiced means of communication, the image.

With this project I will be researching a field of great interest to me and I expect to develop further studies on this subject in my Master’s degree. Being artistic person it will allow me to approach such project from very creative angle and interpret scientific assets from the perspective of the artist. I will be able to draw different patterns of the brain activity and compare them to different users. I will be exploring emotions of people participating in this project and evaluating the feedback happening on the screen. My main interest lies in bridging technology and visual perception and create a new approach in generating imagery with unknown tools.

/////

Here you can download a full version of the Learning Agreement.

Evaluation of the
Specialist Project

BH Digital Media Production

Ilze Kavi Briede
2011

Through the course of my Specialist Project I learned a new set of skills preparing me for the final stage in my BA degree education, namely Extended Major Project. I had to follow strict time management planning in order to meet every small deadline and ensure that my work was up to date and fully progressing. At the start of this project I created a rather small but concise planning sheet with tasks for each week and I succeeded to complete every single one. My online blog is a reflection of the development stages while my sketchbook shows more of my research work and ideas sketching.

For my Specialist Project I have produced a prototype version of a stage installation art piece in conjunction with sound. I have created a 3-dimensional sculptural object consisting of a few pieces suspended in the air and presented using a set design approach:

The idea behind this piece is to create an unusual and artistic solution for projections with prospects to be used on a larger scale for music bands, DJs or an orchestra. This piece should be viewed as a form of art with possibilities to be incorporated in different music events as a visual element. My approach while making it has it’s roots in a necessity to provide a new platform in visual art, using digital tools and to push the boundaries of already existing practice in this medium.

This project consists of a few elements: form, technology and generative visuals. The form is an aspect of the face, technology is explored by use of sensors and the generative side is supported by different software operating in real time and processing data received from sensors. The outcome is presented in the form of a digital canvas mapped onto pieces of a face.

I set out to create a sound responsive visual sculpture. My work consisted of three main stages: exploring the idea of the face, materialising it and working on the style and technicalities of the projections.

Exploring the idea about the face was deepened along with my research subject for my essay. I was looking into aspects of human cognitive faculties of recognising a face, the characteristics of human behaviour in perceiving faces as well as my extensive interest and many years of praxis in drawing faces. All of this contributed to the subject. I came up with very abstract shapes purposely to trigger new sensations in people’s perception which is the underlying purpose of my work. My research was geared towards understanding the science behind the human perception of faces and how I could use these findings to influence the design of my set piece.

Turning my idea into a physical object went quite smoothly. I started off turning the final design into vector graphics and printed it with a laser cutter onto a perspex sheet. Then I added to each piece a relief with the help of a heat gun and sprayed it white to maximise the quality of projection on them. I was very lucky to observe a friend of mine who is a set design student in Costume department and received some valuable advice on how to build a platform of set design of my own. It was my first attempt and I think I have done very well. The idea to use this approach came from looking at different set designs and how clearly they allow a spectator envision a real scene in real theatre set up just by looking at the smaller version of it. This display technique is perfect in being professional and at the same time suitable for showing my idea. I used several sheets of thick black foam-board to build the main structure and PVA glue and metal pins to secure the walls and small copper poles from which I hang the various pieces that make up the face.

The last stage involved aesthetics of the digital generative visuals and programming. I am using four programs which run simultaneously and communicate to each other to satisfy the needs of my project. The first program (Arduino along with the micro controller board) reads analog input (sensors), parses data to a second piece of software called Processing which is generating visual graphics in real time and taking incoming readings from Arduino. The third piece of software called Syphon grabs the screen of visuals and makes it accessible to the fourth piece of software called VPT (Visual Projection Tool) to project these graphical images onto mapped surfaces of the face. VPT is also used to mask the visuals so that they only show up on the face.

The only difficulty I have faced is working with sensors. In order to ensure smooth workflow with electronic components I have to understand the basics of circuit boards and specifications for each sensor. I managed to get the analog readings and implement them into my work but I had difficulty dealing with some errors in readings when the values started to fluctuate on a regular pattern and I couldn’t find the real cause for it. I tried to troubleshoot and rewire the circuit and test for the source of the error and it seemed to be pointing to the arduino board or sensors where my knowledge isn’t broad enough to solve. To overcome this I altered the arduino code to smooth the flow of data coming in to the computer in order to stop the data fluctuating so erratically.

The stylistics of the visuals is minimal and based on simple geometric forms. The reason why I chose working with such shapes is because of inspiration from the artist Bridget Louise Riley and various works from early Op Art (Optical Art) which is rooted in Bauhaus. I found it very exciting and challenging to find the right emphasis on the form using geometrical shapes. In Op Art lines are usually presented on flat surfaces and by specific arrangement they might suggest an optical illusion of 3-dimensions. My work is 3-dimensional already and the reason why I project lines is not to contribute to its shape but to do the opposite. I alter the shape of the line. Straight lines projected on relief surface become curved and it is an optical lie.

From all three stages of my project I found the work with sensors the most challenging. I concluded that in order to achieve an organic union between the sound and the visual sensors had to play a very significant role. It is essential to pair the right sensor with the right musical instrument in order to best represent the instrument through the visuals. Depending upon which sensor is used the musical instrument can be read and interpreted differently by the computer in a number of ways. I found that working with vibration and sound sensors along with a drum kit was not enough. If I will be working with a music band or an orchestra which uses many diverse instruments I will need to have an extensive knowledge in sensor technologies and what is available on the market as well as how practical and reliable they are. If I reflect on my work with the sensors I must say that is is one of the hardest medium I have chosen. I found it hard finding the right sensor to fulfil my exact requirements. This excites me because they are like a representation of the physical world encoded in numbers and it is up to us how we interpret it.

The skills I have learned throughout this unit will allow me to pursue my career as an installation artist. With this piece I am challenging myself to find a form of collaboration with musician or a band which can push my work to a higher level. It will allow me to sell my work and represent it to creative industries as a product with strong artistic value. But to gain such recognition I have to work very hard in order to know the technical side of it. I can see myself working as an artistic director with strong conceptual approach backed up with understanding in technologies and range of possibilities they can be applied to.