Annotated Bibliographies

Moodi93

contentserver_3.pdf

Home >English homework help >Annotated Bibliographies

DISORDERS OF CONSCIOUSNESS

A Brain–Computer Interface Controlled Auditory Event-Related Potential (P300) Spelling System for Locked-In Patients

Andrea Kübler,a,b Adrian Furdea,b,c Sebastian Halder,b

Eva Maria Hammer,b Femke Nijboer,b

and Boris Kotchoubeyb

aClinical and Health Psychology Research Centre, School of Human and Life Sciences, Roehampton University, London, UK

bInstitute of Medical Psychology and Behavioural Neurobiology, University of Tübingen, Tübingen, Germany

cGraduate Institute of Technology, University of Arkansas at Little Rock, Little Rock, Arkansas 72204, USA

Using brain–computer interfaces (BCI) humans can select letters or other targets on a computer screen without any muscular involvement. An intensively investigated kind of BCI is based on the recording of visual event-related brain potentials (ERP). However, some severely paralyzed patients who need a BCI for communication have impaired vision or lack control of gaze movement, thus making a BCI depending on visual input no longer feasible. In an effort to render the ERP–BCI usable for this group of patients, the ERP–BCI was adapted to auditory stimulation. Letters of the alphabet were assigned to cells in a 5 × 5 matrix. Rows of the matrix were coded with numbers 1 to 5, and columns with numbers 6 to 10, and the numbers were presented auditorily. To select a letter, users had to first select the row and then the column containing the desired letter. Four severely paralyzed patients in the end-stage of a neurodegenerative disease were examined. All patients performed above chance level. Spelling accuracy was significantly lower with the auditory system as compared with a similar visual system. Patients reported difficulties in concentrating on the task when presented with the auditory system. In future studies, the auditory ERP–BCI should be adjusted by taking into consideration specific features of severely paralyzed patients, such as reduced attention span. This adjustment in combination with more intensive training will show whether an auditory ERP–BCI can become an option for visually impaired patients.

Key words: locked-in syndrome; brain-computer interface; event-related potentials; P300; auditory; amyotrophic lateral sclerosis; communication

Introduction

Brain–computer interfaces (BCI) are devices that directly connect human (or animal) brain activity with artificial effectors, such as commu- nication programs or neuroprostheses. They al-

Address for correspondence: P.D. Dr. Andrea Kübler, Ph.D., Clinical and Health Psychology Research Centre, School of Human and Life Sci- ences, Roehampton University, Whitelands College, Holybourne Avenue, London SW15 4JD, UK. [email protected]

low patients with lost movement ability after injury or disease to replace a part of the lost motor function (for recent reviews, see Refs. 1–3). The most commonly used neural signal for the purpose of BCI control is electroen- cephalography (EEG).4 The EEG signals are acquired, digitized, filtered, and transformed by various algorithms into an output signal that controls an effector. Depending on the type of BCI, users are required to either learn regula- tion of a specific brain response, for example,

Kübler et al.: BCI Auditory Event-Related Potential Spelling System 91

slow cortical potentials (SCP)5,6 or sensorimo- tor rhythms (SMR),7,8 or to attend to sensory stimulation. This stimulation elicits a character- istic EEG pattern referred to as event-related potential (ERP). In particular, the already exist- ing BCI use steady state–evoked potentials9,10

or the late ERP component P300.11,12 The lat- ter is a broad centroparietal electropositive de- flection usually observed in response to target stimuli (in particular, rare targets) starting about 300 ms poststimulus.

One target group for BCI use is patients who lose the ability to communicate due to progres- sive neurodegenerative disease, such as amy- otrophic lateral sclerosis (ALS). This disease of the first and second motor neurons is character- ized by a progressive motor paralysis, with no curative treatment available. If life-sustaining treatment is withheld or withdrawn, death due to respiratory failure occurs, on average, three to five years after the first symptoms. In several studies it has been shown that patients with ALS can control an effector using different kinds of BCI.5,6,13 However, patients examined in those studies retained at least minimal control of gaze movements. Artificially (via tracheostoma) ven- tilated patients can lose the ability to control ocular muscle activity, which may further re- sult in the decline of visual attention14,15; such completely paralyzed (including gaze paralysis) patients are referred to as being in a complete locked-in state (CLIS).3,14

Recently, efforts have been undertaken to use nonvisual stimulation for BCI control. Audi- tory feedback of SCP and SMR was incorpo- rated into BCI.16,17 Both the SMR– and the SCP–BCI require learning to regulate a brain response that is achieved by means of neuro- feedback. In both studies, the task, target EEG response, and the results were presented and fed back to the participants via auditory stim- uli. The data showed that healthy participants need more time to achieve BCI control with auditory as compared to visual feedback. Vi- brotactile feedback of SMR was realized by Cincotti and colleagues,18 and selection accu- racies were comparable to those achieved with

visual presentation. Müller-Putz and colleagues used vibratory stimulation of left and right hand finger tips to elicit somatosensory steady state– evoked potentials (SSSEP).19 In each trial, both index fingers were stimulated simultaneously at different frequencies, and participants were in- structed by arrows on a computer screen to which finger they should pay attention. Hill and colleagues20 used ERP to auditory stim- uli consisting of 50-ms square-wave beeps with different frequencies, grouped in two different streams, one presented to the left ear and the other to the right ear. Following an oddball paradigm,21 each stream contained rare target and frequent nontarget beeps. The users were able to direct conscious attention to one of the stimulus streams, and the BCI could detect the stream and the target the user attended to.

Almost all of the just-summarized studies in- cluded healthy volunteers only. However, per- formance of patients with severe neurological disease cannot be inferred from healthy sub- jects.1 In a study that included ALS patients, Sellers and colleagues used an auditory odd- ball paradigm in which stimuli were the words “yes,” “no,” “pass,”and “end” presented in a random sequence.12 The participants’ task was to count the number of times the target (ei- ther yes or no) was presented. The authors showed that with a target probability of 25%, a P300 response to these stimuli was reliably de- tected, and that the response remained stable in healthy volunteers as well as in ALS patients over a period of 10 sessions, although in patients selection accuracy was lower.

Some studies have found alteration in la- tency of auditory and somatosensory evoked potentials in ALS patients,22,23 whereas oth- ers have not.24,25 Cosi and colleagues investi- gated somatosensory-evoked potentials (SEP) in ALS patients and found a delay of cer- vical and cortical SEP, namely the N13 and subsequent components, accompanied by de- crease of amplitude, which suggested a slowing of conduction along the central sensory path- ways.23 Alterations in auditory processing were reported by Pekkonen and colleagues, who

92 Annals of the New York Academy of Sciences

TABLE 1. Background Data of Patients

Artificial

Participants Age Sex Time since diagnosis Nutrition Ventilation Degree of Impairment

1 39 F 10 years No No Major 2 51 F 3 years Yes Yes LIS 3 49 F 4 years Yes Yes LIS 4 39 M 8 years Yes Yes LIS/CLIS

ABBREVIATIONS: LIS = locked-in state; CLIS = complete locked-in state; M = male; F = female.

used magnetoencephalography.22 The authors suggest that these abnormalities in stimulus detection and subsequent memory-based com- parison processes in ALS patients, with ad- vanced symptoms, might be due to hyperac- tivity of the cortical excitatory glutamatergic system. Palma and colleagues also found a re- duction of N13 amplitude and a prolonged P22 latency in SEP recordings, but on the ba- sis of a source analysis they concluded that these changes do not implicate the involve- ment of somatosensory pathways.24 Gil and colleagues compared late ERP components in healthy subjects and patients with ALS and did not find significant differences between patients with ALS and a control group in the amplitudes of N100, P200, N200, and P300.25 Taken to- gether, the results of studies on auditory and SEP and the P300 component in ALS patients are ambiguous. We assume that at least in most patients auditory and somatosensory pathways remain functional, offering a possible alterna- tive for stimulus and feedback presentation for BCI control.

All the presented studies with auditory BCI provided the user with the possibility to com- municate a yes or no response. To allow users a more flexible communication, we recently de- veloped an oddball-based spelling system with auditory stimulation, referred to as auditory ERP–BCI, and tested it in healthy volunteers.26

Following the setup of the visual ERP–BCI, as described initially by Farwell and Donchin and later by Sellers and Donchin, users were pre- sented with a 5 × 5 matrix of characters.11,12 Each row and column was coded with numbers

1 to 5 and 6 to 10, respectively, and numbers were presented acoustically. The users’ task was to spell the English word BRAINPOWER. To do this, they had to attend first to the number of the row, and after row selection, to the num- ber of the column (see methods section for de- tailed description). Auditory spelling accuracy of 14 healthy volunteers was worse, letter se- lection needed more time, and peak latency of the ERP was delayed as compared to the visual ERP–BCI. Nevertheless, 9 of the 14 partici- pants achieved spelling accuracies above 70%, which is the lower limit for spelling with an assistant device for communication.6,27

In the current study, we explored whether patients in the late stages of ALS in the locked- in state (LIS), or in the LIS with periods of CLIS, were able to spell words using the audi- tory ERP–BCI.26

Methods

Participants

Background data of patients are listed in Table 1. Three patients were diagnosed with the sporadic and one with the hereditary form of ALS; all patients had the spinal form of ALS. Three patients lived at home and were cared for either by family members or professional caregivers; one patient lived in a nursing home. Three patients lived at a maximum distance of 160 km from our institute and had been trained with the visual P300–BCI since the be- ginning of 2006.28 BCI training was conducted

Kübler et al.: BCI Auditory Event-Related Potential Spelling System 93

exclusively at the patients’ home at a maximum rate of two visits per week. Patient 1’s major im- pairment included tetraplegia and almost in- comprehensible speech.14 Residual speech and head movement were used for communica- tion. Patients 2 and 3 were artificially ven- tilated via tracheostoma and fed via PEG. Communication was established using residual movement of facial muscles and head move- ments. All three patients additionally used PC- based programs for assistant communication and for surfing the Internet. Patient 4 lived at a distance of 300 km, and therefore was trained in weekly blocks, twice a year since early 2006. Gradually decreasing residual muscular control of the right thumb and of the right corner of the mouth were his only means of communication. To the moment of writing this report, he has lost thumb movement and mouth twitches. Some days the patient can still use slight horizontal eye movement to indicate yes or no.

Procedure

Patients were seated in their wheelchairs (pa- tients 1, 2, and 3) or bed (patient 4). Audi- tory stimuli were presented with two separate loudspeakers, positioned in front of the par- ticipants at a distance of approximately 1.2 m. The experimental session started with a calibra- tion run to determine classification weights (see Data Analysis); within this run the participants did not receive any feedback. Data of the fol- lowing runs were collected in the copy spelling mode of the spelling system. In this mode, the software prompts participants to spell a given word character by character, and feedback of the selected letter is provided after each selec- tion.6 We used a 5 × 5 spelling matrix con- taining all letters of the alphabet, except the letter Z.26 The task of each run was to spell a word preset by the trainer. Patients had to select each letter of the word by attending to acous- tically presented number words. A short break followed each run; its duration was determined by the patients.

Figure 1. Design of the acoustically presented matrix. Rows were coded with numbers 1–5, and columns with numbers 6–10. Speakers indicate that numbers were presented auditorily by a male voice. The word to be spelled is depicted in the topmost line—in this case WINTER. The letter in parenthesis following the word indicates the letter to be spelled in each specific trial.

Each character’s position in the matrix was coded by two acoustically presented numbers, that is, words spoken by a male voice: one for the row and one for the column (Fig. 1). To select one target letter, the patients had to first attend to the number that coded the row (num- bers 1–5; see Fig. 1) containing the target letter. In the trial following row selection, numbers 6 to 10 were presented, and patients had to attend to the number that coded the column that contained the target letter. After comple- tion of the second trial a letter was selected. To facilitate focusing of attention in each trial, pa- tients were asked to count how often the num- bers referring to the target coordinates were presented. Since counting during number pre- sentation may be difficult, we also instructed patients to have an “aha-effect” whenever they heard the target number. This design consti- tutes an oddball paradigm, because in one trial of stimuli presentations, only one number out of five refers to the target letter. The rare events in the context of the other irrelevant stimuli were shown in healthy participants to elicit a P300-like event-related response.26 In addition to auditory presentation of numbers, a “visual support matrix” was displayed on a monitor to

94 Annals of the New York Academy of Sciences

help the participants to remember the coordi- nates for the target letter, but no visual stimula- tion occurred.26

One trial consisted of 150 stimuli presenta- tions (30 targets, 120 nontargets), thus each au- ditory stimulus was presented 15 times per trial. First, 75 stimuli (15 targets) were presented, in- dicating the numbers that coded the rows; sec- ond, the next 75 stimuli (15 targets) were the numbers coding the columns. Each auditory stimulus lasted 450 ms, followed by an inter- stimulus interval (ISI) of 175 ms. Row codes were separated from column codes by an inter- val of 1875 ms. Total time needed for selection of one character was 1.5 minutes.

The 5 × 5 visual support matrix was displayed on a 19-inch monitor located about 1.2 m in front of a participant to facilitate identifying the coordinates of the target letter; no visual stimulation occurred. Patient 4 could not see the matrix, because he could not focus gaze. However, due to previous training sessions with the visual P300–BCI, he knew the matrix by heart.

Patients 1 to 3 were presented with the task to spell a 5-character word in each of 3 runs. The task of Patient 4 was to spell in 5 runs one 5- character and four 3-character words that were suggested by him. The first run was used for calibration (see Data Analysis).

Data Acquisition

Stimulus presentation and data collection were controlled by the BCI2000 software29

(http://www.bci2000.org/). The EEG was recorded using a tin electrode cap (Electro-Cap International, Inc., Eaton, Ohio, USA) with 16 or 8 channels [(F3), Fz, (F4), (T7), (T8), (C3), Cz, (C4), (Cp3), (Cp4), P3, Pz, P4, PO7, PO8, and Oz; channels in parenthesis were only available with the 16-channel cap] based on the modi- fied 10–20 system of the American Electroen- cephalographic Society.30 Each channel was referenced to the right mastoid and grounded to the left mastoid. The EEG was amplified us- ing a g-tec 16- or 8-channel amplifier, sampled

at 256 Hz band-pass filtered between 0.01 and 30 Hz. Fifty-hertz noise was filtered using the notch filter implemented in the BCI2000 soft- ware. Data processing, storage, and on-line dis- play of the participants’ EEG were conducted using an IBM ThinkPad laptop.

Data Analysis

We used the stepwise-linear discriminant analysis method (SWLDA) for classification and weight generation.11,31 The method, an ex- tension of the Fisher’s Linear Discriminant (FLD), is established as a successful classifica- tion method for EEG data in general, and more recently for BCI data, for which rapid classifica- tion is essential. Previous studies of classification methods demonstrated that SWLDA provides good overall performance in classifying both visual P30011,12,21,31 and auditory P300 within the frame of BCI.26

For each of the recorded channels, 1200-ms poststimulus data segments were extracted and averaged. A moving average filter was then ap- plied to the segments that were further deci- mated by a factor of 20. The feature vector that resulted from concatenating the data segments was then used to train the classifier.

For each of the 16 channels an r2 coefficient was computed to indicate the portion of signal variance that was due to whether the row or column did (oddball/target) or did not contain the desired character (standard). The r2 was calculated on the base of a point–biserial cor- relation between the ERP signal and the task. Peak latency and amplitude were calculated for the electrode with the highest r2 value.

To quantify the ability to select characters, we determined selection accuracy as a percent- age of correctly selected letters as compared with the total number of letters to be spelled in each run. Selection accuracy comprises cor- rect selection of rows and columns. Classifica- tion accuracy was determined as percentage of correctly selected rows or columns per run.

The data obtained using the visual ERP– BCI has already been presented elsewhere.28

Kübler et al.: BCI Auditory Event-Related Potential Spelling System 95

TABLE 2. Results with the Auditory

SA CA Peak amp Peak lat Number of Patient [%] [%] Loc [μV] [ms] Chars Target Spelled Stimuli

1 25 58.33 Cp4 2.9 457.03 12 WINTER WJNJBS 360 Targets total WINTER WSOEOI 1440 Nontargets total

2 0 25 Po8 2.5 578.03 12 WINTER FEUHGH 1800 Total WINTER UKOJJM

3 0 25 Po8 1.68 941.31 12 WINTER RRKSLB WINTER LFUYCB BRAIN LAIEK 510 Targets total UHR IMQ 2040 Non-targets total

4 23.53 41.17 Pz 0.91 574.12 22 BAD BSI 2550 Total MAI NAI EVA ESM

ERP–BCI. SA = selection accuracy; CA = classification accuracy; Loc = electrode position at which peak amplitude and latency were calculated; amp = amplitude; lat = latency; Chars = characters to be spelled

The method is sufficiently described in the lit- erature (e.g., Refs. 12, 32, 33).

Results

Selection and classification accuracies, elec- trode position of peak amplitude, latency, the word to be spelled, selected characters, and the number of presented stimuli per run are pre- sented in Table 2. As can be seen from this table, performance was generally low. Taking into account the design of the auditory ERP– BCI (5 × 5 matrix), the performance at chance level would be 4% for selection accuracy and 20% for classification accuracy. Therefore, pa- tients 2 and 3 performed at, or very close to, chance level. Patient 1 who had the highest peak amplitude and the shortest peak latency, also performed better than the other patients, although her performance was also not high enough to allow meaningful communication, which requires a selection accuracy of above 70%.6 Interestingly, the same patient also had the largest P3 amplitude with the visual ERP– BCI.28 On the other hand, the results of pa- tient 4 indicate that even a low peak amplitude of around 1 μV may be sufficient for above- chance classification.

Our previous data indicate that the same pa- tients were much more successful when using a visual ERP–BCI.28 They achieved selection accuracy of at least 70%. Patients 1–3 commu- nicated messages of considerable length.

Figure 2 shows ERP to auditory (left column) and visual (right column) stimulation averaged across all runs separated according to whether rows and columns did or did not contain the target letter. In both modalities and all patients, the ERP response to target and nontarget stim- uli can be clearly distinguished. However, in three patients (except GR) the configuration of target responses is more distinct in the vi- sual BCI than in the auditory BCI. The size of the target/nontarget difference is lower and the peak latency larger in the auditory than in the visual ERP–BCI.

Discussion

We presented four patients with ALS with a spelling system based on auditory ERP. Two of the patients were in LIS and one was at the border of CLIS. All patients were previ- ously trained with an oddball-based ERP–BCI using visual stimulation and were thus well fa- miliar with this type of BCI. With the visual

96 Annals of the New York Academy of Sciences

Figure 2. Event-related potentials (ERP) to auditory (left) and visual (right) stimulation with the ERP–BCI (brain–computer interface). Solid line depicts averaged EEG responses to targets, and dashed line to nontarget stimuli. In the auditory ERP–BCI for all patients peak amplitudes were lower and peak latencies delayed as compared to the visual ERP–BCI.28

ERP–BCI, all patients achieved spelling ac- curacies above 70%, some even above 90%, clearly showing that the patients could concen- trate on the task that elicited highly classifiable

brain responses to stimulation (for a detailed report of patients 1, 2, and, 3 see Ref. 28).

In the auditory BCI, however, the patients’ performance was very poor. Although it might

Kübler et al.: BCI Auditory Event-Related Potential Spelling System 97

be suggested that any modification in the functioning BCI would result in a negative performance change, the degree of the ob- served impairment calls for additional explana- tion. Healthy control subjects performed only slightly worse in the auditory ERP–BCI than in the visual ERP BCI.26

It should be taken into account that each number word in the coding system used in the auditory BCI referred to five different letters, whereas in the visual BCI, the target letter it- self served as a stimulus. Thus, the auditory system imposed an additional effort for con- stantly maintaining the target number in short- term memory. Three patients stated that it was very difficult for them to focus their attention on the numbers, and that the visual support matrix was absolutely necessary. ERP studies of working memory clearly indicate that an increasing workload results in a delay of P3 peak latency and the decrease of P3 amplitude (for review, see, e.g., Refs. 34 and 35). Accord- ingly, Figure 2 shows that the target/nontarget differences were of a smaller size and of a larger latency for the auditory than for the visual BCI.

Although the amplitudes were decreased in all patients, two of them achieved classifica- tion accuracy quite above chance level, and we may assume that with training and adaptation to auditory stimulation patients’ performance might improve. It is important to note that pa- tient 4 was one of these two patients despite his low peak amplitude. This patient immediately learned the matrix by heart and wanted to con- tinue using the auditory ERP–BCI. Due to the lack of control over eye movement, the patient may have already learned to rely more on audi- tory input during his daily life, a phenomenon also seen in visually impaired or blind people (e.g., Ref. 36).

Kübler and Kotchoubey suggested a hierar- chical approach to the detection of cognition and consciousness using BCI in nonresponsive patients. The approach comprises five steps4,37

(Fig. 3). Step 1: recording of the rest EEG and auditory-evoked potentials to exclude pa-

tients whose dominant EEG frequency is below 4 Hz and those with hearing loss.38 Step 2: pas- sive auditory stimulation paradigms with EEG recording, including auditory oddballs to as- sess preattentive cortical orientation (mismatch negativity), and deeper cortical analysis of the physical stimulus properties (P300); and second, semantic stimulation leading to N400 or P600 components. Step 3: presentation of the same tasks as in Step 2, with additional instructions to pay attention to specific target stimuli. With participants who are able to understand and follow the instruction, this results in a consid- erable increase of the P300 and N400 effects (e.g., Refs. 39 and 40). Step 4: volitional tasks that require the subject to actively imagine one of two actions according to two corresponding auditory stimuli,16,41 or to focus attention on a specific target in a paradigm designed to elicit ERP (as described in this study and Refs. 12 and 26). If a nonresponsive patient performed above chance level in any of these volitional paradigms, it could be inferred on the presence not only of cognitive processing and conscious awareness but also on active volition.4 Step 5: decision making with a BCI. On the most ba- sic level, this would include answering yes/no questions with the BCI. A more sophisticated level would include communication and inter- action with the environment.5,6

The technology of auditory BCI is the nec- essary prerequisite for the use of this approach. Nonresponsive patients may be in a CLIS due to total paralysis of peripheral muscles as a re- sult of degeneration of motor neurons or due to neuronal loss in brain areas required for con- scious attention and awareness, initiation of ac- tion or motor execution, and goal-directed be- havior.4,37 The present data show that we do not yet have an auditory system that would ful- fill the requirements of such patients. Patients who are already deprived of visual input may benefit more of the auditory ERP–BCI than do patients with intact vision. However, the feasi- bility of the auditory ERP–BCI must be con- siderably improved before it becomes an option for patients approaching or being in the CLIS.

98 Annals of the New York Academy of Sciences

Figure 3. Flow chart of the hierarchical approach to the use of brain–computer interface (BCI) for the detection of conscious awareness and cognitive function in completely locked-in patients. Each level involves higher demands on cognitive processing: Step 1: EEG record- ing, exclusion of patients with a rest EEG below 4 Hz; Step 2: Passive stimulation; Step 3: instruction to focus attention on target stimuli in passive-stimulation tasks;. Step 4: presentation of volitional tasks such as a 4-choice oddball paradigm (EEG recording) or mental imagery (fMRI or EEG recording); Step 5: decision making by means of a BCI. On each specific step, performance above chance level is required in order to proceed to the next level of the hierarchy.

Several possible lines of such modification can be proposed. First, numbers can be replaced by more interesting stimuli such as musical tones varying in pitch and timbre. Second, such stim- uli can be presented from different spatial po- sitions corresponding to the position of the sig-

nified unit (e.g., the leftmost position for the leftmost column of the matrix). Third, a BCI based on the self-regulation of the SMR13 also can be adapted to auditory presentation, which might yield better results because the SMR- based BCI can have lower memory load than

Kübler et al.: BCI Auditory Event-Related Potential Spelling System 99

the ERP-based BCI.16,42 Future studies should test these options.

Acknowledgments

This work was supported by the Deutsche Forschungsgemeinschaft (German Research Council) SFB 550/TB 5. The authors are grateful to the patients who participated in the study. We thank Slavica von Hartlieb for her support in data acquisition.

Conflicts of Interest

The authors declare no conflicts of interest.

References

1. Kübler, A., F. Nijboer & N. Birbaumer. 2007. Brain- computer interfaces for communication and motor control—Perspectives on clinical application. In To- ward Brain-Computer Interfacing. G. Dornhege, J.d.R. Millan, T. Hinterberger, et al., Eds.: 1–25. MIT Press. Cambridge, MA.

2. Lebedev, M.A. & A.M. Nicolelis. 2006. Brain- machine interfaces: past, present and future. Trends Neurosci. 29: 536–546.

3. Birbaumer, N. 2006. Breaking the silence: brain- computer interfaces (BCI) for communication and motor control. Psychophysiology 43: 517–532.

4. Kübler, A. & B. Kotchoubey. 2007. Brain-computer interfaces in the continuum of consciousness. Curr. Opin. Neurol. 20: 643–649.

5. Birbaumer, N., N. Ghanayim, T. Hinterberger, et al. 1999. A spelling device for the paralysed. Nature 398: 297–298.

6. Kübler, A., N. Neumann, J. Kaiser, et al. 2001. Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. Arch. Phys. Med. Rehabil. 82: 1533–1539.

7. Neuper, C., R. Scherer, M. Reiner & G. Pfurtscheller. 2005. Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res. Cogn. Brain Res. 25: 668– 677.

8. McFarland, D.J., A.W. Sarnacki, M.T. Vaughan & R.J. Wolpaw. 2005. Brain-computer interface (BCI) operation: signal and noise during early training ses- sions. Clin. Neurophysiol. 116: 56–62.

9. Gao, X., D. Xu, M. Cheng & S. Gao. 2003. A BCI-based environmental controller for the motion-

disabled. IEEE Trans. Neural Syst. Rehabil. Eng. 11: 137–140.

10. Müller-Putz, G.R., R. Scherer, C. Brauneis & G. Pfurtscheller. 2005. Steady-state visual evoked poten- tial (SSVEP)-based communication: impact of har- monic frequency components J. Neural. Eng. 2: 123– 130.

11. Farwell, L.A. & E. Donchin. 1988. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70: 512–523.

12. Sellers, E.W. & E. Donchin. 2006. A P300-based brain-computer interface: initial tests by ALS pa- tients. Clin. Neurophysiol. 117: 538–548.

13. Kübler, A., F. Nijboer, J. Mellinger, et al. 2005. Pa- tients with ALS can use sensorimotor rhythms to op- erate a brain-computer interface. Neurology 64: 1775– 1777.

14. Kübler, A. & N. Birbaumer. 2008. Brain–computer interfaces and communication in paralysis: extinc- tion of goal directed thinking in paralysis? Clin. Neurophysiol. 19: 2658–2666.

15. Hayashi, H. & S. Kato. 1989. Total manifestations of amyotrophic lateral sclerosis: ALS in the totally locked-in state. J. Neurol. Sci. 93: 19–35.

16. Nijboer, F., A. Furdea, I. Gunst, et al. 2008. An audi- tory brain-computer interface (BCI). J. Neurosci. Meth- ods 167: 43–50.

17. Hinterberger, T., N. Neumann, M. Pham, et al. 2004. A multimodal brain-based feedback and communi- cation system. Exp. Brain Res. 154: 521–526.

18. Cincotti, F., L. Kauhanen, F. Aloise, et al. Vibrotac- tile feedback of brain-computer interface operation. Comput. Intell. Neurosci. 2007: 48937.

19. Müller-Putz, G.R., R. Scherer, C. Neuper & G. Pfurtscheller. 2006. Steady-state somatosensory evoked potentials: suitable brain signals for brain- computer interfaces? IEEE Trans. Neural Syst. Rehabil. Eng. 14: 30–37.

20. Hill, N.J., N.T. Lal, K. Bierig, et al. 2005. An auditory paradigm for brain-computer interfaces. In Advances in Neural Information Processing Systems. 569–576. MIT Press. Cambridge, MA.

21. Fabiani, M., G. Gratton, D. Karis & E. Donchin. 1987. Definition, identification, and reliability of measurement of the P300 component of the event- related brain potential. Adv. Psychophysiol. 2: 1–78.

22. Pekkonen, E., D. Osipova & H. Laaksovirta. 2004. Magnetoencephalographic evidence of abnormal auditory processing in amyotrophic lateral sclerosis with bulbar signs. Clin. Neurophys. 115: 309–315.

23. Cosi, V.M., M. Poloni, L. Mazzini & R. Calieco. 1984. Somatosensory evoked potentials in amy- otrophic lateral sclerosis. J. Neurol. Neurosurg. Psychiatry 47: 857–861.

100 Annals of the New York Academy of Sciences

24. Palma, V.M., M. Guadagnino, V. Brescia Morra & G. Nolfe. 1993. Multimodality evoked potentials in sporadic amyotrophic lateral sclerosis: a statistical approach. Electroencephalogr. Clin. Neurophysiol. 33: 167–171.

25. Gil, R., J.-P. Neau, M. Dary-Auriol, et al. 1995. Event-related auditory evoked potentials and amy- otrophic lateral sclerosis. Arch. Neurol. 52: 890–896.

26. Furdea, A., S. Halder, D. Bross, et al. An auditory oddball (P300) spelling system for brain-computer interfaces (BCI). Psychophysiology. In press.

27. Choularton, S. & R. Dale. 2004. User responses to speech recognition errors: consistency of behaviour across domains. In: 10th Australian International Confer- ence on Speech Science & Technology. Macquire Univer- sity, Sydney, Australia: Australian Speech Science & Technology Association Inc.

28. Nijboer, F., E.W. Sellers, J. Mellinger, et al. 2008. A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clin. Neurophysiol. 119: 1909–1916.

29. Schalk, G., D.J. McFarland, T. Hinterberger, et al. 2004. BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51: 1034–1043.

30. Sharbrough, F., E.G. Chatrian, P.R. Lesser, et al. 1991. American Electroencephalographic Society guidelines for standard electrode position nomencla- ture. J. Clin. Neurophysiol. 8: 200–202.

31. Donchin, E., M.K. Spencer & R. Wijesinghe. 2000. The mental prosthesis: assessing the speed of a P300- based brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 8: 174–179.

32. Sellers, E.W., J.D. Krusienski, D. McFarland, et al. 2006. A P300 event-related potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance. Biol. Psychiatry 73: 242–252.

33. Sellers, E.W., A. Kübler & E. Donchin. 2006. Brain- computer interface research at the University of

South Florida cognitive psychophysiology labora- tory: The P300 Speller. IEEE Trans. Neural Syst. Reha- bil. Eng. 14: 221–224.

34. Kotchoubey, B., J. S. Jordan, B. Grözinger, et al. 1996. Event-related brain potentials in a varied-set mem- ory search task: a reconsideration. Psychophysiology 33: 530–540.

35. Verleger, R. 1997. On the utility of P3 latency as an index of mental chronometry. Psychophysiology 34: 131–156.

36. Després, O., V. Candas & A. Dufour. 2005. The ex- tent of visual deficit and auditory spatial compen- sation: evidence from self-positioning from auditory cues. Brain Res. Cogn. Brain Res. 23: 444–447.

37. Kübler, A. Brain-computer interfaces for com- munication in paralysed patients and implications for disorders of consciousness. In The Neurology of Consciousness—Cognitive Neuroscience and Neuropathology. G. Tononi & S. Laureys, Eds.: Elsevier. Amsterdam. Pp. 217–233.

38. Kotchoubey, B., S. Lang, G. Mezger, et al. 2005. Information processing in severe disorders of con- sciousness: vegetative state and minimally conscious state. Clin. Neurophysiol. 116: 2441–2453.

39. Lang, S., B. Kotchoubey, A. Lutz & N. Birbaumer. 1997. What are you doing when you are doing noth- ing? ERP components without a cognitive task. Z. Exp. Psychol. 44: 138–162.

40. Lang, S. & B. Kotchoubey. 2002. Brain responses to number sequences with and without active task requirement. Clin. Neurophysiol. 113: 1734–1741.

41. Owen, A.M., R.M. Coleman, M. Boly, et al. 2006. Detecting awareness in the vegetative state. Science 313: 1402.

42. Hinterberger, T., F. Nijboer, A. Kübler, et al. 2007. Brain computer interfaces for communication in paralysis: a clinical-experimental approach. In To- wards Brain-Computer Interfacing. G. Dornhege, J.d.R. Millan, T. Hinterberger, et al., Eds.: 43–64. MIT Press. Cambridge, MA.