近年來在醫學診斷和神經生物學研究中，腦電波訊號(Electroencephalogram, EEG)已成為非常有用的非侵入式生理訊號工具，主要因為它能在千分之一秒(milliseconds)的時間裡提供極高的生理訊號解析度，直接反映出細胞群體中動態的變化。在所有量測大腦造影的醫療工具中，量測腦電波訊號最不受任何限制，因為在量測過程中，受試者不需受到固定身體和保持頭不動等限制。然而，若將現今市面上的腦波監測系統應用到日常生活中卻會深深受到許多限制，例如：需在頭皮上塗抹導電膠才能量測腦電波訊號，系統缺乏高精確度的量測，即時訊號處理和有效去除雜訊等功能，皆是主要腦波監測系統的缺失。因此，為了解決這些缺失，本論文主要目的在開發，設計和測試一可直接應用於日常生活環境裡的腦神經人機界面，可讓使用者在日常生活的自然情況下方便使用生理訊號監測系統，即使是在多變的環境中做不同的工作任務，亦能直接擷取大腦活動變化。更重要的是，本論文為了探討此創新的移動式無線腦神經人機界面的應用，我們亦建構一環繞式虛擬實境動態駕駛環境，此環境可真實的模擬日常生活的駕駛情境，無論是應用於神經科學基礎研究上或是應用於日常生活中的警示提醒，皆能有效地測試此腦神經人機界面的效能。在本論文中，我們提出三個與日常生活相關的應用研究，但絕不僅侷限於這些而已，這三個研究分別為：(1) 在虛擬實境動態駕駛環境裡探討受試者駕車時的認知狀態的變化；(2) 探討利用聲音回饋來維持駕駛員的精神狀態和注意力是否集中的腦神經變化；(3) 在虛擬實境動態駕駛環境裡探討動態刺激對身體感覺和知覺的腦神經變化。對目前許多探討複雜腦功能的研究來說，本論文探討的應用能在生活環境裡受限最小並提出許多重要新穎的發現，這些研究成果亦可有效提升正常人每天在反覆工作任務環境下的工作能力表現，也可應用於腦傷、生病或身體不健全等病人更細部的動態認知狀態研究。否則這些成果至今僅能請受試者來傳統腦波實驗室參與實驗，並要求受試者固定身體，頭不能亂動，眼睛不能亂瞄等限制受試者行為。一旦有了創新的可攜式無線腦神經人機界面，這些限制將能一一破除，我們相信這能為認知神經科學和人機界面互動等研究開啟另一新的頁章。

Electroencephalogram is a powerful non-invasive tool widely used by for both medical diagnosis and neurobiological research because it provides high temporal resolution in milliseconds which directly reflects the dynamics of the generating cell assemblies, and it is the only brain imaging modality that does not require the head/body to be fixed. However, the lack of availability of EEG monitoring system capable of high-definition recording, online signal processing and artifact cancellation, without use of conductive gels applied to the scalp, has long thwarted the applications of EEG monitoring in the workplace. This dissertation describes a design, development and testing of a neural human machine interface that allows assessment of brain activities of participants actively performing ordinary tasks in natural body positions and situations within a real operational environment. More importantly, this dissertation also discuss the implications of this innovative mobile wireless brain imaging technology in neuroscience and neuro-technology, through three sample studies: (1) cognitive-state monitoring of participants performing realistic driving tasks in the virtual-reality-based dynamic driving simulator; (2) the efficacy and neural correlates of auditory feedback delivered to participants to maintain participants attention and alertness; (3) the neural correlates of kinesthetic sensation and perception in the dynamic driving simulator. Results of these studies provide many new insights into the understanding of complex brain functions of participants performing ordinary/routine tasks in a minimum constrained environment. These results also allow a better appreciation of the limitations of normal human performance in repetitive task environments, and may allow more detailed study of changes in cognitive dynamics in brain-damaged, diseased, or genetically abnormal individuals. Furthermore, these findings might be difficult, if ever possible, to obtain in a standard EEG laboratory where participants are asked to limit their eye blinks, teeth clinching, head/ body movements. We, thus, believe this work opens a new chapter in neuro-cognitive human-machine interface/interaction.

Contents

摘要 i
Abstract iii
誌謝 v
Contents vi
List of Tables viii
List of Figures ix
1. Introduction 1
1.1 Motivation 1
1.2 Statement of the Problem 2
1.3 Notation 4
2. Materials and Methods 6
2.1 Virtual-reality-based Dynamic Driving Environment 6
2.2 Electroencephalogram Signal Acquisition System 9
2.3 Independent Component Analysis 11
2.4 Event-Related Potential 12
2.5 Event-Related Spectral Perturbation 13
3. EEG Activation of Kinesthetic Perception 15
3.1 Introduction 15
3.2 Experimental Setup 19
3.3 Experimental Results 24
3.4 Discussion 32
4. EEG Activation under Different Cognitive States 36
4.1 Introduction 36
4.2 Experimental Setup 37
4.3 Experimental Results 44
4.4 Discussion 51
5. Portable Brain Computer Interface in Detecting Drowsiness 52
5.1 Introduction 52
5.2 System Architecture 54
5.3 Real-time Drivier’s Drowsiness Detection 64
5.4 Experimental Results and Discussion 66
6. Conclusions 73
References 78

Baudonniere P. M., Belkhenchir S., Lepecq J. C., Mertz S. (1999): Otolith-vestibular-evoked potentials in humans: Intensity, direction of acceleration (Z+, Z-), and BESA modeling of generators. Annals of the New York Academy of Sciences, vol. 871, pp. 384-386.
Bell A. J. and Sejnowski T. J. (1995): An information-maximization approach to blind separation and blind deconvolution. Neural Computation, vol. 7, pp. 1129–1159.
Berthoz A., Israel I., Georges-Francois P., Grasso R., Tsuzuku T. (1995): Spatial memory of body linear displacement: what is being stored? Science, vol. 269, pp. 95-98.
Berthoz A. (2000): The Brain’s Sense of Movement. Cambridge MA: Harvard University Press. 352p.
Bianchi L., Babiloni F., Cincotti F., Arrivas M., Bollero P., and Marciani M. G. (2003): Developing Sensing Bio-Feedback Systems: A General-Purpose Platform. IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, no. 2, pp. 117–119.
Bishop C. M. (1995): Neural Networks for Pattern Recognition, Oxford University Press, Oxford.
Brookhuis K. A., Waard D. D., and Fairclough S. H. (2003): Criteria for driver mpairment. Ergonomics, vol 46, pp. 433-445.
Cardoso J. F. and Souloumiac A. (1993): Blind beamforming for non Gaussian signals. IEE Proceedings F in Radar and Signal Processing, vol. 140, no. 6, pp. 362-370.
Chatterjee S. and Hadi A. S. (1986): Influential Observations, High Leverage Points, and Outliers in Linear Regression. Statistical Science, pp. 379-416.
Cheng M., Gao X., Gao S., and Xu D. (2002): Design and Implementation of a Brain-Computer Interface With High Transfer Rates. IEEE Transactions on biomedical Engineering, vol. 49, no. 10, pp. 1181- 1186.
Chiou J.C., Ko L.W., Lin C. T., Jung T. P., Liang S. F., and Jeng J. L. (2006): Using Novel MEMS EEG Sensors in Detecting Drowsiness Application. Proceedings of IEEE Biomedical Circuits and Systems Conference, London, United Kingdom, Nov. 29-Dec. 1.
Comon P. (1994): Independent component analysis — A new concept? Signal Processing, vol. 36, pp. 287–314.
Edlinger G., Krausz G., Laundl F., Niedermayer I., and Guger C. (2005): Architectures of Laboratory-PC and Mobile Pocket PC Brain-Computer Interfaces. Proceedings of the 2nd International IEEE EMBS Conference on Neural Engineering, Arlington, Virginia, March 16 - 19.
Elidan J., Langhofer L., and Honrubia V. (1987): Recording of short-latency vestibular evoked potentials induced by acceleration impulses in experimental animals: current status of the method and its applications. Electroencephalography and Clinical Neurophysiology, vol. 68, pp. 58-69.
Elidan J., Leibner E., Freeman S., Sela M., Nitzan M., and Sohmer H. (1991): Short and middle latency vestibular evoked responses to acceleration in man. Electroencephalography and clinical neurophysiology, vol. 80, no. 2, pp. 140-145.
Elidan J., Sohmer H., Lev S., and Gay I. (1984): Short latency vestibular evoked response to acceleration stimuli recorded by skin electrodes. The Annals of Otology, Rhinology, and Laryngology, vol. 93, pp. 257-261.
Elidan J., Sohmer H., and Nizan M. (1982): Recording of short latency vestibular evoked potentials to acceleration in rats by means of skin electrodes. Electroencephalography and Clinical Neurophysiology, vol. 53, pp. 501-505.
Eoh H. J., Chung M. K., and Kim S. H. (2005): Electroencephalographic study of drowsiness in simulated driving with sleep deprivation. International Journal of Industrial Ergonomics, vol. 35, pp. 307-320.
Gao X., Xu D., Cheng M., and Gao S. (2003): A BCI-Based Environmental Controller for the Motion-Disabled. IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, no. 2, pp. 137-140.
Girolami M. (1998): An alternative perspective on adaptive independent component analysis. Neural Computation, vol.10, pp.2103–2114.
Groen E. L., Howard I. P., and Cheung B. S. (1999): Influence of body roll on visually induced sensation of self-tilt and rotation. Perception, vol. 28, pp. 287-297.
Guger C., Schlögl A., Neuper C., Walterspacher D., Strein T., and Pfurtscheller G. (2001): Rapid Prototyping of an EEG-Based Brain–Computer Interface (BCI). IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 9, no. 1, pp. 49–58.
Jung T. P., Humphries C., Lee T. W., Makeig S., McKeown M. J., Iragui V., and Sejnowski T. J. (1998): Extended ICA removes artifacts from electroencephalographic recordings. Advances in Neural Information Processing Systems, vol. 10, pp. 894-900.
Jung T. P., Makeig S., Humphries C., Lee T. W., McKeown M. J., Iragui V., and Sejnowski T. J. (2000): Removing electroencephalographic artifacts by blind source separation. Psychophysiology, vol. 37, pp. 163-78.
Jung T. P., Makeig S., Stensmo M., and Sejnowski T. J. (1997): Estimating alertness from the EEG power spectrum. IEEE Transactions on Biomedical Engineering, vol. 44, no. 1, pp. 60–69.
Jung T. P., Makeig S., Westerfield W., Townsend J., Courchesne E., and Sejnowski T. J. (2001): Analysis and visualization of single-trial event-related potentials. Human Brain Mapping, vol. 14, no.3, pp. 166-185.
Jutten C. and Herault J. (1991): Blind separation of sources I. An adaptive algorithm based on neuromimetic architecture. Signal Processing, vol. 24, pp. 1-10.
Kalcher J., Flotzinger D., Neuper C., Gölly S., and Pfurtscheller G. (1996): Graz brain–computer interface II: Toward communication between humans and computers based on online classification of three different EEG patterns. Medical and Biological Engineering and Computing, vol. 34, no. 5, pp. 382–388.
Kemeny A. and Panerai F. (2003): Evaluating perception in driving simulation experiments. Trends in Cognitive Sciences, vol. 7, pp. 31-37.
Kobayashi T. and Takahashi K. (2003): Linux DSP Gateway Specification Rev2.0, Nokia Corporation, Nov. 13.
Lee T. W., Girolami M., and Sejnowski T. J. (1999): Independent component analysis using an extended infomax algorithm for mixed sub- and super-Gaussian sources. Neural Computation, vol.11, pp. 606-633.
Liao R., Krolik J. L., and McKeown M. J. (2005): An information-theoretic criterion for intrasubject alignment of FMRI time series: motion corrected independent component analysis. IEEE Transactions on Medical Imaging, vol. 24, no. 1, pp. 29-44.
Lin C. T., Ko L. W., Chung I. F., Huang T. Y., Chen Y. C., Jung T. P., and Liang S. F. (2006): Adaptive EEG-based Alertness Estimation System by Using ICA-based Fuzzy Neural Networks. IEEE Transactions on Circuits and System I: Regular Papers, vol. 53, no. 11, pp. 2469-2476.
Lin C. T., Wu R. C., Liang S. F., Chao W. H., Chen Y. J., and Jung T. P. (2005): EEG-based drowsiness estimation for safety driving using independent component analysis. IEEE Transactions on Circuits and Systems I, vol. 52, no. 12, pp. 2726 – 2738.
Lin C. T., Wu R. C., Jung T. P., Liang S. F., and Huang T. Y. (2005): Estimating Driving Performance Based on EEG Spectrum Analysis. EURASIP Journal on Applied Signal Processing, vol. 19, pp. 3165-3174.
Liu Jane W. S., Real-Time Systems, Prentice Hall Upper Saddle River, NJ, 2000.
Loose R., Probst T., Tucha O., Bablok E., Aschenbrenner S., and Lange K. W. (2002): Vestibular evoked potentials from the vertical semicircular canals in humans evoked by roll-axis rotation in microgravity and under 1-G. Behavior Brain Research, vol. 134, pp. 131-137.
Makeig S. (1993): Auditory event-related dynamics of the EEG spectrum and effects of exposure to tones. Electroencephalography and Clinical Neurophysiology, vol. 86, pp. 283-293.
Makeig S., Delorme A., Westerfield M., Jung T. P., Townsend J., Courchense E., and Sejnowski T. J. (2004): Electroencephalographic brain dynamics following visual targets requiring manual responses. PLOS Biology, vol. 2, no.6, pp. 747-762.
Makeig S. and Inlow M. (1993): Lapses in alertness: coherence of fluctuations in performance and EEG spectrum. Electroencephalography and Clinical Neurophysiology, vol. 86, no. 1, pp. 23–35.
Makeig S. and Jung T. P. (1995): Changes in alertness are a principal component of variance in the EEG spectrum. Neuroreport, vol. 7, no. 1, pp. 213–216.
Makeig S. and Jung T. P. (1996): Tonic, phasic, and transient EEG correlates of auditory awareness in drowsiness. Cognitive Brain Research, vol. 4, no. 1, pp. 15–25.
McGregor D. K. and Stern J. A. (1996): Time on task and blink effects on saccade duration. Ergonomics, vol. 39, pp. 649-660.
Merfeld D. M., Zupan L., and Peterka R. J. (1999): Humans use internal models to estimate gravity and linear acceleration. Nature, vol. 398, pp. 615-618.
Naganawa M., Kimura Y., Ishii K., Oda K., Ishiwata K., and Matani A. (2005): Extraction of a plasma time-activity curve from dynamic brain pet images based on independent component analysis. IEEE Transactions on Biomedical Engineering, vol. 52, no. 2, pp. 201–210.
NHTSA. National Highway Traffic Safety Administration, Washington, DC. Drowsy driver detection and warning system for commercial vehicle drivers: Field proportional test design, analysis, and progress. [Online] Available: http://www.nhtsa.dot.gov/
NSF. National Sleep Foundation, Washington, DC. Sleep facts and stats. [Online] Available: http://www.sleepfoundation.org/
Obeid I., Nicolelis M., and Wolf P. (2004): A multichannel telemetry system for signal unit neural recording. Journal of Neuroscience Methods, vol. 133, no. 2, pp. 33-38.
Obermaier B., Neuper C., Guger C., and Pfurtscheller G. (2001): Information transfer rate in a five-classes brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 9, no. 3, pp. 283-288.
Orden K. V., Jung T. P., and Makeig S. (2000): Combined eye activity measures accurately estimate changes in sustained visual task performance. Biological Psychology, vol. 52, no. 3, pp. 221-40.
Orden K. V., Limbert W., Makeig S., and Jung T. P. (2001): Eye activity correlates of workload during a visualspatial memory task. Human Factors, vol. 43, no. 1, pp. 111-121.
Page N. G., and Gresty M. A. (1985): Motorist’s vestibular disorientation syndrome. Journal of Neurology, Neurosurgery & Psychiatry, vol. 48, pp. 729-735.
Parker D. E. and Phillips J. O. (2001): Self-motion perception: assessment by real-time computer-generated animations. Applied Ergonomics, vol. 32, pp. 31-38.
Piccini L., Arnone L., Beverina F., Cucchi A., Petrelli L., and Andreoni G. (2004): Wireless DSP architecture for biosignals recording. Proceedings of the 4th IEEE International Symposium on Signal Processing and Information Technology, pp. 487- 490, Dec. 18-21.
Pilutti T. and Ulsoy G., (1999): Identification of Driver State for Lane-Keeping Tasks. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 29, pp. 486-502.
Polich J. and Comerchero M. (2003): P3a from visual stimuli: typicality, task, and topography. Brain Topography, vol. 15, pp. 141-152.
Probst T., Ayan T., Loose R., and Skrandies W. (1997): Electrophysiological evidence for direction-specific rotary evoked potentials in human subjects -- a topographical study. Neuroscience letters, vol. 239, pp. 97-100.
Probst T., Bablok E., Dabrowski H., Dombrowski J. H., Loose R., and Wist E. R. (1996): Position and velocity responses from the otoliths and the canals: results from ESA’s parabolic flights. Aviation, space, and environmental medicine, vol. 67, no. 7, pp. 633-639.
Probst T., Dabrowski H., Liebler G., and Wist E. R. (1993): MARDER -- Multi-Axes Rotation Device for Experimental Research: a new concept for investigations of the vestibular, oculomotor, and visual systems of humans in three-dimensional space. Journal of neuroscience methods, vol. 49, pp. 49-61.
Reymond G. and Kemeny A. (2000): Motion cueing in the Renault driving simulator. Vehicle System Dynamics, vol. 34, pp. 249-259.
Reymond G., Kemeny A., Droulez J., and Berthoz A. (2001): Role of lateral acceleration in curve driving: driver model and experiments on a real vehicle and a driving simulator. Human Factors, vol. 43, pp. 483-495.
Roberts S., Rezek I., Everson R., Stone H., Wilson S., and Alford C. (2000): Automated assessment of vigilance using committees of radial basis function analysers,” IEE Science Measurement and Technology, vol. 147, pp. 333–338.
Seidmann S. H., Telford L., Paige G. D. (1998): Tilt perception during dynamic linear acceleration. Experimental Brain Research, vol. 119, pp. 307-314.
Sterlade M. (1993): Central Core Modulation of Spontaneous Oscillations and Sensory Transmission in Thalamocortical Systems. Current Opinion in Neurobiology, vol. 3, no.4, pp. 619-625.
Stern J. A., Boyer D., and Schroeder D. (1994): Blink rate: Possible measure of fatigue. Human Factors, vol. 36, pp. 285-297.
Stewart D. (1965): A platform with six degrees of freedom. Institution of Mechanical Engineers, Proceedings, vol. 180, pp. 371–386.
Thakor N. V. (1999): Biopotentials and Electro-physiology Measurement. The Measurement, Instrumentation and Sensors Handbook, J. H. School of Medicine.
Thilo K. V., Kleinschmidt A., and Gresty M. A. (2003): Perception of self-motion from peripheral optokinetic stimulation suppresses visual evoked responses to central stimuli. Journal of Neurophysiology, vol. 90, pp. 723-730.
Treisman M. (1984): Temporal Rhythms and Cerebral Rhythms. Annals of the New York Academy of Sciences, vol. 423, pp. 542-565.
Vuckovic A., Radivojevic V., Chen A. C. N., and Popovic D. (2002): Automatic recognition of alertness and drowsiness from EEG by an artificial neural network. Medical engineering & physics, vol. 24, pp. 349-360.
Wexler M., Lamouret I., and Droulez J. (2001): The stationarity hypothesis: an allocentric criterion in visual perception. Vision Research, vol. 41, pp. 3023-3037.
Whitchurch A. K., Ashok B. H., Kumaar R. V., Sarukesi K., and Varadan V. K. (2002): Wireless system for long term EEG monitoring of Absence Epilepsy. Biomedical Applications of Micro- and Nanoengineering, vol. 4937, pp. 343-349.