跳到主要內容

臺灣博碩士論文加值系統

(44.200.194.255) 您好!臺灣時間:2024/07/20 13:40
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:洪彗庭
研究生(外文):Hong, Hui-Ting
論文名稱:異質性資料在人類行為其信息交互過程之探討:疼痛程度評估和多媒體暴力檢測
論文名稱(外文):Investigate Communication Process of Human Behavior from Heterogeneous Data: Pain Level Assessment and Multimedia Violence Prediction
指導教授:李祈均李祈均引用關係
指導教授(外文):Lee, Chi-Chun
口試委員:曹昱賴穎暉陳冠宇
口試委員(外文):Tsao, YuLai, Ying-HuiChen, Kuan-Yu
口試日期:2020-02-20
學位類別:碩士
校院名稱:國立清華大學
系所名稱:電機工程學系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2020
畢業學年度:108
語文別:英文
論文頁數:74
中文關鍵詞:人類行為訊號處理疼痛程度辨識急診檢傷分類傳媒暴力程度預測
外文關鍵詞:Behavioral signal processingpain leveltriageviolence prediction
相關次數:
  • 被引用被引用:0
  • 點閱點閱:237
  • 評分評分:
  • 下載下載:12
  • 收藏至我的研究室書目清單書目收藏:0
人類在現今依存的世界裡透過多種不同的方式進行溝通與交流,從概念的生 成、送訊者的編碼、傳達的媒介一直到收訊者的解碼,是一連串複雜而多層次 的程序,其中亦可能因噪音(訊息之扭曲、干擾)或是回饋的產生,而為其增 添更多未知之可能性。本研究基於人類行為本質的複雜性,著重探討編碼與解 碼的過程以及行為者與現象發生之關聯性,並實際驗證於兩個任務:ㄧ)疼痛 程度評估以及 二)多媒體暴力檢測。在醫療臨床疼痛程度診斷的應用上,部分 臨床紀錄之資訊在工程研究中尚未列入考慮,本次運用多模態機器學習之模型 設計技術,提出軟層排序任務特定(Task Specific Encoder with Soft Layer Order- ing, TSEN-SLO)架構,透過參閱疼痛部位的臨床紀錄,提升在診斷疼痛程度的 準確率,而為理解各個特徵在不同族群的重要程度及其影響,該研究亦透過多 變量變異數分析,了解語音特徵在多個獨立變項(年齡、性別、疼痛程度、疼 痛部位)影響下的顯著性差異。而在多媒體內容檢索的應用上,過去研究多著 重彌合語義(semantic)資訊、擷取影視內容之摘要片段等等,而較少探討創作 者與受眾之間的關係脈絡,本次使用英國影集—黑鏡之觀者與劇本撰寫者之辭 彙特徵,運用外類別(extra-genre multimodal embedding)模型架構,初步驗證 受眾回饋具有評估多媒體暴力程度的可行性。
Living world nurtures diverse ways on how people communicate with one another. Its concepts could be structured from encoding, message, medium to decoding. This communication process is a series of complicate and multilayered procedures which also include the generation of noise (distortion and interference) or feedback. Base on the multidimensionality of human behavior, this research focused on investigating the pro- cess from encoding to decoding, and the relationship between the human beings and their corresponding phenomenon. We verified the results on two tasks: 1) automatic pain level recognition and 2) multimedia violence prediction. In the first task, triage pain assessment, clinical parameter like pain-site, is yet not considered in previous engineer studies. We utilized multi-modal machine learning technique and proposed the Task Specific Encoder with Soft Layer Ordering (TSEN-SLO) structure with the auxiliary information from pain-site and demonstrates improvement in the pain-level recognition. Moreover, to further reveal the variability of voice quality conditioned on clinical parameters (age, gender, pain-level, pain-site), the multivariate analysis of variance was conducted to understand the significant differences of acoustic features with respect to multiple independent variables. In the second task, multimedia violence pre- diction, previous lines of work aim to bridge the semantic gap and cut down the content of multimedia to its essential parts. We, however, conducted the experiment on the British TV series – Black Mirror, revealing the relationship between screenplay writers and audiences. The extra-genre multimodal embedding structure is proposed in this task and preliminarily verify the feasibility of assessing multimedia violence extent by considering audience feedback.
摘要································································································ I ABSTRACT ···················································································· II
誌謝······························································································ IV
CHAPTER 1 INTRODUCTION·························································3
CHAPTER 2 TASK1: AUTOMATIC PAIN LEVEL RECOGNITION ········6
2.1 Introduction·················································································6
2.2 Database ·····················································································9
2.2.1 Data Acquisition········································································ 11
2.2.2 Processed Data········································································ 13
2.3 Research Methodology·································································· 14
2.3.1 Discriminative Features Extractions ········································· 15
2.3.1.1 Visual Features ································································ 16
2.3.1.2 Acoustic Features ··························································· 20
2.3.2 Session-Level Behavior Encoding ··········································· 22
2.3.2.1 Gaussian Mixture Models Fisher Vector Encoding (GMMs-FV) ······ 22 2.3.2.2 Functional Encoding······································································ 25 2.3.3 Model Architecture ·········································································· 25 2.3.3.1 Multi-task Learning (MTL)························································· 26 2.3.3.2 Soft Layer Ordering [63] ·························································· 28 2.3.3.3 Task Specific Encoder with Soft Layer Ordering (TSEN- SLO)························ 31
2.4 Experiments and Results ······························································· 33
2.4.1 Experimental Settings······························································ 33
2.4.2 Recognition Results································································· 36
2.5 Statistical Analysis······································································· 38
2.5.1 Variability of Facial Expressions ··············································· 39
2.5.2 Variability of Acoustic Expressions ·········································· 41
2.5.2.1 Clinical Parameters: Age, Gender, Pain-Site························ 42
2.5.2.2 Voice Quality······································································ 43
2.5.2.3 MANOVA Settings······························································ 45
2.5.2.4 Result (I): Main / Interaction Effect ···································· 47
2.5.2.5 Result (II): Univariate Analysis and Pairwise Comparison ···· 51
CHAPTER 3 TASK2: MULTIMEDIA VIOLENCE PREDICTION ·········· 56
3.1 Introduction··············································································· 56
3.2 Database ··················································································· 56
3.3 Research Methodology·································································· 59
3.3.1 Features Extractions ································································ 59
3.3.1.1 Linguistic Inquiry and Word Count (LIWC) ························· 59
3.3.1.2 BERT ················································································ 62
3.3.2 Model Architecture ·································································· 63
3.4 Experiments and Results ···························································· 64 CHAPTER 4 CONCLUSION ·························································· 67 REFERENCE ················································································· 69
[1] Popescu, Gabriela. "Human behavior, from psychology to a transdisciplinary insight." Procedia- Social and Behavioral Sciences 128 (2014): 442-446.
[2] Giddens, Anthony, et al. Introduction to sociology. New York: Norton, 1991.
[3] Rose, Arnold M. Human behavior and social processes: An interactionist approach. Routledge,
2013.
[4] Segall, Marshall H., et al. Human behavior in global perspective: An introduction to cross-cul-
tural psychology. Pergamon Press, 1990.
[5] Zimbardo, Philip G., and Floyd L. Ruch. "Psychology and life." (1975).
[6] Rosenzweig, Mark R., S. Marc Breedlove, and Arnold L. Leiman. Biological psychology: An in-
troduction to behavioral, cognitive, and clinical neuroscience. Sinauer Associates, 2002.
[7] Alland, Alex. Evolution and human behaviour: an introduction to Darwinian anthropology.
Routledge, 2012.
[8] Cronk, Lee. "Human behavioral ecology." Annual Review of Anthropology 20.1 (1991): 25-53.
[9] Lepri, Bruno, et al. "Human behavior understanding for inducing behavioral change: Social and
theoretical aspects." International Joint Conference on Ambient Intelligence. Springer, Berlin,
Heidelberg, 2011.
[10] Deleanu,I.,(1983),Biologieşidrept,DaciaPublishing-House,Cluj-Napoca,p.53
[11] Gastil,RaymondD."Thedeterminantsofhumanbehavior."AmericanAnthropologist63.6
(1961): 1281-1291.
[12] Narayanan,Shrikanth,andPanayiotisG.Georgiou."Behavioralsignalprocessing:Derivinghu-
man behavioral informatics from speech and language." Proceedings of the IEEE 101.5 (2013):
1203-1233.
[13] Lee,Chi-ChunJeremy.ModelingHumanBehaviorsinPsychologyUsingEngineeringMethods.
River Publishers, 2014.
[14] Lee,ChulMin,andShrikanthS.Narayanan."Towarddetectingemotionsinspokendia-
logs." IEEE transactions on speech and audio processing 13.2 (2005): 293-303.
[15] Narayanan,Shrikanth,andPanayiotisG.Georgiou."Behavioralsignalprocessing:Derivinghu-
man behavioral informatics from speech and language." Proceedings of the IEEE 101.5 (2013):
1203-1233.
[16] Bieri,Daiva,etal."TheFacesPainScalefortheself-assessmentoftheseverityofpainexperi-
enced by children: development, initial validation, and preliminary investigation for ratio scale
properties." Pain 41.2 (1990): 139-150.
[17] Hicks,CarrieL.,etal."TheFacesPainScale–Revised:towardacommonmetricinpediatricpain
measurement." Pain 93.2 (2001): 173-183.
[18] Carlsson,AnnaMaria."Assessmentofchronicpain.I.Aspectsofthereliabilityandvalidityof the visual analogue scale." Pain 16.1 (1983): 87-101.
[19] Bond,M.R.,andI.Pilowsky."Subjectiveassessmentofpainanditsrelationshiptothe
A., Alison R. Snyder, and Brian G. Leggin. "Responsiveness of the numeric pain rating scale in pa-
tients with shoulder pain and the effect of surgical status." Journal of sport rehabilitation 20.1
(2011): 115-128.
[23] Farrar,JohnT.,etal."Clinicalimportanceofchangesinchronicpainintensitymeasuredonan
11-point administration of analgesics in patients with advanced cancer." Journal of psychoso-
matic research (1966).
[20] Shields,B.J.,etal."Predictorsofachild'sabilitytouseavisualanaloguescale."Child:care,
health and development 29.4 (2003): 281-290.
[21] Bijur,PollyE.,ClarkeT.Latimer,andE.JohnGallagher."Validationofaverballyadministered
numerical rating scale of acute pain for use in the emergency department." Academic Emergency
Medicine 10.4 (2003): 390-392.
[22] Michener,Lorinumericalpainratingscale."Pain94.2(2001):149-158.
[24] Weideman,AnnMarie,etal."Meta-analysisoftheage-dependentefficacyofmultiplesclerosis treatments." Frontiers in neurology 8 (2017): 577.
[25] Edwards,RobertR.,etal."Paintoleranceasapredictorofoutcomefollowingmultidisciplinary treatment for chronic pain: differential effects as a function of sex." Pain 106.3 (2003): 419-426.
[26] Prkachin,KennethM."Theconsistencyoffacialexpressionsofpain."PaulEkmanErikaL.Ros- enberg (1997): 198.
[27] Kaltwang,Sebastian,OgnjenRudovic,andMajaPantic."Continuouspainintensityestimation from facial expressions." International Symposium on Visual Computing. Springer, Berlin, Hei- delberg, 2012.
[28] Ashraf,AhmedBilal,etal."Thepainfulface–painexpressionrecognitionusingactiveappear- ance models." Image and vision computing 27.12 (2009): 1788-1796.
[29] Olugbade,TemitayoA.,etal."Howcanaffectbedetectedandrepresentedintechnologicalsup- port for physical rehabilitation?." ACM Transactions on Computer-Human Interaction (TO- CHI) 26.1 (2019): 1-29.
[30] Rivas,JesúsJoel,etal."Unobtrusiveinferenceofaffectivestatesinvirtualrehabilitationfrom upper limb motions: A feasibility study." IEEE transactions on affective computing (2018).
[31] Tsai,Fu-Sheng,etal."EmbeddingstackedbottleneckvocalfeaturesinaLSTMarchitecturefor automatic pain level classification during emergency triage." 2017 Seventh International Confer- ence on Affective Computing and Intelligent Interaction (ACII). IEEE, 2017.
[32] Li,Jeng-Lin,etal."LearningConditionalAcousticLatentRepresentationwithGenderandAge Attributes for Automatic Pain Level Recognition." Interspeech. 2018.
[33] Cervero,Fernando."Visceralversussomaticpain:similaritiesanddifferences."DigestiveDis- eases 27.Suppl. 1 (2009): 3-10.
[34] Kischner,S.,andR.C.McMyne."Dermatomesanatomy."URL:http://emedicine.medscape. com/article/1878388-overview[accessed June 1, 2017] (2015).
[35] Mantyh,PatrickW."Theneurobiologyofskeletalpain."EuropeanjournalofNeuroscience39.3 (2014): 508-519.
[36] AndrewRice,Clinicalpainmanagement,HodderArnold,London,2008.
[37] Tsai,Fu-Sheng,etal."TowardDevelopmentandEvaluationofPainLevel-RatingScalefor
Emergency Triage based on Vocal Characteristics and Facial Expressions." Interspeech. 2016.
[38] Eriksson,Kerstin,etal."Numericratingscale:patients'perceptionsofitsuseinpostoperative
pain assessments." Applied nursing research 27.1 (2014): 41-46.
[39] Perronnin,Florent,JorgeSánchez,andThomasMensink."Improvingthefisherkernelforlarge-
scale image classification." European conference on computer vision. Springer, Berlin, Heidel-
berg, 2010.
[40] Prkachin,KennethM.,andKennethD.Craig."Expressingpain:thecommunicationandinterpre-
tation of facial pain signals." Journal of Nonverbal Behavior 19.4 (1995): 191-205.
[41] Craig,KennethD."ThefacialexpressionofpainBetterthanathousandwords?."APSJour-
nal 1.3 (1992): 153-162.
[42] Fuller,BarbaraF."Acousticdiscriminationofthreetypesofinfantcries."NursingRe-
search (1991).
[43] Oshrat,Yaniv,etal."Speechprosodyasabiosignalforphysicalpaindetection."ConfProc8th
Speech Prosody. 2016.
[44] Ren,Zhao,etal."Evaluationofthepainlevelfromspeech:Introducinganovelpaindatabaseand
benchmarks." Speech Communication; 13th ITG-Symposium. VDE, 2018.
[45] Baltrusaitis,Tadas,PeterRobinson,andLouis-PhilippeMorency."Constrainedlocalneuralfields
for robust facial landmark detection in the wild." Proceedings of the IEEE international confer-
ence on computer vision workshops. 2013.
[46] Xing,Junliang,etal."Towardsmulti-viewandpartially-occludedfacealignment."Proceedings
of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.
[47] Peng,Jian,LiefengBo,andJinboXu."Conditionalneuralfields."Advancesinneuralinfor-
mation processing systems. 2009.
[48] Hjortsjö,Carl-Herman.Man'sfaceandmimiclanguage.Studenlitteratur,1969.
[49] Ekman,Paul,WallaceV.Freisen,andSoniaAncoli."Facialsignsofemotionalexperience."Jour-
nal of personality and social psychology 39.6 (1980): 1125.
[50] McIntyre,Gordon,etal."Anapproachforautomaticallymeasuringfacialactivityindepressed subjects." 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 2009.
[51] Cohn,JeffreyF.,etal."Detectingdepressionfromfacialactionsandvocalprosody."20093rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 2009.
[52] LeResche,Linda."Facialexpressioninpain:astudyofcandidphotographs."JournalofNonver- bal Behavior 7.1 (1982): 46-56.
[53] Lucey,Patrick,etal."Automaticallydetectingpainusingfacialactions."20093rdInternational Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 2009.
[54] Rathee,Neeru,andDineshGanotra."Anovelapproachforpainintensitydetectionbasedonfa- cial feature deformations." Journal of Visual Communication and Image Representation 33 (2015): 247-254.
[55] https://www.cs.cmu.edu/~face/facs.htm
[56] 徐雅玲. "利用多模態模型混合CNN 和LSTM 影音特徵以自動化偵測急診病患疼痛程度." (2018): 1-44.
[57] Schuller,Björn,etal."Theinterspeech2012speakertraitchallenge."ThirteenthAnnualConfer- ence of the International Speech Communication Association. 2012.
[58] Reynolds,DouglasA."GaussianMixtureModels."Encyclopediaofbiometrics741(2009).
[59] Moon,ToddK."Theexpectation-maximizationalgorithm."IEEESignalprocessingmaga-
zine 13.6 (1996): 47-60.
[60] Kaya,Heysem,AlexeyA.Karpov,andAlbertAliSalah."Fishervectorswithcascadednormali-
zation for paralinguistic analysis." Sixteenth Annual Conference of the International Speech
Communication Association. 2015.
[61] Zhang,Yu,andQiangYang."Asurveyonmulti-tasklearning."arXivpreprint
arXiv:1707.08114 (2017).
[62] Ruder,Sebastian."Anoverviewofmulti-tasklearningindeepneuralnetworks."arXivpreprint
arXiv:1706.05098 (2017).
[63] Meyerson,Elliot,andRistoMiikkulainen."Beyondsharedhierarchies:Deepmultitasklearning
through soft layer ordering." arXiv preprint arXiv:1711.00108 (2017).
[64] Nair,Vinod,andGeoffreyE.Hinton."Rectifiedlinearunitsimproverestrictedboltzmannma-
chines." Proceedings of the 27th international conference on machine learning (ICML-10). 2010.
[65] Sherman,R.O.G.E.R."Abdominalpain."(1990).
[66] Sluka,K.A."Stimulationofdeepsomatictissuewithcapsaicinproduceslong-lastingmechanical
allodynia and heat hypoalgesia that depends on early activation of the cAMP pathway." Journal of Neuroscience 22.13 (2002): 5687-5693.
[67] Taverner,Deryck."Musclespasmasacauseofsomaticpain."Annalsoftherheumaticdis- eases 13.4 (1954): 331.
[68] Sikandar,Shafaq,andAnthonyH.Dickenson."Visceralpain–theinsandouts,theupsand downs." Current opinion in supportive and palliative care 6.1 (2012): 17.
[69] https://bodyinmind.org/pain-communication-body-posture/
[70] Walsh,Joseph,ChristopherEccleston,andEdmundKeogh."Paincommunicationthroughbody posture: The development and validation of a stimulus set." PAIN® 155.11 (2014): 2282-2290.
[71] Blondell,RichardD.,MohammadrezaAzadfard,andAngelaM.Wisniewski."Pharmacologic therapy for acute pain." American family physician 87.11 (2013): 766-772.
[72] Nicholson,Bruce."Differentialdiagnosis:nociceptiveandneuropathicpain."TheAmericanjour- nal of managed care 12.9 Suppl (2006): S256-62.
[73] Teixeira,JoãoPaulo,CarlaOliveira,andCarlaLopes."Vocalacousticanalysis-jitter,shimmer and HNR parameters." (2013).
[74] Fusaroli,Riccardo,etal."IsvoiceamarkerforAutismspectrumdisorder?Asystematicreview and meta- analysis." Autism Research 10.3 (2017): 384-407.
[75] Vaiciukynas,Evaldas,etal."DetectingParkinson’sdiseasefromsustainedphonationandspeech signals." PloS one 12.10 (2017).
[76] Silva,DarcioG.,LuísC.Oliveira,andMarioAndrea."Jitterestimationalgorithmsfordetection of pathological voices." EURASIP Journal on Advances in Signal Processing 2009 (2009): 1-9.
[77] Biemans,Monique.Gendervariationinvoicequality.NetherlandsGraduateSchoolofLinguis-
tics, 2000.
[78] Lortie,CatherineL.,etal."Effectsofageontheamplitude,frequencyandperceivedqualityof
voice." Age 37.6 (2015): 117.
[79] Mendoza,Elvira,etal."Differencesinvoicequalitybetweenmenandwomen:Useofthelong-
term average spectrum (LTAS)." Journal of voice 10.1 (1996): 59-66.
[80] Weninger,Felix,etal."Ontheacousticsofemotioninaudio:whatspeech,music,andsound
have in common." Frontiers in psychology 4 (2013): 292.
[81] Teixeira,JoãoPaulo,andPaulaOdeteFernandes."Jitter,ShimmerandHNRclassificationwithin
gender, tones and vowels in healthy voices." Procedia technology 16 (2014): 1228-1237.
[82] Devlin,Jacob,etal."Bert:Pre-trainingofdeepbidirectionaltransformersforlanguageunder-
standing." arXiv preprint arXiv:1810.04805 (2018).
[83] Ko,Ming-Ya,Jeng-LinLi,andChi-ChunLee."LearningMinimalExtra-genreMultimodalEm-
bedding from Trailer Content and Reactor Expressions for Box Office Prediction." 2019 IEEE
International Conference on Multimedia and Expo (ICME). IEEE, 2019.
[84] Zhang,Ying,andHuchuanLu."Deepcross-modalprojectionlearningforimage-textmatch-
ing." Proceedings of the European Conference on Computer Vision (ECCV). 2018.
[85] Sudhakaran,Swathikiran,andOswaldLanz."Learningtodetectviolentvideosusingconvolu- tional long short-term memory." 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 2017.
[86] AlfredLKroeberandTalcottParsons.1958.Theconceptsofcultureandofsocialsystem.Americansocio- logical review 23, 5 (1958), 582–583.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top