跳到主要內容

臺灣博碩士論文加值系統

(34.204.176.71) 您好!臺灣時間:2024/11/10 20:34
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:楊程翔
研究生(外文):YANG, CHENG-XIANG
論文名稱:認知負荷對於手勢概念化之影響
論文名稱(外文):The Influences of Cognitive Load on Co-speech and Co-thought Gestures for Conceptualization
指導教授:林彥良林彥良引用關係
指導教授(外文):LIN, YEN-LIANG
口試委員:郭政淳許展嘉
口試委員(外文):KUO, CHENG-CHUENHSU, CHAN-CHIA
口試日期:2019-01-16
學位類別:碩士
校院名稱:國立臺北科技大學
系所名稱:應用英文系
學門:人文學門
學類:外國語文學類
論文種類:學術論文
論文出版年:2019
畢業學年度:107
語文別:英文
論文頁數:58
中文關鍵詞:意象型手勢搭配思考之手勢認知負荷任務複雜度
外文關鍵詞:representational gesturesco-thought gesturescognitive loadtask complexity
相關次數:
  • 被引用被引用:0
  • 點閱點閱:253
  • 評分評分:
  • 下載下載:31
  • 收藏至我的研究室書目清單書目收藏:0
現今多項研究對於認知負荷是否影響手勢之使用皆有不同的看法。部分研究(如Melinger & Kita, 2007)認為提高認知負荷為導致增加手勢使用的因素,然而也有研究(Sassenberg & Van Der Meer, 2010)認定心智表徵才是主要原因。不同於前者所述,手勢的增加有助於減低認知負荷已被研究所證實(如Goldin-Meadow, Nusbaum, Kelly, & Wagner, 2001)。此外,新研究指出搭配思考之手勢與搭配言談之手勢皆能反映問題處理的策略(Alibali, Spencer, Knox, & Kita, 2011)及增強問題處理的表現(Chu & Kita, 2011)。建立在此概念之上,Kita、Alibali以及Chu(2017)提出Gesture-for-Conceptualization Hypothesis,指出手勢具備以下四種功能:1. activation 2. manipulating 3. packaging 4. exploration。此研究不僅1. 檢視手勢之頻率,更率先探討手勢之持續時間是否也受到認知負荷所影響,同時討論 2. 特定手勢是否有特定功能。實驗結果指出認知負荷影響搭配言談之手勢之頻率以及持續時間,但對於搭配思考之手勢,僅有頻率受到影響。此外,此研究更發現iconic手勢、metaphoric手勢及deictic手勢主要功能皆為activation;如除外而activation,iconic手勢與packaging較有關聯、metaphoric手勢之於manipulation、deictic手勢之於exploration;而co-thought手勢則主要為exploration。
There is disagreement in the literature as to whether cognitive load influences gesture. While some hold that increasing of cognitive load causes more production in gesture (e.g., Melinger & Kita, 2007), others argue that mental simulation increases the production of gestures (Sassenberg & Van Der Meer, 2010). Conversely, use of gestures has been shown to be a mechanism to decrease cognitive load (e.g., Goldin-Meadow, Nusbaum, Kelly, & Wagner, 2001). Similar to co-speech gesture, co-thought gesture is found to reflect problem solving strategies (Alibali, Spencer, Knox, & Kita, 2011) and enhance problem solving performance (Chu & Kita, 2011). Encapsulating this, the Gesture-for-Conceptualization Hypothesis (Kita, Alibali, & Chu, 2017) proposes four conceptual functions: gestures (1) activate, (2) manipulate, (3) package, and (4) explore spatio-motoric information. However, to date most studies have only examined gesture occurrence and not gesture duration. Building on existing research but including duration as a variable, this study aimed to: (1) clarify whether increased cognitive load changed gesture frequency and duration, (2) describe the functions of the gestures observed, and (3) understand how types of gestures correspond to specific functions. Twenty participants were recruited and required to retell the story in English after watching a pre-determined cartoon. In order to distinguish task complexity, participants were separated into two equal groups: one group was given a structure for story re-telling, while the other was not. The results of this study showed that: (1) increased cognitive load (task structure) significantly influenced the frequency and the duration of co-speech gesture; (2) the frequency of co-thought gestures was influenced while duration was not; (3) activation dominated in iconic, metaphoric, and deictic gesturesalthough iconic gestures were shown to have a closer relationship to packaging, metaphoric gestures to manipulation, and deictic gestures to exploration; and (4) co-thought gestures mostly functioned as exploration. This study provides necessary clarification on the relationship between cognitive load and gestures, including novel interactions of gesture duration and gesture function.
Abstract (Chinese)............................................i
Abstract (English)............................................iii
Acknowldgements............................................v
Table of Contents............................................vi
List of Tables............................................viii
CHAPTER 1 INTRODUCTION............................................1
1.1 Background............................................1
1.2 Research Questions............................................4
CHAPTER 2 LITERATURE REVIEW............................................6
2.1 Language and gesture............................................6
2.2 Types of Gestures............................................9
2.3 Co-thought gestures............................................11
2.4 Cognitive load and gesture............................................11
2.5 Information Packaging Hypothesis............................................16
2.6 Gesture-for-Conceptualization Hypothesis............................................17
2.7 Task complexity............................................21
CHAPTER 3 METHODOLOGY............................................24
3.1 Participants............................................24
3.2 Stimuli............................................25
3.3 Procedure............................................26
3.4 Gesture Coding............................................28
3.5 Data analysis............................................29
Chapter 4 RESULTS AND DISCUSSION............................................31
4.1 Cognitive Load and Frequency of Gestures............................................31
4.2 Cognitive Load and Duration of Gestures............................................35
4.3 Functions of Gestures............................................37
Chapter 5 CONCLUSION............................................44
5.1 Summary of the findings............................................44
5.2 Limitation and future research............................................46
References............................................49
Appendix A. Structures of Story Retelling of Sylvester and the Tweety Bird........................55
Appendix B. Transcription Conventions............................................58
Alibali, M. W., Bassok, M., Solomon, K. O., Syc, S. E., & Goldin-Meadow, S. (1999). Illuminating mental representations through speech and 1 gesture. Psychological Science, 10(4), 327-333.

Alibali, M. W., Spencer, R. C., Knox, L., & Kita, S. (2011). Spontaneous gestures influence strategy choices in problem solving. Psychological Science, 22(9), 1138-1144.

Alibali, M. W., Kita, S., & Young, A. J. (2000). Gesture and the process of speech production: We think, therefore we gesture. Language and Cognitive Processes, 15(6), 593-613.a

Anthony, L., Carrington, P., Chu, P., Kidd, C., Lai, J., & Sears, A. (2011). Gesture dynamics: Features sensitive to task difficulty and correlated with physiological sensors. Stress, 1418(360), 312-316.

Beattie, G., & Shovelton, H. (1999). Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech. Journal of Language and Social Psychology, 18(4), 438-462.

Beilock, S. L., & Goldin-Meadow, S. (2010). Gesture changes thought by grounding it in action. Psychological Science, 21(11), 1605-1610.

Butcher, C., & Goldin-Meadow, S. (2000). Gesture and the transition from one- to two-word speech: When hand and mouth come together. In D. McNeill (Ed.), Language and gesture (pp. 235-257). Cambridge: Cambridge University Press

Chen, F., Ruiz, N., Choi, E., Epps, J., Khawaja, M. A., Taib, R., ... & Wang, Y. (2012). Multimodal behavior and interaction as indicators of cognitive load. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(4), 22.

Chu, M., & Kita, S. (2011). The nature of gestures' beneficial role in spatial problem solving. Journal of Experimental Psychology: General, 140(1), 102.

Chu, M., & Kita, S. (2016). Co-thought and co-speech gestures are generated by the same action generation process. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(2), 257.

Cook, S. W., Yip, T. K., & Goldin-Meadow, S. (2012). Gestures, but not meaningless movements, lighten working memory load when explaining math. Language and Cognitive Processes, 27(4), 594–610.

de Ruiter, J. P. (1998). Gesture and speech production. Ph.D. Dissertation, Nijmegen Nijmegen University.

de Ruiter, J. P. (2000). The production of gesture and speech. Language and Gesture, 2, 284.

Freedman, N. (1977). Hands, words and mind: On the structuralization of body movements during discourse and the capacity for verbal representation. In N. Freedman & S. Grand (Eds.), Communicative structures and psychic structures: A psychoanalytic interpretation of communication (pp. 109-132). New York: Plenum.

Goldin-Meadow, S., & Alibali, M. W. (2013). Gesture's role in speaking, learning, and creating language. Annual Review of Psychology, 64, 257-283.

Goldin-Meadow, S., & Brentari, D. (2017). Gesture, sign, and language: The coming of age of sign language and gesture studies. Behavioral and Brain Sciences, 40.

Goldin-Meadow, S., Cook, S. W., & Mitchell, Z. A. (2009). Gesturing gives children new ideas about math. Psychological Science, 20(3), 267-272.

Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., & Wagner, S. (2001). Explaining math: Gesturing lightens the load. Psychological Science, 12(6), 516-522.

Goldin-Meadow, S., & Wagner, S. M. (2005). How our hands help us learn. Trends in Cognitive Sciences, 9(5), 234-241.

Gullberg, M. (2003). Gestures, referents, and anaphoric linkage in learner varieties. Information structure and the dynamics of language acquisition, 311-328.

Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137(2), 297.

Hostetter, A. B., Alibali, M. W., & Kita, S. (2007). I see it in my hands’ eye: Representational gestures reflect conceptual demands. Language and Cognitive Processes, 22(3), 313-336.

Hadar, U., & Butterworth, B. (1997). Iconic gestures, imagery, and word retrieval in speech. Semiotica, 115(1-2), 147-172.

Hussain, M. (2014). Cross linguistic variation in the gestures accompanying manner of motion event descriptions by native speakers of English and Urdu. Balochistan Journal of Linguistics, 2, 139–167.

Iverson, J. M., & Goldin-Meadow, S. (2005). Gesture paves the way for language. development. Psychological science, 16(5), 367-371.

Kellerman, E., & Hoof, A. V. (2003). Manual accents. International Review of Applied Linguistics, 41(3), 251–269.

Kita, S. (2000). How representational gestures help speaking. Language and gesture, 1, 162-185.

Kita, S., Alibali, M. W., & Chu, M. (2017). How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis. Psychological review, 124(3), 245.

Kendon, A. (1994). Do gestures communicate? A review. Research on Language and Social Interaction, 27(3), 175-200.

Kendon, A. (1988). Sign languages of Aboriginal Australia: Cultural, semiotic and. communicative perspectives. Cambridge and New York: Cambridge University Press.

Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: CambridgeUniversity Press.

Krauss, R. M. (1998). Why do we gesture when we speak?. Current Directions in Psychological Science, 7(2), 54-54.

Krauss, R. M., & Hadar, U. (1999). The role of speech-related arm/hand gestures in word retrieval. In R. Campbell & L. Messing (Eds.), Gesture, speech, and sign (pp. 93–116). Oxford: Oxford University Press.

Krauss, C., & Chen, J. Gottesman (2000) Lexical Gestures and Lexical Access: A Process Model. Language and gesture, 261-283.

Lee, Hao-Hsiang (2017), Linguistic-gestural Representations in Motion Event Narrations: Mandarin Chinese as L1 and English as L2. Published Doctoral Dissertation, National Taipei University of Technology

Lindholm, K (1998). English question development in second language learners: Relationship between semantic context and linguistic complexity. Interface Journal of Applied Linguistics, 2(2), 67-68

Logan, T., Lowrie, T., & Diezmann, C. M. (2014). Co-thought gestures: Supporting students to successfully navigate map tasks. Educational Studies in Mathematics, 87(1), 87-102.

Long, M. H. (1990). Task, Group, and Task-Group Interactions. University of Hawaii Working Papers in English as a Second Language, 8(20), 1-25

Melinger, A., & Kita, S. (2007). Conceptualisation load triggers gesture production. Language and Cognitive Processes, 22(4), 473-500.

McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.

McNeill, D. (2000). Language and gesture. Cambridge: Cambridge University Press.

Nicoladis, E., Pika, S., Yin, H. U. I., & Marentette, P. (2007). Gesture use in story recall by Chinese–English bilinguals. Applied Psycholinguistics, 28(4), 721-735.

Özçalışkan, Ş., & Goldin-Meadow, S. (2005). Gesture is at the cutting edge of early. language development. Cognition, 96(3), B101-B113.

Sassenberg, U., & Van Der Meer, E. (2010). Do we really gesture more when it is more difficult?. Cognitive Science, 34(4), 643-664.

Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press.

So, W. C., Ching, T. H. W., Lim, P. E., Cheng, X., & Ip, K. Y. (2014). Producing gestures facilitates route learning. PLOS One, 9(11), e112543.

Sterelny, K. (2012). Language, gesture, skill: the co-evolutionary foundations of. language. Phil. Trans. R. Soc. B, 367(1599), 2141-2151.

Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and instruction, 4(4), 295-312.

Talmy, L. (2000). Toward a cognitive semantics (Vol. 2). MIT press.

Taub, S. F. (2001). Language from the body: Iconicity and metaphor in American Sign Language. Cambridge: Cambridge University Press.

Tellier, M. (2009). The development of gesture. Language development over the lifespan, de Bot (Ed.), 1(27), 191-216

Mayherry, R. I., & Jaques, J. (2000). Gesture production during stuttered speech: Insights into the nature of gesture-speech integration. Language and gesture, 2(199), 15.

Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational psychologist, 38(1), 1-4.

Ping, R., & Goldin‐Meadow, S. (2010). Gesturing saves cognitive resources when talking about nonpresent objects. Cognitive Science, 34(4), 602-619.

Rauscher, F. H., Krauss, R. M., & Chen, Y. (1996). Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science, 7(4), 226-231.

Rimé, B., Schiaratura, L., Hupet, M., & Ghysselinckx, A. (1984). Effects of relative immobilization on the speaker's nonverbal behavior and on the dialogue imagery level. Motivation and Emotion, 8(4), 311-325.

Robinson, P. (2001). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied linguistics, 22(1), 27-57.

Robinson, P. (2003). Attention and memory during SLA. In C. Doughty & M. Long (Eds.), Handbook of second language acquisition (pp. 631–678). Oxford: Blackwell.

Robinson, P. (2005). Cognitive complexity and task sequencing: A review of studies in a componential framework for second language task design. International Review of Applied Linguistics in Language Teaching, 43(1), 1–32.

Robinson, P. (2007). Criteria for classifying and sequencing pedagogic tasks. In M. P. Garcia Mayo (Ed.), Investigating tasks in formal language learning (pp. 7–27). Clevedon, UK: Multilingual Matters.

Wagner, S. M., Nusbaum, H., & Goldin-Meadow, S. (2004). Probing the mental representation of gesture: Is handwaving spatial?. Journal of Memory and Language, 50(4), 395-407.

QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top