[1] Chen Qingrong, Jiang Zhijie,. Eye movements during inspecting pictures of natural scenesfor information to verify sentences [J]. Journal of Southeast University (English Edition), 2010, 26 (3): 444-447. [doi:10.3969/j.issn.1003-7985.2010.03.014]

Copy

Eye movements during inspecting pictures of natural scenesfor information to verify sentences()

Share：

Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]

Volumn:

26

Issue:

2010 3

Page:

444-447

Research Field:

Computer Science and Engineering

Publishing date:

2010-09-30

Info

Title:

Eye movements during inspecting pictures of natural scenesfor information to verify sentences

Author(s):

Chen Qingrong¹;  Jiang Zhijie²

¹Department of Psychology, Nanjing Normal University, Nanjing 210097, China
²School of Geography, Nanjing University, Nanjing 210093, China

Keywords:

natural scene;  semantic mismatch;  double negation;  eye movement

PACS:

TP391

DOI:

10.3969/j.issn.1003-7985.2010.03.014

Abstract:

As eye tracking can be used to record moment-to-moment changes of eye movements as people inspect pictures of natural scenes and comprehend information, this paper attempts to use eye-movement technology to investigate how the order of presentation and the characteristics of information affect the semantic mismatch effect in the picture-sentence paradigm. A 3(syntax)×2(semantic relation)factorial design is adopted, with syntax and semantic relations as within-participant variables. The experiment finds that the semantic mismatch is most likely to increase cognitive loads as people have to spend more time, including first-pass time, regression path duration, and total fixation duration. Double negation does not significantly increase the processing difficulty of pictures and information. Experimental results show that people can extract the special syntactic strategy from long-term memory to process pictures and sentences with different semantic relations. It enables readers to comprehend double negation as affirmation. These results demonstrate that the constituent comparison model may not be a general model regarding other languages.

References:

[1] Carpenter P A, Just M A. Sentence comprehension: a psycholinguistic processing model of verification [J]. Psychological Review, 1975, 82(1):45-73.
[2] Clark H H, Chase W G. On the processing of comparing sentences against pictures [J]. Cognitive Psychology, 1972, 3(3):472-517.
[3] Reichle E D, Carpenter P A, Just M A. The neural bases of strategy and skill in sentence-picture verification [J]. Cognitive Psychology, 2000, 40(4):261-295.
[4] Kaup B, Zwaan R. Effects of negation and situational presence on the accessibility of text information [J]. Journal of Experimental Psychology: Learning, Memory, and Cognition, 2003, 29(3):439-446.
[5] Schroyens W, Verschueren N, Schaeken W, et al. Conditional reasoning with negations: implicit and explicit affirmation or denial and the role of contrast classes [J]. Thinking and Reasoning, 2000, 6(3):221-251.
[6] Tramow D, Finlay K A. The prediction of attitudes from beliefs and evaluations: the logic of the double negative [J]. British Journal of Social Psychology, 2002, 41(1):77-86.
[7] Underwood G, Jebbett L, Roberts K. Inspecting picture for information to verify a sentence: eye movements in general coding and in focused search [J]. Quarterly Journal of Experimental Psychology, 2004, 57(1):165-182.
[8] Underwood G, Crundall D, Hodson K. Confirming statements about pictures of natural scenes: evidence of the processing of gist from eye movements [J]. Perception, 2005, 34(9):1069-1082.
[9] Knoeferle P, Crocker M W, Pickering M, et al. The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye-movements in depicted events [J]. Cognition, 2005, 95(1):95-127.
[10] Chen Q R, Deng Z, Tan D L. Sentence-picture integration in Chinese: evidence from the measurement of eye movement [J]. Acta Psychologica Sinica, 2008, 40(5): 543-551.
[11] Traxler M, McElree B, Williams R S, et al. Context effects in coercion: evidence from eye movements [J]. Journal of Memory and Language, 2005, 53(1): 1-25.
[12] Calvo M G. Working memory and inferences: evidence from eye fixations during reading [J]. Memory, 2001, 9(4): 365-381.
[13] Rayner K, Warren T, Juhasz B, et al. The effect of plausibility on eye movement in reading [J]. Journal of Experimental Psychology: Learning, Memory, and Cognition, 2004, 30(6): 1290-1301.
[14] Caplan D, DeDe G, Brownell H. Effects of syntactic features on sentence-picture matching in Broca’s aphasics: a replay to Drai and Grodzinksy [J]. Brain and Language, 2006, 96(2):129-134.
[15] Knoeferle P, Crocker M W. Incremental effects of mismatch during picture-sentence integration: evidence from eye-tracking [C]//Proceedings of the 26th Annual Conference of the Cognitive Science Society. Mahwah, NJ, USA: Lawrence Erlbaum, 2005:1166-1172.

Memo

Memo:

Biography: Chen Qingrong(1980—), male, doctor, jscqr80@sina.com.
Foundation items: The National Social Science Foundation of China(No.CBA080236), the Graduate Innovation Project of Jiangsu Province(No.CX08B_-016R).
Citation: Chen Qingrong, Jiang Zhijie.Eye movements during inspecting pictures of natural scenes for information to verify sentences[J].Journal of Southeast University(English Edition), 2010, 26(3):444-447.

Common functions