臺灣博碩士論文加值系統

由於電腦化適性測驗的測驗時間可彈性安排，若不同考生間的施測試題重複性高，已應試的考生可能向其他考生分享試題，將危及測驗題庫的效度與測驗分數的公平性。因此，在電腦化適性測驗中，試題曝光控管是非常重要的議題之一。過去常用a分層法進行試題曝光控管，每當題庫中的試題汰舊換新，便須重新模擬以選擇合適的分層數，造成實務者使用不便。為了改善a分層法的問題，Huebner、Wang、Daly與Pinkelman（2018）提出連續a分層指標，題庫不須再事先分層；但在他們的研究中，當考生能力來自常態分佈且題長20、30題的情境下，連續a分層指標仍有近7%、12%的試題超過0.2的曝光率，這表示連續a分層指標仍無法良好控制試題曝光。此外，實務組卷經常需納入數個選題條件，例如內容平衡、答案平衡等，若組卷未能滿足選題條件，將導致測驗效度受到質疑；過去許多研究以最大優先指數（Cheng & Chang, 2009）來達成組卷的選題條件，但尚未有研究將連續a分層指標與之進行結合。
本研究包含兩部分，研究一將曝光控管方法加入連續a分層指標中，期望能改善連續a分層指標試題過度曝光的問題；研究二將連續a分層指標與最大優先指數結合，期望在滿足數個選題條件的同時也控管試題曝光。研究結果顯示，連續a分層在加入曝光控管法後，確實有效改善試題過度曝光的情況但測量精確度略降低；當以連續a分層指標與最大優先指數結合進行選題時，研究發現可以良好控管試題曝光且具有可接受的測量精準度，同時也僅違反極少的選題條件。

Because test takers can arrange the testing time according to their schedule, the former test takers might share the test information with the later ones. If test overlap rates between the former and later test takers were high, it could threaten test security and validity. Hence, item exposure control plays a critical role in computerized adaptive testing. A commonly used method for item exposure control is a-stratification. Whenever the item pool is updated frequently, it is inconvenient for practitioners to partition the item bank and determine appropriate strata. Huebner, Wang, Daly, and Pinkelman (2018) proposed the continuous a-stratification index (CAI), which incorporates exposure control as one building block intrinsic to the index itself, therefore having greater flexibility when applying in an operational framework. They found that when examinees come from the normal distribution, the CAI yielded 7% and 12% items overexposed for 20- and 30-item test length, respectively. It meant that the CAI still cannot control the item exposure rate well. Besides, assembling tests usually requires fulfilling various constraints, such as content balancing, key balancing, and so on; while composing a quiz should include all the constraints to avoid being dubious test validity. Previous researches adopted the maximum priority index (MPI; Cheng & Chang, 2009) to meet many constraints simultaneously; however, no research has been conducted to combine the CAI and MPI for item selection while meeting the constraints.
The purpose of this paper has two fold: (a) to add the item exposure control method into the CAI for reducing the item exposure rates, and (b) to integrate the CAI with the MPI for item selection. Results show that after incorporating the CAI into the exposure control method, the item exposure rate was indeed effectively decreased, but the measurement precision slightly declined. When the CAI combined with the MPI, the item exposure and measurement precision were controlled well, while only a few constraints were violated.

摘要 i
Abstract ii
目次 iii
表次 iv
圖次 v
第一章 緒論 1
第二章 文獻探討 7
第一節 電腦化適性測驗 7
第二節 試題的曝光控管 11
第三節 試題的內容平衡 17
第三章 研究方法 20
第一節 研究一：連續a分層指標的曝光控管 20
第二節 研究二：連續a分層指標的條件加權選題 25
第四章 研究結果與討論 29
第一節 研究一：連續a分層指標的曝光控管 29
第二節 研究二：連續a分層指標的條件加權選題 52
第五章 結論與建議 76
第一節 結論 76
第二節 研究限制與建議 78
參考文獻 80

朱怡君、陳淑英（2008）。「a-分層」電腦適性測驗之曝光率控管。測驗學刊，55（4），793-811。doi:10.7108/PT.200812.0015
吳玫玲、陳淑英（2008）。電腦化適性測驗線上曝光率控管之研究。測驗學刊，55（1），1-32。doi:10.7108/PT.200804.0001
許嘉凌、陳淑英（2007）。控管「變動長度」電腦適性測驗之試題曝光率與測驗重疊率。測驗學刊，54（2），403-427。doi:10.7108/PT.200712.0403
Armstrong, R. D., Jones, D. H., & Kunce, C. S. (1998). IRT test assembly using network-flow programming. Applied Psychological Measurement, 22(3), 237-247. doi:10.1177/01466216980223004
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick, Statistical theories of mental test scores (pp. 397-479). Reading, MA: Addison-Wesley.
Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443-459. doi:10.1007/BF02293801
Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied psychological measurement, 6(4), 431-444. doi:10.1177/014662168200600405
Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213-229. doi:10.1177/014662169602000303
Chang, H.-H., & Ying, Z. (1999). a-stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23(3), 211-222. doi:10.1177/01466219922031338
Chen, S.-Y. (2004). The Analytical Determination of Item Exposure Rates and Trait Estimate Precision in Computerized Adaptive Testing. Psychological Testing, 51(1), 103-115. doi:10.7108/PT.200406.0103
Chen, S.-Y., Ankenmann, R. D., & Spray, J. A. (2003). The relationship between item exposure and test overlap in computerized adaptive testing. Journal of Educational Measurement, 40(2), 129-145. doi:10.1111/j.1745-3984.2003.tb01100.x
Chen, S.-Y., Lei, P.-W., & Liao, W.-H. (2008). Controlling item exposure and test overlap on the fly in computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 61(2), 471-492. doi:10.1348/000711007X227067
Cheng, Y., & Chang, H.-H. (2009). The maximum priority index method for severely constrained item selection in computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 62(2), 369-383. doi:10.1348/000711008X304376
Cheng, Y., Chang, H.-H., Douglas, J., & Guo, F. (2009). Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: Balancing measurement efficiency and exposure control. Educational and Psychological Measurement, 69(1), 35-49. doi:10.1177/0013164408322030
Cheng, Y., Chang, H.-H., & Yi, Q. (2007). Two-phase item selection procedure for flexible content balancing in CAT. Applied Psychological Measurement, 31(6), 467-482. doi:10.1177/0146621606292933
Educational Testing Service. (2018). A Snapshot of the Individuals Who Took the GRE® General Test July 2013–June 2018. Retrieved from https://www.ets.org/s/gre/pdf/snapshot_test_taker_data_2018.pdf
Georgiadou, E. G., Triantafillou, E., & Economides, A. A. (2007). A review of item exposure control strategies for computerized adaptive testing developed from 1983 to 2005. The Journal of Technology, Learning and Assessment, 5(8). Retrieved from https://ejournals.bc.edu/ojs/index.php/jtla/article/view/1647
Hambleton, R. K., & Swaminathan, H. (1985). Item Response Theory: Principles and applications. Boston: Kluwer-Nijhoff Publishing.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Item and Test Information and Efficiency Functions. (Eds.), Fundamentals of Item Response Theory (pp. 91-98). Newbury Park: SAGE.
Huebner, A., Wang, C., Daly, B., & Pinkelman, C. (2018).A Continuous a-Stratification Index for Item Exposure Control in Computerized Adaptive Testing. Applied psychological measurement, 42(7), 523-537. doi:10.1177/0146621618758289
Leung, C. K., Chang, H.-H., & Hau, K.-T. (2005). Computerized adaptive testing: A mixture item selection approach for constrained situations. British Journal of Mathematical and Statistical Psychology, 58(2), 239-257. doi:10.1348/000711005X62945
Lord, F. M. (1977). A broad-range tailored test of verbal ability. Applied Psychological Measurement, 1(1), 95-100. doi:10.1177/014662167700100115
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum. doi:10.4324/9780203056615
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
McBride, J. R., & Martin, J. T. (1983). Reliability and validity of adaptive ability tests in a military setting. In New horizons in testing (pp. 223-236). Academic Press. doi:10.1016/B978-0-12-742780-5.50022-6
Owen, R. J. (1969). A Bayesian approach to tailored testing (Research Bulletin No. 69-92). Princeton, NJ: Educational Testing Service. doi:10.1002/j.2333-8504.1969.tb00771.x
Owen, R. J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70(350), 351-356. doi:10.2307/2285821
Paap, M. C. S., Born, S., & Braeken, J. (2019). Measurement efficiency for fixed-precision multidimensional computerized adaptive tests: Comparing health measurement and educational testing using example banks. Applied psychological measurement, 43(1), 68-83. doi:10.1177/0146621618765719
Patton, J. M., Cheng, Y., Yuan, K.-H., & Diao, Q. (2013). The influence of item calibration error on variable-length computerized adaptive testing. Applied Psychological Measurement, 37(1), 24-40. doi:10.1177/0146621612461727
Revuelta, J., & Ponsoda, V. (1998). A comparison of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement, 35(4), 311-327. doi:10.1111/j.1745-3984.1998.tb00541.x
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph supplement, 17. doi:10.1007/BF02290599
Samejima, F. (1973). A comment on Birnbaum’s three-parameter logistic model in the latent trait theory. Psychometrika, 38(2), 221-233. doi:10.1007/BF02291115
Segall, D. O. (2005). Computerized adaptive testing. Encyclopedia of social measurement, 1, 429-438. doi:10.1016/B0-12-369398-5/00444-8
Stocking, M. L. (1994). Three practical issues for modern adaptive testing item pools (ETS Research Report RR-94-5). Princeton, NJ: Educational Testing Service. doi:10.1002/j.2333-8504.1994.tb01578.x
Stocking, M. L., & Lewis, C. (1995). A new method for controlling item exposure in computerized adaptive testing (ETS Research Report RR-95-25). Princeton, NJ: Educational Testing Service. doi:10.1002/j.2333-8504.1995.tb01660.x
Stocking, M. L., & Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17(3), 277-292. doi:10.1177/014662169301700308
Su, Y.-H. (2016). A comparison of constrained item selection methods in multidimensional computerized adaptive testing. Applied psychological measurement, 40(5), 346-360. doi:10.1177/0146621616639305
Sympson, J. B., & Hetter, R. D. (1985, October).Controlling item-exposure rates in computerized adaptive testing. In Proceedings of the 27th annual meeting of the Military Testing Association (pp. 973-977).
Thissen, D. & Mislevy, R. J. (2000). Testing algorithms. In H. Wainer (Ed.), Computerized adaptive testing: A primer (pp. 101-133). (2nd ed.). Mahwah, NH: Lawrence Erlbaum Associates.
van der Linden, W. J. (2000). Constrained adaptive testing with shadow tests. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 27–52). Boston, MA: Kluwer-Nijhoff. doi:10.1007/0-306-47531-6_2
van der Linden, W. J., & Reese, L. M. (1998). A model for optimal constrained adaptive testing. Applied Psychological Measurement, 22(3), 259-270. doi:10.1177/01466216980223006
Wainer, H. (2000). Computerized adaptive testing: A primer (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. doi:10.4324/9781410605931
Wang, T., & Vispoel, W. P. (1998). Properties of ability estimation methods in computerized adaptive testing. Journal of Educational Measurement, 35(2), 109-135. doi:10.1111/j.1745-3984.1998.tb00530.x
Wang, W.-C., & Chen, P.-H. (2004). Implementation and measurement efficiency of multidimensional computerized adaptive testing. Applied Psychological Measurement, 28(5), 295-316. doi:10.1177/0146621604265938