Common Framework for Mathematics – Discussions of Possibilities to Develop a Set of General Standards for Assessing Proficiency in Mathematics

Jari Metsämuuronen

doi:10.12973/iejme/2693

Full Text (PDF)

Jari Metsämuuronen ¹ ^*

More Detail

¹ NLA University College, Bergen, NORWAY^* Corresponding Author

Abstract

The article discusses the challenges of and solutions of developing a common, general standards-referenced student assessment framework for mathematics. Two main challenges are faced. First, the main challenge is the lack of commonly accepted standards as the basis for the criterion- and standards-referenced assessment. Second challenge is that, even if having criteria and standards for the assessment, the descriptions of the standards are, in many cases, so vaguely worded that it is not possible to create unambiguous test items on the basis of a specific level of the standards. An initial common framework for mathematics standards is introduced on the basis of Common European Framework in Reference for Languages (CEFR) – a common framework for mathematics (CFM).

Keywords

assessment
standards-referenced assessment
criterion-referenced assessment
Mathematics

License

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Article Type: Research Article

INT ELECT J MATH ED, Volume 13, Issue 2, May 2018, 13-39

https://doi.org/10.12973/iejme/2693

Publication date: 31 May 2018

Article Views: 4457

Article Downloads: 3078

Open Access References How to cite this article

References

Acharya, D. R., & Metsämuuronen, J. (2014). Nepali Achievement at Grade 3 in NASA 2012. In ERO, Student achievement in Mathematics, Nepali and English at grade 3 and 5 in 2011. Ministry of Education, Kathmandu, Nepal.
Acharya, D. R., Metsämuuronen, J., & Adhikari, H. (2013). Nepali Achievement in NASA 2011. In J. Metsämuuronen & B. R. Kafle (Eds.), Where Are We Now? Student achievement in Mathematics, Nepali and Social Studies in 2011 (pp. 113–170). Ministry of Education, Kathmandu, Nepal.
AERA, APA, & NCME. (2014). Standards for Educational and Psychological Testing. Washington DC: American Educational Research Association, American Psychological Association. National Council of Measurement in Education.
Angoff, W. H. (1974). Criterion referencing, norm referencing and the SAT. College Board Review, 92, pp. 3–5, 21.
Béguin, A. (2000). Robustness of Equating High-Stake Tests. Enschede: Febodruk B.V.
Brown, S. (1988). Criterion referenced assessment: what role for research? In H. D. Black, & W. D. Dockerell, New developments in educational assessment, British Journal of Educational Psychology, Monograph series, 3, 1–14.
Bruner, J. S. (1986). Actual Minds, Possible Worlds, Cambridge, Massachusetts: Harvard University Press.
Bruner, J. S. (1990) Culture and Human Development: A New Look. Human Development 33 (6), 344–355. https://doi.org/10.1159/000276535
Bruner, J. S. (1996). The Culture of Education, Cambridge, London: Harvard University Press.
CLS. (2014). Research impact case study. The impact of adult literacy and numeracy research based on the 1970 British Cohort Study. Centre for Longitudinal Studies. Institute of Education, University of London. Retrieved on October 18, 2016 from http://www.cls.ioe.ac.uk/library-media%5Cdocuments%5CFINAL%20-%20CLS%20impact%20case%20study%20-%20Adult%20literacy%20and%20numeracy.pdf
Council of Europe. (2011). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Council of Europe. Retrieved on October 18, 2016 from http://www.coe.int/t/dg4/linguistic/Source/Framework_EN.pdf
Cox, B. (1995). The Battle for the English Curriculum. London: Hodder & Stoughton.
Criterion-referenced Test. (2014). In S. Abbott (Ed.), The glossary of education reform. Retrieved on October 18, 2016 from http://edglossary.org/criterion-referenced-test/
Daro, P., McCallum, W., & Zimba, J. (2012). The Structure is the Standard. In Tools for the Common Core Standards. Retrieved on October 18, 2016 form http://commoncoretools.me/2012/02/16/the-structure-is-the-standards/
Dziuban, C. D., & Vickery, K. V. (1973). Criterion-referenced Measurement: Some Recent Developments. Educational Leadership, February 1973, 483–486. Retrieved on October 18, 2016 from http://www.ascd.org/ASCD/pdf/journals/ed_lead/el_197302_dziuban.pdf
ERO. (2015). Unpublished output of a workshop of creating National standards-reference assessment system for Nepali, English, Science and Mathematics at 4. – 22.12.2015 in Educational Review Office (ERO), Sanothimi, Bakhtapur, Nepal.
FNBE. (2003). National Core Curriculum for Upper secondary Schools 2003. Finnish Board of Education. Vammala: Vammalan Kirjapaino Oy. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/download/47678_core_curricula_upper_secondary_education.pdf
FNBE. (2004). National Core Curriculum for Basic Education 2004. Finnish Board of Education. Helsinki: Yliopistopaino. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/english/curricula_and_qualifications/basic_education/curricula_2004
FNBE. (2014). National Core Curriculum 2014. Finnish Board of Education. Määräykset ja ohjeet 2014:96. Helsinki: Next Print Oy. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/download/163777_perusopetuksen_opetussuunnitelman_perusteet_2014.pdf [In Finnish]
FNBE. (2015). National Core Curriculum for Upper secondary Schools 2015. Finnish Board of Education. Määräykset ja ohjeet 2015:48. Helsinki: Next Print Oy. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/download/172124_lukion_opetussuunnitelman_perusteet_2015.pdf [In Finnish]
Glaser, R. (1963). Instructional Technology and the Measurement of Learning Outcomes. American Psychologist, 18, 519–521. Reprinted in Educational Measurement: Issues and Practice, 13(4), 6 – 8. https://doi.org/10.1111/j.1745-3992.1994.tb00561.x
Glaser, R., & Klaus, D. J. (1962). Proficiency measurements: Assessing human performance. In R. M. Gagne (Ed.), Psychological Principles in Systems Development (pp. 419 – 474). New York: Holt, Reinhart & Winston.
Glaser, R., & Nitko, A. (1971). Measurement in Learning and Instruction. In R. L. Thorndike (Ed.), Educational Measurement (pp. 625 –670). Washington DC: American Council of Education.
Glass, G. V. (1978). Standards and criteria, Journal of Educational Measurement, 15, 237–261. https://doi.org/10.1111/j.1745-3984.1978.tb00072.x
Gray, W. M. (1978). A comparison of Piagetian theory and criterion-referenced measurement. Review of Educational Research, 48, 223–249. https://doi.org/10.3102/00346543048002223
Green, S. (2002). Criterion Referenced Assessment as a Guide to Learning – The Importance of Progression and Reliability. A paper presented at the Association for the Study of Evaluation in Education in Southern Africa International Conference Johannesburg, 10 July – 12 July 2002. Retrieved on October 18, 2016 from http://www.cambridgeassessment.org.uk/images/109693-criterion-referenced-assessment-as-a-guide-to-learning-the-importance-of-progression-and-reliability.pdf
Hambleton, R. K. (1980). Contributions to Criterion-referenced Testing Technology: An Introduction. Applied Psychological Measurement, 4(4), 421–424. Doi: http://dx.doi.org/10.1177/014662168000400401. Retrieved from https://conservancy.umn.edu/bitstream/handle/11299/100198/v04n4p421.pdf?sequence=1
Hambleton, R. K. (1981). Advances in criterion-referenced testing technology. In C. R. Reynolds & T. B. Gutkin (Eds.), Handbook of school psychology (pp. 351–379). New York: Wiley.
Hannula, M. (2015). Email discussion at November 6th, 2015.
K-8 Publishers. (2013). K–8 Publishers’ Criteria for the Core State Standards for Mathematics. Spring 2013 Release 04/09/2013. Retrieved on October 18, 2016 from http://achievethecore.org/content/upload/Math_Publishers_Criteria_K-8_Spring_2013_FINAL.pdf
Kaftandjieva, F. (2004). Standard Setting. Reference Supplement, Section B. In S. Takala (Ed.) (2009), Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR). A Manual. Language Policy Division, Strasbourg. Retrieved on October 18, 2016 from https://www.coe.int/t/dg4/linguistic/CEF-refSupp-SectionB.pdf
Kane, M. T. (1986). The role of reliability in criterion-referenced tests. Journal of Educational Measurement, 23(3), 221–224. https://doi.org/10.1111/j.1745-3984.1986.tb00247.x
Kuukka, K., & Metsämuuronen, J. (2016). Perusopetuksen päättövaiheen suomi toisena kielenä (S2) -oppimäärän oppimistulosten arviointi 2015 [Assessment of the learning outcomes of the L2 Finnish 2015.] Kansallinen koulutuksen arviointikeskus. Julkaisut 13:2016. Tampere: Juvenes Print – Suomen Yliopistopaino Oy. https://doi.org/10.13140/RG.2.1.2569.0005 Retrieved on October 18, 2016 from http://karvi.fi/app/uploads/2016/05/KARVI_1316.pdf [In Finnish].
Linn, R. L. (1993). Linking results of distinct assessment. Applied Measurement in Education, 6(1), 83–102. https://doi.org/10.1207/s15324818ame0601_5
Linn, R. L., & Gronlund, N. E. (2000). Measurement and assessment in teaching (8th ed.). Upper Saddle River, NJ: Prentice Hall.
Lithner, J., Bergqvist, E., Bergqvist, T., Boesen, J., Palm, T. & Palmberg, B. (2010). Mathematical competencies: A research framework. In C. Bergsten, E. Jablonka, & T. Wedege (Eds.), Mathematics and mathematics education: Cultural and social dimensions. Proceedings of MADIF 7, the Seventh Mathematics Education Research Seminar, Stockholm, January 26-27 (pp. 157–167). Linköping, Sweden: Svensk förening för matematikdidaktisk forskning, SMDF.
MEB. (2016). The Finnish Matriculation Examination. The Matriculation Examination Board. Retrieved on October 18, 2016 from https://www.ylioppilastutkinto.fi/fi/english
Mehrens, W. A., & Lehmann, I. J. (1991). Measurement and Evaluation in Education and Psychology. 4th Ed. Fort Worth, TX: Harcourt Brace College Publishers.
Metsämuuronen, J. & Mattsson, M. (2013). Effect of repeated testing to the development of Vocabulary, Nominal Structures, and Verbal morphology. Journal of Educational and Psychological Development, 3(2), 89–101. https://doi.org/10.5539/jedp.v3n2p89 Retrieved on October 18, 2016 from http://www.ccsenet.org/journal/index.php/jedp/article/view/28330
Metsämuuronen, J. (2009). Ratkaisuprosentin muuntaminen taitotasoksi – A-Finska-oppiaineen aineiston erityiskysymyksiä [Changing the percentage of correct answers into proficiency level – Specific questions of A-Finnish assessment]. Unpublished memorandum 28.12.2009. [In Finnish]
Metsämuuronen, J. (2010). Omvandlingen av poängtal till färdighetsnivåer inom det receptive delområdet av provet i A-lärokursen och i den modersmålsinriktade lärokursen. [Changing the percentage of correct answers into proficiency level in Finnish language for Swedish speakers A-course and native speakers’ course test]. Appendix VII. In O. Toropainen (2010). Utvärdering av läroämnet finska i den grundläggande utbildningen. Inlärningsresultaten i finska enligt A-lärokursen och den modersmålsinriktade lärokursen i årskurs 9 våren 2009 (pp. 164–168). Uppföljningsrapporter 2010:1. Utbildningsstyrelsen. Helsinki. Retrieved from http://www.oph.fi/publikationer/2010/utvardering_av_laroamnet_finska_i_den_grundlaggande_utbildningen [In Swedish].
Metsämuuronen, J. (2013). A New Method to Setting Standard for the Wide Range of Language Proficiency Levels. International Education Research, 1(1). https://doi.org/10.12735/ier.v1i1p01 Retrieved on October 18, 2016 from http://www.todayscience.org/IER/v1/Jari%20Metsamuuronen.pdf
Metsämuuronen, J. (2014). English Achievement at Grade 5 in NASA 2012. In ERO, Student achievement in Mathematics, Nepali and English at grade 3 and 5 in 2011. Ministry of Education, Kathmandu, Nepal.
Metsämuuronen, J. (2016a). Challenges in the National Level Criterion-referenced assessment. Nepalese Journal of Educational Assessment, 1(1), 17 – 36.
Metsämuuronen, J. (2016b). Oppia ikä kaikki. Matemaattinen osaaminen toisen asteen koulutuksen lopulla. [Learning all my days. Mathematical achievement at the end of the upper secondary education.] Kansallinen koulutuksen arviointikeskus. Julkaisut 2016. Tampere: Juvenes Print – Suomen Yliopistopaino Oy. [forthcoming] [in Finnish]
Metsämuuronen, J., & Salonen, R. V. (2016). Student achievement in Mathematics in Vocational Education. Kansallinen koulutuksen arviointikeskus. Julkaisut 2016. Tampere: Juvenes Print – Suomen Yliopistopaino Oy. [forthcoming] [In Finnish]
Metsämuuronen, J., Acharya, S. P., & Aryal, H. (2013). Setting Standard in Nepali Language – CEFR levels. In J. Metsämuuronen, & B. R. Kafle (Eds). Where Are We Now? Student achievement in Mathematics, Nepali and Social Studies in 2011 (pp. 393–422). Ministry of Education, Kathmandu, Nepal.
Mislevy, R. J. (1992). Linking educational assessments: Concepts, issues, methods, and prospects. Princeton, NJ: ETS Policy Information Center.
Niss, M. & Jensen, T. H. (Eds.). (2002). Kompetencer og matematiklæring: Ideer og inspiration til udvikling af matematikundervisning i danmark. København: Undervisningsministeriets forlag.
Niss, M., & Højgaard, T. (Eds.) (2011). Competencies and Mathematical Learning. Ideas and Inspiration for the development of mathematics teaching and learning in Denmark. English Edition, October 2011. Roskilde University, Department of Science, Systems and Models, IMFUFA. nr. 485 – 2011. Retrieved on October 18, 2016 from http://rudar.ruc.dk//bitstream/1800/7375/1/IMFUFA_485.pdf
Nitko, A. J. (1980). Distinguishing the many varieties of criterion-referenced tests. Review of Educational Research, 50, 461–485. https://doi.org/10.3102/00346543050003461
OECD. (2014). PISA 2012 Technical report. Paris: OECD. Retrieved on October 18, 2016 from http://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf
Oxford English Dictionary (1983). Oxford University Press.
Parsons, S. (2012). User guide to accompany. The 1970 British Cohort Study 2004 adult literacy and numeracy assessment data. Centre for Longitudinal Studies. Institute of Education, University of London. Retrieved on October 18, 2016 from www.cls.ioe.ac.uk/bcs70_userguide_literacynumeracy
Poldrack, R. A. & Packard, M. G. (2003). Competition among multiple memory systems: Converging evidence from animal and human brain studies. Neuropsychologia, 41(3), 245–251. https://doi.org/10.1016/S0028-3932(02)00157-4
Pollitt, A. (1994). Measuring and evaluating reliability in national curriculum assessments. In D. Hutchinson & I. Schagen, Eds, How reliable is national curriculum assessment? (pp. 60 – 70). London: NFER.
Popham, W. J. (1978a). Well-crafted criterion-referenced tests. Educational Leadership, 36(2), 91–95.
Popham, W. J. (1978b). Criterion-referenced measurement. Eaglewood Cliffs, NJ: Prentice-Hall.
Popham, W. J. (1980). Domain specification strategies. In Berk, R.A. (Ed,), Criterion-referenced measurement: the state of the art (pp. 15–31). Baltimore and London: John Hopkins University Press.
Popham, W. J. (1981). Modern educational measurement. Englewood Cliffs, NJ: Prentice-Hall.
Popham, W. J. (2014). Criterion-referenced Measurement: Half a Century Wasted? Educational Leadership, 71(6), 62–66.
Popham, W. J., & Husek, T. R. (1969). Implications of criterion-referenced measurement. Journal of Educational Measurement, 6(1), 1–9. https://doi.org/10.1111/j.1745-3984.1969.tb00654.x
QCA. (2006). The National Qualifications Framework. Qualifications and Curriculum Authority, Qualifications, Curriculum and Assessment Authority for Wales, and Council for the Curriculum, Examinations and Assessment. Retrieved on October 18, 2016 from http://web.archive.org/web/20090126170205/http://qca.org.uk/libraryAssets/media/qca-06-2298-nqf-web.pdf
Räkköläinen, M., & Metsämuuronen, J. (2016). Student achievement in Sustainable Development in Vocational Education. FINEEC. [Forthcoming]
Räsänen, P. (2015). Email discussion at 2015 November 9th.
Raynolds, C. R., & Fletcher-Janzen, E. (Eds.) (2004). Criterion-referenced Testing. In Concise Encyclopedia of Special Education: A Reference for the Education of the Handicapped and Other Exceptional Children and Adults. 2nd Edition. New York, NY: Wiley.
Sadler, D. R. (1987). Specifying and Promulgating Achievement Standards. Oxford Review of Education, 13(2), 191—209. https://doi.org/10.1080/0305498870130207 Retrieved on October 18, 2016 from http://www.clinteach.com.au/assets/Specifying-and-Promulgating-Achievement-Standards.pdf
Sadler, D. R. (2009) Indeterminacy in the use of preset criteria for assessment and grading in higher education. Assessment and Evaluation in Higher Education, 34, 159–179. https://doi.org/10.1080/02602930801956059
Sadler, D. R. (2012). Assessment, evaluation and quality assurance: Implications for integrity in reporting academic achievement in higher education. Education Inquiry. 3(2), 201–216. Doi: https://doi.org/10.3402/edui.v3i2.22028 Retrieved on October 18, 2016 from http://www.education-inquiry.net/index.php/edui/article/view/22028
Säftström, A. I. (2013). Exercising Mathematical Competence. Practising Representation Theory and Representing Mathematical Practice. Thesis for the Degree of Doctor of Philosophy. Göteborgs Universitet. Retrieved on October 18, 2016 from http://hdl.handle.net/2077/32484
Shakya, S., & Metsämuuronen, J. (2014). Nepali Achievement at Grade 5 in NASA 2012. In ERO, Where Are the grade 3 and 5 students now? Student achievement in Mathematics, Nepali and English at grade 3 and 5 in 2011. Ministry of Education, Kathmandu, Nepal.
Squire, L. R. (2009). Memory and brain systems: 1969–2009. The Journal of Neuroscience, 29(41), 12711–12716. https://doi.org/10.1523/JNEUROSCI.3575-09.2009 Retrieved on October 18, 2016 from http://www.jneurosci.org/content/29/41/12711.full
Takala, S., & Kaftandjieva, F. (2009). Reseptiivisen kielitaidon kokeen muuntaminen taitotasoiksi. [Changing the receptive language test to proficiency levels]. In E Tuokko, Miten Ruotsia osataan peruskoulussa? Perusopetuksen päättövaiheen ruotsin kielen B-oppimäärän oppimistulosten kansallinen arviointi 2008 (p. 118). [How Swedish language is spoken in general education? National assessment 2008 of the student achievement in Swedish language B-course at the end of basic education]. Oppimistulosten arviointi 2/2009. Opetushallitus. Helsinki: Edita Prima Oy. [in Finnish]. Retrieved on October 18, 2016 from http://www.oph.fi/julkaisut/2009/miten_ruotsia_osataan_peruskoulussa
Toropainen, O. (2010). Utvärdering av läroämnet finska i den grundläggande utbildningen. Inlärningsresultaten i finska enligt A-lärokursen och den modersmålsinriktade lärokursen i årskurs 9 våren 2009. Uppföljningsrapporter 2010:1. Utbildningsstyrelsen. Helsinki. [In Swedish]. Retrieved on October 18, 2016 from http://www.oph.fi/publikationer/2010/utvardering_av_laroamnet_finska_i_den_grundlaggande_utbildningen
Tulving, E. (1983). Elements of Episodic Memory, New York: Oxford University Press.
Ullman, M.T. (2004) Contributions of memory circuits to language: the declarative/procedural model. Cognition. 92, 231–270.
Wiliam, D. (1993) Validity, dependability and reliability in National Curriculum assessment, The Curriculum Journal, 4(3), 335–350. https://doi.org/10.1080/0958517930040303

How to cite this article

APA

Metsämuuronen, J. (2018). Common Framework for Mathematics – Discussions of Possibilities to Develop a Set of General Standards for Assessing Proficiency in Mathematics. International Electronic Journal of Mathematics Education, 13(2), 13-39. https://doi.org/10.12973/iejme/2693

Vancouver

Metsämuuronen J. Common Framework for Mathematics – Discussions of Possibilities to Develop a Set of General Standards for Assessing Proficiency in Mathematics. INT ELECT J MATH ED. 2018;13(2):13-39. https://doi.org/10.12973/iejme/2693

AMA

Metsämuuronen J. Common Framework for Mathematics – Discussions of Possibilities to Develop a Set of General Standards for Assessing Proficiency in Mathematics. INT ELECT J MATH ED. 2018;13(2), 13-39. https://doi.org/10.12973/iejme/2693

Chicago

Metsämuuronen, Jari. "Common Framework for Mathematics – Discussions of Possibilities to Develop a Set of General Standards for Assessing Proficiency in Mathematics". International Electronic Journal of Mathematics Education 2018 13 no. 2 (2018): 13-39. https://doi.org/10.12973/iejme/2693

Harvard

Metsämuuronen, J. (2018). Common Framework for Mathematics – Discussions of Possibilities to Develop a Set of General Standards for Assessing Proficiency in Mathematics. International Electronic Journal of Mathematics Education, 13(2), pp. 13-39. https://doi.org/10.12973/iejme/2693

MLA

Metsämuuronen, Jari "Common Framework for Mathematics – Discussions of Possibilities to Develop a Set of General Standards for Assessing Proficiency in Mathematics". International Electronic Journal of Mathematics Education, vol. 13, no. 2, 2018, pp. 13-39. https://doi.org/10.12973/iejme/2693