The Grainger College of Engineering
Siebel School of Computing and Data Science

Computing Education Research Area Qualifying Exam

Last Updated: August 1, 2024
Previous Computing Education Research Quals website 

Qualifying Exam Process

Why do we have this quals process?
As the Computers and Education research area is in a School of Computing and Data Science, most of us come to this research area with breadth and depth in computing but little formal experience with the field of education or education research. While the process of conducting dissertation research helps give our PhD students depth in one or two aspects of education research, we have few mechanisms for helping all graduate students develop the breadth that can sustain your research over a career. We are changing our qualifying exam process to help students gain this breadth of knowledge. We believe that this breadth of perspectives on education research can help you be a reviewer who is able to comment on a wider variety of studies, give you more tools to tackle the complex problems in education research, support your collaborations with more people, and help you stand out in the computing education research community. We hope that this new qual process will provide you with a wide foundation on which to build upon.

What are we doing to help you prepare for the qual?
First, we are offering a new CS 598 course (section KMC) that will provide you with course credit for reading a significant portion of the literature we expect you to know for the qualifying exam. The course will also focus on giving you feedback on your writing to prepare you to write the essay part of your exam. We are also reaffirming that we expect all PhD students in the area to attend the research group and reading group every week. The CS598 course and required quals readings focus on the foundations within the field, which forms the trunk of your research skills tree. These weekly meetings are the branches that will grow from the trunk provided in the CS 598 course. We will help you make connections between newer papers and new research back to the central trunk. The full reading list can be found here.

Overview
The qualifying exam for the Computers and Education Research Area has two parts: a written exam and oral exam. Students must submit a qualifying exam statement by the departmental deadline. The area chair will assign a qualifying exam committee for the student and the committee will decide upon a set of question prompts for the written exam. At least one week before the oral exam, the student will complete their written exam during a 36-hour period. The committee will then review the students’ written exam and discuss the students’ responses during a follow-up 2-hour oral exam.

Requirements

  • The student must have a signed advisor agreement with a faculty member in the Computers and Education Group.
  • The student must take at least one 400- or 500-level course in educational theory with a minimum grade of B+. Students are particularly encouraged to take the Qualifying Exam Course.
  • The student must take one 400- or 500-level research methods (e.g., quantitative, qualitative, human-computer interactions methods, and/or educational research methods) course with a minimum grade of B+.
  • Pass the qualifying exam consisting of a written exam and oral exam

Written Exam

  • Due 1 week before the oral exam
  • The student will schedule 2 4-hour periods within a 36-hour period during which to complete their written exam.
  • The written exam will comprise responses to three questions. Questions may cover readings from the required reading list, topics discussed in the Qualifying Exam Course, and providing a peer review of a research article from the field of Computers and Education.
    • One 4-hour period will focus on two questions about the readings. The student may use whatever notes they have collected prior to the written exam to assist them in answering the written exam questions, but may not use internet searches or other online resources during this part of the exam. The committee will send the student their questions at the start of the 4-hour period.
    • One 4-hour period will focus on providing a peer review of a research article, using the ICER Review Criteria. The student may use whatever notes they have collected prior to the written exam and may use internet searches to look up references cited in the paper. The committee will send the student their article to review at the start of the 4-hour period.
  • The student will write all of the qualifying exam essays in a Google doc shared with the committee.
  • Students may work and consult with other students and their advisor in preparation for the written exam but may not consult with other students or faculty about the exam during the 36-hour period. Students may ask their committees clarifying questions.

Oral Exam

  • The student will begin by briefly summarizing (< 20 minutes) their responses to the written exam
  • The remainder of the oral exam will be a Q&A by the committee focused primarily on responses to the written exam and may also explore questions related to the required readings not explored by the written exam.
  • The oral exam will take about 1.5 hours but students should schedule 2 hours for the exam.

Qualifying Exam Committee
The Area Chair will assemble a qualifying exam committee (henceforth the “committee”) that will include 3-4 members of the Core Faculty. Other faculty members with graduate advising privileges with relevant computing and/or education expertise may be substituted. The PhD advisor is not a member of the committee.  If the Area Chair is the PhD advisor, then the list of proposed committee members will be assembled by another Core Faculty member.  The Chair of the committee must be a Core Faculty member.

History
The Computers and Education area qual was modified to its current form in Summer 2023.

 

Computers and Education Core Reading List

  • Why teach CS? 
    Lewis, C. M. (2017). Good (and Bad) Reasons to Teach All Students Computer Science. In S. B. Fee, A. M. Holland-Minkley, & T. E. Lombardi (Eds.), New Directions for Computing Education (pp. 15–34). Springer International Publishing. https://doi.org/10.1007/978-3-319-54226-3_2
  • History – Overview of the field:
    Guzdial, M., & du Boulay, B. (2019). The History of Computing Education Research. In S. A. Fincher & A. V. E. Robins (Eds.), The Cambridge Handbook of Computing Education Research (pp. 11–39). Cambridge University Press. https://doi.org/10.1017/9781108654555.002
  • Overview: Novice programmers overview: 
    Robins, A. V. (2019). Novice Programmers and Introductory Programming. In S. A. Fincher & A. V. E. Robins (Eds.), The Cambridge Handbook of Computing Education Research (pp. 327–376). Cambridge University Press. https://doi.org/10.1017/9781108654555.013
  • Cognitive Science:
    Robins, A. V., Margulieux, L. E., & Morrison, B. B. (2019). Cognitive Sciences for Computing Education. In S. A. Fincher & A. V. E. Robins (Eds.), The Cambridge Handbook of Computing Education Research (pp. 231–275). Cambridge University Press. https://doi.org/10.1017/9781108654555.010
     save the comment
  • Plans/Patterns: 
    Soloway, E. (1986). Learning to program = learning to construct mechanisms and explanations. Communications of the ACM, 29(9), 850–858. https://doi.org/10.1145/6592.6594
  • Conversational Programmers:
    Cunningham, K., Ericson, B. J., Agrawal Bejarano, R., & Guzdial, M. (2021). Avoiding the Turing Tarpit: Learning Conversational Programming by Starting from Code’s Purpose. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–15. https://doi.org/10.1145/3411764.3445571
  • Transfer:
    Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn?: A taxonomy for far transfer. Psychological Bulletin, 128(4), 612–637. https://doi.org/10.1037/0033-2909.128.4.612
  • Computational Thinking:
    Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining Computational Thinking for Mathematics and Science Classrooms. Journal of Science Education and Technology, 25(1), 127–147. https://doi.org/10.1007/s10956-015-9581-5
  • Conceptual Change: 
    diSessa, A. A. (2014). A History of Conceptual Change Research: Threads and Fault Lines. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (2nd ed., pp. 88–108). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.007
  • Overview of Learning Sciences:
    Margulieux, L. E., Dorn, B., & Searle, K. A. (2019). Learning Sciences for Computing Education. In S. A. Fincher & A. V. E. Robins (Eds.), The Cambridge Handbook of Computing Education Research (pp. 208–230). Cambridge University Press. https://doi.org/10.1017/9781108654555.009
  • Embodied Cognition/Distributed Cognition: 
    Abrahamson, D., & Lindgren, R. (2022). Embodiment and Embodied Design. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (3rd ed., pp. 301–320). Cambridge University Press. https://doi.org/10.1017/9781108888295.019
    https://ccl.northwestern.edu/2022/Dor%20Embod.pdf 
  • Constructionism:
    Tissenbaum, M., Weintrop, D., Holbert, N., & Clegg, T. (2021). The case for alternative endpoints in computing education. British Journal of Educational Technology, 52(3), 1164–1177. https://doi.org/10.1111/bjet.13072
  • BPC – K-12 Overview: 
    Margolis, J., Estrella, R., Goode, J., Jellison Holme, J., & Nao, K. (2008). Stuck in the shallow end: Education, race, and computing. MIT Press.
  • BPC – Racial Climate: 
    Harper, S. R., & Hurtado, S. (2007). Nine themes in campus racial climates and implications for institutional transformation. New Directions for Student Services, 2007(120), 7–24. https://doi.org/10.1002/ss.254
  • Representations: 
    Johnson‐Glauch, N., Choi, D. S., & Herman, G. (2020). How engineering students use domain knowledge when problem‐solving using different visual representations. Journal of Engineering Education, 109(3), 443–469. https://doi.org/10.1002/jee.20348
  • Implicit Learning/Dual Process 
    Kahneman, D. (2013). Thinking, fast and slow (1st pbk. ed). Farrar, Straus and Giroux. Chapter 1 
  • Tutoring / self-explanation effect:
    Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25(4), 471–533. https://doi.org/10.1207/s15516709cog2504_1
  • Student perception of learning: 
    Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257. https://doi.org/10.1073/pnas.1821936116
  • Active Learning:
    Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64–74. https://doi.org/10.1119/1.18809
  • Problem Based learning:
    Barron, B., Schwartz, D. L., Vye, N., Moore, A. L., Petrosino, A. J., Zech, L. K., & Bransford, J. D. (1998). Doing with Understanding: Lessons from Research on Problem- and Project-Based Learning. The Journal of the Learning Sciences, 7, 271–311.
  • Cognitive apprenticeship: 
    Collins, A., & Kapur, M. (2014). Cognitive Apprenticeship. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (2nd ed., pp. 109–127). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.008
  • Collaboration: 
    Nokes-Malach, T. J., Richey, J. E., & Gadgil, S. (2015). When Is It Better to Learn Together? Insights from Research on Collaborative Learning. Educational Psychology Review, 27(4), 645–656. https://doi.org/10.1007/s10648-015-9312-8
  • Inquiry vs. Direct Instruction
    Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 41(2), 75–86. https://doi.org/10.1207/s15326985ep4102_1
  • Order of learning activities: 
    Xie, B., Loksa, D., Nelson, G. L., Davidson, M. J., Dong, D., Kwik, H., Tan, A. H., Hwa, L., Li, M., & Ko, A. J. (2019). A theory of instruction for introductory programming skills. Computer Science Education, 29(2–3), 205–253. https://doi.org/10.1080/08993408.2019.1565235
  • Double Bind – Women of Color:
    Ong, M., Wright, C. A., Espinosa, L. L., & Orfield, G. (2011). Inside the Double Bind: A Synthesis of Empirical Research on Undergraduate and Graduate Women of Color in Science, Technology, Engineering, and Mathematics. Harvard Educational Review, 81, 172–209.
  • Paradigms: 
    Greeno, J. G., Collins, A. M., & Resnick, L. (1996). Cognition and learning. In Cognition and Learning (pp. 15–46).
    Gutiérrez, R. (2013). The Sociopolitical Turn in Mathematics Education. Journal for Research in Mathematics Education, 44(1), 37–68. https://doi.org/10.5951/jresematheduc.44.1.0037
 

Note – Papers after this point are included in other classes, but not the quals class Fall 2023. 

 
  • Quantitative Methods: 
    Haden, P. (2019). Descriptive Statistics. In S. A. Fincher & A. V. E. Robins (Eds.), The Cambridge Handbook of Computing Education Research (pp. 102–132). Cambridge University Press. https://doi.org/10.1017/9781108654555.006
    Haden, P. (2019). Inferential Statistics. In S. A. Fincher & A. V. E. Robins (Eds.), The Cambridge Handbook of Computing Education Research (pp. 133–172). Cambridge University Press. https://doi.org/10.1017/9781108654555.007
  • Qualitative Methods: 
    Merriam, S. B., & Tisdell, E. J. (2015). Qualitative research: A guide to design and implementation (Fourth edition). John Wiley & Sons.
  • Learner models:
    Rosé, C. P., McLaughlin, E. A., Liu, R., & Koedinger, K. R. (2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943–2958. https://doi.org/10.1111/bjet.12858
  • Databases data mining: 
    Yang, S., Wei, Z., Herman, G. L., & Alawini, A. (2021). Analyzing Patterns in Student SQL Solutions via Levenshtein Edit Distance. Proceedings of the Eighth ACM Conference on Learning @ Scale, 323–326. https://doi.org/10.1145/3430895.3460979
  • Expert-novice differences 
    Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and Representation of Physics Problems by Experts and Novices*. Cognitive Science, 5(2), 121–152. https://doi.org/10.1207/s15516709cog0502_2
  • Notional machines:
    Fincher, S., Jeuring, J., Miller, C. S., Donaldson, P., Du Boulay, B., Hauswirth, M., Hellas, A., Hermans, F., Lewis, C., Mühling, A., Pearce, J. L., & Petersen, A. (2020). Notional Machines in Computing Education: The Education of Attention. Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education, 21–50. https://doi.org/10.1145/3437800.3439202
  • Intelligent Tutors: 
    Koedinger, K. R., & Corbett, A. (2005). Cognitive Tutors: Technology Bringing Learning Sciences to the Classroom. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 61–78). Cambridge University Press. https://doi.org/10.1017/CBO9780511816833.006
  • Learning trajectories in CS: 
    Rich, K. M., Strickland, C., Binkowski, T. A., Moran, C., & Franklin, D. (2017). K-8 Learning Trajectories Derived from Research Literature: Sequence, Repetition, Conditionals. Proceedings of the 2017 ACM Conference on International Computing Education Research, 182–190. https://doi.org/10.1145/3105726.3106166
  • Assessment/Testing:
    Chen, B., Azad, S., Fowler, M., West, M., & Zilles, C. (2020). Learning to Cheat: Quantifying Changes in Score Advantage of Unproctored Assessments Over Time. Proceedings of the Seventh ACM Conference on Learning @ Scale, 197–206. https://doi.org/10.1145/3386527.3405925
  • Frequent Assessments: 
    Smith, D. H., Emeka, C., Fowler, M., West, M., & Zilles, C. (2023). Investigating the Effects of Testing Frequency on Programming Performance and Students’ Behavior. Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, 757–763. https://doi.org/10.1145/3545945.3569821
  • Defensive climates: 
    Barker, L. J., & Garvin-Doxas, K. (2004). Making Visible the Behaviors that Influence Learning Environment: A Qualitative Exploration of Computer Science Classrooms. Computer Science Education, 14(2), 119–145. https://doi.org/10.1080/08993400412331363853
  • Belonging in CS: 
    Lewis, C. M., Anderson, R. E., & Yasuhara, K. (2016). “I Don’t Code All Day”: Fitting in Computer Science When the Stereotypes Don’t Fit. Proceedings of the 2016 ACM Conference on International Computing Education Research, 23–32. https://doi.org/10.1145/2960310.2960332
  • Spatial Reasoning:
    Margulieux, L. E. (2019). Spatial Encoding Strategy Theory: The Relationship between Spatial Skill and STEM Achievement. Proceedings of the 2019 ACM Conference on International Computing Education Research, 81–90. https://doi.org/10.1145/3291279.3339414
  • BPC – Overview: 
    Lewis, C. M., Shah, N., & Falkner, K. (2019). Equity and Diversity. In S. A. Fincher & A. V. E. Robins (Eds.), The Cambridge Handbook of Computing Education Research (pp. 481–510). Cambridge University Press. https://doi.org/10.1017/9781108654555.017
  • Motivated Reasoning: 
    Epley, Nicholas, and Thomas Gilovich. 2016. “The Mechanics of Motivated Reasoning.” Journal of Economic Perspectives, 30 (3): 133-40. DOI: 10.1257/jep.30.3.133

Computing Education Research Area Email: glherman@illinois.edu.