The Grainger College of Engineering
Siebel School of Computing and Data Science

AI Qualifying Exam Guidelines

Last updated Fall 2024
Previous AI Qual Exam website

The AI qualifying exam consists of brief written materials and a 90-minute oral exam in which the student presents their own research and a tutorial focused on background material to a committee of three professors. The student’s presentation serves as a starting point for questions from the committee, which generally will focus on the presented material but could range more broadly. The exam is structured to ensure that students are evaluated in a consistent manner and serves multiple purposes:

  1. Learning: Encourage students to study fundamental background topics. Time spent in preparation should be a good investment for research success.

  2. Diagnostic: Identify weaknesses in a student’s knowledge or abilities that can be improved to ensure a more successful Ph.D. outcome. The student receives clear and actionable feedback, including areas to focus development and, if applicable, the main reason(s) for failure.

  3. Qualification: Identify students who are unlikely to succeed in the program. This is mainly a consideration if there is a retake of the exam, in which a pass is required.

 

Formation of Committee

  1. The committee consists of three faculty members.

  2. At least one faculty member should be familiar enough with the candidate’s general research area to ask in-depth questions about the research. If no one meeting that criterion (other than advisor) is in the AI group, then someone from outside AI/CS should be recruited with the help of the advisor.

 

Exam Preparation

Each student must prepare a presentation consisting of two parts: their own research and background tutorial. The order of the two components is up to the student, and the detailed expectations and target times are given below.

  1. Presentation of research topic: 15 min prepared (i.e., excluding questions)

    1. This should include a clear problem definition, motivation for research, major challenges, outline of proposed or in-progress work, highlights of any preliminary results and/or method for evaluation.

    2. During this portion of the exam, the committee should aim to evaluate whether the student understands and can explain the motivation, significance, and key technical concepts behind their own work. It is not the committee’s role to judge the novelty, scope, or promise of the chosen direction.

  2. Presentation of background tutorial: 30 min prepared

    1. The topic and scope of the background tutorial should be appropriate for a half-lecture in a graduate course, i.e. relevant to current researchers, accessible to those with general AI background, and able to be adequately covered in 30 minutes. 

    2. The tutorial should focus on a small number (around 2-5) of key background papers forming the context for the student’s work. Key background papers should not include works from the student's research group or close collaborators.

    3. Besides papers, books and monographs are an acceptable source for presentation materials. Preprints or technical reports may also be used if especially relevant, though the presentation should not rely too heavily on non-peer-reviewed sources.

    4. As much as possible, the tutorial should be accessible to committee members not directly in the student’s area and should give them a self-contained way to understand the student’s research area and its immediate context. 

    5. The student should begin this presentation by explaining why the topic and key background papers were chosen.

    6. In preparing the tutorial, the student should take care to find an appropriate technical level. It is insufficient to summarize the content of papers in a “bullet list” format -- the student must identify a few technical concepts or methods that can be explained in some technical detail so as to permit the committee to judge the student’s analytical ability and formal thinking skills. 

    7. In addition to exposition of key technical content, the tutorial should include an interpretation and critical evaluation of the presented work. Papers should be placed in context, and historical background should be explained as necessary. Limitations of existing work (where applicable) and directions for future research should be clearly indicated. Where relevant, the students are also highly encouraged to address real-world implications of their topic, e.g., likely applications or ethical/societal concerns.



Written Materials Prepared by Student

The student must send the materials listed below to the committee at least one week in advance of the qual. Sending them farther in advance is recommended in case the committee has suggestions. Failure to send the materials in a timely fashion may result in forfeiture of the qual attempt. 

  1. C.V. with academic background and publications to date, including papers in submission and in preparation.

  2. Research statement (1-2 pages) on a topic related to the student's interests that summarizes motivation, background, expected contribution, methods, and results (if any). There is no expectation that the described work be published (or submitted for publication). This statement should be more detailed than the initial one required at qual signup; its primary purpose is to provide context to the committee ahead of the exam, allowing faculty to start formulating questions and setting expectations. The student may or may not receive specific feedback on the statement (see Evaluation below).

  3. List of references for the tutorial portion of the presentation. This list should be in two parts: first, the short list of papers (2-5) that will be directly covered in the presentation, and second, an extended list of 10-20 papers giving a more comprehensive overview of the student’s background reading in the area. As stated above, books and monographs are also suitable for inclusion in the reference list.

 

Exam Format

The following is the recommended timeline for the exam. Examiners should do their best to adhere to the time guidelines to prevent the exam from getting off track. The student should prepare slides ahead of time and present on their own laptop.

  1. Preliminaries (5 min)

    1. This can include introductions and any required committee consultation.

  2. Presentation of research topic and background tutorial (45 min prepared, 70 min total)

  1. As specified above, the student’s own research should be one-third of the overall presentation and the tutorial should be two-thirds. The order of the two portions is up to the student.

  2. It is expected that the committee will frequently ask questions throughout the presentation.

  3. This portion of the exam should end ~75 minutes after the start of the exam.

  1. Further questions (15 min)

    1. Examiners can ask general knowledge questions, or re-examine topics from the research or tutorial. Generally the questions should be at a level that one would expect most 2nd year PhD students in AI to know the answer.  Any single line of questions should be curtailed if the student is stuck. 

 

Evaluation

The committee will determine whether the student passed or failed the exam and provide detailed feedback on the following aspects:

  • Knowledge of the topic of presentation and closely related areas 

  • Quality of prepared presentation and associated slides

  • Oral communication, based primarily on the ability to answer questions

  • Written communication, based on the 1-2 page research statement sent to committee ahead of exam

 

Guidelines for the Student

  • You should carefully familiarize yourself with the background literature. Be sure to understand the motivations, techniques, and experiments of the background papers and their relation to your work. Beyond technical understanding, you should be able to identify contributions, analyze whether conclusions are well-supported by experiments, and suggest ways to improve or extend existing research. You should also be able to interpret papers within a historical context. Even if you focus on recent papers in your tutorial, you should be aware of early or seminal work in the field and should be able to answer questions about it.

  • Practice the exam multiple times with students who have taken it before. Encourage the audience to question you extensively.

  • When a committee member asks you a question, it is often good to restate the question to be sure that you understand it. If you are not sure what is being asked, ask for clarification. 

  • You are not expected to know the answer to every question. If you don’t know the answer to a question or are unsure, it is best to say so up front. It’s a good idea to pause for a moment to think before responding.

  • In preparing your slides, feel free to draw on publicly available materials and external sources (e.g., figures from your chosen papers, slides posted by their authors, teaching materials from the web, etc.). However, you must clearly attribute any external sources directly on the respective slides. Reuse of materials without attribution can be grounds for failing the exam or being subjected to academic integrity proceedings. You are also expected to go significantly beyond external materials in your presentation – e.g., it is not acceptable to take a slide deck from the web and present it with only minor changes, even with attribution.

  • You can expect general background questions to be focused mainly on machine learning (roughly the equivalent of CS446 or a similar introductory-level ML class), plus core material from your own research area (computer vision, NLP, etc.). All AI students are expected to have good familiarity with ML, but knowledge of other sub-areas of AI is not expected if it is not directly related to the student’s research or survey topic: e.g., an ML theory student is not expected to know computer vision or NLP, and vice versa.

 

Guidelines for the Committee Chair

The most important role of the chair is to ensure that the exam follows the above guidelines. The chair also drafts the decision/feedback letter. To enforce the guidelines:

  • Try to make the candidate feel at ease at the start of the exam.

  • Keep the exam on schedule. 

  • If an examiner’s question is not clear, ask the examiner to clarify.

  • If a line of questions has become unproductive or is taking longer than five minutes, interpose and move on.

  • If the candidate says that they do not know about a particular topic, ensure that questions move on to other topics.

  • If the candidate looks flustered, ask if they would like a short break.

 

Guidelines for the Committee Members

  • Listen to the Committee Chair.

  • If a line of questions has become unproductive or is taking longer than five minutes, move on.

  • Try not to pepper the student with questions. Give the student time to reflect and respond before asking another question.

  • Confine questions that are not directly related to the candidate’s research and tutorial topics to the “Further Questions” section at the end of the exam.

  • For the Research Presentation section of the exam, the committee’s goal is primarily to evaluate whether the student has adequate background preparation for working in their area, and whether they can clearly communicate the “what”, “why” and “how” of their work. It is not the qual committee’s job to evaluate the novelty, significance, scope, or promise of the research direction. If a committee member wishes to comment on these aspects or give the student advice, it is best to do so during a separately arranged meeting afterwards.

 

Guidelines for the Advisor

 

  • Avoid extensively coaching your own students for their qualifying exam. Many of the area faculty feel it is appropriate to give high-level advice on the exam format, or to provide feedback on the portion having to do with the student’s own research. However, the advisor should refrain from giving feedback on the survey portion or doing practice exams with the student. Students are strongly encouraged to practice with each other instead.