Experiences serving on the RISS Admissions Committee

I recently experienced serving on an admissions committee for the CMU Robotics Institute Summer Scholars Program (RISS). The program is structured to immerse undergraduate students in ongoing research projects and give them a taste of graduate level research. Previous alums from the program have gone on to grad school at CMU, and other similar institutions around the country. My motivation in volunteering for the program was two-fold:

1. I’ve always been interested in how admissions committees structure their decision-making processes. Unconscious bias regularly influence our decisions¹ , however it is possible to actively correct for them. How do we design systems that factor this in, and enforce an unbiased and rational decision-making process?
2. I had applied to this same program over 3 years ago and didn’t get through :) I thought it’d be insightful to have a ‘behind the scenes’ experience, and better understand where/how I lacked at the time.

In many ways, the admissions process for the RISS program is structured similarly to that of a graduate program application. Applicants are required to submit a recent copy of their academic transcripts, recommendation letters, and a statement of purpose. There is no single decisive factor that guarantees admission to the program, which ensures a more holistic decision-making process. Given these components of the application, reviewers try to sketch together an overall portrait of the candidate and decide whether they’d be a good fit for the program. The admissions process occurs in roughly 3-4 different stages, and I’ll try to structure this post roughly in chronological order of the events that occur. I’ll attempt to touch on the finer details within each section, and try to give my take on how you should go about structuring your application. Though my experiences were with the RISS admissions specifically, I’m certain many of the insights can transfer across to similar fellowship programs.

Pre-Committee Stage:

In this step, the program administrators review the candidates in aggregate and address any outstanding issues including:

• incomplete applicant information
• sending a reminder to recommenders who have not yet submitted their letters
• sorting and flagging applications that are incomplete
• working with applicants that had technical issues during completing their application
• addressing any special requests/inquiries.

I would like to emphasize that candidates should be as thorough and particular as possible when submitting their application. During the review process, I’ve come across many candidates who had one or more pieces of their application incomplete. The most common of these is the recommendation letter. Though the candidate may think the blame for this lies on the letter writer, that is not true. It falls squarely on the candidate’s shoulders since there is plenty of time before the deadline to remind (and re-remind), the letter writer to submit the letter. The application portal clearly displays whether or not we’ve received a letter from your writers (and/or receive an e-mail once the letter has been submitted), so you should have up to date information about which of your letters have been submitted. Additionally, for some reason, if the letter writer has not been responsive in the days prior to the deadline, you can still inform the committee and change your letter writer². From what I’ve seen, applicants usually manage to submit the other parts of the application on time, since these are solely under their control.

Ensuring that you have a complete application is the bare minimum that you should do.

Review Stage

At this stage, the pre-filtering process has been finished, and a complete list of the applicants are then reviewed by the admissions committee. Each member is usually assigned a subset of applications from a common pool, with each application being reviewed by at least 2 committee members. These applications are then evaluated holistically, and then assigned a score. Since there will be a variance in the scores assigned by members³, there is a normalization step which is carried out at the end, to rank all the candidates on an absolute basis. Since this stage consists of committee members evaluating each component of the candidate’s application, I thought it’d be best to give general advice on how to focus on each part of your application.

General advice

In this section, I’ll attempt to give general advice for each component of your application. I’d recommend the reader to take all of this with a pinch of salt since you’ll get varying degrees of mileage depending on who is reviewing your application, and what they have formed in their mind to be the ‘ideal candidate’.⁴ That said, one generally applicable advice I’d give is being as detailed as possible when describing his/her past work. Since RISS is trying to search for the best matching candidates, it is in the applicants best interest to give as much relevant information as possible⁵.

Background

The committee tends to look for signs of whether the candidate is prepared at a technical level to contribute to research. This may be through courses or prior project work, so there’s no standard template to gauge preparedness. For RISS specifically, it’s recommended to have a strong math + computing background with a fair amount of exposure to programming. If you’ve done relevant projects, that’s a great value addition, and you should definitely try to explain the technical details and showcase your results. If you don’t have strictly relevant projects, that’s alright. Even if you can clearly explain your previous projects, and make a deep dive into the technical details, this sends a strong signal to the committee.

Resume/CV

This component of your application is given different weight depending on the preferences of the reviewer. However, this is a great place to see a high-level overview/summary of all your achievements. Depending on how much effort you’re willing to put into the application, I’d recommend tailoring your resume⁶ to fit the application⁷. One important point I’d like to stress is making sure that your CV is accomplishment oriented. Many candidates make the mistake of writing about what they did, instead of what they accomplished. The difference is subtle, but important.

Recommendation Letter

I’d argue that this is the most crucial part of your application, and is also the one over which you have the least direct control. I’d recommend requesting and informing your letter writers well in advance of the application deadline so that they have plenty of time to write a detailed letter. Ensuring that your letter writers have as many data points from which to base your letter off of, is important. Additionally, you’d want to make sure that you are requesting a letter from someone who knows you quite well and can vouch for your abilities. Admission committees tend to read and re-read your letters quite carefully and weigh the following properties (in no specific order)

• How well the recommender knows the applicant
• Specific evidence of the applicant’s capabilities.
• Comments on technical ability, teamwork, interaction, and motivation.

As a general thumb rule, to maximize the chance of your letter writer covering important aspects of your I’d advise supplementing your request with your CV, a letter stating the types of programs that you are applying to (and why), and related documents/data that help the referee to recollect your association with him/her.

Statement of Purpose

This is often the section of the application that I’ve observed many candidates don’t spend enough time on. You can use the SOP as a great place to talk about other strengths that you feel will be beneficial for the program, which have not been adequately covered by the other parts of your application. Personally, I recommend candidates think hard about the following questions prior to drafting an SOP:

1. What is your background story? What topics interest you and why?
2. Are there examples of where you’ve learned important life lessons?
3. What is your motivation behind applying to the program?
4. If selected whose work aligns the most with your interests? Why?

I’d recommend that all the above questions be addressed in your SOP. This shows the committee that you’ve done due diligence while applying for the program. It also gives you a sense of self-satisfaction of having been able to clearly articulate your background and future goals in the context of the program.⁸ On a side note, the SOP is also a great place to talk about your potential shortcomings (lower grades? These can be justified, provided you have a strong reason). Due to this, the SOP is a crucial part of your application, since it can help you overcome your shortcomings and add a personal touch to your application.

Post Review Stage

At this stage, all of the evaluated candidates along with their normalized scores (per committee member) are ranked into a list. The committee then extensively discusses candidates where there is a large standard deviation in the magnitude of the score assigned by the committee members. This may have occurred due to a multitude of reasons, since usually reviewers may place varying weights on different factors of the application. However, this stage of the process forces an active discussion between all the committee members and the administrators and results in the application being extensively scrutinized. This stage is a crucial part of the review process since committee members are encouraged to voice their opinions and vouch for candidates that they feel deserve to be assigned a higher normalized score. From what I’ve seen ~%5 of the applications fall in this category, since by this stage after multiple rounds of filtering and unbiased normalization, there starts to emerge a fairly distinct ranking among the applicants (based on a combination of the above-described metrics).

The finalized candidate list is then sent out to all labs within RI, and candidates with a good match are sent out offers.

Concluding remarks

Admitting students is an inherently imperfect process. Most committees tend to err on the conservative side, however the RISS admissions guiding principles actively encouraged us to be considerate and try to take into account the candidate’s environmental factors during evaluation⁹. Applicants may not have been privileged enough to have access to the excellent technical facilities of world-class research labs and professors with whom to do research. This in turn directly influences the quality of work that the applicant has been able to do, and can result in a strong bias while evaluating candidates if not actively corrected for.

In retrospect, serving on the admissions committee was definitely fruitful. I gained valuable insights towards my original motivation of serving on the committee and learned important skills in objectively evaluating a candidate. I’m certain I’ll be able to draw more insights from the experience, as time passes.

Writing long posts, trying to convey information in an unbiased way is hard, but I think what kept me going was the fact that a reader like you would someday read this post, and hopefully gain something from it. Even if I could help aid a few students help achieve their potential and go on to do great things, well then that’s a goal worth striving for :)

Thanks to Rachel Burcin and John Dolan for reading drafts of this.

Footnotes

---