What do you look for (part one) ?

I am a faculty member in a university physics department, who finds himself periodically involved in faculty searches and hires. How do I evaluate the curriculum vitae of an applicant for a tenure-track position?
What do I look for, and what are the red flags? Does it really boil down to counting (first-authored) articles, impact factor of journals where they were published, citations, invited talks, or maybe places where and individuals by whom the applicant has been mentored, as a student and postdoctoral associate?
Do I even look at the research plan? If so, how do I judge it?
What about teaching potential and/or experience?