At two of my previous jobs, we’d typically get around 150 applicants for every design position. Our HR department would make the first cut based upon criteria we told them were essential — relevant university degrees, minimum years of experience, and so forth.
This worked reasonably well since a quick scan through the remainder would enable me to eliminate, probably, an additional 75 percent.
Even so, I always wondered how many good designers HR eliminated based on the black and white cutoffs we used.
We decided to experiment with using Robert Half and Ladders (if I remember correctly). Both offered prequalification testing similar to yours. I was not satisfied with the results.
For example, their pre-qualification exams included questions like, “In Photoshop, what is the keyboard shortcut used to zoom in?” This is a key combination that most that every designer uses, but the ability to recall the exact keys is evidence of nothing. Even though I might use that shortcut 50 times each day, it’s all muscle memory, not a sequence of keystrokes I could recall in the context of a quiz question.
Other questions involved basics from design school that are typically replaced through experience by other approaches. For example, “What is the complementary color to red?” That might be an easy question for a university design senior, but for an experienced professional who hasn’t looked at a color wheel in years and who relies on experience and gut instinct to choose colors, the answer might not be easily recalled.
In other examples, I disagreed with the premise of the questions. For example “Which of the following are 'modern fonts?” First, modern, when used in typography, can have very different definitions. Second, the correct answer might differ depending on whether the term font was intentionally or mistakenly used in place of the term typeface.
Several freelancer sites I’m familiar with also use these kinds of pre-qualification questions. The few that I’ve tested have suffered from the same types of deficiencies that I mentioned above.
As I said, in a previous post, I can see considerable value in pre-qualification testing, but I’ve yet to encounter the kinds of savvy and insightful questions that I could depend upon to yield good results.