This is such a perfect story of doing it all wrong - thank you!
I just replied to HotButton about this as well. Our tests are meant to be used early in the hiring process, for example, when you have hundreds or thousands of applicants and need a way to start removing the completely unqualified applicants from the larger pool so that you can spend your time with candidates who at least possess the bare minimum of skills required. To use your example, avoiding wasting time on copywriter applicants who don’t know how to use an en dash but interview well and impress HR with their culture fit, etc. These tests are not meant to be used to tell you who to hire amongst a small group of finalists - and they’re not meant to do that for any field, not just design.
What I think is probably hard for many people to appreciate is just how many people without any relevant skills will apply for jobs they are completely unqualified for. Online job listings tend to exacerbate these numbers, but it’s an issue that predates their use. It’s so common in some fields that, for instance, it’s often cited that 199 out of 200 applicants for programming positions can’t code at all. That’s not to say that 199 out of 200 programmers can’t code - just 199 out of 200 applicants. There is some strange cohort of people that will apply to jobs they are completely unqualified for - any job. It is these people we would like to fail such a test, while passing any actual designer, regardless of the subjective quality of their work. Candidates who pass could then move along in the hiring funnel and have their work evaluated by actual humans.
Those of you who are at more senior levels might not ever encounter such a test except when you’re in the hiring seat. But junior-level candidates, especially those applying for entry-level positions, are quite likely to encounter some form of testing meant to remove unskilled people from the pool, since those positions tend to attract huge numbers of applicants.
Deciding who to move past the first step in a hiring funnel based on automated resume scans just rewards people who design their resumes to pass automated resume filters (or hire someone else to do that for them, or steal someone else’s resume, etc.). Resume-writing skills aren’t all that pertinent for most jobs, though (in terms of the actual work), so it’s not a great way to separate skilled candidates from the unskilled masses. That’s a problem I’m trying to help solve.
Unfortunately, as has been mentioned in this and other threads, many companies use the wrong kind of testing, with way too much of a time-ask, and at the wrong part of the funnel, leading to lots of unhappy candidates.