The typical modern credential (i.e., standard worker quality sign of widely understood significance) is based on a narrow written declared test of knowledge given early in one’s career on a pre-announced date at a quiet location. In this test, there is a list of questions to which one gives answers, answers then graded by independent judges who supposedly look only at the answers, and don’t take into account other things they know about the testee. In this post I want to point out that a much larger space of credentials are possible.

For example, you could be evaluated on actual products and contributions, based on your efforts over a long period, instead of being evaluated on short tests. You could be tested via tasks you must perform, instead of questions you must answer. After all, mostly we want to know what workers can do, not what questions they can answer. Since much of real question answering in the world is done verbally, test question-answering could also be done verbally, instead of in writing. And it could be done with frequent distractions and interruptions, as with most real question-answering.

However expressed, judges could take your first response as a starting point to ask you more questions (or give you more tasks), and dig deeper into your understanding. Judges could know you well, and choose questions specifically for you, and interpret your answers given all they know about you. This is, after all, closer to how most question-answering in the world actually goes.

Tests could be done at random days and times, and spread all through your career. Tests might be disguised as ordinary interactions, and not revealed to be tests until afterward. These approaches could discourage cramming for tests and other strategies that makes you good only at tests, and not so much at remembering or using your knowledge at other times.

Finally, you could be tested on your ability to integrate knowledge from a wide range of topic areas, instead of on your knowledge of a narrow topic area. Yes you could show that you know many areas via passing tests for many areas, but that won’t show that you have integrated these diverse areas usefully together in your mind.

Of course I’m not saying that these variations are never explored, just that they are used much less often than the standard credential test. This vast space of possible credentials suggests that a lot of innovation may be possible, and I’m naturally especially interested in helping to develop better credentials for abilities that I have which are neglected by the usual credentials. For example, I’d love to see a polymath credential, for those who can integrate understanding of many fields, and a conversation credential, on one’s ability to get to the bottom of topics via a back & forth interaction.

The narrow range of most credentials compared to the vast possible space also seems to confirm Bryan Caplan’s emphasis on school as emphasizing and screening conformity. Yes the the usual kinds of tests can often be cheaper in many ways, but the lack of much variation even when credentials are very important, and so worth spending a bit more on, suggests that conformity is also an issue. It really does seem that people see non-standard tests as illicit in many ways.

The dominance of the usual credential test can also be seen as a way our society is unfairly dominated by the sort of writing-focused book-smart narrowly-skilled people who happen to be especially good at such tests. These people are in fact usually in charge of designing such tests.