Nassim N. Taleb, author of The Black Swan: The Impact of the Highly Improbable Fragility, has this concept of silent evidence. That is, we tend to make decisions mistakenly taking a limited subset of the information as the whole dataset.
It is not that we don’t acknowledge the existence of silent data. We do. The thing is that, in practice, we unconsciously ignore it again and again.
Silent evidence is what events use to conceal their own randomness, particularly the Black Swan type of randomness. (Nassim N. Taleb, The Black Swan)
Something similar can be said about evaluating people’s performance in an organization. Managers often have to consider if their reports are ready to be promoted. Depending of the company, a more or less formal processes for this may exist. But in the end, it is the manager’s responsibility to judge if the employee is ready to be promoted or not.
If you are a manager, do you pause to think if there is silent data about your candidate that you are not considering? Is her “success” a consequence of chance? In the light of someone consistently exceeding her sales goals, other considerations may not seem important. How did she achieve her goals? How were the goals set?
Perfect information is rarely available.
Developing the habit of looking for silent data is a really important asset.