I think that would be a nice feature, but it’s separate from deciding when you earn that trust.
Stack Exchange does have this feature: anonymous feedback. If you vote on a post without having the privilege or even while not logged in, the “vote” is added to a database field which is basically not shown anywhere. 10k users can see some statistics and outliers (e.g. Meta.SE). As an example, Stack Overflow has about 27M anonymous pseudo-upvotes, 0.8M pseudo-upvotes from accounts without the upvote privilege, and 133M actual upvotes; 17M anonymous pseudo-downvotes, 0.1M logged-in pseudo-downvotes, and 7M actual downvotes. (Anonymous visitors are much more negative than logged-in users!) Ask Ubuntu has more anonymous feedback than actual votes, and again a lot more more negative feedback than actual downvotes. Feedback on a given post is generally correlated to votes.
The problem with giving visibility to anonymous feedback is that you can’t prevent vote stuffing.
Would it? Stack Exchange already does it, except for the conversion to votes part, which has been requested.
The primary audience for scores is anonymous visitors. It’s meant to be a quality indicator, not a poll.