You might not have a manager , but you always have a Boss to a certain extent. There is always someone who is named as the companies MD/CEO , someone who has the authority to legally hire you / fire you. So therefor their opinion will always be more important.

Also the problem with having the team evaluate someones performance is that teams are vulnerable to cliqueness and groupthink and you may therfor often end up with decisions that are based more on popularity than anything else.

In theory at least a good manager should be able to step back from the inter-employee politics and make judgements with some degree of perspective.