What you're saying makes sense, but it's impractical. Someone could spam a thousand likes on your note, then you'll download all of that to get only two in your web-of-trust? Most relays won't even allow you to fetch more than 500 events directly.

At the same time an influencer might want to know a full count of likes, and if he is famous most of the likes will come from outside his WoT. But still that number is meaningless if it can be easily spammed.

The only way to reconcile all the different use cases and preferences while also providing some efficiency is by using relays, and relays with different policies for different users.

Client-side filtering has its place too, but it's a secondary barrier.