building a 'truth db'.
the idea is generating claims from regular text. some texts will be considered ground truth. ground truth texts will get initial scores of 0.7 - 0.9. and claims that match ground truth will start with higher scores.
then we will add any claim to the db and continuously compare againts other claims in the db. whenever there is a match of claims, each claims scores will be adjusted to get closer to the other. since ground truth claims will have static scores, they wont move much.
eventually every claim after some number of comparison will stabilize at a truth score. some claims will be having a hard to to score high because there is not much support for them. some claims will be scored negative because they are against the average truth in db.
then we can calculate a person's truth score. a person's truth score can affect other things he said. claims of a veracious person will be buffed because of his other claims.
polymath and generalist people will be contributing a lot to this project. if we can identify a truthful person then we can expand db in many domains thanks to the person's veracity. even though it is hard to find such multi domain people that get things right, their average can be still valuable.
this work can be huge. can be used to align ai. benchmark ai. many things.
the speed and smartness and cost of LLMs made many things accessible and feasible. exciting times.
the idea is generating claims from regular text. some texts will be considered ground truth. ground truth texts will get initial scores of 0.7 - 0.9. and claims that match ground truth will start with higher scores.
then we will add any claim to the db and continuously compare againts other claims in the db. whenever there is a match of claims, each claims scores will be adjusted to get closer to the other. since ground truth claims will have static scores, they wont move much.
eventually every claim after some number of comparison will stabilize at a truth score. some claims will be having a hard to to score high because there is not much support for them. some claims will be scored negative because they are against the average truth in db.
then we can calculate a person's truth score. a person's truth score can affect other things he said. claims of a veracious person will be buffed because of his other claims.
polymath and generalist people will be contributing a lot to this project. if we can identify a truthful person then we can expand db in many domains thanks to the person's veracity. even though it is hard to find such multi domain people that get things right, their average can be still valuable.
this work can be huge. can be used to align ai. benchmark ai. many things.
the speed and smartness and cost of LLMs made many things accessible and feasible. exciting times.
1👍1
