There’s no way around when you want to use Android Auto on GrapheneOS.
Ok, your answer might be: don’t use Android Auto.
I need a Navi and the Graphene option is better than right out Google Map...
Did you try Organic Maps? Has navigation, and works offline.
1