What people think of as authority can be one two things:

- Respect for experience in voluntary relationships

- Ability of unleashing brutal violence without having to fear any consequences in involuntary relationship like between individuals and state actors

So I guess if you, like Neo, you have nothing to lose and everything to gain, you don't have to fear the system's violence, destroying any illusion of authority