I've only made mayo with a blender, but the process is emulsifying oil in egg yolks, then giving it a tangy flavor. So if you're just stirring it, I don't think you're getting enough agitation, but I also don't know how to emulsify things by hand. If it were me, I'd pop it in a blender or food processor to see if it sets.