Yes, it literally means “from Rum.” But, during both the Seljuk and Ottoman periods, Anatolia was widely referred to as Rum. So when someone was described as Rumi in those times, it did not necessarily mean they were ethnically Rum/Greek. More commonly, it simply meant a person from Rum, that is from Anatolia.