Just because somebody asks you a question, it doesn't mean they have a right to your answer. To refuse to answer a question isn't rude. Rather, to ask questions about things that are none of your business is.