Don't worry too much about what others think of your labels.
Much of the infighting and discussion about who gets to use which labels in what way is restricted to the anonymity of the internet.
There are exceptions, of course, but out in the world most people are far more relaxed.