I can't vouch for how others think, but it is a disparity, isn't it.
I catch myself suspecting any group that restricts as "caucasian" or "white" as potentially racist.
But other groups can do that with blacks, hispanic, etc, etc, etc.

Interesting phenomenon. I dare say some of it is due to history. Groups that listed themselves as "white" were racist.
But that by no means should imply that all groups that do so are.

But regardless, a double standard...