Who should decide really whether the Australasian terms 'Pom' and 'Pommie' are acceptable or not?
Clearly the English immigrants who have suffered it the most? 

The singularly Australasian term Pom (or Pommie) was most often used from the 1960s onwards along with two other words 'whingeing' and 'bastard'.
The Oxford English dictionary mildly describes them as a 'derogatory terms' for (originally) British and (latterly) English immigrants.
In fact these terms were frequently used in the phrase "Whingeing Pommy Bastard".
That would today be called 'hate speech' and if used to in a physical confrontation would be 'direct racism' or racial abuse that would involve a police prosecution.
Australians and New Zealanders try hard to convince the gullible that these are actually terms of endearment by inventing various definitions of the word 'Pom' in an attempt to persuade everyone that it is in no way racist. Both countries (mainly Australia) had dubious racism reputations during the 60s, 70s and 80s, and most older English immigrants now clearly understand that they were certainly subjected to racial abuse.
Like any other racist or semi-racist terms It is not just the definition of the word itself, (although, noticeably, other derogatory terms such as Kraut, Chink, Frog, Jock, and Wog were removed from both countries' vernacular long ago) but the context in which the words are used and the intent of the abuser. user. 

At one time or another various Australian bodies have "authoritatively" declared that Pom and Pommie are not racist terms including Cricket Australia and Australian Television watchdogs but these bodies have no legal right to do so. In fact they have a vested interest to challenge that suggestion.
The reality is that a racist will always deny being a racist and just because a word has been used for decades does not make it racially or legally acceptable.  
In most civilised societies today racism is loosely defined as any action inflicted on a person(s) that causes them to feel NOT OK about their race, culture or ethnicity. Many English people would say that "Whingeing Pommie Bastard" fits that description. Legally the final arbiter of what is and is not racism even in Australia is the country's Race Relations legislation and the Commission responsible for applying it.  

These Commissions can only take action when it receives official complaints and what English folk in Australia learned long ago is that it is dangerous to complain if you are an immigrant and particularly an early English immigrant, described affectionately as a "10 pound Pom".  

Ten years ago I presented these facts to the major television and radio companies in New Zealand. All of them graciously agreed to have their presenters replace "Pom" and "Pommie" with "English", and have honoured that commitment. My fellow English immigrants to NZ should be grateful.
Sky Television NZ was able to agree to remove it from its local NZ content, but Sky Television Australia flatly refused.
In the end it is up to English immigrants (not visiting English cricketers or their supporters) to demands the removal of these words and report instances where they feel racially abused to the appropriate Race Relations authorities.

Until this happens and ethnic English immigrants use legal redress, as many other immigrant groups have done anti-Englishness will just continue.