LONDON, England (CNN) -- British prison officers who wore a St. George's Cross tie-pin have been ticked off by the jails watchdog over concerns about the symbol's racist connotations.

The pins showing the English flag -- which has often raised hackles due to its connection with the Crusades of the 11th, 12th and 13th centuries -- could be "misconstrued," Chief Inspector of Prisons Anne Owers said in a section on race in a report on a jail in the northern English city of Wakefield.

The banner of St. George, the red cross of a martyr on a white background, was adopted for the uniform of English soldiers during the military expeditions by European powers to recapture the Holy Land from Muslims, and later became the national flag of England.

A section on race relations in Owers' report said: "We were concerned to see a number of staff wearing a flag of St. George tie-pin.

"While we were told that these had been bought in support of a cancer charity there was clear scope for misinterpretation, and Prison Service Orders made clear that unauthorized badges and pins should not be worn."

As one of her formal recommendations Owers said: "Staff should not wear unauthorized badges or pins."

Chris Doyle, director of the Council for the Advancement of Arab-British Understanding, said Tuesday the red cross was an insensitive reminder of the Crusades.

"A lot of Muslims and Arabs view the Crusades as a bloody episode in our history," he told CNN. "They see those campaigns as Christendom launching a brutal holy war against Islam.

"Muslim or Arab prisoners could take umbrage if staff wore a red cross badge. It's also got associations with the far-right. Prison officers should be seen to be neutral."

Doyle added that it was now time for England to find a new flag and a patron saint who is "not associated with our bloody past and one we can all identify with."

CNN.com gives you the latest stories and video from the around the world, with in-depth coverage of U.S. news, politics, entertainment, health, crime, tech and more.

CNN.com gives you the latest stories and video from the around the world, with in-depth coverage of U.S. news, politics, entertainment, health, crime, tech and more.