A group of about 30 people gathered to demonstrate against the Home Office's use of a hotel to house migrants in a Suffolk town.
However, they were outnumbered by about 100 people supporting those inside the Novotel in Ipswich, on Saturday.
They waved banners with the slogan "hate racism" and chanted "refugees welcome here".
Margaret Bulaitis president of Ipswich Trades Union Council said there was "no place for hatred… in Ipswich".
Ipswich Borough Council had previously said housing migrants in the hotel would put an "unsustainable" strain on local services.
Joining those supporting the migrants in the hotel, Ms Bulaitis said: "We're here primarily to stand in solidarity with the refugees in the [hotel], to say there's no place for hatred and division in Ipswich."
"I've lived in this town for 20 years and I'm confident that… the average person in this town is horrified at attempts to divide us."
Phanuel Mutumburi, from the Ipswich and Suffolk Council for Racial Equality, said: "It is worrying because we have seen in other places where there's been violence and acts of vandalism, but this is so important, and so everyone is brave and has come out here to support the people who are suffering."
Police said the event passed off peacefully and no arrests were made.
Find BBC News: East of England on Facebook, Instagram and Twitter. If you have a story suggestion email eastofenglandnews@bbc.co.uk
Hotel migrant plan unsustainable – council
Here are the 56 Suffolk street parties set to take place over coronation weekend
5 shops in Suffolk people would bring back
'We've loved each other ever since' – 73 years for Stowmarket couple
Swimmers pay tribute to women who led the way for disabled sport
Here's some of Suffolk's quirkiest places to eat and drink
More traffic chaos in town as major route to be shut for nearly two weeks to fix water main
US Supreme Court preserves abortion drug access
Dame Edna star Barry Humphries dies at 89
Tributes paid to Dame Edna Everage creator Barry Humphries
What’s happening on the ‘frightened’ streets of Khartoum?
Killed holding off waves of Russians in Bakhmut
What comes next for the abortion pill in the US?
Why South Koreans want the nuclear bomb
Eid, phantoms and flowers: Photos of the week
'Like warm mud': New Murakami book draws in crowds
What children want us to know about climate change. Video
The incredible humans who've evolved to thrive underwater
Canada's latest gold heist was far from the first
Full story: The royal line of succession. Video
Eight of the best films of 2023 so far
An ancient trick to think more wisely
© 2023 BBC. The BBC is not responsible for the content of external sites. Read about our approach to external linking.