The woman, in her 30s, was sharing a taxi home with a man after going in the medieval town of Visby on Sweden's Gotland island when she asked to stop off.
Her fellow passenger invited her inside before she was brutally set upon in the toilet, her lawyer said.
The assault sparked anger in the town and extra police were called to the island when an angry mob attacked the asylum centre with rocks.
The victim's lawyer Staffan Freriksson said the abuse started in the toilet and several other men joined in.
He added: 'Where they came from we don't know. This was going on for a couple of hours.
'She got paralysed in this situation and was not able to bring herself to resist physically, other than saying no.'
He said his client is unable to walk long distances and need a wheelchair. She was eventually able to escape the building.
Six men aged in their 20s were arrested in the days that followed they were later released.
Sweden's Expressen newspaper reported that the suspects were all asylum seekers and that the attack allegedly took place at asylum accommodation.
Large crowds gathered in protest and marched in the town.
The head of Gotland police Torbjörn Nilsson said: 'Gotland is safe, compared to other places, and there are few crimes compared to the rest of the country. But the way people have now acted is unprecedented.'