The young migrant, named as Pateh Sabally, a 22-year-old Gambian, was seen jumping into the water from a pier and swimming to the middle of the large canal.
Hundreds of people witnessed the leap and at least three life rings were thrown into the water near the man, but he refused to hold on to them, raising speculation he wanted to commit suicide. No-one jumped in to help him.
Italian magistrates have opened an investigation into the incident after a video of the scene was posted on the Internet amid claims were shouting racist abuse.
The recording appears to pick up some bystanders yelling at the flailing man. 'He is stupid. He wants to die,' says one person.
One person allegedly shouted: 'Go on, go back home.
In the video, onlookers start to laugh and one man can be heard saying, 'Let him die at this point.'
Hundreds witnessed the migrant's death and later saw his dead body being pulled out of the water wrapped in a blue bag and tied to a ladder.
A taxi driver told local media: 'A young man threw himself into the water and two or three life jackets were thrown from the boat but he refused the help.
A fellow cabbie added: 'This is despair. I do not know the story of this guy. They have said he is African. Who knows what he's been through, how many seas and deserts he has crossed? And he came to die in the most beautiful city in the world.'
Witness Abdus Sattar from Bangladesh confirmed that the drowning had seemed to be a suicide.
He said: 'He threw himself from the pier. At first I thought he slipped but when I saw they threw lifebuoys and saw that he refused, I realised that he had decided to call it quits.'
More than 181,000 migrants arrived in Italy by boat in 2016, most of them from sub-Saharan Africa, an increase of almost 18 percent compared with 2015.
Italian media said Sabally had residency papers for Italy.