Nie Shubin was killed by firing squad in 1995 at the age of 20 after being found guilty of killing a woman in Shijiazhuang, in Hebei province.
The supreme court ruled that the facts used in Mr Nie's trial were "unclear and the evidence insufficient".
Mr Nie's family, who have been campaigning for two decades to clear his name, have thanked his supporters.
Eleven years ago another man also said he had carried out the crime but the claim was rejected.
Chinese courts have a conviction rate of more than 99%. The official number of executions are a state secret, but is believed to be in the thousands every year.
Rights groups allege that confessions used in court are forced or extracted under torture.
It is highly unusual for convictions to be overturned.
In 2014, a teenager from Inner Mongolia was cleared of rape and murder, 18 years after his execution.
His parents were given 30,000 yuan ($4,850; £3,080) in compensation while 27 officials involved in his trial were later punished.
The BBC's John Sudworth in Beijing says Mr Nie's case is well-known in China.
Rather than a sign of the justice system's ability to right wrongs, many will see his exoneration as exposing continuing flaws and weaknesses in the justice system, he adds.