It will also close two compounds used for Russian intelligence-gathering.
President Barack Obama had vowed action against Russia amid US accusations it directed hacks against the Democratic Party and Hillary Clinton's campaign.
Russia has denied any involvement and called the decision "ungrounded".
The US state department declared the 35 Russian diplomats from the Washington DC embassy and the consulate in San Francisco "persona non grata", and gave them and their families 72 hours to leave the US.
The move follows calls from senior US senators to sanction Russian officials who are believed to have played a role in the hacking, which some lawmakers referred to as America's "political Pearl Harbor".
Republican senators John McCain and Lindsey Graham, who led the calls for sanctions, said they "intend to lead the effort in the new Congress to impose stronger sanctions on Russia".
A Kremlin spokesman told journalists in Moscow that President Vladimir Putin would consider retaliatory measures.
Dmitry Peskov said the actions were "a manifestation of unpredictable and aggressive foreign policy", and called them "ungrounded and not legal".
And the Russian embassy in the UK tweeted a visual gag calling the Obama presidency a lame duck.