Barrow faces an uphill task after taking over from Jammeh, who left behind a dysfunctional economy and allegedly emptied state coffers ahead of his departure.
Rights group blame the notorious National Intelligence Agency (NIA) under his longtime control for forced disappearances and torture.
Barrow said the NIA was "an institution that has to continue", but that its name would be changed and training would be given to its operatives.
"The rule of the law, that will be the order of the day," he said.
Barrow also addressed one of Jammeh's most controversial declarations, from 2015, that The Gambia was an "Islamic republic".
Barrow, in contrast, insisted the country -- whose population is 90 percent Muslim, with the rest Christian and animist -- was a republic, "not the Islamic republic".
Civil servants would likely return to a five-day work week, breaking with Jammeh's rule that Friday was a day off in line with his Islamic republic rules.
"My government is going to look at every avenue and there will be a complete overhaul of the system," Barrow said, speaking at his first press conference since arriving back from Senegal.
The president promised his cabinet would be named early next week so that he could "get the ball rolling", adding he would receive the first comprehensive information about the state of the nation's finances also on Monday or Tuesday.