Several top MPs reportedly received nuisance calls after their profiles were accessed on the official mobile application for the annual party conference.
The security breach saw members of the public able to enter the profiles using just the politicians' email addresses -- easily available online -- to view and edit the data stored within.
Former foreign secretary Johnson had his profile picture briefly swapped for pornography and his job title changed to an profane insult, according to several Twitter users.
Meanwhile Environment Secretary Gove's picture was changed to one of media tycoon Rupert Murdoch, his former employer when he was a journalist.
Among the first to report the flaw was Dawn Foster, a columnist for daily The Guardian.
"The Tory conference app allows you to log in as other people and view their contact details just with their email address, no emailed security links, and post comments as them," she wrote on Twitter, using a colloquial name for the party.
"They've essentially made every journalist, politician and attendee's mobile number public. Fantastic."
A Conservative party spokesperson apologized for the breach, saying the technical issue had "been resolved and the app is now functioning securely".
But the opposition Labor Party said the blunder showed the ruling party could not be trusted in matters of security.