Lawrence John Ripple gave a note to a Bank of Labor teller on Friday in Kansas City, Kansas, demanding cash and warning he had a gun, court documents say.
The teller complied with his demands and gave Ripple the money.
However, instead of fleeing, the 70-year-old man went to sit in the lobby where he told a security guard he was the 'guy he was looking for.'
The guard took the money back from Ripple and called police who arrived quickly.
An FBI agent wrote in the affidavit that Ripple had earlier been arguing with his wife and that he 'no longer wanted to be in that situation.'
He told investigators he wrote the note in front of his wife, telling her he would 'rather be in jail than at home.'
Ripple was charged with bank robbery on Tuesday.