Walter Freeman was apprehended in a building across the street from the bank.
Police said a witness notified officers that the suspect ran into the Department of Motor Vehicles office, catty-corner from bank.
Officers reportedly located the suspect in the building's bathroom changing clothes.
Police said the stolen money was recovered, along with the suspect's clothing.
The suspect was taken into custody and transported to the Eunice Police Department
He allegedly had about $4,000 in stolen money.
According to those on the scene, Freeman apparently indicated to bank employees that he had a weapon before demanding money.
A weapon was not recovered.