Seriously, I would change banks before I would let then tell me to use such an old browser version, one that isn't even supported by Mozilla any longer. Sounds like they don't have a true grasp of what security is all about.

I recommend that you install the Portable Firefox 3.5.19 version to your hard drive, to use at their website, and continue using Firefox 5.0 everywhere else. The "Portable" version won't interact with the "regular" installation, saving you a lot of grief with "incompatibility" between the latest version and that old version.
http://portableapps.com/apps/internet/firefox_portable/localization#legacy35

The old Firefox 3.5.19 branch has been EOL since April (barely got a security update then even) and is inscure now.

Instead of downgrading to a old insecure version perhaps you could try this User Agent Switcher extension to fool the site into thinking you are using Firefox 3.5.19 and not 5.0 as your Bank should really update their browser sniffing if they suggest Firefox 3.5.* as the most current version to use over 5.0 or even 3.6.* or 4.0.1

User Agent Switcher extension https://addons.mozilla.org/en-US/firefox/addon/user-agent-switcher/