Normally release versions of Firefox will overwrite previous versions. Have you installed Firefox in the default location? (C:\Program Files\Mozilla Firefox\)

If there are multiple versions listed in the Windows control panel add/remove programs dialog, the entries may be false and only one version may be installed. Check the C:\Program Files\ directory for multiple versions of Firefox. If there is more than one version listed you can delete the folders containing the old versions of Firefox. Make sure that it is an old version of Firefox before deleting them, if unsure it is best not to delete them.

You can check the file application.ini to see which version of Firefox is installed in a Mozilla Firefox program folder.

Why do you download a version instead if updating what you have installed?

Help > Check for Updates = doesn't that work for you?