Unfortunately, I'm not aware of an easy way to determine this. The hard way is to look each of them up on addons.mozilla.org and check their version range. (There must be an easier way.)

Maybe the developers will take note and figure something out -- I upgraded to 4.0 when it first came out and had it not been for that wonderful program ( freeware ) Moz Backup, I would have had no recourse at all.

As it was, when I found out that 1/3 or more of my addons didn't work in 4. , I simply did a fresh install of 3.6 and then restored via Moz Backup.

I did find something else. You can see whether your extensions are compatible with Firefox 4 by installing the Extension Manager Extended add-on in Firefox 3.6. For each extension, the range of compatible versions will be listed. If your extensions list version 4.0.* or higher, they should work after the upgrade.

You can also look at this chart. Unfortunately, it's not in alphabetical order, so it'll take sometime to look up your add-ons:

https://addons.mozilla.org/en-US/firefox/compatibility/report?from=%2Fzh-CN%2Ffirefox%2Fcompatibility%2Freport

I digress, but I found a way to sort it. ;-)

TableSorter for Greasemonkey - userscripts.org

I found another solution to sort it alphabetically:

Copy the whole page and paste it into a Microsoft Word document, then highlight the first column (where the names are), click on "Table" and then "Sort" and choose the "1st column". The whole table is now sorted alphabetically. I don't know why I didn't think about this earlier.

The "Extension Manager Extended" addon is a big help as is the MS Word trick.

Big thanks to both silkphoenix and jscher2000