You can do that just like you rename a normal bookmark.
Open the parent folder of that folder and select that folder in the right pane.
Change the name in the Details pane at the bottom right.

How do I rename a bookmark or bookmark folder while in library mode ?:

Select the folder (or bookmark) at the bottom of the library window you will see the properties, change it there.

This is where that will have to be added:

http://kb.mozillazine.org/Sorting_and_rearranging_bookmarks_-_Firefox#Viewing_bookmarks_in_the_Bookmarks_Manager_or_Library_window

Similar list for working history but you don't rename anything in history

http://kb.mozillazine.org/Viewing_the_browsing_history_-_Firefox

I'm not sure the replies given to the "rename bookmark" topic are the expected ones. I need to rename the bookmark folder in the bookmarks list. When I right click on a bookmark folder in the list, I expect to see a rename bookmark feature in a list like it happens in ie . So why is this not happening here ??

I'm running FF6

Use Properties in the right-click context menu to make such changes. If that entry is missing or grayed then you can't change the name.