Solved my own issue. Right click on the menu bars to bring up the options. Select customize. Drag the offending button off of the toolbar and into the Customize Toolbars box just like any other button.

The toolbar stays active but that button is gone, restoring space on the toolbar.

I, too, want to remove the "display your bookmarks" button from the bookmarks toolbar, because it takes up too much space and is redundant with the more compact (icon-only) "display your bookmarks" button on the navigation bar. (Zespectre's solution didn't work for me: When I use Customize to remove the "display your bookmarks" button from the bookmarks toolbar, it also hides the bookmarks toolbar!)

I had the same problem as bjtalbot, but when I looked in my "Customize Toolbar" window I noticed an extra "Bookmarks" option without an image. By dragging this onto the bookmark button already on there and then dragging it back off I was able to remove the button but keep the toolbar.