I assume you don't want to actually delete the history -- just get rid of the drop-down.

Many parts of the Firefox interface are configurable with style rules.

The following rule will prevent the list from appearing when you click and hold the back button, but a right-click will still display the list. Not sure whether that is useful for your purposes.

@namespace url(http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul);
toolbarbutton#back-button > menupopup,
toolbarbutton#forward-button > menupopup {
  display:none !important; 
  visibility:hidden !important
}

The two ways to apply style rules to the UI are (1) the userChrome.css file and (2) the Stylish add-on. More info available if you want to try either one.

You can also the set the Integer pref browser.sessionhistory.max_entries to "0" on the about:config page.

• http://kb.mozillazine.org/about:config