There is an add-on which re-attaches a detached tab for you --

https://addons.mozilla.org/firefox/addon/disable-tab-detach/

-- but that might not meet your needs because it sounds like you still want to be able to drag a tab from window A to window B. The problem arises when the tab doesn't attach successfully to window B. In that case, you don't want it to create a new window C, but what should happen instead?

Update: There's now a built-in preference for this in Firefox 74.

Correct. I do want to be able to move tabs between windows by drag and drop.

The problem occurs when I miss the small tab list on drop and hit anywhere else in the target window instead.

What would happen instead: Preferably: still move the tab as a tab to that target window. Preferably last or first. Second, still much better than current behavior, but not as good as above: Nothing happens and I have to drag the tab again(and maybe again until I hit the small tab row).

Try the extension I mentioned and see what you think. It seems to do what you want.

Edit: I mean this part:

Luvirini said

Second, still much better than current behavior, but not as good as above: Nothing happens and I have to drag the tab again(and maybe again until I hit the small tab row).

Thank you.

The extension description did not seem to imply it, but it works as option 2(the less optimal one) though with the new window popping up and then closing with the tab moving back. So the end result is as with option 2 above.

You could use code in userChrome.css to increase the height of the Tab bar to make it easier to drop the tab.

By the way, Firefox 74 now has a built-in preference to turn off the "tear-off tabs" feature:

(1) In a new tab, type or paste about:config in the address bar and press Enter/Return. Click the button accepting the risk.

(2) In the search box above the list, type or paste detach and pause while the list is filtered

(3) Double-click the browser.tabs.allowTabDetach preference to switch the value from true to false

You can still move a tab to a new window when needed using the tab context menu: right-click the tab > Move Tab > Move to New Window