If only one tab is opened in a Firefox window and you close that tab, the whole Firefox browser will close too. This is the default behaviour of Firefox. So naturally when you start Firefox again, it opens the startup homepage set by you.

Set the pref browser.tabs.closeWindowWithLastTab to false on the about:config page to prevent closing the last tab from closing that window, you need to use "File > Close Window" or the close X on the title bar.

To open the about:config page, type about:config in the location (address) bar and press the "Enter" key, just like you type the url of a website to open a website.
If you see a warning then you can confirm that you want to access that page.

• Use the Filter bar at to top of the about:config page to locate a preference more easily.
• Preferences that have been modified show as bold(user set).
• Preferences can be reset to the default or changed via the right-click context menu.

You may need to wait a bit longer before restarting Firefox.

See "Hang at exit":

• http://kb.mozillazine.org/Firefox_hangs
• https://support.mozilla.com/kb/Firefox+hangs