That is OK and expected behavior

"Use custom settings for history" allows to see the current history and cookie settings, so you can inspect and see them. Selecting the custom settings doesn't make any changes to history and cookie settings. Firefox will show "Use custom settings for history" as an indication that at least one of the history and cookie settings is not the default to make you aware that changes were made. If all History settings are default then the custom settings are hidden and you see "Firefox will: (Never) Remember History". "Never Remember History" means that Private Browsing is active and "Always use private browsing mode" gets a checkmark. "Use custom settings for history" stays selected as long as one of the History or Cookie settings is not the default to make you aware that changes from the default setting have been made.