That's very helpful.

I wonder whether this is a Let's Encrypt specific issue? Two threads mentioned either them or a "free" certificate:

• MOZILLA_PKIX_ERROR_REQUIRED_TLS_FEATURE_MISSING caused by ocsp must staple, SSL labs says stapling is OK
• MOZILLA_PKIX_ERROR_REQUIRED_TLS_FEATURE_MISSING

However, that still doesn't explain the odd inconsistency between requests where some work (logged out) and other do not (logged in).

That's very helpful. I wonder whether this is a Let's Encrypt specific issue?

Let's put this differently:

1. does any commercial CA provide certificates with the must-staple feature?
2. would anybody who is willing to pay for a cert, and who therefore most likely expects some return-on-investment, be willing to activate a feature that is not mandatory, and that is known to cause problems in some circumstances?

Two threads mentioned either them or a "free" certificate:

• MOZILLA_PKIX_ERROR_REQUIRED_TLS_FEATURE_MISSING caused by ocsp must staple, SSL labs says stapling is OK
• MOZILLA_PKIX_ERROR_REQUIRED_TLS_FEATURE_MISSING

The main difference here: these people actually see this error! Their TLS handshake does effectively fail, because must-staple is engaged and the stapling OCSP response is someway missing (for whatever reason), or Firefox is made believe it would be missing. And a competent guy from letsencrypt forum stated that it is difficult to get must-staple properly working because of flaws in nginx. No idea how the situation is with apache.

But our case here is stranger: the TLS handshake works, the webpage works flawlessly, and only deep in the guts of Firefox is a (second?) verification done, which fails, and that failure is reported to nowhere (and can only be found by spamming the codebase with fprintf), but manages to occasionally(!) disturb the internal certChain store and subsequently the UI.

I don't get the rationale of this: when stapling is required, then either the reponse is present and good, or the response is bad or missing, and then the connection must be aborted.

However, that still doesn't explain the odd inconsistency between requests where some work (logged out) and other do not (logged in).

I gave up wondering about such. Today I cannot reproduce it at all, except when in turn accessing two individual applications with different hostnames and different certs.

The question now is: how do we get this thing fixed?

The current state of affairs is: people may connect to my application, then they may want to check the security, and then they may experience that the lock icon has gone dead.

People without detailed background knowledge - and even people with some technical knowledge - will then most likely think that my application has somehow hacked their browser.

So this is very very bad, and has some urgency.

I have prepared an issue statement for the issue. What is next? Do I need a sponsor, or what?

Issue statement:

When loading a page with many ressources, firefox opens additional connections to the server, to fetch files in parallel. These connections do carry the 0x5 TLS extension (OCSP stapling), and they do invoke TLS verification. When verification fails, however, these connections are not necessarily abandoned. When must_staple is set in the certificate, that TLS verification will also invoke TLSFeaturesSatisfiedInternal(). For TLSv1.2 this works. With TLSv1.3 this results in error ERROR_REQUIRED_TLS_FEATURE_MISSING because the OCSPStapling response appears not to be available. A perceivable difference is that with TLSv1.3 these additional connections are opened with TLS extension 0x29 (pre_shared_key), which is not the case with TLSv1.2. The consequence of the failing verification is not an abandonment of the connection, neither is it reported to anywhere. Nevertheless, the consequence is that Firefox does not copy the certificate data to the presentation layer. At a subsequent change of the security context, e.g. when a new cookie is received, in the case when the TCP connections did stay open (a new TLS negotiation did NOT happen), this certificate data (that was NOT copied and therefore is undefined) is relied on from UI code. At that point it may appear that the lock icon is rendered nonfunctional, and no certificate data at all is shown.

You can file a bug report on https://bugzilla.mozilla.org/

There might be a mailing list (now a Google Group) that others read more regularly, but I'm not sure which one is the most relevant:

https://lists.mozilla.org/

You can file a bug report on https://bugzilla.mozilla.org/

Done. #1732476

What are the chances anyone will read it? (With other products, bug reports -even those with ready-made fixes attached- stay unread for ten or twenty years.)

Looks like some progress on the bug report. However, Firefox 94 release comes in November, so it's some time away.

Yeah, I admit I was wrong in not hoping on much progress.

November is no problem at all - I am rather used to OS release cycles where it can take a year or more to the next major version. The important thing is, it goes away from my stack, it is no longer me who must carry it along.

So, thanks a lot for Your help!