Generally, any amount of heat that would change the temper of the metal components would leave the polymer frame a puddle in the gun safe.
The risk here is that the barrel and slide assembly, and the metal components such as the slide rails and locking block that the barrel and slide depend on may no longer be in proper alignment. At the instant of firing, the barrel and slide are probably just fine, but the slide may not stay on the frame, or the barrel link engagement notch may not be keeping it locked.
I think the temptation is to use such a gun if it were proven "reliable", as a tackle-box gun, trunk gun, or a deep storage last-ditch tertiary backup pistol of some kind. Because of the environment of storage, or the lack of security, you wouldn't mind (too much) if it got cosmetic rust, or were stolen. Of course, it can't be proven "reliable", since you don't know that it might fail on the 51st round, the 101st round, or the 11th. etc. And IMO, if things were so dire that you needed to go to that gun, wouldn't you want one you could trust absolutely?