Avaya 4850 - certificate issues when opening https console in IE/Firefox

[GoodEnoughThen]

4850GTS-PWR+, HW:18, FW:5.8.0.1, SW:v5.9.3.023

I've got (60) 4850's, all on the same current/latest firmware. (10) of them have (self-signed) SSL certificate issues.

Prompts/warnings I get when attempting to open the https web console to the switch in IE 11:

- There is a problem with this website’s security certificate.
- Continue to this website (not recommended)
- You are using an unsupported browser/version and may experience latency. The supported browsers are IE 7.0/8.0 and Firefox 3.0+.
- IE will then allow me to log in!

Prompts/warnings I get when attempting to open the https web console to the switch in FF 52.x:

- Secure Connection Failed -- An error occurred during a connection to 10.8.5.152. The server presented a certificate that contains an invalid encoding of an integer. Common causes include negative serial numbers, negative RSA moduli, and encodings that are longer than necessary. Error code: MOZILLA_PKIX_ERROR_INVALID_INTEGER_ENCODING.
The page you are trying to view cannot be shown because the authenticity of the received data could not be verified. Try again
- I *cannot* login, and FF does not offer to ADD AN EXCEPTION

On the switches, I've tried resetting ServerControl and then creating a new SSL cert but no joy.

Configuration > Security > SSH/SSL > SSL > ServerControl - reset
Configuration > Security > SSH/SSL > SSL > CertificateControl - Create
 
Anyone seen this before and have a fix? TIA!

[krc]

I think this is fixed in the latest Firefox....

This has been a huge pain, I basically run an older ver of Firefox (v36), make sure it's set not update, and use that only to mange the avaya switches.

[MatzeKS]

... this is fixed on "both ends", Firefox v57.0.x and BOSS v5.11.0.011.

Make sure your SSL certificate on ERS has the actual size of 2048 bits:


ERS4926-02#sho ssl
WEB Server SSL secured: Yes
SSL server state      : Active
SSL Certificate       :
        Generation in progress: No
        Saved in NVRAM        : Yes
        Certificate file size : 804 bytes
        RSA host key length   : 2048 bits

If not, you can re-create this certificate by:

conf t
ssl certificate
ssl reset
end

Good luck