Inspect the connection error and the Engine/session logs on the relevant side. A client-side trust file controls verification of the server; the acceptor uses its own trust configuration to verify client certificates.
| Symptom | Checks and corrective action |
|---|---|
| CA file cannot be opened | Check the effective file path, application working directory, read permissions, and PEM format. Check both Session::sslCaFile and Engine-level SSL.CaFile; an unset session value falls back to the Engine value. |
| Certificate chain is untrusted | Confirm that the effective CA file trusts the issuer and that the peer supplies the required certificate chain. Check both configuration levels and obtain the correct trust material through the agreed provisioning process. |
| Certificate has expired | Check the certificate validity dates and the system clock. Renew the affected certificate and update deployed credentials or trust material as appropriate. |
| Acceptor rejects the client during mutual TLS | With global SSL.VerifyPeer=true, the acceptor requires a client certificate and verifies it using its own trust configuration. Supply the initiator's certificate and matching private key, and confirm that the acceptor trusts its issuer. Configuring only the initiator's CA file is insufficient. |
| Encrypted private key cannot be loaded | Check that the key matches the certificate and that the application supplies its correct password through sslPrivateKeyPassword as an OnixS::Cryptography::SecureString. For global credentials, check SSL.PrivateKeyPassword or EngineSettings::sslPrivateKeyPassword. |
| SNI rejects an IP address | With sslClientSni("*"), use a DNS name as the connection URL. If connecting to an IP address, configure the explicit SNI name required by the counterparty. |
Correct the trust or credential configuration while keeping peer verification enabled. See Engine-level Fallback, Optional Client Credentials for Mutual TLS, and Acceptor Configuration.
openssl command-line tool can be used as a generic TLS/SSL client/server which connects to a remote host or accepts incoming connections using TLS/SSL. It's Windows version can be downloaded at https://www.slproweb.com/products/Win32OpenSSL.html. For example for the client:
or
For example for the server:
or
More details could be received using the -msg and -debug options. The detailed description of these modes can be found at:
This article provides a detailed explanation of the TLS/SSL protocol, specifically the handshake protocol, its associated messages and alerts, and the record protocol.
For regular TCP connections, the MSG_NOSIGNAL flag is used to avoid sending the SIGPIPE signal to the application in case of sending data to a socket closed ungratefully by another side. The OpenSSL library does not support a way to pass the MSG_NOSIGNAL flag. As a result, the application can receive the SIGPIPE signal, and the application can be terminated if there is no signal handler. Therefore, the application that is used the SSL/TLS encryption should ignore or process the SIGPIPE signal, e.g.:
You can find the proper handling example of Linux signals in BuySide and SellSide samples from the FIX Engine distributions package: Getting Started Samples. Please see the manageLinuxSignals() function in Helpers.h.
There are third-party SSL/TLS tools which can be used as an SSL/TLS proxy. For example, stunnel can be used for this purpose.