SSL/TLS with curl command: Guide to Secure Connections

curlcommand is a tool for making network requests, it uses SSL/TLS when communicating with secure servers via HTTPS.

By default, curl attempts to use secure connections when available, but it’s essential to understand how to control and diagnose these connections.

 

 

Using Client Certificates

You can use the --cert option when you need to authenticate with a remote server using an SSL client certificate.

A client certificate is a way to confirm the identity of the client to the server. It’s particularly useful in setups where more than just a username and password are required for increased security.

curl --cert /path/to/certificate.pem:password https://secure.example.com

Output:

*   Trying 192.168.1.10...
* TCP_NODELAY set
* Connected to secure.example.com (192.168.1.10) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
...
< HTTP/1.1 200 OK
...

The lines following it are a detailed breakdown of the SSL/TLS handshake process, which includes the protocol version being used (in this case, TLSv1.3) and other related metadata.

When the server responds with “HTTP/1.1 200 OK,” it indicates a successful connection and data transfer.

If the certificate is password protected, the password is provided after the colon.

 

Specifying Certificate Type

curl does an excellent job at autodetecting the right type, there are occasions where you need to explicitly specify the certificate type.

curl --cert /path/to/certificate.der --cert-type DER https://secure.example.com

This command tells curl to use a client certificate in DER format when connecting to secure.example.com.

Output:

*   Trying 192.168.1.11...
* TCP_NODELAY set
* Connected to secure.example.com (192.168.1.11) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
...
< HTTP/1.1 200 OK
...

Here, the --cert option points to the location of the client certificate. In this instance, it’s a DER encoded certificate.

The option --cert-type DER tells curl that the provided client certificate is of type DER.

While PEM is the default (and most common) format, DER is a binary form of PEM and is used in specific environments.

 

Using Private Keys

The private key is used with the certificate to prove the client’s identity to the server without revealing the key itself.

curl --cert /path/to/certificate.pem --key /path/to/privatekey.pem https://secure.example.com

This command instructs curl to use both a client certificate and its associated private key when connecting to secure.example.com.

With the --key option, you’re directing curl to the location of the private key that corresponds to the client certificate.

Remember, while the client certificate is public and can be shared, the private key must remain confidential.

If a malicious actor gains access to the private key, they can impersonate the certificate owner in SSL/TLS communications.

 

Specifying Private Key Type

You can explicitly specify the type of the private key to avoid any ambiguity.

curl --cert /path/to/certificate.pem --key /path/to/privatekey.der --key-type DER https://secure.example.com

This command signals curl to utilize a client certificate and its associated private key in DER format when making a connection to secure.example.com.

Specifying CA Bundle

Trusting remote servers requires a foundation of trust. This foundation is usually based on certificates from Certificate Authorities (CAs).

In many systems, there’s a pre-defined bundle of CA certificates trusted by default.

However, there are times when you need to work with custom or private CAs or just want to specify a different CA bundle. curl offers the --cacert option for such cases.

curl --cacert /path/to/ca-bundle.crt https://custom-ca.example.com

Here, you’re telling curl to use a specific CA bundle for validating the server’s certificate when connecting to custom-ca.example.com.

Output:

*   Trying 192.168.1.14...
* TCP_NODELAY set
* Connected to custom-ca.example.com (192.168.1.14) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /path/to/ca-bundle.crt
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
...
< HTTP/1.1 200 OK
...

In this output, curl is making a secure connection to custom-ca.example.com, and it’s leveraging the CA bundle provided to validate the server’s certificate during the SSL/TLS handshake.

Breaking down the primary elements of the command:

The --cacert /path/to/ca-bundle.crt option specifies the path to the CA bundle that curl should use. The CA bundle contains one or more CA certificates that curl will trust.

 

Directory for Multiple CA Certificates

If you have multiple CA certificates, it’s not practical to combine them all into a single CA bundle file.

Instead, you should keep each CA certificate in its own file and point curl to a directory where all these certificates reside. This is where the --capath option comes in.

curl --capath /path/to/ca-directory/ https://multi-ca.example.com

With this command, curl will use the specified directory to validate the server’s certificate when connecting to multi-ca.example.com.

Output:

*   Trying 192.168.1.15...
* TCP_NODELAY set
* Connected to multi-ca.example.com (192.168.1.15) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: /path/to/ca-directory/
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
...
< HTTP/1.1 200 OK
...

From the output, you can see curl establishing a secure connection to multi-ca.example.com. Notably, during the SSL/TLS handshake, it utilizes the CA certificates in the specified directory to validate the server’s certificate.

 

Using Certificate Revocation Lists

A Certificate Revocation List (CRL) is a list of certificates, usually maintained by a Certificate Authority (CA), that have been revoked before their expiration dates and should no longer be trusted.

If a private key is compromised or a certificate was misissued, it’s placed on a CRL. curl provides an option to utilize a CRL to ensure it doesn’t trust a revoked certificate.

curl --crlfile /path/to/crlfile.pem https://revoked-cert.example.com

In this command, curl employs the given CRL to verify that the server’s certificate isn’t revoked before establishing a connection to revoked-cert.example.com.

Output:

*   Trying 192.168.1.16...
* TCP_NODELAY set
* Connected to revoked-cert.example.com (192.168.1.16) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* Certificate did not match CRL
* Closing connection 0
* SSL certificate problem: certificate has been revoked
curl: (60) SSL certificate problem: certificate has been revoked

From this output, curl attempts a secure connection to revoked-cert.example.com.

During the SSL/TLS handshake, it detects that the server’s certificate matches an entry on the provided CRL, and as a result, the connection is terminated.

 

Public Key Pinning

By “pinning” a public key, you’re explicitly defining which public key or set of keys a client should expect when establishing a secure connection.

This mitigates risks of Man-in-the-Middle (MitM) attacks leveraging rogue certificates, even if they’re issued by a trusted CA.

curl --pinnedpubkey /path/to/pubkey.pem https://pinned-server.example.com

With this command, curl will only trust a connection to pinned-server.example.com if the server’s public key matches the pinned key provided.

Output:

*   Trying 192.168.1.17...
* TCP_NODELAY set
* Connected to pinned-server.example.com (192.168.1.17) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* SSL certificate problem: public key does not match pinned public key
* Closing connection 0
curl: (60) SSL certificate problem: public key does not match pinned public key

In this example output, curl is attempting to create a secure connection to pinned-server.example.com.

However, during the SSL/TLS handshake, it identifies that the server’s public key doesn’t match the one provided in pubkey.pem, causing the connection to terminate.

 

Choosing Specific Cipher Suites

Sometimes, for either security or compatibility reasons, you want to specify which cipher suites curl should use (or avoid) when establishing a connection.

The --ciphers option allows you to make this specification.

curl --ciphers 'ECDHE-RSA-AES128-GCM-SHA256' https://cipher-specific.example.com

By executing this command, curl will attempt to use the ECDHE-RSA-AES128-GCM-SHA256 cipher suite when establishing a secure connection to cipher-specific.example.com.

Output:

*   Trying 192.168.1.18...
* TCP_NODELAY set
* Connected to cipher-specific.example.com (192.168.1.18) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* SSL connection using TLSv1.3 / ECDHE-RSA-AES128-GCM-SHA256
...
< HTTP/1.1 200 OK
...

The output indicates that curl has established a secure connection to cipher-specific.example.com using the specified cipher suite.

It’s possible to list multiple cipher suites separated by colons, allowing curl to use any of them depending on server support.

 

Force TLS version

In some cases, want to enforce the use of a particular version of TLS, either due to server requirements, testing needs, or security policies.

You can force curl to use a specific version of TLS. This can help determine if there’s a specific version causing the issue:

• For TLS 1.0: curl --tlsv1.0 https://example.com
• For TLS 1.1: curl --tlsv1.1 https://example.com
• For TLS 1.2: curl --tlsv1.2 https://example.com
• For TLS 1.3: curl --tlsv1.3 https://example.com

TLS 1.3 is the latest version and introduces various improvements and security enhancements over its predecessors.

 

Setting the Maximum Allowed TLS Version

While it’s generally a good idea to use the latest version for enhanced security, there are some scenarios where you need to limit the maximum TLS version, be it for compatibility, testing, or transitional reasons. curl offers the --tls-max option to define this upper limit.

curl --tls-max 1.2 https://tls-specific.example.com

This command instructs curl to connect to tls-specific.example.com using at most TLS version 1.2, even if both the client and server support a higher version.

Output:

*   Trying 192.168.1.19...
* TCP_NODELAY set
* Connected to tls-specific.example.com (192.168.1.19) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* SSL connection using TLSv1.2 / [Cipher Suite]
...
< HTTP/1.1 200 OK
...

From this output, it’s evident that curl has successfully established a secure connection to tls-specific.example.com but will use TLS version 1.2, even if version 1.3 or any newer version is available and supported.

 

Resuming a Previous TLS Session

By reusing session parameters from a previous connection, you can skip parts of the handshake process, resulting in faster connection times.

curl offers the --tls-session option to utilize session IDs from prior sessions for this purpose.

curl --tls-session /path/to/session-id-file https://session-resume.example.com

Here, curl will attempt to resume a previous TLS session with session-resume.example.com using the session details stored in session-id-file.

Output:

*   Trying 192.168.1.20...
* TCP_NODELAY set
* Connected to session-resume.example.com (192.168.1.20) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* SSL connection using TLSv1.3 / [Cipher Suite]
* Session resumption (using session ID)
...
< HTTP/1.1 200 OK
...

The output shows that curl successfully connected to session-resume.example.com and resumed the TLS session based on the session ID provided, resulting in a potentially quicker connection setup.

However, for session resumption to work, both the client and the server must support it and have compatible session details.

 

Perform TLS Authentication

In addition to the server providing its certificate (as in traditional SSL), the client also presents a certificate that the server verifies.

curl supports this extended form of security and the --tlsauthtype option lets you specify the type of TLS authentication to carry out.

curl --cert /path/to/client-cert.pem --key /path/to/client-key.pem --tlsauthtype SRP --tlsuser 'tlsusername' --tlspassword 'tlspassword' https://mutual-auth.example.com

In this example, curl will attempt to connect to mutual-auth.example.com using the Secure Remote Password (SRP) type of TLS authentication, while also providing a client certificate and key.

Output:

*   Trying 192.168.1.21...
* TCP_NODELAY set
* Connected to mutual-auth.example.com (192.168.1.21) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* SSL connection using TLSv1.3 / [Cipher Suite]
* Server certificate:
*  ...
* Client certificate:
*  ...
* TLS SRP authentication successful
...
< HTTP/1.1 200 OK
...

The output showcases a successful mutual authentication using SRP. Both the server’s and the client’s certificates are verified, ensuring a higher level of trust for the established connection.

The --tlsuser and --tlspassword options allow you to provide the username and password for TLS authentication.

 

Require SSL/TLS

The --ssl-reqd option in curl ensures that a connection only gets established if SSL/TLS is used.

curl --ssl-reqd https://secure-only.example.com

Output:

*   Trying 192.168.1.24...
* TCP_NODELAY set
* Connected to secure-only.example.com (192.168.1.24) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* SSL connection using TLSv1.3 / [Cipher Suite]
...
< HTTP/1.1 200 OK
...

The output confirms a successful connection to secure-only.example.com over an SSL/TLS encrypted channel.

 

Trying to Upgrade to SSL/TLS

In situations where a connection starts without encryption but has the capability to upgrade to a secure SSL/TLS connection, the --ssl option in curl comes into play.

When used, curl will try to upgrade the connection to use SSL/TLS encryption, giving you the benefit of a secure channel even when the initial handshake is not be encrypted.

curl --ssl http://try-ssl.example.com

Output:

*   Trying 192.168.1.25...
* TCP_NODELAY set
* Connected to try-ssl.example.com (192.168.1.25) port 80 (#0)
* Issuing SSL/TLS upgrade request...
* Successfully upgraded to SSL/TLS
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
...
< HTTP/1.1 200 OK
...

From this output, you can see the connection initially started on port 80 (standard HTTP port) and then an SSL/TLS upgrade request was issued.

Then, the connection was successfully upgraded to an SSL/TLS-encrypted channel.

The --ssl option is a powerful tool, especially when interacting with servers that support the “Upgrade” header, allowing for a transition from an insecure to a secure connection.

 

Disable Certificate Revocation Checks

Certificate revocation is a security feature in which certificates that have been deemed unreliable (due to compromise, expiration, or other reasons) are invalidated before their actual expiration dates.

However, there are some scenarios, especially in testing or controlled environments, where you want to disable these checks.

In curl, the --ssl-no-revoke option allows for this, but you should use it with caution.

curl --ssl-no-revoke https://revocation-test.example.com

This command instructs curl to ignore certificate revocation checks when connecting to revocation-test.example.com.

Output:

*   Trying 192.168.1.27...
* TCP_NODELAY set
* Connected to revocation-test.example.com (192.168.1.27) port 443 (#0)
* WARNING: Certificate revocation checks disabled!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
...
< HTTP/1.1 200 OK
...

The output shows a warning that certificate revocation checks have been disabled.

 

Troubleshooting Common SSL/TLS Errors

This section will guide you through some common SSL/TLS errors you might encounter while using curl and offer solutions to address them.

Curl Error SSL Verify Failed

This means that the certificate presented by the server couldn’t be verified against a known Certificate Authority (CA) or there might be other issues related to the SSL certificate.

Here are some potential causes and how to address them:

1. Expired Certificate: The SSL certificate presented by the server is expired.
2. Self-Signed Certificate: The certificate was self-signed and not issued by a known Certificate Authority.
3. Mismatched Hostname: The certificate’s Common Name (CN) or Subject Alternative Name (SAN) doesn’t match the server’s hostname.
4. Outdated CA Store: The CA store on your system might be outdated, and might not have the required certificates to verify the server’s certificate.

Troubleshooting Steps:

Check the Certificate: You can view the server’s certificate details using:

echo | openssl s_client -connect servername:443 | openssl x509 -text

This will give you details about the certificate, such as its expiration date and the issuer.

Specify a CA Bundle: If you have a specific CA bundle you want to use for verification, you can use the --cacert option:

curl --cacert /path/to/ca-bundle.crt https://example.com

Update the CA Store: On some systems, you might need to update the CA store. The method varies based on the system. For example, on Debian-based systems, you can use:

sudo apt-get update && sudo apt-get install --reinstall ca-certificates

Check for Proxies: Ensure that there aren’t any proxy configurations that might be interfering with your connection. If a proxy is in place, it should be properly configured to handle SSL/TLS traffic.

 

SSL 403 Forbidden

The “403 Forbidden” is an HTTP status code that indicates the server understands the request, but it refuses to authorize it. This could be due to several reasons:

1. Lack of Permissions: The server may be set up to deny access to the specific resource you’re trying to reach.
2. Geo-Restrictions: Some web servers restrict access based on the geographical location of the client.
3. IP Blacklisting: Your IP address might be blacklisted due to suspicious activities.
4. Hotlink Protection: Some web servers prevent direct access to resources (like images) if the referrer isn’t from the same domain.
5. Server Misconfiguration: An improperly configured .htaccess file or other server configuration files.
6. Access Rule Restrictions: The server may have specific rules for user-agents and might block requests from curl by default.
7. SSL Certificate Issues: Even though 403 is generally about authorization, a misconfigured SSL/TLS setup on the server or client side might interfere with proper access.

Troubleshooting Steps:

Use Verbose Mode: Using -v with curl can give more details about the request and response process:

curl -v https://example.com

Change User-Agent: Some servers may block default curl user-agents. You can change it with -A:

curl -A "Mozilla/5.0" https://example.com

Check with a Browser: Open the URL in a web browser to see if the issue persists. This can help determine if the problem is with curl or server-wide.

Check Headers Only: Use the -I option with curl to fetch only the headers. This can give clues, especially if there’s a ‘Location’ header that might be redirecting you:

curl -I https://example.com

Examine Server Logs: If you have access to the server, check the server logs (like Apache’s error.log or Nginx’s error.log) for more details on why the request was forbidden.

SSL/TLS Configuration: If you suspect the error is tied to SSL/TLS, inspect the server’s configuration. Tools like Qualys SSL Labs can provide insights into a server’s SSL setup.

 

Handling Expired Certificates

When a server’s certificate is expired, curl will refuse to establish a connection due to the certificate’s invalidity.

curl https://expired-cert.example.com

Output:

*   Trying 192.168.1.28...
* TCP_NODELAY set
* Connected to expired-cert.example.com (192.168.1.28) port 443 (#0)
* SSL certificate problem: certificate has expired
* Closing connection 0
curl: (60) SSL certificate problem: certificate has expired

Solutions:

• The most secure way to resolve this is to update the certificate on the server side.
• For testing purposes, you can bypass this check using --insecure flag, but it’s not recommended for production environments.

Addressing Mismatched Hostnames

This error arises when the hostname provided in the URL doesn’t match the Common Name (CN) or Subject Alternative Name (SAN) in the certificate provided by the server.

curl https://mismatched-host.example.com

Output:

*   Trying 192.168.1.29...
* TCP_NODELAY set
* Connected to mismatched-host.example.com (192.168.1.29) port 443 (#0)
* SSL: no alternative certificate subject name matches target host name 'mismatched-host.example.com'
* Closing connection 0
curl: (51) SSL: no alternative certificate subject name matches target host name 'mismatched-host.example.com'

Solutions:

• Verify the URL and ensure you’re connecting to the correct server.
• Update the server’s SSL certificate to include the correct CN or add the required hostnames to the SAN field.
• For temporary troubleshooting, the --insecure flag can be used to ignore this error, but it’s not a secure solution for the long term.

Dealing with Unsupported Protocols or Ciphers

If the client (in this case, curl) and the server don’t have a mutual SSL/TLS protocol version or cipher suite, the connection will fail.

curl https://unsupported-protocol.example.com

Output:

*   Trying 192.168.1.30...
* TCP_NODELAY set
* Connected to unsupported-protocol.example.com (192.168.1.30) port 443 (#0)
* SSL handshake failed due to unsupported protocol or cipher
* Closing connection 0
curl: (35) SSL handshake failed due to unsupported protocol or cipher

Solutions:

• Check the server’s supported protocols and ciphers, then ensure your curl client supports them.
• Upgrade your curl version or the server’s SSL/TLS software to support modern and secure protocol versions and ciphers.
• As a temporary measure, you can specify a particular protocol version using flags like --tlsv1.2 or --tlsv1.3. However, always ensure the version you’re forcing is supported by both and is secure.

 

Practical Examples

Putting all the theory aside, the real value of understanding curl‘s SSL/TLS capabilities comes into play when applied in real-world scenarios.

Let’s go through a couple of practical examples to showcase how you can use curl in everyday tasks related to SSL/TLS.

Securely Connecting to a Website

Connecting securely to a website is the primary use case for curl when dealing with SSL/TLS.

Here’s how you can ensure the best security when doing so:

curl -I --tlsv1.3 --cacert /path/to/trusted/ca/certificate.pem https://secure-website.example.com

Output:

HTTP/2 200 
date: Fri, 25 Aug 2023 12:00:00 GMT
server: Apache/2.4.29
strict-transport-security: max-age=63072000; includeSubDomains

By looking at the headers, you can see the site uses HTTP/2 and has the “Strict-Transport-Security” header set, indicating it’s well-configured for security.

• -I fetches only the headers, making it a lightweight way to check the website’s connectivity and its security headers.
• --tlsv1.3 ensures that you’re using the most recent and secure version of TLS.
• --cacert lets you specify a custom certificate authority, ensuring the server’s certificate is validated against a certificate you trust.

Debugging SSL/TLS Problems

If you face issues connecting to a site, turning on verbose logging can help you identify the problem:

curl -v --tlsv1.3 https://problematic-website.example.com

Output:

*   Trying 192.168.1.31...
* TCP_NODELAY set
* Connected to problematic-website.example.com (192.168.1.31) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, handshake failure (512):
* error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure
* Closing connection 0
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

The verbose output (-v flag) provides detailed information about the connection process.

Here, you can see there’s an “alert handshake failure”, suggesting there might be no shared cipher suites between curl and the server.

--tlsv1.3 specifies the TLS version, which need to be adjusted based on the server’s capabilities.