Managing Cookies with Linux curl command

When interacting with web applications, you’ll encounter the need to handle cookies and sessions.

In this tutorial, you will learn how to use curl command to send, receive, and manage cookies.

 

 

Send Cookie Header Manually

The -I option sends a HEAD request, returning only the headers from the response. Among the headers, if the server is setting a cookie, you’ll notice the Set-Cookie header.

curl -I https://www.example.com/login

The Set-Cookie header from the server indicates that it wants the client (in this case, curl) to store a cookie.

This cookie will typically contain key-value pairs, expiration times, and other attributes.

If you need to manually set this header, you can do so as follows:

curl -H "Cookie: name=value" https://www.example.com/dashboard

With the -H option, you can manually set any header in your request. In the command above, you’re manually setting the Cookie header with a specific cookie value.

 

Storing Cookies Received from a Server

Using the curl -c option, you can store these cookies in a file for future use.

curl -c session_cookie.txt -d "username=john&password=secret" https://www.example.com/login

This command attempts to login with the username and password specified, and if successful, the server’s session cookie will be saved in session_cookie.txt.

Viewing Stored Cookies

You can view the contents of the saved cookies file using a simple cat command:

cat saved_cookies.txt

The output will provide a list of cookies in a structured format.

 

Sending Cookies with a Request

You can use the -b option to send saved cookies back to the server:

curl -b cookies.txt https://www.example.com/user-profile

In this command, curl reads the cookies.txt file and includes all the stored cookies in the request. This is especially useful for maintaining sessions across multiple curl commands.

Sending Individual Cookies

For cases where you only want to send a specific cookie:

curl -b "username=john_doe" https://www.example.com/settings

Here, you’re sending a single cookie named username with the value john_doe.

Sending Multiple Individual Cookies

To send multiple cookies without referencing a cookie file:

curl -b "token=abc123; session_id=456xyz" https://www.example.com/dashboard

In this scenario, you’re sending two separate cookies (token and session_id) with their respective values to the server.

When you utilize the -b option, curl automatically sets the Cookie HTTP header in the request.

 

Updating Cookie Values Between Requests

In dynamic web applications, cookies can change between requests due to session renewals, authentication tokens being refreshed, or simply user preferences being updated.

With curl, you can update cookie values between requests.

If you want to maintain a single cookie file and continuously update it with new values:

curl -b cookies.txt -c cookies.txt https://www.example.com/modify-session

By specifying the same file for both -b and -c, the initial cookies are sent to the server, and any updates from the server overwrite the original entries.

 

Handling Timeouts and Re-authentication

Detecting session timeouts often involves examining the response from a server. This could be an HTTP status code, a specific error message, or a redirect to a login page.

response=$(curl -b session_data.txt -o /dev/null -w "%{http_code}" https://www.example.com/dashboard)
if [ "$response" == "401" ]; then
    echo "Session has expired."
fi

In the snippet above, you’re checking for an HTTP 401 (Unauthorized) status code, which often indicates an expired or invalid session.

Automated Re-authentication

When a timeout is detected, immediately attempt to re-authenticate:

if [ "$response" == "401" ]; then
    curl -c session_data.txt -d "username=john&password=secret" https://www.example.com/login
fi

Here, if the session is expired, you make a login request to renew the session data.

Handling Multiple Timeouts

For long-running scripts, it’s possible to encounter multiple session timeouts. Implement a loop that checks for session validity and re-authenticates as needed:

for attempt in {1..3}; do
    response=$(curl -b session_data.txt -o /dev/null -w "%{http_code}" https://www.example.com/action)
    if [ "$response" != "401" ]; then
        break
    else
        echo "Re-authenticating attempt $attempt..."
        curl -c session_data.txt -d "username=alice&password=secret" https://www.example.com/login
    fi
done

This loop will try to re-authenticate up to three times if session timeouts are detected.

Avoiding Rapid Session Timeouts

Some web services may have very short session durations, especially if they’re designed for high security. In such cases, regularly sending “keep-alive” or “heartbeat” requests can help:

while true; do
    curl -b session_data.txt https://www.example.com/heartbeat
    sleep 300  # send a request every 5 minutes
done

This sends a request every 5 minutes, which can prevent short session timeouts.

 

Save Multiple Domains Cookies in Cookie Jar

curl can store and manage cookies for multiple domains in a single file.

Start by creating a cookie jar, which is essentially a file where curl will store all cookies:

curl -c cookie_jar.txt https://www.example.com

Inspecting the Cookie Jar

The cookie jar file has a standard format, with each line representing a cookie:

# Netscape HTTP Cookie File
.example.com    TRUE    /   FALSE   1694496488  USER_TOKEN  abc123xyz
.another-domain.com FALSE   /   FALSE   1694497488  AUTH_ID def456uvw

This format shows the domain, path, cookie name, and value, among other attributes.

 

Understand Cookie Jar Format

The curl cookie jar uses a plain text format. Each line of the file represents a single cookie and contains several fields, separated by tabs.

domain flag path secure expiration name value

• Domain: This indicates the domain for which the cookie is valid. It can be a specific subdomain or a higher-level domain.
• Flag: A TRUE/FALSE value indicating if all machines within a given domain can access the cookie. Typically set to FALSE.
• Path: Specifies the path within the domain for which the cookie is valid. Cookies are often set to be accessible within specific directories.
• Secure: This field will have the word Secure if the cookie should only be transmitted over secure (HTTPS) connections.
• Expiration: This numeric value represents the UNIX timestamp when the cookie will expire. If the cookie is a session cookie (expires when the browser closes), this field will be zero.
• Name: The name or key of the cookie.
• Value: The actual content or value associated with the cookie name.

Let’s break down a sample entry:

.example.com TRUE / FALSE 1694496488 USER_TOKEN abc123xyz

• Cookie is valid for .example.com.
• Machines within the domain can’t universally access the cookie.
• It’s valid for the root path (/).
• Not restricted to secure connections.
• Expires at UNIX timestamp 1694496488.
• Cookie’s name is USER_TOKEN.
• Cookie’s value is abc123xyz.

 

Security Considerations

Cookies often encapsulate user sessions, preferences, and authentication data. Exposure or mishandling of these can lead to:

• Session Hijacking: Malicious actors can use the session data to impersonate users, gaining unauthorized access to accounts or sensitive information.
• Cross-site Scripting (XSS) Attacks: If cookies aren’t properly secured, they can be targeted in XSS attacks, where intruders trick a user’s browser into running malicious scripts.
• Data Breaches: In cases where cookies store or provide access to personal data, their exposure can lead to broader data leaks.

Two crucial attributes can enhance cookie security:

Secure Flag: A cookie set with this flag ensures it’s only transmitted over secure HTTPS channels. Thus, even if an attacker is listening, they can’t intercept the cookie over insecure channels.

.example.com    TRUE    /   FALSE   1694496488  USER_TOKEN  abc123xyz   Secure

HttpOnly Flag: A cookie with this flag ensures it’s inaccessible to JavaScript running in the browser, mitigating the risk of XSS attacks.

.example.com    TRUE    /   FALSE   1694496488  USER_TOKEN  abc123xyz   HttpOnly

When you inspect cookies in your jar or in HTTP headers, look out for these flags and ensure they exist for better security.