Remove newlines from JSON Strings in Python

In this tutorial, we will explore several methods to remove newlines from JSON strings in Python.

We’ll cover methods like str.replace(), regular expressions, and str.strip() in combination with list comprehensions.

Also, we’ll perform a benchmark test to compare the performance of these methods.

 

 

Using str.replace() Method

The str.replace() method allows you to specify a substring to be replaced and the string to replace it with.

In our case, we’ll replace newline characters, represented as "\n", with an empty string.

import json
json_string_with_newlines = '{\n    "id": "001",\n    "name": "Customer A",\n    "service": "Telecom"\n}'
clean_json_string = json_string_with_newlines.replace("\n", "")
print(clean_json_string)

Output:

{    "id": "001",    "name": "Customer A",    "service": "Telecom"}

In this output, you’ll notice that the newlines are removed, making the JSON string a single line.

 

Using Regular Expressions

Regular expressions can handle different types of newline characters (like \r\n in Windows or \n in Unix/Linux).

Here’s how to apply regular expressions to remove newlines:

import json
import re
json_string_with_newlines = '{\n    "id": "001",\n    "name": "Customer B",\n    "service": "Telecom"\n}'
clean_json_string = re.sub(r'\s*\n\s*', '', json_string_with_newlines)
print(clean_json_string)

Output:

{"id": "001","name": "Customer B","service": "Telecom"}

In this output, the re.sub(r'\s*\n\s*', '', json_string_with_newlines) line uses a regular expression to target newline characters.

The pattern \s*\n\s* matches newline characters possibly surrounded by other whitespace characters (like spaces or tabs).

 

Using str.strip() with List Comprehensions

Another method to remove newlines from a JSON string in Python is by using str.strip() in combination with list comprehensions.

Here’s how you can do this:

import json
json_string_with_newlines = '{\n    "id": "001",\n    "name": "Customer C",\n    "service": "Telecom"\n}'
clean_json_string = ''.join([line.strip() for line in json_string_with_newlines.splitlines()])
print(clean_json_string)

Output:

{"id": "001","name": "Customer C","service": "Telecom"}

In this output, the multi-line JSON string is first split into individual lines using .splitlines().

Then we used list comprehension to apply .strip() to each line, removing any leading or trailing whitespace (including newline characters).

Finally, ''.join(...) joins the lines back into a single string.

 

Benchmark Test

This test will compare the execution time of the str.replace() method, the Regular Expressions method, and the use of str.strip() with list comprehensions.

We’ll use the timeit module to measure execution time in Python.

import timeit
json_string_with_newlines = '{\n    "id": "001",\n    "name": "Customer D",\n    "service": "Telecom"\n}' * 1000

def using_replace():
    return json_string_with_newlines.replace("\n", "")

def using_regex():
    import re
    return re.sub(r'\s*\n\s*', '', json_string_with_newlines)

def using_strip():
    return ''.join([line.strip() for line in json_string_with_newlines.splitlines()])

time_replace = timeit.timeit(using_replace, number=10000)
time_regex = timeit.timeit(using_regex, number=10000)
time_strip = timeit.timeit(using_strip, number=10000)
print(f"Time using str.replace(): {time_replace} seconds")
print(f"Time using Regular Expressions: {time_regex} seconds")
print(f"Time using str.strip() with List Comprehensions: {time_strip} seconds")

Output:

Time using str.replace(): 1.3522527999994054 seconds
Time using Regular Expressions: 35.05227589999777 seconds
Time using str.strip() with List Comprehensions: 7.689232900000206 seconds

As you can see, str.replace() is the fastest method.