In 2017, 24% of surveyed companies claimed that downtimes cost them between $301,000 and $400,000, according to the Statista website. In most cases, these situations of downtimes might have arisen from a simple issue that escalated into significant setbacks.
Were these issues solved at an earlier stage, such businesses could be reporting smaller losses from these minor setbacks.
In the world of network monitoring, packet loss can be one of the initial signs of an ailing IT infrastructure. Ideally, your network should never, at any time, lose packets. It can be a sign that one of the company’s IT assets needs to be evaluated and fixed soon.
However, to eliminate the problem of packet loss, you need to understand what causes it and some of the best ways to eliminate it.
What Exactly is a Packet Loss?
Packet loss occurs when packets (pieces of data) making their way through a network fail to get to the receiving end. In most cases, this results in the loss of some of the transferred information through a network.
For instance, a VoIP call might have parts of the entire message missing. Well, the acceptable rate of packet loss will vary and depend on your application.
While losing a few packets might cause little to no harm in the message being sent, a considerable loss can make the message incomprehensive. Losses that start from five to ten percent will easily catch the eye of the user as they start to notice missing pieces on their sentences as well as awkward pauses.
In the case of files being shared within a network, a single loss can mean that the intended document will not get delivered.
In the world of gaming, on the other hand, a loss can mean painful delays, especially if the game is fast-paced. Well, monitoring your free syslog server will ensure that you can spot these issues from a mile away.
The Security Risks Brought About By Packet Loss
Packet loss can easily be a sign that your organization is under attack by hackers. On the other hand, it could also expose you to some risk that can easily invite hackers and cybercriminals to take hold of your security systems.
For these situations to happen, however, the level of packet loss should exceed the acceptable rate. Here are some few ways that excessive packet loss will affect your security:
1. Packet Drop Attacks
Packet drops are a common DoS (denial of service) attack that has been used by hackers recently. They mainly occur when a malicious user manipulates the normal functioning of a router and makes it drop packets within particular streams. You will typically notice this by experiencing a high level of packet loss in your systems.
Some attacks might be so sophisticated that they may make themselves invisible to outdated antiviruses. They might lead to the antivirus dropping your packets instead of doing away with those that are leading the attack. The best way to be prepared for such attacks is always to use updated antiviruses.
2. Opening the Doors for a Network Attack
During peak business days or seasons, your network might not be able to buffer traffic in the right manner. This is often the result of a bad queuing strategy. Sadly, hackers can easily find a way to use this Achilles heel to their advantage.
In a situation where a network struggles to set a priority for incoming information or traffic, malicious users can circumvent your security systems through a low priority backdoor, since every part of the network is treated in the same manner.
Additionally, when your network is running on insufficient bandwidth, hackers might flood the network with infected packets, which the system will most likely not drop all of them to maintain optimal network performance.
In case you are experiencing network problems, it is always vital to pay attention to the allocated resources. Additionally, ensure that your network can easily differentiate between the high priority data from the low priority data.
This way, it will become easier to weed out infected packets that malicious users disguise in the packet explosion before they make their way through your network.
3. Packet Loss Can Be a Domino Effect
The more complex your IT infrastructure is, the more the possible attack areas. In the case of packet loss in a security system, then vital information might fail to make its way to security reports, which might make remediating a situation tough.
What’s even worse is that once a packet loss situation occurs, it can easily escalate into a huge problem.
One massive and successful packet loss can easily make its way throughout the entire IT infrastructure leading to many more packet losses. This will leave your organization vulnerable to attacks from multiple vectors.
What Causes Packet Loss?
Before getting to the final destination, a piece of data might have to travel through a set of devices. In case one of these devices tends to be running at full capacity, it will queue this new packet and send it back into the network once its turn reaches. However, if the device is doing too much and has an excessive backlog, then the data will be dropped.
In the case of data that isn’t being transferred in real-time, the end-user might not even notice that there was any form of packet loss. The sending application will slow down its speed of transferring the information and resend the lost packet.
For transactions such as emails and file transfer, you will still get the data as initially intended as long as the packet loss doesn’t continue.
On the other hand, it might be tough for transactions that are happening in real-time to recover from a packet loss. For instance, a packet loss during a phone call will mean that some of the audio will be distorted. In the case of video calls, the video will have some inconsistencies.
How to Remedy the Situation
To counter this, you can increase the bandwidth of the links undergoing packet loss to limit the loss. On the other hand, you should adjust the prioritization of the network to prioritize real-time packets. Although this will not eliminate the congestion, it will reduce the chances of there being a drop.
2. Software Bugs on Network Devices
In some cases, the packet loss might result from a bug in one of your network devices. Such a bug might cause some form of malfunction leading to the loss.
The best way to remedy this loss is to use updated software versions on every network device that you have. Additionally, you should look to fix these bugs as soon as possible in case the software you use is in-house.
3. Dismal Device Performance
In some situations, you might have a device such as a firewall, router, or switch, which is currently performing dismally. This could be the situation even if you upgrade your bandwidth. For instance, you might switch to a bandwidth of about 15 GB from one of 1 GB because you expect some spikes in traffic.
However, the device in question might only record 1.5 GB of bandwidth usage. The extra traffic will be lost through packet loss. In such cases, the device in question might have already reached its maximum capacity.
More specifically, the hardware on the device might be struggling to keep up with the current spikes in traffic. This means that any traffic spikes that exceed the capacity of the device will potentially be lost.
How to Remedy This
Such hardware might have reached the end of its useful life. The best remedy would be to replace it with other hardware options that will perform better in this position. Alternatively, you can add more devices to help ease the stress caused by the extra traffic.
In case you are using a software firewall like iptables, you should recheck the rules that may drop packets.
4. Faulty Cabling or Hardware
In other situations, your links might neither be over-utilized nor is the hardware’s utilization exceeding the expected levels. This means that the packet drops are a result of the malfunction of one of the physical components of your hardware. In case a part of your hardware is malfunctioning, you will easily notice error messages on its interface.
On the other hand, committing to log monitoring among other performance management activities will provide you with enough insight into the parts of your hardware that are performing dismally. In the case of a link malfunction, you will find the error messages on its interface. This is most common in both fiber and copper cabling.
5. Excessive Hops
The network software that initiates the transfer of data from one point to another might have a minor influence on how packets travel, though it might not be a positive influence.
The sender has the option of choosing the maximum number of hops that a packet can go through before reaching the final destination, commonly known as the Time To Travel (TTL), and is found on a packet’s IP header.
Despite its name, however, the TTL doesn’t point out the expected travel time, but it points out the number of routers that a packet is expected to go through before being dropped. Once the packet passes through a router, the TTL number reduces by one.
Once this number gets to zero, the packet gets dropped by the network. Ideally, packets are never meant to expire, but if they are rerouted to go around a malfunctioning router, they might go through too many hops.
In most cases, the router problem will be resolved in good time, and only a few packets will be lost.
However, if the issue endures, the routers will have to find a workaround. While a few packets will be dropped in the stream, the workaround will bring about an organized and more efficient route.
How to Remedy This
The root cause of the packet loss should be established at an early stage. This can be best done by paying attention to the log information of your tools. In case you find that a piece of hardware is the issue, this should be fixed soon. If the issue is faulty links, they should also be repaired.
Diagnosing and Measuring Packet Loss
Packet loss measurement is a task best reserved for those who are tech-savvy. In case you would like a shorter method to measure the extent of the loss, then you can always use the automated method where you can utilize packet loss measuring software. If you do not mind taking the manual path, then here is how to go about it:
How To Measure Packet Loss Manually
You can then proceed to measure the latency of the network (the time that it will take to send your message or a signal and receive another one back. It is typically measured in milliseconds.
Once you open the prompt, you should key in the command below:
ping -n 100 <hostname>
The hostname should be a server or website of your choice. For a simpler approach, you can use Google.com. The idea is to send out 100 pings to the host and ideally receive back the same number.
However, if you send out 100 pings and only receive 50 back, this means that your system is going through a 50% packet loss.
The response you are to receive should contain details about the packet transmitted, the packets you receive, the packet loss percentage, and the latency. However, you shouldn’t rely too much on a single test.
The idea is to test on several other hosts and maybe a few more times on a single host. This will at least give you enough data to derive an average measurement of your packet loss.
Furthermore, a 50% packet loss when interacting with one host doesn’t necessarily mean that all other hosts will react in the same manner. Your network might not be the one contributing to the packet loss in the initial tests.
Communication can only be effective enough if the message sent out on a network reaches the recipient in a comprehensible manner. While losing a few packets in a stream might not be alarming, it is the excessive loss that should be eliminated.
Study your IT network infrastructure to identify the root cause of excessive packet loss to prevent exposing yourself to security risks and downtimes.
Keep coming back. Thank you.
Mokhtar is the founder of LikeGeeks.com. He works as a Linux system administrator since 2010. He is responsible for maintaining, securing, and troubleshooting Linux servers for multiple clients around the world. He loves writing shell and Python scripts to automate his work.