secure Linux Server using Hardening Best Practicies
Server Administration

Secure Linux Server Using Hardening Best Practices

In the previous post we talked about some Linux security tricks and as I said, we can’t cover everything about Linux hardening in one post, but we are exploring some tricks to secure Linux server instead of searching for ready Linux hardening scripts to do the job without understanding what’s going on, However, the checklist is so long so let’s get started.


Disable Ctrl-Alt-Delete

This is important if you don’t have the best physical security for your server.

If you are using Systems prior to CentOS 7, all you have to do is to comment out the following line in /etc/inittab file.

ca::ctrlaltdel:/sbin/shutdown -t3 -r now

Otherwise, if you are using CentOS 7 use the following command:

$ ln -s /dev/null /etc/systemd/system/

Secure Linux Mounting

Each of your Linux file systems is mounted to allow you to access the data stored on it. You can mount your file systems using different options.

These options allow you to lock down the capabilities and functionalities of each of your mounted file systems. You can control this using the /etc/fstab file.

LABEL=/      /       ext4     defaults       1 1

The first column is just a label for your device.

The second column is the directory or location on your system where you want to mount the file system.

The third column is the type of file system like ext4.

The fourth column contains security options which is the most important one for us.

The last two columns control the options for the dump and fsck commands.

There are many different ways to control how file systems are mounted and the following list shows some of them:

auto                       It will be mounted automatically at boot time.

noauto                   It will not be mounted automatically at boot time.

exec                       You can execute binaries on this file system.

noexec                  You can’t execute binaries on this file system.

suid                       setuid bits are permitted.

nosuid                  No setuid bits.

user                       non-root users can mount this device.

nouser                  root users can mount this device.

owner                   Allows the owner of the device to mount the file system.

ro                          Mount device read-only.

rw                          Mount device read-write.

defaults                Make your file system’s options: rw, suid, dev, exec, auto, nouser, and async.

The exec and noexec options enable you to control whether binary execution is allowed or not.

Be careful, setting this option on some file systems such as /boot will prevent your system from booting. Since /boot must be executable to boot the system.

You can mount /home securely with noexec like this:

/dev/hda1     /home       ext4      noexec      0 2

Keep in mind that this line will prevent the execution of binaries on /home, so if you have any executables, you should take care of that.

You can mount /tmp with noexec option as a step of hardening, but keep in mind that some programs might not work properly because they use /tmp to execute. So you can test your software with this mount option, if it goes well then it’s OK.

The suid and nosuid options control the functioning of binaries having setuid or setgid bits on them. If you set the nosuid option, the setuid and setgid bits will be ignored.

The user, nouser, and owner options provide control over who is allowed to mount your file systems. Only root users can mount file systems. If your file systems have the user option specified, then any user can mount or unmount file systems.

Any user other than root shouldn’t be allowed to mount file systems.

The ro and rw, read-only and read-write allow you to control whether your users and applications can write to the specific file system.

You can mount any file system as read-only like this:

/dev/hda2         /usr        ext4       ro,nodev        0 2

You can mount /boot as read-only using the same way, but keep in mind that in case of any kernel update, you have to remount it as rw to apply the update like this:

$ mount -o remount,rw /boot

You know mount options and you should be wise enough to take the decision about which directory needs which option to mount with.

Protect /etc/services File

The /etc/services file allows server and client programs to convert service names to port numbers.

Only “root” user is allowed to make modifications to this file.

To improve security, we can set the immutable attribute on this file to prevent unauthorized deletion or modification.

Also, that prevents accidental deleting or overwriting of such a vital file.

$ chattr +i /etc/services

Remove Unused Accounts

These vendor accounts are preinstalled on your system for some Linux system activity.

If you don’t need those accounts, it’s preferred to remove them using the userdel command, and these are some of the unused users for me.

Also, you will need to remove the groups belongs to those accounts if exist using groupdel command

If you check /etc/passwd file, you’ll see that the users are deleted.

If you run your own server or VPS you can set the immutable bit on /etc/passwd and /etc/shadow to prevent any unwanted changes.

If you need to add new users to the system or install a program that will add users, consider removing the immutable flag first.

Hardening Cron Scripts

Some scripts under /etc/cron.d doesn’t have the secured permissions, they are readable to normal users.

Consider fixing the permission for the scripts that are responsible for executing scheduled job on our server so root only can read it.

$ chmod 0700 /etc/cron.daily/*

Normal users don’t need to look at those scripts.

Keep in mind that if you update a program that provides a cron file on your system, consider updating the permission, or you can make a shell script that does the job for you instead.

And the same for the other cron directories like:

Securing SUID Programs

SUID (Set User ID) is a special type of file permissions given to a file. When you want to use a tool like passwd command which writes on files belong to root such as /etc/passwd and /etc/shadow, the passwd command must have this SUID permission to enable normal users to use that command.

You may take a look at all programs that have this permission and consider removing that permission from unnecessary programs that you think that normal users won’t need it.

All these programs have SUID bit and normal users can run them as root. To remove that permission, you can use this command:

$ chmod a-s /bin/mount

Keep in mind that some programs need that permission to work so be careful when doing that.

Risky World-Writable Files and Directories

World-writable directories and files can lead to serious problems if the attacker gains access to them.

He will be allowed to modify or delete any file, and this is a serious problem.

To get all writable files on your system use this command:

Or maybe you want to get writable files only in the web folders like public_html:

To get world writable directories on the whole system, use the following command:

And the same for web folders:

You may find writable directories and files in some locations like /var/mail which has no problem, but on web folders, you have to be careful about that much.

You can use some integrity check tool like tripwire.

This tool will scan the system for any public writable files and directors and warn you, so you can take action about them.

Risky Symlinks

Symlinks or symbolic links are useful if they used for a good purpose to simplify your work, but the attacker in some cases uses any scripting language on your server to build a symlink to travel between directories and see your files, steal passwords and gain access to all websites on the server, so it’s very important to keep any eye on that.

The following command searches for any symlink and deletes it.

$ find -L /home/*/public_html -type l delete

You can change the path based on your server paths, you may also create a shell script to find those symlinks and send to your email so you can investigate how it was created.

There are many ways to stop symlink creation, if you are using PHP, you can disable some serious functions, and apply Symlinks only if owner matches for your server if you are using apache.

This trick is very useful, especially when dealing with compromised systems.

There is a lot to talk about securing PHP; maybe we should make another post about that, but let’s keep simple for now.

Securing Log Files

Your last line of defense is the log files. Log files for each running service tell you everything about that service, so you can keep track of everything happened on your system.

In worst scenarios (like gaining root access), the attacker might delete those log files and left you without any evidence of what had happened.

Consider copying your log files to a different place or schedule a regular backup of log files to somewhere else that shouldn’t be accessible to the attacker if he gains access to your system.

Securing Linux Resources

Securing Linux Resources is a must because users can jeopardize the stability of your server if they left to use server resources without limits.

You can allocate how much memory for each user, how many processes and other server resources.

Under /etc/security, there is a file called limits.conf, in this file you can specify the limits for your users like this:

The first line says for all users, limit the memory usage to 500 MB.

The second line says for all users, limit the number of processes to 50 processes.

All these restriction rules applied to all users expect root user.

The asterisk on both lines means all users, and some systems have users running services like www or mysql users and these service users are used by all users on the system and if we apply our restriction rules for them too, that can lead to problems.

A good solution for this problem is to add a special group and add our users to that group and apply our restriction rules to that group.

In this case, the rules will be applied for every user in this group and not to the whole users of the group.

Hardening /proc Directory

The /proc directory or as they call it (process information pseudo-file system) contains a lot of information about the currently running processes. Linux is installed by default to allow normal users to see that information. You can see what processes belong to root and all other user’s processes.

Before you use this trick, as you can see that normal user can see all processes even root processes:

Secure Linux Server ps -ef

The hidepid mount option allows you to hide process IDs. It takes a value of 0, 1, 2.

$ mount -o remount,rw,hidepid=2 /proc

And you can write it to /etc/fstab to make it permanent so after reboot, the process IDs remains hidden.

proc    /proc    proc    defaults,hidepid=2     0 0

proc directory Hardening Best Practices

After that command, you are only allowed to see your processes. Only root users can see all processes for all users.

$ ps -ef

Secure Linux Server ps -ef

Another mount option is gid which allows users in a specific group to see /proc directory.

If the group you want to assign the permission to has ID of 100, you can write it like this:

$ mount -o remount,rw,gid=100 /proc

Also, you can write it in /etc/fstab file:

proc    /proc    proc    defaults,gid=100     0     0

The last advice for you is to keep your system and software updated always, that will protect you from many threats.

I hope you find these hardening tricks useful. Keep coming back.

Thank you.

  • Pingback: Secure Linux Server Using Hardening Best Practices – TechNow()

  • Pingback: Linux iptables Firewall Simplified Examples » TechNotes()

  • Jim Dennis

    That find command is way too complicated. This will suffice: find / -type f -perm +6000 -ls

    … the + prefix on the permissions (octal) causes find to evaluate as true if *any* of the bits are set for a given file. (The more usual – prefix requires that *all* of the bits be set). Since permission 4000 (SUID) and 2000 (SGID) can be combined to a single octal value of 6000 we can use +6000 here.

    There are numerous other issues with these tips. In general it’s not useful to use the noexec mount option as there are a number of ways to force Linux to execute files (including compiled binaries) as arguments to command interpreters. noexec merely prevents Linux from honoring the execute bit, not from loading and executing the code in a file through the use of some other interpreter (debugger or other such utility). There are options to disable the honoring of SUID/SGID bits which could more useful. But you still only get marginal benefit from employing them. Sometimes SUID/SGID programs *are* useful (usually when they are NOT root owned nor root-group associated).

    Symlinks are not particularly risky. It’s the permissions on the target which determine access and as for symlinks from under a directory tree that’s being exposed by a web server … most web servers are configured to use a “chroot” system call to lock their serving processes into a directory tree. Symlinks can’t cross chroot boundaries and are treated as dangling if they do so.

    The /etc/services file is not particularly sensitive and I would consider *many* other files as much higher priority for protection with the chattr immutable attribute (your kernels, init and shell binaries, ps and related utilities … pretty much everything your system depends on to boot and every executable that’s trusted by root and/or SUID root and trusted by users on the system.

    Also disabling Ctrl-Alt-Del is only marginally useful for systems being exposed as “kiosks” or student terminals or similar situations where the hardware power switches, reset switches, power cables and such are all secure and yet the console is exposed. It’s basically useless for servers and for desktop systems where the user can power cycle the system without the operating system’s co-operation.

    • Thank for your comment.
      I said SUID/SGID programs *are* useful and you should take care of that, so you may not notice it.
      Symlinks are too dangerous if you’ve ever run a server before that uses Cpanel which is reconfigured not to use chroot at all.
      The /etc/services file was a sample of the files that need to be secured, the idea is the immutable attribute and I discssed how to secure the kernel binaries but you didn’t notice it also.