firewalls – Is there a bulletproof way to configure your AWS system?

I am learning how to configure a multi-region AWS system. However, everything I read says “if you want/need more security then do X”. Why would anyone want less security (is what I start thinking)… Is there a way to create a bullet proof AWS system? If so, what are the key pieces? If not, how can I prove that to someone who asks?

Protect Your VPS With an Electronic Pitbull: Active Firewalls

A firewall is a security tool that blocks network traffic, with many different configuration options.  For example, you could configure your firewall to block all traffic except attempts to connect to port 80 and 443 (for a web server) as well as your ssh port.  This is an example of the best practice of “blocking everything except what is explicitly allowed”. 

However, even with this sort of policy in place, the security gains for a basic firewall are limited.  If you only have a web server and sshd server running, then there is nothing to block because connecting to a different port will fail anyway.  Chiefly the benefit from this kind of simple firewall is to prevent inadvertent opening of ports.  For example, let’s say you install a piece of software that has a dependency that starts up a service you hadn’t intended to run, or perhaps you change configurations an accidentally set MySQL to expose itself on your public IP.  With a basic firewall, traffic to those port will still be blocked so you benefit from this kind of safety net.

Read more to continue…

However, the real power of firewalls becomes evident when we deploy an active firewall that intelligently monitors traffic and blocks traffic to/from specific clients.  For example:

  • If you are running an IMAP service, you have no choice but to have it publicly-facing where it may become a magnet for people trying to brute-force (guess passwords).  If you deploy an active firewall, the firewall software will watch the IMAP logs and after a configured number of failed logins, the client IP trying to connect will be temporarily blocked. 
  • Likewise, you could have someone trying passwords on your ssh ports all day but they’ll be quickly stopped by an active firewall after a few failed logins.
  • Although an active firewall cannot stop a distributed denial of service, it can end some limited denial of service attacks that are coming from a small number of attacking hosts.

Active firewalls are critical when you have less sophisticated users on your system, such as if you’re running a web host or mail server.  While you may conscientiously pick good passwords, use SSH keys, etc., your more naive users might not.   An active firewall helps protect you by eliminating brute-forcing.

Installing CSF

There are number of different products but one of the best known is ConfigServer Firewall.

To set it up, first install some required perl modules.  On Debian-based systems:

apt-get install libwww-perl liblwp-protocol-https-perl libgd-graph-perl

On CentOS-based systems:

yum install perl-libwww-perl.noarch perl-LWP-Protocol-https.noarch perl-GDGraph

If you’re running CentOS, that distro’s native firewall (firewalld) comes pre-enabled.  You’ll want to disable it before setting up CSF:

systemctl stop firewalld
systemctl disable firewalld

Now download and extract the tarball:

cd /usr/src
wget https://download.configserver.com/csf.tgz 
tar xzf csf.tgz
cd csf
sh install.sh

Next make sure you have all the required kernel modules:

# perl /usr/local/csf/bin/csftest.pl 
Testing ip_tables/iptable_filter...OK
Testing ipt_LOG...OK
Testing ipt_multiport/xt_multiport...OK
Testing ipt_REJECT...OK
Testing ipt_state/xt_state...OK
Testing ipt_limit/xt_limit...OK
Testing ipt_recent...OK
Testing xt_connlimit...OK
Testing ipt_owner/xt_owner...OK
Testing iptable_nat/ipt_REDIRECT...OK
Testing iptable_nat/ipt_DNAT...OK
RESULT: csf should function on this server

In this case, all required kernel modules were present.  On your VPS you might see an error or missing module but as long as you see the result that “csf should function on this server,” you’re good to go.

Configuring CSF

CSF starts in “TESTING” mode.  This is so you do not accidentally lock yourself out. 

Take a look at /etc/csf/csf.conf.  You probably want to adjust the following:

  • TCP_IN has a list of ports you allow.  Remove any services you’re not using.  For example, if you’re not running an FTP server, you can remove ports 20 and 21.  If you’ve changed your SSH port, make sure it is in this list and remove port 22.
  • TCP_OUT should match TCP_IN
  • You can probably pare down UDP_IN and UDP_OUT, perhaps just to port 53 (DNS)
  • If you’re not using IPv6, set IPV6 to 0.  Otherwise, adjust TCP6_IN, TCP6_OUT, UDP6_IN, and UDP6_OUT to match the ipv4 versions.
  • You will be mailed for each blocked IP.  You can modify the templates in /etc/csf/alerts to set an appropriate To: address, or you can set LF_ALERT_TO to one master email address.

There are many, many more ways to customize CSF.  You should take a look through the documentation on configserver.com to see the plethora of CSF capabilities.

Starting CSF

By default, CSF is in TESTING mode.  Before you start it for real, it’s handy to pre-set a time to turn it off.  You could use a crontab entry like this:

*/10 * * * *   systemctl stop lfd ; systemctl stop csf

Don’t forget to remove this crontab entry when you are ready to use CSF for real!

This will disable CSF entirely every 10 minutes on the 10s (so 1:00, 1:10, etc.)  You can then fire up CSF with the knowledge that if you lock yourself out, you will have to wait a maximum of 10 minutes to get back in.  Alternatively, you can login from your VPS provider’s console.

When you’re ready to go, modify /etc/csf/csf.conf and set TESTING to 0.  Then:

systemctl restart csf
systemctl restart lfd

Testing CSF

Here’s a test of CSF so you can see how it works.  I’ve replaced the IP of the server with ‘s.s.s.s’ and the IP of the client with ‘c.c.c.c’.

I ssh’d with a bogus user to the VPS:

$ ssh nonexistant@s.s.s.s
nonexistant@s.s.s.s's password:
$ ssh nonexistant@s.s.s.s
nonexistant@s.s.s.s's password:
$ ssh nonexistant@s.s.s.s
nonexistant@s.s.s.s's password:
$ ssh nonexistant@s.s.s.s
nonexistant@s.s.s.s's password:
$ ssh nonexistant@s.s.s.s
nonexistant@s.s.s.s's password:

CSF (specifically the lfd daemon) detected someone trying to brute-force a login by watching the system logs.  In /var/log/lfd.log this entry appeared:

Apr  7 16:31:23 debian10 lfd(6780): (sshd) Failed SSH login from c.c.c.c (US/United States/some.example.com): 5 in the last 3600 secs - *Blocked in csf* (LF_SSHD)

CSF then created a firewall rule to block the IP.  It also made a notation in /etc/csf/csf.deny (so that if the system restarts, the firewall rule is recreated):

c.c.c.c # lfd: (sshd) Failed SSH login from c.c.c.c (US/United States/some.example.com): 5 in the last 3600 secs - Tue Apr  7 16:31:23 2020

Everything after the # is a comment so you know why this IP was blocked.  CSF also looks up the location of the IP address (this is not 100% perfect) and notes it.

Behind the scenes, CSF is manipulating the kernel’s firewall rules to create blocks.  You can see the entire CSF ruleset at any time by issuing this command:

csf -l

You may also see them in dmesg and /var/log/messages, depending on your syslog config.

If you configured an email address as mentioned above, you’ll get an email like this one:

Time:     Tue Apr  7 16:31:23 2020 -0700
IP:       s.s.s.s (US/United States/some.example.com)
Failures: 5 (sshd)
Interval: 3600 seconds
Blocked:  Permanent Block (LF_SSHD)
Log entries:
Apr  7 16:31:07 debian10 sshd(6722): Invalid user nonexistant from c.c.c.c port 42858
Apr  7 16:31:08 debian10 sshd(6722): pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=c.c.c.c
Apr  7 16:31:10 debian10 sshd(6722): Failed password for invalid user nonexistant from c.c.c.c port 42858 ssh2
Apr  7 16:31:13 debian10 sshd(6722): Failed password for invalid user nonexistant from c.c.c.c port 42858 ssh2
Apr  7 16:31:18 debian10 sshd(6722): Failed password for invalid user nonexistant from c.c.c.c port 42858 ssh2

After receiving this email I tried sshing from the same client:

$ ssh nonexistant@s.s.s.s

…and there was an infinite pause.  Here I am blocked not just from SSH but at the IP level, so nothing from my client would be able to connect to this server (web, FTP, etc.)

To enable the product, issue these commands:

systemctl enable csf
systemctl enable lfd

Again, if you’ve put any kind of temporary disabling command in cron during testing, be sure to remove it before relying on the firewall.

An active firewall like CSF is a wonderful security tool but as always, security is the inverse of convenience.  There are two headaches that are common: a flood of block notifications and accidental blocks.

Reducing Block Notifications

At first, you may be surprised at how many emails you get about blocked IPs.  Welcome to the reality of the public Internet!  Previously you were in blissful ignorance – now you can see how many attempts to attack your box are routine every day.

If you’re running a web host, then there are many ports that you are kind of stuck with because users expect FTP on port 20/21, email on 25/587, etc.  However, you might consider changing your SSH port.

This advice is mildly controversial because changing your SSH port does not prevent someone from finding it and trying to login on that port.  However, it will radically cut down how many automated attacks you get.  Many attackers will mass-scan IPs and hone in on those that answer on port 22.  If your system doesn’t respond on port 22, they will move on to another host. 

Some sysadmins take the point of view that once they’ve secured the box by allowing logins only via ssh key and disabling root logins, there is little to fear from script kiddies wasting their time banging on the SSH port.  While this is true, the numerous emails you’ll receive daily are tedious and may drown out alerts you truly do want to review.

You can change sshd to run on any port, but ideally an unused high port (above 1024).

For example, let’s say you wanted to use port 32222.  To do this, modify /etc/ssh/sshd_config:

# Port 22
Port 32222

In this case, we’ve commented out the default and added our port. 

Next, be sure to update TCP_IN and TCP6_IN in /etc/csf/csf.conf to both remove port 22 and add your custom port.

Then restart sshd:

systemctl restart sshd

Your ssh command from your client will now look something like this if your account is ‘joe’:

ssh -p 32222 -i my_ssh_key joe@myvps.example.com

Handling Mistaken Blocks

When CSF blocks a client IP, it adds a firewall rule and also writes it to/etc/csf/csf.deny.  If a client is accidentally are blocked by CSF, you can immediately unblock them with this CSF command (here I’m using ‘c.c.c.c’ again for the client’s IP).  This will both remove the kernel’s firewall rule and clear it from /etc/csf/csf.deny:

# csf --denyrm c.c.c.c
Removing rule...
DROP  all opt -- in !lo out *  c.c.c.c  -> 0.0.0.0/0 
LOGDROPOUT  all opt -- in * out !lo  0.0.0.0/0  -> c.c.c.c

You can also whitelist any IP by adding it to /etc/csf/csf.allow.  Note that CSF whitelists the IP you’re connected from when you first set the product up.  You’ll see an entry like this in /etc/csf/csf.allow:

c.c.c.c # csf SSH installation/upgrade IP address - Tue Apr  7 22:56:37 2020

Consider whitelisting your home IP and/or any VPN IPs you use to connect.

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

firewalls – Is it possible to intentionally cause Google and AWS to block my IP address?

It may sound a little weird. I am validating one of my possible research ideas where I want to see if I can intentionally and effectively make websites such as Google and AWS to block my IP. By “block”, I mean it won’t let me directly access the service, but not necessarily blacklist my IP. For example, the website will ask me to solve a ReCaptcha before I can access its service, instead of telling me service is unavailable.

I know if I send a large number of requests in a short time (i.e., using DoS) it is very likely that I can make it work, but I wonder if there is any other “efficient” way to make it happen. From what I have found here: https://support.google.com/websearch/thread/2596872?hl=en, it mentioned Google may block the following:

  • Sending searches from a robot, computer program, automated service, or search scraper
  • Using software that sends searches to Google to see how a website or webpage ranks on Google
  • Using an app, program or script to perform a large number of searches in a short time

Is it possible that I mimic such a request and cause myself to be blocked in just one or a few requests?

firewalls – What are the downsides to having PostgreSQL database available directly from public network?

When auditing a machine for one of my clients I noticed that they have PostgreSQL listening and accessible from public network. This is not necessary for their application, seems they simply neglected to set up the firewall properly.

PostgreSQL seems to be running default pg_hba.conf file from the distribution package, and it does not accept connections without valid credentials.

I still feel that it is wrong to unnecessarily expose the database to the outside world. What arguments are for/against such approach?

firewalls – (Public) IP address [x.x.x.x] of Synology has been blocked by SSH

For the past few months I regularly see alerts on my Synology about SSH connection being blocked. Somebody (here a nice Chinese guy from 222.186.15.158) was attempting to connect to my NAS with the root account (Fortunately PermitRootLogin is disabled).

What I am a bit worried because if I see a public address here, it means my NAS is somehow reachable from the internet. However all ports are closed on my front router, NAT is disabled, DMZ is disabled.

When I try to nmap my router from the outside I get this :

$ nmap -Pn x.x.x.x

Starting Nmap 7.60 ( https://nmap.org ) at 2020-05-17 01:07 CEST
Nmap scan report for x.x.x.x
Host is up (0.012s latency).
Not shown: 997 filtered ports
PORT     STATE  SERVICE
113/tcp  closed ident
2000/tcp open   cisco-sccp
5060/tcp open   sip

Nmap done: 1 IP address (1 host up) scanned in 8.23 seconds

So there is no SSH entry point.

How would it be technically possible to see a public IPv4 address attempting to connect to my LAN NAS?

Firewalls – VPN out of one company

I had to recently bypass my company policy by establishing a VPN connection from a LAN device (Raspberry Pi) in my company to a WAN server.

Since externally accessible devices have to be placed in the DMZ, I was not able to get the appropriate authorization. In fact, it is too complex and requires too many permits. So because I'm lazy, it's me just Use openvpn to connect my Raspberry Pi to the outside via a virtual machine in the cloud. From this VM I can reach my device without opening a port on the master firewall.

To avoid problems with IT, I do not use the standard 1194 port, but a standard port: 443.

It made me realize how weak this concept of the DMZ is. Even with a strong firewall, it is still possible to place a spy mole in a company. Is my company's IT security policy really bad or is it very difficult to prevent such a mechanism from being set up?

Firewalls – Disable MDNS publishers on Linux?

I just installed a firewall and other security related software on my Linux computer (Ubuntu 18.04) and after reading it decided to disable two services that I don't need:

sudo systemctl stop cups-browsed avahi-daemon
sudo systemctl disable cups-browsed avahi-daemon

After the restart, no ports are opened for these services, which I expected. Good!

However, there is an application mdns-publisher which appends UDP:5353. I think this is the same port that Avahi normally listens to, and a quick online search suggests that this is related to Avahi. I tried to run:

sudo systemctl stop mdns-publisher

but that returns "Error stopping mdns-publisher.service: unit mdns-publisher.service not loaded.".

What is this application; why is it running again avahi-daemon is disabled; and if I don't need it how can I disable it mdns-publisher?

Firewalls – Why is the "Not Recommended" option used by default in the Windows security warning dialog?

When I start a program that requires network communication for the first time, the following dialog is displayed:

Block Windows Defender Firewall dialog box

It has been a part of Windows for some time (this runs on Windows 10, but note the Vista / 7 design language) and, as far as I know, has always activated the selection of public networks by default. But based on the text, I don't understand:

Allow (application) to communicate on these networks:

() Private networks like my home or work network

(X) Public networks … (Not recommended as these networks often have little or no security)

Why should it be the default not recommended Choice? Given that people tend to click and ask questions first (including myself, included from time to time), this seems to be an obvious vulnerability. To do the recommended action, I need to check the first box and uncheck the second. Then click the Allow Access field. However, this appears to have been unchallenged or unchanged for several years.

Am I misunderstanding this dialog or is there a legitimate reason for its default selection?

How can I set up a dual SSH tunnel so that two machines A and B behind firewalls are ssh into a central server so that A can VNC in B?

I need to connect my computer A to another friend B's computer using VNC.
We both have no way of opening incoming ports. ISPs do not allow tunneling unless you buy a static IP-based business plan.
By the way, both are Linux systems

We have a full access web server that we can both ssh on.
How do we set up a tunnel from both ends to the server for VNC to work?

I know how to set up an SSH tunnel between two systems, and I think it's not difficult to do A-> B-> C by doing SSH first on A to B and then on B to C.

But I need A-> C and B-> C instead of A-> B and B-> C.

Basically, I think I need a proxy server – can this be done with SSH or even some kind of Linux network magic?