Ugly bug in Draytek Vigor firewall?

One day I came across a unique error. A client reported that they were unable to query any nameserver outside their network, except for the case they query standard A records. So, A records worked fine but, e.g. NS or MX records failed with timeout. Local DNS servers was properly set with valid forwarders.
So, we experienced:
nslookup    
Default Server:  dc01.hq.local           
Address:  192.168.80.248                                                                 

> google.org
Server:  dc01.hq.local                   
Address:  192.168.80.248

Non-authoritative answer:                       
Name:    google.org                             
Address:  216.239.32.27                                                                         

> set type=mx 
> google.org                                 
Server:  dc01.hq.local                   
Address:  192.168.80.248                                                                       

DNS request timed out.                              
timeout was 2 seconds.                      
*** Request to dc01.hq.local timed-out   

> server 8.8.8.8                                   
Default Server:  google-public-dns-a.google.com           
Address:  8.8.8.8        

> google.org                            
Server:  google-public-dns-a.google.com        
Address:  8.8.8.8

DNS request timed out.                              
timeout was 2 seconds.                      
*** Request to google-public-dns-a.google.com timed-out      

What a riddle! Guess that! :)
After three hours it turned out that in their Vigor 2925 firewall router there was a built-in rule called "xNETBios > DNS" in the section called "Data filter" (very informative names by Draytek guys, phuhh). That blocked such special DNS queries - even if it was DISABLED!

Default factory settings

Factory settings

In the end I had to disable the entire Data Filter section - in that way, external DNS queries got to work as expected. I'm still unable to find any explanation for this.

Model Name : Vigor2925n
Firmware Version : 3.7.6
Build Date/Time : Nov 17 2014 17:20:57

Working

Living with IPFire (bye-bye pfSense)

In the first part of this article I discussed some interesting facts about pfsense. I, again, strongly recommend not to use pfSense 2.2.* in production environments because it is a totally unreliable and buggy system. Okay but what to use then ?
For instance, one can choose IPFire. Yep, I did. It's rock solid, lightning fast and easy to use system. Everything that can't be told about pfSense. I like it.
Except for one minor thing... And that thing is, sadly, not that minor.
For anyone who is familiar with standard iptables chains and logic (I mean input/output/forward/etc) it's very confusing the way pfsense and IPFire virtually handles the traffic.
IPFire consists lots of built-in chains that can be troublesome at the first glance. But you will never get to know about those ones if you use only the GUI based rules editor. I've spent 3 days, frankly, on creating some very basic allow and deny rule on the red0 interface, without any success. That totally screwed me up. You can just never be sure where (I mean, which chain) your web edited rules will be put in. E.g. below shown rules are all faulty, God knows why.

Playing with basic IPFire rules

So I ended up with editing the /etc/sysconfig/firewall.local file and tadaaam, that worked. If you are an expert on iptables, forget your firewall fancy GUI editor forever.

case "$1" in
  start)
        iptables -A CUSTOMINPUT -d 255.255.255.255 -p udp --dport 7437 -j DROP
        iptables -A CUSTOMINPUT -i red0 ! -s 192.168.1.1 -p udp -j DROP
        ;;
  stop)
        iptables -D CUSTOMINPUT -d 255.255.255.255 -p udp --dport 7437 -j DROP
        iptables -D CUSTOMINPUT -i red0 ! -s 192.168.1.1 -p udp -j DROP
        ;;

Just a small side note: reloading the rules with the GUI also reloads your .local defined rules.

Pfsense, Transparent Squid and Dansguardian - a piece of crap

How to set up a transparent Squid (here: http only) proxy with an advanced level security filtering add-in for your local network ?

What is Pfsense? What is a proxy? If you don't know the answer to these questions this is not for you.

1. Install Pfsense
2. Set up your interfaces, default gateway, DNS resolvers or forwarders, etc.
3. Install Squid3 and Dansguardian (at the time of this writing Squidguard is broken in recent Pfsense and won't work with Squid3. In systemlog we can see lots of:

squid[81808]: Squid Parent: (squid-1) process 45089 exited with status 1
squid[81808]: Squid Parent: (squid-1) process 63729 started
(squid-1): The redirector helpers are crashing too rapidly, need help!

and in cache.log:

Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
kid1| WARNING: redirector #Hlpr0 exited
FATAL: The redirector helpers are crashing too rapidly, need help!

Of course we have libldap-2.4.so.2 is right there, in /usr/pbi/squidguard-devel-amd64/lib/libldap-2.4.so.8. So after some hours of struggling I decided to give squidGuard up and switch further. Dansguardian is a more advanced and complex filter system anyway.

4. Setup your (transparent) Squid, for example:

5. Setup your Dansguardian

Remember to edit your regexp URL filters because the default ones will surely block some nice part of your harmless favourite pages. In the log (did you turn logging on?) search for:
[2.2.2-RELEASE][admin@my.proxy.local]/var/log/dansguardian: grep DENIED access.log

6. You need an additional port forwarding rule to get it go because, as you can see, Dansguardian listens only on TCP 8080. Pay attention on the Destination address: you should not access Pfsense via Dansguardian. If Dansguardian dies for whatever reason (this happens frequently if you want to upgrade it manually, yeah I've permanently killed it several times in my lab), you won't be able te reach the default webadmin interface. If you use SquidGuard this step is not required because Squid creates its "hidden" firewall rule and SquidGuard does not use any TCP port as DG does.

That's all. If you don't have any blocking firewall rule, your advanced (but not-yet-fine-tuned!) HTTP proxy system works now.

UPDATE: actualy, it does not. Another irritating, ugly, hideous bug here. It's 2015 and this bug still exists for more than 2 years, still in the latest stable release: *DENIED* Web upload is banned.
I've tried these recompiled binaries, written in this forum thread, but after three days of digging deep, I could say that more problems encountered that solved. I'm too pissed off to detail all the hacks I've done.
If anyone asked me if he could give pfsense a try I would say: DO NOT. NEVER. 
YOU SHOULD AVOID using PFSENSE.  Latest "stable" is an ANNOYING, unthinkably BUGGY system, mindlessly designed GUI, full of outdated, incompatible and deprecated packages - what's more, its whole package managing system is broken or, if you are lucky enough, "just" failing - and if packages somehow accidentaly work with each other, pray everyday for the Lord to keep this thing in such a working condition and never think about any system update! I can't imagine that Pfsense is in production use by anyone. How could a sysadmin be so fearless? Looking back to the far past I admit that Pfsense was a great software. But this is the case no more.
I wish I could get these days back of my life wasted on this piece of sh*. More to come in this topic.

IPTABLES - how to allow or deny certain countries of the world

It's a usual request for a sysadmin to ban or allow only a certain country in firewalls or .htaccesses of apache. Here are two common ways to do that.

Method 1.
Using xtables and maxmind

apt-get install libtext-csv-xs-perl module-assistant geoip-database libgeoip1

module-assistant --verbose --text-mode auto-install xtables-addons

mkdir /usr/share/xt_geoip

cd /usr/share/xt_geoip

# this is a rather old package but for free

wget http://terminal28.com/wp-content/uploads/2013/10/geoip-dl-build.tar.gz
tar xvf geoip-dl-build.tar.gz
./xt_geoip_dl
./xt_geoip_build -D . *.csv

##EXAMPLE ##EXAMPLE ##EXAMPLE ##EXAMPLE ##EXAMPLE ##EXAMPLE ##EXAMPLE 

iptables --flush # BEWARE

iptables -A INPUT -p tcp --dport 443 -m geoip --src-cc HU,CZ,PL,RO -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j DROP

Method 2.
Simply using https://www.countryipblocks.net/country_selection.php to get ranges to allow/deny

IPSET for heavy use

What is ipset?
According to the official page: "IP sets are a framework inside the Linux 2.4.x and 2.6.x kernel, which can be administered by the ipset utility. Depending on the type, currently an IP set may store IP addresses, (TCP/UDP) port numbers or IP addresses with MAC addresses in a way, which ensures lightning speed when matching an entry against a set."
It's worth mentioning that this cool tool is mainly written by Jozsef Kadlecsik, a Hungarian Linux kernel expert.

Why and when to use ipset?
If you have plenty of IP rules in your iptables and their number is growing, one day you are going to experience a heavy performance drop. In practice if you have more than ca. ~1000-1500 rules you should worry about this. Anyway, it's more neater to use ipset above dozens of sets.

How does it work? 
You don't have to know and don't want to know. It's enough to know that it generates hashes from the rules and flipping thru these hashes is so efficient that it doesn't matter how many rules you have, the fastness of searching the whole set remains almost the same.

How to use?


For beginners

Assuming you have a modern .deb based distro, here are some simple steps.
apt-get update
apt-get install ipset
ipset create SET1 hash:net (for example)
ipset add SET1 91.83.231.25 (for example)
ipset add SET1 80.249.172.0/24 (for example)
iptables -I INPUT -m set --match-set SET1 src -j DROP (to drop all matching packets)
To save all your sets:
ipset save > backupfile
To delete:
ipset del SET1 91.83.231.25 - deletes a single line from a set
ipset flush SET1 - deletes a whole set
ipset destroy - deletes all the sets
BEFORE deleting a set you should delete the links in your iptables pointing to your set, e.g.
iptables -D INPUT -m set --match-set SET1 src -j DROP
To see your sets in different ways:
ipset -n list
ipset -t list
ipset list
To check if an IP address exists in a set:
ipset test 10.10.10.10
To restore your sets (assuming that sets in the file don't exist already)
ipset restore < mybackupfile

Some tricks

To create a new ruleset being the type of hash (thats the type because you want ipset, more info here), append some addresses to it and deny them based on the source IP address.
ipset -N set2 hash
ipset -A set2 10.10.10.0/24
ipset -A set2 80.249.172.62
iptables -A INPUT -m set --myset set2 src -j DROP
To fast delete a rule (don't forget to delete the relevant iptables rule before)
ipset -F set2
ipset -X set2 
or simply:
ipset f  
ipset x
To auto-deny a host that wants to connect to your SSH port is so simple that:
ipset -N denied hash
iptables -A INPUT -p tcp --dport 22 -j SET --add-set denied src
iptables -A INPUT -m set --set denied src -j DROP

To block IP addresses based on geo location (country) here is a simple shellscript:
#!/bin/sh
ipset -N geoblock nethash
for IP in $(wget -O – http://www.ipdeny.com/ipblocks/data/countries/{cn,kr,pk,tw,sg,hk,pe}.zone)
do
ipset -A geoblock $IP
done
iptables -A INPUT -m set –set geoblock src -j DROP

To auto-timeout a rule (and not generate any message if it already exists):
ipset create test hash:ip timeout 10
ipset add --exists test 91.83.231.25 120 (overwriting the default 10 seconds value)

To auto-learn a MAC address: (and define a range)
ipset create test bitmap:ip,mac range 192.168.0.0/24
ipset add test 192.168.0.1,11:11:22:22:11:11
ipset add test 192.168.0.2 (this one will auto-learn)

More advanced WAN/LAN/DMZ firewall example

We define our client (source) IPs and ports they want to communicate to. We define our server IP address and ports. We allow established tcp sessions. Here, things are getting interesting.
We allow all packets coming in my external (internet) interface heading towards to my dmz server ip address and ports. (see dst,dst. That means destination IP AND destination port. (Here HTTP and HTTPS and udp only DNS and ping [it will reply the echo].)
Then we allow our LAN clients to access the internet web based on src,dst. (source IP address and destination port). In our case, anyone in the LAN can browse the web but only 192.168.0.10 can use https.
In the last line we allow our trusted administrator to connect to tcp ports 22020 to 22022 anywhere in our system.

ipset n dmzservers hash:ip,port
ipset n mynetworks hash:ip,iface
ipset n lanusers hash:ip,port
ipset n remoteadmin hash:ip,port
ipset a dmzservers 195.195.195.195,http
ipset a dmzservers 195.195.195.195,https
ipset a dmzservers 195.195.195.195,udp:53
ipset a dmzservers 195.195.195.195,icmp:ping 
ipset a mynetworks 192.168.0.0/24,eth0
ipset a mynetworks 8.8.8.0/24,eth1
ipset a mynetworks 195.195.195.193/30,eth2 (these network definitions are not used)
ipset a lanusers 192.168.0.0/24,http
ipset a lanusers 192.168.0.10,https
ipset a remoteadmin 82.112.112.112,tcp:22020-22022
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT

iptables -A FORWARD -i $EXTERNAL -m set --match-set dmzservers dst,dst -m state --state NEW -j ACCEPT 
iptables -A FORWARD -i $INTERNAL -m set --match-set lanusers src,dst -m state --state NEW -j ACCEPT
iptables -A FORWARD -i $EXTERNAL -m set --match-set remoteadmin src,dst -m state --state NEW -j ACCEPT