Hokay, so I'm wondering if somebody can help me out here.
We're a little picky about losing internet connectivity at the office because that means we can't process a credit card and when that happens things get pear shaped really quick.
So we keep two internet connections (cable and dsl) and the Linux router will flip between them if it detects a problem with connecting to our credit card provider or Google. And because I'm a dork I keep a running check of our ability to connect to the CC processor via Nagios.
What I want to do are two things, and they are related:
Be able to run an ICMP ping from the firewall that connects to the CC processor from each separate interface regardless of which one is the current default gateway. Problem I have there is when the default gateway is the cable company all I can get to on the DSL side is my next top.
The other thing I'd like is to be able to ssh in from either side regardless of which one is currently the default gateway. That's also giving me trouble.
With all that out of the way, here's what I've got
Routing table:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 71.13.68.221 0.0.0.0 UG 0 0 0 eth4
71.13.68.220 0.0.0.0 255.255.255.252 U 0 0 0 eth4
72.35.32.74 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0
172.21.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.21.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
172.21.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
172.21.5.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
eth4 being the cable connection and ppp0 being the PPPoE DSL connection.
I tried getting this to work today by switching away from using MASQUERADE in favor of SNAT, which is easy because I've got static IPs for both sides.
Here's where my script is at:
#!/bin/sh
# eth0 - Office
# eth1 - Servers
# eth2 - Registers
# eth3 - Cameras
# eth4 - Charter
# ppp0/eth5 - DSL
# eth6 - Sign
OFFICE=eth0
SERVER=eth1
REGISTER=eth2
CAMERA=eth3
SIGN=eth6
CHARTERIP=71.13.68.222
ACEIP=72.35.34.4
#POSGATEWAY=`/usr/bin/host posgateway.secureexchange.net | awk '{print $4}'`
POSGATEWAY=65.118.49.56
#CGWEBSITE=`/usr/bin/host www.countrysidegreenhouse.com | awk '{print $4}'`
CGWEBSITE=216.120.135.92
DBMASTER=`/usr/bin/host dbmaster.cgfm.local | awk '{print $4}'`
FIDDLEBOX=`/usr/bin/host fiddlebox.cgfm.local | awk '{print $4}'`
SECURITYCAMS=`/usr/bin/host securitycams.cgfm.local | awk '{print $4}'`
# Reference material: http://oceanpark.com/notes/firewall_example.html
EXT_ETH=$1
# Delete all existing rules
/sbin/iptables -F
/sbin/iptables -F -t nat
# Set default chain policies
/sbin/iptables -P INPUT DROP
/sbin/iptables -P FORWARD DROP
/sbin/iptables -P OUTPUT ACCEPT
# NAT
/sbin/modprobe iptable_nat
# TFTP
/sbin/modprobe nf_conntrack_tftp
# Enable masq for the office, servers, and registers.
/sbin/iptables -t nat -A POSTROUTING -o eth4 -s 172.21.2.0/24 -j SNAT --to-source $CHARTERIP
/sbin/iptables -t nat -A POSTROUTING -o eth4 -s 172.21.3.0/24 -j SNAT --to-source $CHARTERIP
/sbin/iptables -t nat -A POSTROUTING -o eth4 -s 172.21.4.0/24 -j SNAT --to-source $CHARTERIP
# Re-write source address on stuff going out the backup interface
/sbin/iptables -t nat -A POSTROUTING -o ppp0 -s 172.21.3.0/0 -j SNAT --to-source $ACEIP
echo 1 > /proc/sys/net/ipv4/ip_forward
# Forward all packets to the external (EXT_ETH) network from the $OFFICE Office network
/sbin/iptables -A FORWARD -i $OFFICE -o $EXT_ETH -j ACCEPT
# Forward existing connections betweeen $OFFICE (office) and internet (EXT_ETH)
/sbin/iptables -A FORWARD -i $EXT_ETH -o $OFFICE -m state --state ESTABLISHED,RELATED -j ACCEPT
## BEGIN PUBLIC INTERNET ##
# Forward all packets to the external (EXT_ETH) network from the $SERVER Server network
/sbin/iptables -A FORWARD -i $SERVER -o $EXT_ETH -j ACCEPT
# Forward existing connections betweeen $SERVER (servers) and internet (EXT_ETH)
/sbin/iptables -A FORWARD -i $EXT_ETH -o $SERVER -m state --state ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -A FORWARD -i $SERVER -o ppp0 -j ACCEPT /sbin/iptables -A FORWARD -i ppp0 -o $SERVER -m state --state ESTABLISHED,RELATED -j ACCEPT
# Forward only packets on a white list to the public network for registers
/sbin/iptables -A FORWARD -i $REGISTER --dst $CGWEBSITE/32 -o $EXT_ETH -j ACCEPT
/sbin/iptables -A FORWARD -i $REGISTER --dst $POSGATEWAY/32 -o $EXT_ETH -j ACCEPT
# .. And once they're established let the response back in
/sbin/iptables -A FORWARD -i $EXT_ETH -o $REGISTER -m state --state ESTABLISHED,RELATED -j ACCEPT
## END PUBLIC INTERNET ##
## BEGIN OFFICE ##
# Office can see servers
/sbin/iptables -A FORWARD -i $OFFICE -o $SERVER -j ACCEPT
/sbin/iptables -A FORWARD -i $SERVER -o $OFFICE -m state --state ESTABLISHED,RELATED -j ACCEPT
# Office can see regiters
/sbin/iptables -A FORWARD -i $OFFICE -o $REGISTER -j ACCEPT
/sbin/iptables -A FORWARD -i $REGISTER -o $OFFICE -m state --state ESTABLISHED,RELATED -j ACCEPT
# Office can see cameras
/sbin/iptables -A FORWARD -i $OFFICE -o $CAMERA -j ACCEPT
/sbin/iptables -A FORWARD -i $CAMERA -o $OFFICE -m state --state ESTABLISHED,RELATED -j ACCEPT
# Forward everything from the office to the sign network and vice versa
/sbin/iptables -A FORWARD -i $OFFICE -o $SIGN -j ACCEPT
/sbin/iptables -A FORWARD -i $SIGN -o $OFFICE -m state --state ESTABLISHED,RELATED -j ACCEPT
# END OFFICE ##
## BEGIN SERVERS ##
# Servers can see cameras
/sbin/iptables -A FORWARD -i $SERVER -o $CAMERA -j ACCEPT
/sbin/iptables -A FORWARD -i $CAMERA -o $SERVER -m state --state ESTABLISHED,RELATED -j ACCEPT
# Servers can see office because that's where some cameras are.
/sbin/iptables -A FORWARD -i $SERVER -o $OFFICE -j ACCEPT
/sbin/iptables -A FORWARD -i $OFFICE -o $SERVER -m state --state ESTABLISHED,RELATED -j ACCEPT
## END SERVERS ##
## BEGIN REGISTERS
# Registers can contact DBMASTER only
/sbin/iptables -A FORWARD -i $REGISTER -o $SERVER --dst $DBMASTER/32 -p tcp --dport 3306 -j ACCEPT
# ... and Fiddlebox's TFTP server
/sbin/iptables -A FORWARD -i $REGISTER -o $SERVER --dst $FIDDLEBOX/32 -j ACCEPT
# Forward existing connections betwen registers and servers
/sbin/iptables -A FORWARD -i $SERVER -o $REGISTER -m state --state ESTABLISHED,RELATED -j ACCEPT
## END REGISTERS ##
## BEGIN CAMERAS ##
# Yeah, they can't inniate a connection to anything, so they don't get any rules
## END CAMERAS
# Permit packets from the firewall to itself
/sbin/iptables -A INPUT -i eth4 -m state --state ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -A INPUT -i ppp0 -m state --state ESTABLISHED,RELATED -j ACCEPT
# Accept all local traffic
/sbin/iptables -A INPUT -i $OFFICE -s 0/0 -d 0/0 -j ACCEPT
/sbin/iptables -A INPUT -i $SERVER -s 0/0 -d 0/0 -j ACCEPT
/sbin/iptables -A INPUT -i $REGISTER -s 0/0 -d 0/0 -j ACCEPT
/sbin/iptables -A INPUT -i $CAMERA -s 0/0 -d 0/0 -j ACCEPT
/sbin/iptables -A INPUT -i $SIGN -s 0/0 -d 0/0 -j ACCEPT
/sbin/iptables -A INPUT -i lo -s 0/0 -d 0/0 -j ACCEPT
# Accept SYN packets for SSH connections
/sbin/iptables -A INPUT -p tcp -s 0/0 -d 0/0 --destination-port 22 --syn -j ACCEPT
# SSL
/sbin/iptables -A FORWARD -p tcp -s 0/0 -d 0/0 --destination-port 443 --syn -j ACCEPT
#Bluecherry
/sbin/iptables -A FORWARD -p tcp -s 0/0 -d 0/0 --destination-port 7001 --syn -j ACCEPT
/sbin/iptables -A FORWARD -p tcp -s 0/0 -d 0/0 --destination-port 7002 --syn -j ACCEPT
#Nagios
/sbin/iptables -A PREROUTING -t nat -i eth4 -p tcp --dport 443 -j DNAT --to $FIDDLEBOX:443
#BlueCherry
/sbin/iptables -A PREROUTING -t nat -i eth4 -p tcp --dport 7001 -j DNAT --to $SECURITYCAMS:7001
/sbin/iptables -A PREROUTING -t nat -i eth4 -p tcp --dport 7002 -j DNAT --to $SECURITYCAMS:7002
# These are attempts at getting the non-primary connection to still forward stuff to internal servers. Doesn't work.
/sbin/iptables -A PREROUTING -t nat -i ppp0 -p tcp --dport 443 -j DNAT --to $FIDDLEBOX:443
/sbin/iptables -A PREROUTING -t nat -i ppp0 -p tcp --dport 7001 -j DNAT --to $SECURITYCAMS:7001
/sbin/iptables -A PREROUTING -t nat -i ppp0 -p tcp --dport 7002 -j DNAT --to $SECURITYCAMS:7002
# Close off a few last things
/sbin/iptables -A INPUT -s 0/0 -d 0/0 -p udp -j DROP
/sbin/iptables -A INPUT -s 0/0 -d 0/0 -p tcp --syn -j DROP
Pretty sure the problem is more to do with my routing table only having one gateway, but I don't want any internal traffic going out the "backup" one unless it originated from the firewall itself. So not even 1 out of 1,000 internal connections to the public network should go through it. So just giving them metric labels of 1 and 1000 doesn't work for me. If I have to, yeah, I will, but I want to avoid that.
Any suggestions?
