Ethernet Bridge + netfilter Howto



Ethernet Bridge +netfilter Howto

http://www.tldp.org/HOWTO/Ethernet-Bridge-netfilter-HOWTO.html

1.Introduction

Ethernet bridges connect twoor more distinct ethernet segments transparently.
An ethernetbridge distributes ethernet frames coming in on one port to otherports associated to the bridge interface. This is accomplished withbrain: Whenever the bridge knows on which port the MAC address towhich the frame is to be delivered is located it forwards this frameonly to this only port instead of polluting all ports together.

Ethernet interfaces can beadded to an existing bridge interface and become then (logical) portsof the bridge interface.

Putting a netfilter structureon top of a bridge interface renders the bridge capable of servicingfiltering mechanisms. This way, a transparent filtering instance canbe created. It even needs no IP address assigned to work. Of course,you can assign an IP address to the bridge interface for maintenancepurposes ( certainly, with ssh only ;-).

The advantage of this systemis evident. Transparency alleviates the network administrator of thepain of restructuring the network topology. And users may not noticethe existence of the bridge but their connection beeing blocked.Also, users are not disturbed while working (think of a company wherenetwork connection loss pays alot).

The other common case is aclient beeing connected to the global web via a leased router. As theproviders seldomly grant administration privileges on their leasinghardware, the client cannot change the interconnecting configuration.But, of course, the client has a network running, and wants to spendat least as possible, he does not want to reconfigure his entirenetwork. And he does not need to if he uses a bridging device.

2.Requiredsoftware

This software setup is neededon the ethernet bridge computer. According to ourTestinggrounds.

2.1FeaturedLinux kernel

Use of kernel 2.6 is not yet agood idea. Yes, it's astonishing. The why the bridging code breaksand where it does so has not yet come to my and others attention, Icannot recommend kernels of the 2.6 series. You have the clou? Assureyourself the credit, mail the solution to me (e-mail address at entrypage). See alsoKernel-Notesfor additional information on this. So far, use kernel 2.4 series.
Asof kernel version 2.4.18 there's already support for theEthernet Bridge capability built-in. No patches needed so far.Regarding later kernel versions, it must be stated that2.4.23might be less recommendable, especially in conjunction with ebtablesand netfilter-bridging. Later versions seem advisable.
Thefollowing paragraph is outdated now (2005-07-12) as all we need ispresent in kernel. You may skip this paragraph, it is only retainedfor legacy:
But if we intend to use netfilter capabilities,because we want to run iptables on our new Linux router/fw box, westill need to apply a patch. Any patches needed can be found anddownloaded on thesourceforgeEthernet Bridge homepage.

root@bridge:~> cd /usr/src/
root@bridge:~> wget -c http://bridge.sourceforge.net/devel/bridge-nf/bridge-nf-0.0.7-against-2.4.18.diff
root@bridge:~> cd /usr/src/linux/
root@bridge:~> patch -p1 -i ../bridge-nf/bridge-nf-0.0.7-against-2.4.18.diff
        

Supposedly we want netfiltersupport on our bridge interface and we have already patched thevanillal kernel we may now activate some necessary kernelconfiguration items. On how to build a private kernel image see theCD-Net-Install-HOWTO,Toolbox. Oh, yeah, it's still in German only. Hm, I should fixthis some time, but time lacks... Any volunteers? (deadly silence iscracking.. ;)

Nevertheless, we start by now:In

        Code maturity level options
        

we activate

        [*] Prompt for development and/or incomplete code/drivers
        

and in

        Loadable module support
        
        [*] Enable loadable module support 
        [*] Set version information on all module symbols
        [*] Kernel module loader
        

Ok, so far so good. Now, we go to

        Networking options
        

and mark

        [*] Network packet filtering (replaces ipchains)
        [ ] Network packet filtering debugging
        

Note:

Previously, the above debuggingoption had been selected. For now, unless you want your/var/log/-partition being filled up inshort-time distance, deactivate this option.
If this options isactivated, messages similar to the following appear in counts ofthousands in dmesg and/var/log/{kern.log,debug,syslog,messages}:

skb: pf=2 (unowned) dev=br0 len=52
PROTO=6 156.136.32.121:3709 192.168.101.2:112 L=52 S=0x00 I=35470 F=0x4000 T=51 
nf_hook: hook 1 already set.
skb: pf=2 (unowned) dev=br0 len=52
PROTO=6 156.136.32.121:3709 192.168.101.2:112 L=52 S=0x00 I=35470 F=0x4000 T=51 
nf_hook: hook 0 already set.
skb: pf=2 (unowned) dev=br0 len=52
PROTO=6 192.168.101.11:2828 192.168.101.2:202 L=52 S=0x10 I=63 F=0x4000 T=64 
nf_hook: hook 1 already set.
skb: pf=2 (unowned) dev=br0 len=52
PROTO=6 192.168.101.11:2828 192.168.101.2:202 L=52 S=0x10 I=63 F=0x4000 T=64 
nf_hook: hook 3 already set.
skb: pf=7 (owned) dev=eth1 len=1500
        

Furthermore, in

          IP: Netfilter Configuration --->
        

we mark any item we need asmodule. Now the long awaited item: activate

        <M> 802.1d Ethernet Bridging
        

as well as

        [*] netfilter (firewalling) support
        

Note:

The above entry is availableonly if we successfully patched our kernel!

Finally, we just need asuccessful

root@bridge:~> make dep clean bzImage modules modules_install
        

cycle and we're done. Don'tforget to edit/etc/lilo.conf and do

root@bridge:~> lilo -t
root@bridge:~> lilo
root@bridge:~> reboot
        

, though.

Hint:

Perhaps we might mark our newkernel as the bridge kernel? Wevi thetoplevel Makefile in our kernel sources and edit the head linecalledEXTRAVERSION =. We may actuallyset it to, say bridge? ;-)
After the modules_installwe find the fresh modules in/lib/modules/2.4.18bridge
Fordebian users (eventually use exportPATCH_THE_KERNEL=YES before and --added_patches your_patcheswith make-kpkg):

root@bridge:~> make-kpkg --revision=tf.1.0 kernel_image
          

2.2Userspacetool: brctl

Once our kernel has thecapabilities needed to perform Ethernet Bridge and netfilter actions,we prepare the user space toolbrctl.brctl is the configuration tool we usetosetup anything to suit our needs.

We downloadthe source tarball, unpack it and change directory into it.

root@bridge:~> wget -c http://bridge.sourceforge.net/bridge-utils/bridge-utils-0.9.5.tar.gz
root@bridge:~> tar xvzf bridge-utils-0.9.5.tar.gz
root@bridge:~> cd bridge-utils-0.9.5
        

At this time, read the READMEand the files in the doc/ subdirectory.Then do a simple make and copy the resultingbrctl/brctlexecutable to /sbin/.

root@bridge:~> make
root@bridge:~> cp -vi brctl/brctl /sbin/
        

This is it. Go for Setupnow.

2.3Kernel-Notes

Symptom: Anything during setupworks but packets do no longer traverse as they did in 2.4 the bridgeinterfaces.
ipuk s (qasuari_ @ _yahoo.com) wrote (about june2005):

[...]
I have to compile my kernel from 2.4.18-14 to 2.6.0 and activate
bridge-netfilter&ebtables.
After compiling, i can't ping from a host to interface of linux box.
Linux box just have 1 interface.whats wrong with my compilation ???
[...]

3.SetLinux up to serve

3.1Settingup the bridge

We need Linux to know aboutthe bridge. First tell it that we want one virtual ethernet bridgeinterface: (this is to be executed on hostbridge,of course. See Testinggrounds)

root@bridge:~> brctl addbr br0
        

Second, we do not need the STP(Spanning Tree Protocol). I.e. we do only have one single router, soa loop is highly improbable. We may then deactivate this feature.(Results in less polluted networking environment, too):

root@bridge:~> brctl stp br0 off
        

After these preparations, we nowdo finally some effective commands. We add our two (or even more)physical ethernet interfaces. That means, we attach them to the justborn logical (virtual) bridge interfacebr0.

root@bridge:~> brctl addif br0 eth0
root@bridge:~> brctl addif br0 eth1
        

ImportantNote:

People sent me emails that itwould have helped them if I stressed more clearly the risk of beingcut off. So listen at this point to my warnings:
If you readthis, you are one (small) step before you _might_ cut yourself offyour box you are going to subverse to a bridging device.
If youlove living on bleeding edges, it is now the instant to prepare yourfirst aid material. You will likely need it.
If you do not havephysical access, nor does another person within your range:
DONOT PROCEED UNLESS YOUR FINGERS LEFT THE KEYBOARD IN FRONT OF YOUAND YOUR EYES FIXED REFLECTIVELY SOMETHING OTHER THAN YOURCONSOLE.
You have been warned, now. No responsability is assumedfor anything at all.

Now, our two previously physicalethernet interfaces became a logical bridge port each. Erm, ok, therewere and will be the physical devices. They are still there, go havea look ;-) But now they became part of the logical bridge device andtherefore need no IP configuration any longer. So release the IPs:

root@bridge:~> ifconfig eth0 down
root@bridge:~> ifconfig eth1 down
root@bridge:~> ifconfig eth0 0.0.0.0 up
root@bridge:~> ifconfig eth1 0.0.0.0 up
        

Great! We now have a box w/o anyIP attached. So if you were configuring your future fw/router via TP,go for your local console now ;-)) You have a serial console? Happyone :-)

Optional:

We tell Linux the new (logical)interface and associate one single IP with it:

root@bridge:~> ifconfig br0 10.0.3.129 up
        

And we're done.
Read theImportantNote!

3.2Settingup the routing

Incase we are configuring a gateway we enable the forwarding in thelinux kernel.

root@bridge:~> echo "1" > /proc/sys/net/ipv4/ip_forward
        

Our box already has an IPassigned but no default route. We solve this now:

root@bridge:~> route add default gw 10.0.3.129
        

Finally, we should have a workingnet from, to and through the gateway.

3.3Makeit happen again!

Aka: We need the changes topersist reboots.
To do so, you need some sh-style script and putthis in the appropriate system boot-up directory:/etc/init.d/
Secondly, you create thelink in your runlevel directory. The correct directory depends onyour gusto and of course on your linux distribution. Common runlevelvalues on workstations are2, 3and 5. Examples are: /etc/rc?.d/(replace the ? with the right runlevel)
Also, you need an idea aswhen your network interfaces are torn up. For now, we assume, yournetwork interfaces are activated at system prioritySso we need not to care of. If you ever should feel the need to knowexactly, look in/etc/rcS.d/. We justwant the bridge to be up and operable as soon as possible and sochose our priority to be10. (Make sure,no service requiring bridging devices is started before, read: withpriority-values less than10)
Fornow, we assume, your runlevel is 5:

root@bridge:~> mv -i bridge.sh /etc/init.d/
root@bridge:~> cd /etc/rc5.d/
root@bridge:~> ln -s ../init.d/bridge.sh S10bridge.sh
        

Virtually any distributionprovides you with some runlevel-checker or equivalent tool thatassists you in the tedious job of administering runlevel links.Consult your distro-documentation on this.
Hint: debian hasupdate-rc.d, redhat and successors have chkconfig. Finally, SuSEevidentally has also it's own tool, too (of which I don't recall thename easily..).
Wondering about the contents of bridge.sh? ;-)

#!/bin/bash
PATH="/sbin:/usr/sbin:/usr/local/sbin";
slaveIfs="1 2 3 4 6 7 8 9 10";
cmd="$1";
[ -z "$cmd" ] && cmd="start";
case "$cmd" in
  start)
    brctl addbr br0;
    brctl stp br0 on;
    brctl addif br0 eth0;
    brctl addif br0 eth1;
    (ifdown eth0 1>/dev/null 2>&1;);
    (ifdown eth1 1>/dev/null 2>&1;);
    ifconfig eth0 0.0.0.0 up;
    ifconfig eth1 0.0.0.0 up;
    ifconfig br0 10.0.3.129 broadcast 10.0.3.255 netmask 255.255.255.0 up ### Adapt to your needs.
    route add default gw 10.0.3.129; ### Adapt to your needs.
    for file in br0 eth0 eth1;
    do
      echo "1" > /proc/sys/net/ipv4/conf/${file}/proxy_arp;
      echo "1" > /proc/sys/net/ipv4/conf/${file}/forwarding;
    done;
    echo "1" > /proc/sys/net/ipv4/ip_forward;
    ;;
  stop)
    brctl delif br0 eth0;
    brctl delif br0 eth1;
    ifconfig br0 down;
    brctl delbr br0;
    #ifup eth0; ### Adapt to your needs.
    #ifup eth1; ### Adapt to your needs.
    ;;
  restart,reload)
    $0 stop;
    sleep 3;
    $0 start;
    ;;
esac;
        

And, yes, make it executable..

root@bridge:~> chmod 700 /etc/init.d/bridge.sh
        

After all, make sure your bridgesurvives unattended reboots. It's the same story as with backups: youshould test it before you need it.

4.Testyour new bridged environment!

4.1TestingGrounds

We imagine this scenario orsimilar:

                                                          /\
          Ethernet Ethernet ATM /-/ \
--------- --------- --------- /-/ |
| Box |----------|Bridge |----------|Router |-----| Inter- \
--------- --------- --------- \ net ---|
         ^ ^ ^ ^ \ /
         | | | | \---/
        eth0 eth0 eth1 if0 ^
         | | | | |
      10.0.3.2 none/10.0.3.1 195.137.15.7 anything else
                  \ /
                   \ /
   ^ \-br0-/
   | ^ ^
   | ^ | |
   | | | |
  own own foreign hostile
        

Our administrative power includesonly machines marked withown, theRouter is completely off-limits and so is the Internet, ofcourse.
That means, if we want to control the flying bits'n'byteson the ethernet wire we can chose to integrate a common firewall orfile in a bridge.
Drawback of the standard way is you have tochange the default gateway route on every and any single host in yournet. And this is really a heavy weighting drawback, nobody wants tochange more than 5 default routes on 5 different hosts more than onetime. Keep the time in mind, this will consume, also! Not to forget,this is a error-prone way to handle the more about security..
Theother way is clean, less time-consuming, more secure and lesserror-prone. More secure in that we won't have the need to assign anyIP address. No IP, no danger. So far the theory, we hope, our stacksare safe. (Although this hope should better not relied on..) Theoverall advantage is, this bridge-setup is completely transparent, noIP, MAC, .. changes at all.
So it's up to you to chose yourpreferred method. But we will handle just the fancy one here ;-)

4.2Pingit, Jim!

Wewill configure the Box' eth0 as usual. The bridge's interfaces areconfigured as described inSetup.
Ifwe are to use forwarding we might perhaps do this one: ;-)

root@bridge:~> echo "1" > /proc/sys/net/ipv4/ip_forward
        

Optionally, we set up a defaultroute:

root@bridge:~> route add default gw 10.0.3.129
        

Thenwe set up some iptables rules on hostbridge:

root@bridge:~> iptables -P FORWARD DROP
root@bridge:~> iptables -F FORWARD
root@bridge:~> iptables -I FORWARD -j ACCEPT
root@bridge:~> iptables -I FORWARD -j LOG
root@bridge:~> iptables -I FORWARD -j DROP
root@bridge:~> iptables -A FORWARD -j DROP
root@bridge:~> iptables -x -v --line-numbers -L FORWARD
        

The last line gives us thefollowing output:

Chain FORWARD (policy DROP 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 0 0 DROP all -- any any anywhere anywhere
2 0 0 LOG all -- any any anywhere anywhere LOG level warning
3 0 0 ACCEPT all -- any any anywhere anywhere
4 0 0 DROP all -- any any anywhere anywhere
        

The LOGtarget logs every packet viasyslogd.Beware, this is intended for testing purposes only, remove inproduction environment. Else you end up either with filled logs andharddisk partitions by you yourself or anyone else does this Denialof Service to you. You've been warned.
Test this ruleset now. Pingthe router interface's IP (195.137.15.7) on host box:

root@box:~> ping -c 3 195.137.15.7
PING router.provider.net (195.137.15.7) from 10.0.3.2 : 56(84) bytes of data.
--- router.provider.net ping statistics ---
3 packets transmitted, 0 received, 100% loss, time 2020ms
^C
root@box:~> 
        

By default, we DROPeverything. No response, no logged packet. This netfilter setup isdesigned toDROP all packets unless wedelete the rule that drops every packet (rule no. 1 above) before theLOG target matches:

root@bridge:~> iptables -D FORWARD 1
root@bridge:~> iptables -x -v --line-numbers -L FORWARD
        

Now, the rules are:

Chain FORWARD (policy DROP 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
2 0 0 LOG all -- any any anywhere anywhere LOG level warning
3 0 0 ACCEPT all -- any any anywhere anywhere
4 0 0 DROP all -- any any anywhere anywhere
        

And any packet may pass through.Test it with a ping on hostbox:

root@box:~> ping -c 3 195.137.15.7
PING router.provider.net (195.137.15.7) from 10.0.3.2 : 56(84) bytes of data.
64 bytes from router.provider.net (195.137.15.7): icmp_seq=1 ttl=255 time=0.103 ms
64 bytes from router.provider.net (195.137.15.7): icmp_seq=2 ttl=255 time=0.082 ms
64 bytes from router.provider.net (195.137.15.7): icmp_seq=3 ttl=255 time=0.083 ms

--- router.provider.net ping statistics ---
3 packets transmitted, 3 received, 0% loss, time 2002ms
rtt min/avg/max/mdev = 0.082/0.089/0.103/0.012 ms
root@box:~> 
        

Yippeah! The router is alive, upand running. (Well it has been all day long.. ;-)

Important Note:

Whenwe just fired up the bridge interface it takes about roughly 30seconds until the bridge is fully operational. This is due the30-seconds-learning phase of the bridge interface. During thisphase, the bridge ports are learning what MAC addresses exist onwhat port. The bridge author, Lennert, tells us in his TODO file,the 30-seconds-learning phase is subjected to some improvement in atimely manner some time.
During the test phase, no packet will weforwarded. No ping be answered. Remind this!

4.3Actualconfiguration

Thissection is intended to give you, dear reader, some hints about howyour system should look and feel after having processed this howtosuccessfully.

Interfaceconfiguration

The output of your ifconfigcommand might look similar to this:

root@bridge:~> ifconfig
br0 Link encap:Ethernet HWaddr 00:04:75:81:D2:1D
          inet addr:10.0.3.129 Bcast:195.30.198.255 Mask:255.255.255.128
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:826 errors:0 dropped:0 overruns:0 frame:0
          TX packets:737 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:161180 (157.4 Kb) TX bytes:66708 (65.1 Kb)

eth0 Link encap:Ethernet HWaddr 00:04:75:81:ED:B7
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:5729 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3115 errors:0 dropped:0 overruns:0 carrier:656
          collisions:0 txqueuelen:100
          RX bytes:1922290 (1.8 Mb) TX bytes:298837 (291.8 Kb)
          Interrupt:11 Base address:0xe400

eth1 Link encap:Ethernet HWaddr 00:04:75:81:D2:1D
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:1 frame:0
          TX packets:243 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:342 (342.0 b) TX bytes:48379 (47.2 Kb)
          Interrupt:7 Base address:0xe800

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:1034 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1034 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:82068 (80.1 Kb) TX bytes:82068 (80.1 Kb)
        

Routing configuration

The output of your routecommand might look similar to this:

root@bridge:~> route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.0.3.129 0.0.0.0 255.255.255.128 U 0 0 0 br0
0.0.0.0 10.0.3.129 0.0.0.0 UG 0 0 0 br0
root@bridge:~>
        

Iptables configuration

Please have a look at the Pingit, Jim! section.

4.4Finalnote (Important!)

I'd like to hear from you!:-)
Did you enjoy the trip?
Do you miss anything?
Needhelp? (Call you local assistant ;-) or rtfm.
You are still online?Then drop me a msg via email. I'd be really glad.
Wanna send me acheque? Pitty, Don't accept these.. (Just kidding;)
Make it worthmy time, just send me some nice words, that's enough.
Nothingmotivates more than happy participants giving you valuablefeedback.
So, go on, invest a minute and hack me a mail!
Thankyou!

Nils
        

4.5Bug-Notes

Apparently,there must have been a bug in the br-nf code:

From: Bart De Schuymer <bart.de.schuymer_@_pandora.be>
Date: Sun, 1 Sep 2002 21:52:46 +0200
To: Nils Radtke <Nils.Radtke_@_Think-Future.de>
Subject: Re: Ethernet-Brigde-netfilter-HOWTO

Hello Nils,

[...]
Also, network packet filtering debugging is generally a bad idea with the
br-nf patch. It can gives a lot of false warnings (about bugs) in the logs.
[...]
        

Personally, I never had falsepositives in my log. Maybe, that bug has been fixed. This mailed toBart, he wrote:

From: Bart De Schuymer <bart.de.schuymer_@_pandora.be>
Date: Mon, 2 Sep 2002 18:30:25 +0200
To: Nils Radtke <Nils.Radtke_@_Think-Future.de>
Subject: Re: Ethernet-Brigde-netfilter-HOWTO

On Monday 02 September 2002 00:39, Nils Radtke wrote:
> Will the revision of the nf-debug code in br-nf be subject of improvement?

I must admit I haven't been running any kernel with netfilter debugging
lately. It sure used to give false positives a few months ago (the bridge
mailing list has posts about that), I've been lacking time to see why and if
it is still the case. It's on my todo list.
[...]
        

But (as of writing this2002-09-19) I haven't found an official announcement, this particularbug has been closed. So have a constant look at this topic on theethernetbridge mailinglist , if you are interested in it's cure.

5.Userexperiences

5.1FedoraCore 3

James Dinkel (jdinkel_ @_gmail.com) wrote on Tue, 8 Mar 2005 10:59:22 -0600:

[...]
I am using Fedora Core 3 and all I had to do was "yum install bridge-utils" 
to use the brctl command. I didn't have to do any kernel recompiling or 
configurations or messing with kernel modules. 
It was very easy.
[...]

6.Links

The Howto's author may becontacted viae-mail.
HowtoAuthor's homepage.

6.1Ethernet-Bridge

• EthernetBridge Mailinglist

• Userspace utilities, patches, etc.:Homeof Linux kernel Ethernet Bridge

• Bridge-STP-HOWTO

• Firewallingfor Free, Shawn Grimes

6.2RelatedTopics

• Filtering on frame level,Ethernet-Bridging-Tables:
  ebtables,sourceforge
ebtables,supported features
ebtables,examples: basic,advanced

• IP mode, Linux Bridgeextension:IPmode, LVS

• Linux inHigh-Availability environments:High-AvailabilityLinux

• Linux Virtual Server: LVS