Bonding Active-Active (High Availability)

[mziad.h]

I have 2 debian firewalls in my network. I have also 2 switches Nortel 8306.
For each firewall there are 4 interfaces: eth0, eth1, eth2 and eth3. I would like to bond interfaces eth0 and eth2 as bond0 and eth1 and eth3 as bond1.
The architecture is currently based on Active-Failover concept.
I would like to implement the High Availability structure. In order to achieve this project, i know that the ports on the switches should be converted from NORMAL mode to SMLT.

I have already done it on the 2 firewalls. On 1 firewall, the operation has passed successfully, but on the 2nd firewall, i have got problems. Can anyone try to explain me where is my fault?

Here is a little schema of my network:

newbielink:http://img291.imageshack.us/img291/5594/smlt.png [nonactive]

I am using the JDM software to access my nortel switches @work.
When I try to convert port (1-8) from NORMAL mode to SMLT on switch-core 1, the port (2-8) changes color immediately to yellow on the same switch-core 1.
I log on switch-core-2 and convert the port (1-8) from NORMAL mode to SMLT, I dont find any changes on port 2-8 on Switch-Core 2.
When I relog on switch-core 1, I tried to convert port 2-8 from NORMAL mode to SMLT mode, and it worked. However, I lost immediate connection to my switch-core 2. The switch-core 2 was down, and I lost all log files on there.
I was unable to complete the operation. The same procedure, I did it with my 1st firewall, and everything passed on well.

Before I converted a port from mode normal to mode SMLT, I was given a prompt: DO U WANT TO DISABLE STP?. Can anyone explain me the message?

Can you explain me in the best way how to ensure that the network dont go down and that everything pass on smoothly?

Thanks

[normski]

Hi

For SMLT to function, you will need an inter switch trunk (IST) between the 2 core switches, the 8300s. Which is probably why you're getting the spanning tree message and the core switch dropping off the network. You probably could get something to work without using smlt/ist but it would be using spanning tree, have traffic passing through only one core switch and would have slow failover times.

And their is no mention of layer 3 stuff, have you thought about that?

 I will bow to others greater knowledge.

Normski

[nightwatch]

# both
config lacp enable

#switch 1
mlt 10 create
mlt 10 add ports 2/47-2/48 #IST Ports
mlt 10 perform-tagging enable
mlt 10 ist create ip 1.1.1.2 vlan-id 500
mlt 10 ist enable
vlan 500  create byport
vlan 500  add-mlt 10
vlan 500  ports add 2/47-2/48 member portmember
vlan 500  ip create 1.1.1.1/255.255.255.252

#switch 2
mlt 10 create
mlt 10 add ports 2/47-2/48 #IST Ports
mlt 10 perform-tagging enable
mlt 10 ist create ip 1.1.1.3 vlan-id 500
mlt 10 ist enable
vlan 500  create byport
vlan 500  add-mlt 10
vlan 500  ports add 2/47-2/48 member portmember
vlan 500  ip create 1.1.1.2/255.255.255.252


#now you can create smlt ports
# in each switch perform
ethernet 2/1 smlt 1 create

i didn't test this config so, i might have missed something. use at your own risk


### linux ###
use mode 0 for your bonds

edit: corrected the vlan 500 ip for each switch (Thanks Michael)

[mziad.h]

Hi

For SMLT to function, you will need an inter switch trunk (IST) between the 2 core switches, the 8300s. Which is probably why you're getting the spanning tree message and the core switch dropping off the network. You probably could get something to work without using smlt/ist but it would be using spanning tree, have traffic passing through only one core switch and would have slow failover times.

And their is no mention of layer 3 stuff, have you thought about that?

 I will bow to others greater knowledge.

Normski

First of all i want to thank you for the quick reply Sir.
My switches are already configured IST between them. The spanning tree message occurs when i try to convert any port (in normal mode) to SMLT. The 1st time i did it, my firewall functionned properly in a round-robin way (mode 0) [ I was able to verify it by using netgraph utility to monitor my firewall interfaces, eth0 and eth2 had the same amount of flux passing in real time ]
I totally agree with you regarding the fact that if only 1 switch is active, the network becomes very very slow.
I have another similar type of intervention to do in next month, and i want to do it smoothly without disrupting the network.
I would like to know also the reason why the switch core 2 went down while i was converting the desired ports to mode SMLT?

Regards
mzh

[Michael McNamara]

# both
vlan 500  create byport
vlan 500  add-mlt 10
vlan 500  ports add 2/47-2/48 member portmember
vlan 500  ip create 1.1.1.1/255.255.255.252

I don't want to knock your efforts, I was far better than I could have done without a switch in front of me but the piece I quoted above is incorrect. You need to assign a unique IP address to each ERS 8600 so the IST will form.

+1

[Michael McNamara]

You'll need to help diagnose what's going on... without looking to hard or asking for a lot of information you are probably throwing a loop into your network. You may have the firewall ports configured improperly. Remember the ports in question need to be configured as a mulit-port trunk (LACP?) on the firewall. I'm assuming the firewalls are Layer 3 and not Layer 2 right? If they are Layer 2 then you need STP/RSTP/MSTP to block the redundant path since the firewalls don't incorporate Nortel/Avaya's IST/SMLT technology.

You realize that when you change the MLT type to SMLT you need to assign a unique SMLT ID right?

Good Luck!

[mziad.h]

You'll need to help diagnose what's going on... without looking to hard or asking for a lot of information you are probably throwing a loop into your network. You may have the firewall ports configured improperly. Remember the ports in question need to be configured as a mulit-port trunk (LACP?) on the firewall. I'm assuming the firewalls are Layer 3 and not Layer 2 right? If they are Layer 2 then you need STP/RSTP/MSTP to block the redundant path since the firewalls don't incorporate Nortel/Avaya's IST/SMLT technology.

You realize that when you change the MLT type to SMLT you need to assign a unique SMLT ID right?

Good Luck!

Hello man,
The information u gave me is very precise. I guess a loop is being formed in my switches. But on the intervention made on my 1st firewall, everything went perfect. It was on the 2nd firewall that i got problems. Here is a mini description of my smlt id configured (knowing that they should be unique:)

Firewall         Switch-Core     Port Number     SMLTID
eth0              Switch Core 1          1-8                108
eth2              Switch Core 2          1-8                108

eth1              Switch Core 1          2-8                208
eth3              Switch Core 2          2-8                208

Kindly note that eth0 and eth2 are forming bond0 and eth1 and eth3 are forming bond1
IST is already configured and the current network is functionning properly.
The goal of the project is to implement high availabily to improve the performance of the network.

I am currently using JDM software to manipulate my nortel switches
Upon converting port 1-8 to SMLT mode on switch core 1, port 2-8 automatically changes from green to yellow. Is it normal?
I then logged on switch-core-2 and converted the port 1-2 to mode SMLT, it worked. I relogged on switch core 1 and changed the port 2-8 (which was yellow) to SMLT. This action unfortunately made switch core 2 shut down. This is the part which i am not understanding. Everything went fine the 1st time. The 2nd time i was puzzled that i got problems.

Here r my log files:

Switch Core 1:

HW INFO portLinkDownEvent starting 06/10/10 05:47:28 on ports 2/8<000>

HW INFO portLinkDownEvent starting 06/10/10 05:47:29 on ports 1/8<000>

HW INFO portLinkUpEvent starting 06/10/10 05:47:31 on ports 1/8<000>

SNMP INFO Spanning Tree Topology Change(StgId=1, PortNum=1/8, MacAddr=00:1a:8f:2b:10:01)<000>

MLT INFO SMLT 108 DOWN<000>

MLT INFO SMLT 108 UP<000>

SNMP INFO Smlt Link Down Trap(SmltId=108)<000>

SW INFO Save config to file /flash/config.cfg successful.<000>

SNMP INFO Save config file /flash/config.cfg  successful.<000>

MLT INFO SMLT 108 UP<000>

SNMP INFO Smlt Link Up Trap(SmltId=108)<000>

SW INFO Save config to file /flash/config.cfg successful.<000>

SNMP INFO Save config file /flash/config.cfg  successful.<000>

MLT INFO smltIstSessionDown<000>

SNMP INFO Smlt Link Down Trap(SmltId=20)<000>

SNMP INFO Smlt Link Down Trap(SmltId=14)<000>

SNMP INFO Smlt Link Down Trap(SmltId=15)<000>

SNMP INFO Smlt Link Down Trap(SmltId=1)<000>

SNMP INFO Smlt Link Down Trap(SmltId=208)<000>

SNMP INFO Smlt Link Down Trap(SmltId=209)<000>

SNMP INFO Smlt Link Down Trap(SmltId=4)<000>

SNMP INFO Smlt Link Down Trap(SmltId=24)<000>

SNMP INFO Smlt Link Down Trap(SmltId=218)<000>

SNMP INFO Smlt Link Down Trap(SmltId=33)<000>

MLT INFO All the SMLTs are down<000>

SNMP INFO Smlt Link Down Trap(SmltId=505)<000>

SNMP INFO Smlt Link Down Trap(SmltId=506)<000>

SSH INFO New connection from ip 172.30.50.2 port 57735<000>

SSH INFO New connection from ip 172.30.50.2 port 57737<000>

SSH ERROR key exchange failed<000>

SNMP INFO Smlt Link Down Trap(SmltId=108)<000>

SW INFO user admin connected via console port<000>

HW INFO portLinkDownEvent starting 06/10/10 07:42:06 on ports 2/23-2/24<000>

HW INFO portLinkDownEvent starting 06/10/10 07:42:07 on ports 1/23-1/24<000>

HW INFO portLinkUpEvent starting 06/10/10 07:43:26 on ports 2/23<000>

HW INFO portLinkUpEvent starting 06/10/10 07:43:26 on ports 1/23,2/24<000>

HW INFO portLinkUpEvent starting 06/10/10 07:43:28 on ports 1/24<000>

HW INFO portLinkDownEvent starting 06/10/10 07:43:35 on ports 2/23-2/24<000>

HW INFO portLinkDownEvent starting 06/10/10 07:43:36 on ports 1/23-1/24<000>

HW INFO portLinkUpEvent starting 06/10/10 07:43:37 on ports 2/23<000>

HW INFO portLinkUpEvent starting 06/10/10 07:43:37 on ports 1/23,2/24<000>

HW INFO portLinkUpEvent starting 06/10/10 07:43:39 on ports 1/24<000>

MLT INFO smltIstSessionUp<000>

SNMP INFO Smlt Ist Link Up trap received<000>

SW INFO user admin logged out from console port<000>

Here are the log files for Switch Core 2. Since the switch core 2 went down, i lost most of the log files

NTP INFO Synchronization with the server failed<000>

NTP INFO Synchronization with the server failed<000>

MLT INFO SMLT 108 DOWN<000>

MLT INFO SMLT 108 UP<000>

SNMP INFO Smlt Link Down Trap(SmltId=108)<000>

MLT INFO SMLT 108 UP<000>

SNMP INFO Smlt Link Up Trap(SmltId=108)<000>

SW INFO Save config to file /flash/config.cfg successful.<000>

SNMP INFO Save config file /flash/config.cfg  successful.<000>

I hope u can gt a better overview of my problem and that u can try to help me. I have a similar intervention during september and i cant afford to down my network this time.

Thanks again. U guys are real hardworkers

With a lot of respect

mzh

[Michael McNamara]

A quick response... you realize that you can't run STP on an SMLT port right?

SNMP INFO Spanning Tree Topology Change(StgId=1, PortNum=1/8, MacAddr=00:1a:8f:2b:10:01)<000>

You need to disable STP on all ports that are going to be connected to the firewalls for the SMLT link to form.

Cheers!

[nightwatch]

A quick response... you realize that you can't run STP on an SMLT port right?

SNMP INFO Spanning Tree Topology Change(StgId=1, PortNum=1/8, MacAddr=00:1a:8f:2b:10:01)<000>

You need to disable STP on all ports that are going to be connected to the firewalls for the SMLT link to form.

Cheers!

i think that stp is disabled when the port is converted to SMLT, from the command line it is at least.

#on both
ethernet 1/8 stg 1 stp disable
ethernet 1/8 smlt 108 create

by the way, i edited the previous configuration i've made since it it was incorrect as Michael mentioned

[mziad.h]

A quick response... you realize that you can't run STP on an SMLT port right?

SNMP INFO Spanning Tree Topology Change(StgId=1, PortNum=1/8, MacAddr=00:1a:8f:2b:10:01)<000>

You need to disable STP on all ports that are going to be connected to the firewalls for the SMLT link to form.

Cheers!

Yes man, upon converting a port to mode smlt, there's a prompt telling me that the action performed will disable STP on that port.
I have sent you a pm regarding my procedure. Can you tell me ur way to ensure that the conversion to smlt will work?
Kindly note that the configurations on my firewalls are already set to mode 0.

Regards
mzh