MLT to LACP on Cisco

[Matt]

I have managed to link our 2 core 8010 switches to Cisco 6509 and 4506 switches using LACP with an MLT at the Nortel end and a LACP port channel on the Cisco end and we have 1 VLAN traversing the link, which has allowed me to to dispense with the previous layer 3 link between the devices.

The link is such that 1 8010 links to the 6509 and 1 8010 to the 4506
MLT id is 30
Port channel is 30
VLAN traversing is 1150

It all works fine

What I now wish to do is to add a VLAN 'hosted' on the 8010 to the link so that i can present that VLAN on the Cisco side and have a device plugged into the Cisco switch using that VLAN as an access port

the new VLAN id is 1100

Adding to the Cisco side is simple create the VLAN and do an 'allowed vlan add 1100' to the port channel.

However on the Nortel side, I cannot seem to find a way of simply associating the VLAN with the MLT (MLT ID 10) created to link to the port channel (ID 10). I have tried config vlan 1100 add-mlt 30 but that is rejected with a mesage saying the MLT is used for link aggregation.

can anyone shed light on what i am obviosuly missing here please?

[Michael McNamara]

Initially reading your message I was going to ask the following:

On the Nortel switches is the MLT (or switch ports themselves) configured as "trunks"? If you issue the "show mlt info" command is the PORT TYPE for that MLT configured as 'Access' or 'Trunk'? I'm going to guess that it's configured as 'Access' and you'll need to change it to 'Trunk' before you can add multiple VLANS to either the ports or the MLT.

However, upon re-reading your post I now know what your issue is. What software release are you running?

LACP configurations are not dynamic in the sense that you can't change them while they are enabled. You must disable LACP, make your VLAN changes and then re-enable LACP. It's a well known pain for those of use that need LACP/MLT functionality.

Cheers!

[Matt]

Thanks for the reply which ismuch appreciated and below is the output of the command you supplied.

The version of Nortel software is 4.1.4 and yes I know its old but it is the lowest supported version and i had a hell of job persuading people here to upgrade to that.

The changes I made I did so after config lacp disable had been executed and I have found that also doing a config ethernet 1/16 lacp disable is also sometimes necessary as well. But in this case that made no difference as you can see below

OS_2:5# config lacp disable
HOS_2:5# config ethernet 1/3,2/3,1/16,2/16 lacp disable
HOS_2:5# config vlan 1100 add-mlt 30

Error: This MLT is used for link aggregation

HOS_2:5# config ethernet 1/3,2/3,1/16,2/16 lacp enable
HOS_2:5# config lacp enable
HOS_2:5#

================================================================================
                         Mlt Info
================================================================================
                        PORT    SVLAN  MLT   MLT        PORT         VLAN
MLTID IFINDEX NAME      TYPE    TYPE  ADMIN CURRENT    MEMBERS      IDS
--------------------------------------------------------------------------------
1   4096  HOS Core MLT trunk   normal norm   norm     1/1,2/1           14 104 1                                                                             07 108 109 110 111 112
30  4105  L2 ESM-SW02  trunk   normal norm   norm     1/16              1150

               MULTICAST             DESIGNATED   LACP      LACP
MLTID IFINDEX  DISTRIBUTION  NT-STG  PORTS        ADMIN     OPER
--------------------------------------------------------------------------------
1      4096     disable      enable   1/1        disable   down
30     4105     disable      disable  1/16       enable      up

[Michael McNamara]

You had the right idea but the wrong commands....

disable LACP on the Ethernet port
add the VLAN to the Ethernet port
enable LACP on the Ethernet port

I just tried this in JDM and it worked fine.

Let me know how you make out.

Cheers!

[Matt]

Ah ok thanks Michael.

So if I have this right the second step is to config vlan 1100 ports add 1/16 etc?

But this vlan is already associated with a port that links to my server stack of 450-24T switches, so 1100 is associated with port 1/15. Will this change mean that 1100 in JDM show that it is on ports 1/15, 1/16 etc and the first VLAN 1150 will only show as being on the original 4 ports? The reason I ask is that when I tried last night to do this in JDM I would select 1/15 under VLAN 1150 and it would drop from VLAN 1100. Maybe I should stick with CLI

BTW something extra VLAN 1100 whilst being a VRRP VLAN also has an IPX network defined on it. I found today that while IP works on the VRRP as master/backup, IPX does not and in effects load balances itself over the two physical interfaces on each core switch. Because I had left default vlan tagging in place on the backup VRRP device IPX traffic was intermittently dropping today until i took the VLAN tag off. I also found out the stack using VLAN 1100 on the core ports actually has all its ports in VLAN 1. (OK I hear the howls of pain and anguish, but I am only 2 weeks into this job and trying to pick my way through the inheritance plus I have not used Nortel kit for many years).

Anyway back to the main problem, which is that eventually I will want to have several VLAN that are defined on the Nortel kit using VRRP supporting IP and IPX carried across the LACP link to Cisco so that I can plug new servers into the many free ports on the Cisco side and they will see their default gateway even though it is on the 8010. These VLAN are used on the 8010 to associate physical ports either with server stacks or directly conencted servers. In the case of the directly conencted servers (using fibre) the intention is to make the VLAN on the Nortel side available on the trunk links that the servers (ESX) use on the Cisco side so that they will only have physical links to Cisco, leaving Nortel will all the stacks.

Apparently there is a good reason for this.

Thank you again for your help

Regards

[Michael McNamara]

Yes, you are correct in your logic... just disable LACP on the Ethernet port (the Ethernet port is still a member of the MLT), add the Ethernet port to the VLAN in question and then enable LACP on the Ethernet port.

As long as the Ethernet port is configured as a trunk you should be able to add as many VLANs as you like. If the Ethernet port is configured as an access then you will only be allowed to have a single VLAN on that port and if you add a second VLAN it will replace the original VLAN.

Good Luck!

[Matt]

Right so my config is as follows

ethernet 1/3 perform-tagging enable
ethernet 1/16 perform-tagging enable
ethernet 2/3 perform-tagging enable
ethernet 2/16 perform-tagging enable

mlt 30 create
mlt 30 name "L2 ESM-SW02"
mlt 30 ntstg disable
mlt 30 lacp enable
mlt 30 lacp key 30

vlan 1150 create byport 1 name "CNL2-Management"
vlan 1150 ports remove 1/1-1/2,1/4-1/15,2/1-2/2,2/4-2/15 member portmember
vlan 1150 ports add 1/3,1/16,2/3,2/16 member portmember
vlan 1150 ip create 10.155.36.10/255.255.255.248 mac_offset 22
vlan 1150 ip vrrp 31 address 10.155.36.11
vlan 1150 ip vrrp 31 enable
vlan 1150 ipx create 0x190032 raw mac_offset 22 tick 1

vlan 1100 create byport 1 name "Hos_CR_1"
vlan 1100 ports remove 1/1-1/14,1/16,2/1-2/16 member portmember
vlan 1100 ports add 1/15 member portmember
vlan 1100 ip create 10.170.50.252/255.255.255.0 mac_offset 6
vlan 1100 ip dhcp-relay enable
vlan 1100 ip ospf enable
vlan 1100 ip vrrp 106 address 10.170.50.254
vlan 1100 ip vrrp 106 enable
vlan 1100 ipx create 0x170050 raw mac_offset 6 tick 1

The following section is replicated in config.cfg for the other 3 ports in the LACP link, with the obviosu difference of the partner-port key value & description

ethernet 1/3 default-vlan-id 310
ethernet 1/3 auto-negotiate disable
ethernet 1/3 name "LACP-30 to ESM-4506-01 2/2"
ethernet 1/3 stg 1 stp disable
ethernet 1/3 lacp key 10
ethernet 1/3 lacp aggregation true
ethernet 1/3 lacp partner-key 10
ethernet 1/3 lacp partner-port 6
ethernet 1/3 lacp partner-port-priority 32768
ethernet 1/3 lacp partner-system-id 0:12:1:14:26:80
ethernet 1/3 lacp enable

ethernet 1/15 auto-negotiate disable
ethernet 1/15 name "HOS Computer room Stack"
ethernet 1/15 stg 1 stp disable

And to make the change I should do

config lacp disable
config ethernet 1/3,2/3,1/16,2/16 lacp disable

config vlan 1100 ports add 1/3,1/16,2/3,2/16 member portmember

or

config vlan 1100 ports add 1/15,1/3,1/16,2/3,2/16 member portmember to keep the original port in its original VLAN. I take it that the 4 ports 1/3 etc will also remain in VLAN 1150

config ethernet 1/3,2/3,1/16,2/16 lacp eable
config lacp enable

[Michael McNamara]

Here are the commands that I believe you'll need to issue. I'm assuming that the ports in the LACP bundle are 1/3,1/15,1/16,2/3,2/16. Also you need to be careful on where you issue these commands. If you are connecting to the switch over the LACP/LAG you'll end up cutting yourself off. You also might want to disable all but one of the ports to prevent any loops from forming while the LACP is disabled.

config ethernet 1/3,1/15,1/16,2/3 state disable

config ethernet 1/3,1/15,1/16,2/3,2/16 lacp disable
config van 110 ports add 1/3,1/15,1/16,2/3,2/16
config ethernet 1/3,1/15,1/16,2/3,2/16 lacp enable

config ethernet 1/3,1/15,1/16,2/3 state enable

There's no need to disable LACP globally (in case you have other LACP groups that are active).

Cheers!

[Matt]

Thanks for the comments and advice

The ports in the bundle are 1/3, 1/16, 2/3 & 2/16 The port 1/15 is the port associated with VLAN 1100 which I want to allow across the LACP link so other ports on switches at the far end of the LACP link can use that VLAN.

The 4 ports are tag enabled and in VLAN 1150

1/15 is at present in VLAN 1100 and is not tag enabled.

I am assuming that I do not need LACP on 1/15 - assumption based on the way this works on Cisco and othr vendors.

When its finished and working I will send the whole config both sides with a diagram (might be nice to actually have some network documentation here anyway)

Cheers

[Matt]

Ok well there is still a problem with adding the new VLAN to the link

I tried JDM, but what ever I did there I could only have the ports currently in the LACP link in 1 VLAN at a time.

The commands below are those executed at CLI.
config ethernet 1/3,2/16,1/16,2/3 state disable - these 4 ports make up the current MLT/LACP link & exist in VLAN 1150
config ethernet 1/3,2/16,1/16,2/3 lacp disable
config vlan 1100 ports add 1/3,2/16,1/16,2/3
config ethernet 1/3,2/16,1/16,2/3 lacp enable
config ethernet 1/3,2/16,1/16,2/3 state enable

I am clearly missing something here. 

Just to recap the 4 ports in the MLT are in VLAN 1150 and this VLAN traverse the LACP link to the Cisco kit. I know this works because I have static routes defined on either side of the link using the VRRP or HSRP address as the next hop gateway for the static routes. e.g. static routes on the Cisco side have 10.155.36.11 as the next hop address, which is the VRRP address on VLAN 1150, as you can see from the config posted earlier.

So I need the 4 ports to stay in the VLAN 1150, but I also need the 4 ports to be in VLAN 1100 as well, so that traffic
from that VLAN 1100 can cross the LACP link and I can then assign 1100 as a switchport access vlan on the Cisco side, whilst at the same time keeping VLAN 1150 active on the same link.(Apologies if it appears I am talking down to readers, that is hoestly not the intention)

Do I need to tag enable the physical ports for VLAN 1150, as well as those for VLAN 1100 (which are the 4 ports making up the LACP link)?

Do I need to enable LACP on the physical ports in VLAN 1100 as well as those for VLAN 1150?

As ever all help, sugegstions, comemnts and advise appreciated

[Michael McNamara]

Where do we start... this is certainly possible... this certainly works... let see why not.

Did you get any errors from the switch when you issued the commands detailed above?

You already set the ports to trunk correct? (config ethernet 1/3,2/16,1/16,2/3 perform-tagging enable)

You can set tagging/trunking on ports not VLANs. You can set LACP on ports not VLANs.

A word of caution I would not disable ALL your links as you detailed above, that will disconnect you from the other network. I would disable all but 1 of the links so you avoid any Spanning Tree issues. Speaking of Spanning Tree do you have Spanning Tree disabled on each port? Are you trying to run Spanning Tree between 6500 and 8600?

You can use JDM if you wish but you must perform the 'same' commands only through JDM. At a minimum you need to disable LACP on the ports, add the VLANS in question to the ports, enable LACP on the ports. This assumes that you've already enabled tagging/trunking (perform-tagging enable) on the ports in question. Do you have tagging enabled on the ports?

Let me summarize some questions;
 1) did you get any errors when issuing those CLI commands?
 2) do you have tagging (perform-tagging enable) enabled on the links?

Would you be able to attach the configuration (config.cfg) of the 8600 to this thread?

Cheers!

[Matt]

One note it is over 10 years since I had to work on Nortel kit, hence I am a little rusty, so please bear with me.

Now in answer to the questions raised

1. No error messages received

2. From the config.cfg (I am not allowed to supply the whole thing by order of IT Security team)
ethernet 1/3 perform-tagging enable
ethernet 1/16 perform-tagging enable
ethernet 2/3 perform-tagging enable
ethernet 2/16 perform-tagging enable

3. Spanning Tree is disabled on the Nortel equipment, the 4 ports noted above are in an STG, called stg 1 and then on each port the command config ethernet n/n stg 1 stp disable is set.  There are no issues with STP on either side of the link, no loops etc. I know this by looking at the logs on the Nortel & Cisco kit and from the monitoring tools and also the netwok is up and running.

So at this stage I have 4 ports configured almost identically - the only differences are port name and the value for lacp partner-port.

In the post of June 18th I placed the relevant sections of the config.cfg. ethernet 1/3 configuration is there and that configuration is repated on ports 2/3, 1/16 & 2/16, with the two exceptions noted above.

Now those ports do work when LACP is enabled on the ports and globally within the configuration.

I know they work because I am running VLAN 1150 with VRRP. On the Cisco side there are some static routes that have the VRRP address as the next hop and those routes work. (If they didn't I would have over 2,000 users screaming at me)

Also in the post of June 18th is the configuration of port 1/15. that port is in VAN 1100 which I now want to add to the link, so that I can use that VLAN on the Cisco side e.g. switchport access vlan 1100. I know this will work when I can put a laptop into a port so configured and I will get a DHCP address allocated for VLAN 1100. (DHCP is enabled on the VLAN VRRP instance).

Port 1/15 is linked to a switch stack (Nortel 47048T) so I obviosuly do not want that physical port to form part of the LACP link, just allow traffic from elsewhere in the network to reach that port.

I will do the work from the CLI - call me old fashioned but I prefer that

So hopefully I have answered the questions and can now pose some of my own.

1. Given what I want to do with VLAN 1100 do I need to tag enable port 1/15? (I think I probably do in order to distinguish traffic traversing the LACP link when that link carries multiple VLAN.)

2. How do I configure the 8600 switch such that, what I think needs to happen is that the 4 ports that form the LACP link exist in two VLAN at the same time, the original VLAN 1150 and the new VLAN 1100. I would have though and indeed expected the comamnds in the post of une 18th from Michael to work. They do work but take take the 4 LACP ports out of VLAN 1150, which causes the static routes to fail.

What I think the configuration should look like in JDM VLAN's is that for VLAN 1150 there will be 4 active port members 1/3,1/16,2/3,2/16 and for VLAN 1150 there will be 5 active members: 1/3,1/15,1/16,2/32/16, then all should be lovely

One final complication, which I already probably have the answer to, port 1/15 on the 8600 is in VLAN 1150, but at present the port is not tag enabled. raffic going via 1/15 to the attached stack works even though all ports on the stack are in VLAN 1. Now this is something I have inherited and do not like as a configuration practice. I understand that if I have to tag enabled port 1/15 as part of the process to get VLAN 1150 traversing the LACP link I will need to change all ports on the stack to be in VLAN 1150 as well. But should I do that anyway in order to achieve my final aim?

As ever thank you for your time and trouble, advise and comments.

[Michael McNamara]

I have an idea...

Let me answer your questions first...

1. Given what I want to do with VLAN 1100 do I need to tag enable port 1/15? (I think I probably do in order to distinguish traffic traversing the LACP link when that link carries multiple VLAN.)

You will need to bridge VLAN 1100 out to all the switches that you wish to participate in VLAN 1100. As you commented the port 1/15 doesn't need to be a member of the LAG but it needs to be configured as a trunk (perform-tagging enable) so you can add all the necessary VLANs to that downlink.

2. How do I configure the 8600 switch such that, what I think needs to happen is that the 4 ports that form the LACP link exist in two VLAN at the same time, the original VLAN 1150 and the new VLAN 1100. I would have though and indeed expected the comamnds in the post of une 18th from Michael to work. They do work but take take the 4 LACP ports out of VLAN 1150, which causes the static routes to fail.

So VLAN 1150 get's replaced with VLAN 1100 instead of having both VLANs in the MLT/LACP/LAG.

Looking back at the MLT configuration you posted above, I believe your LACP key need to match up with your MLT ID.

I'm not sure if you've reviewed this technical configuration guide, but it does a good job of explaining the basics.

http://www.michaelfmcnamara.com/files/2008_09_24_LACP_802_3ad_and_VLACP_for_ES_and_ERS_TCG_J_Vant_Erve_NN48500502.pdf

Keep us informed.

Cheers!

[Matt]

Thanks for the update.

I have corrected the LACP key value and to make life simpler I have changed the Cisco side so that I am using MLT 30 with a LACP key of 30 and the partner key is 30 to match the use of Cisco port channel 30

The document you mentioned is an update of one I have from Nortel and looking at the update, as you say, I will need to tag enable ethernet 1/15 and change the attached stack on that port to use VLAN 1100 for all its ports rather than VLAN 1.

Hopefully once I have done that I can then add VLAN 1100 to the LACP link and have both VLAN 1150 and 1100 on ports 1/3,1/16,2/3 & 2/6, as well as 1100 on port 1/15

Will let you know in a week or so, as this will be a major impact change on the network here.