• May 22, 2012, 08:26:50 PM
Welcome, Guest. Please login or register. Registration is free.
Did you miss your activation email?

Author Topic: Network outage.  (Read 540 times)

0 Members and 1 Guest are viewing this topic.

Offline vishu

  • Rookie
  • **
  • Posts: 7
Network outage.
« on: November 17, 2011, 06:30:18 AM »
we have encountered system instabilities and network outage on 28 october
at 01:02 :

CPU6 [10/28/11 01:02:29] MLT INFO smltIstSessionDown

Additionally, one of the ERS has the following message on the logs:

CPU6 [10/22/11 20:37:38] HW WARNING HwCheck: Continuous tmux history error, Reset
tmux on  slot 3

This message appears many times since august. Does it mean that we have a hardware
problem with the card ?

please advice.
Appreciate you input!!!!!!!

Thanks in advance.


Offline Flintstone

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 584
Re: Network outage.
« Reply #1 on: November 17, 2011, 08:56:27 AM »
Hi vishu,

What hardware are you having the problem with?

CheerZ

Online Michael McNamara

  • Administrator
  • Hero Member
  • *****
  • Posts: 2517
    • Michael McNamara
Re: Network outage.
« Reply #2 on: November 17, 2011, 08:53:12 PM »
Hi vishu,

I'm guessing that your IST ports were on the card in slot 3? It's really important to make sure that you have multiple ports in your IST (MLT) spread out across multiple cards. If you don't and you loose that card (which it appears you did in this scenario) your IST will go down and you'll be left with a split brain scenario which not good.

You should probably look to RMA the card in slot 3 but also redesign your IST (MLT) so you have ports from other cards in the IST.

Cheers!
We've been helping network engineers, system administrators and technology professionals since June 2009.
If you've found this site useful or helpful, please help me spread the word. Link to us in your blog or homepage - Thanks!

Offline vishu

  • Rookie
  • **
  • Posts: 7
Re: Network outage.
« Reply #3 on: November 18, 2011, 06:46:05 AM »
Hi,

Thanks for your response.

@flintstone, we are using "8x1000BaseSXBB-E" module in slot 1 to 4 and in slot 7
                 and the software version we are using is 4.1.7.1

@michael,

No Michael we are having IST ports in all the modules i.e from 1 to 4, in each module we have 1 port dedicated to IST in MLT 1.

Could you please suggest some more?

Thanks in advance!

vishu

Online Michael McNamara

  • Administrator
  • Hero Member
  • *****
  • Posts: 2517
    • Michael McNamara
Re: Network outage.
« Reply #4 on: November 18, 2011, 11:56:43 AM »
That's really old software... there were issues with the older software were a problem could pre-empt the IST session from getting the necessary priority in the CPU/SF scheduling. I would advise that you upgrade to at least 4.1.8.3 software.

Good Luck!
We've been helping network engineers, system administrators and technology professionals since June 2009.
If you've found this site useful or helpful, please help me spread the word. Link to us in your blog or homepage - Thanks!

Offline vishu

  • Rookie
  • **
  • Posts: 7
Re: Network outage.
« Reply #5 on: November 19, 2011, 01:28:38 PM »
 Thanks Michael,

We are planning for an upgrade.
Lets see if it works.

-Vishu

Online Michael McNamara

  • Administrator
  • Hero Member
  • *****
  • Posts: 2517
    • Michael McNamara
Re: Network outage.
« Reply #6 on: November 19, 2011, 04:07:48 PM »
Hopefully you already know but there are some changes to how the IST/SMLT works in 4.1.8.0 and beyond so make sure you read the release notes carefully since you are upgrading from 4.1.7.x software. You will need a downtime window where both ERS 8600 switches will be down. You can not upgrade one switch at a time, the software is not compatible so both ERS 8600s must be upgraded before the IST will re-establish.

Good Luck!
We've been helping network engineers, system administrators and technology professionals since June 2009.
If you've found this site useful or helpful, please help me spread the word. Link to us in your blog or homepage - Thanks!