• February 09, 2012, 12:41:46 PM
Welcome, Guest. Please login or register.
Did you miss your activation email?

Author Topic: <SOLVED > 8600 CPU dormant mode  (Read 1302 times)

0 Members and 1 Guest are viewing this topic.

Offline dafle_ro

  • Rookie
  • **
  • Posts: 6
<SOLVED > 8600 CPU dormant mode
« on: September 02, 2010, 03:57:50 AM »
Hi All,

I have some problems with two 8600, each 8600 have 2 cpu card 8692SF. My problem is, i cannot wakeup card 6, if i try to make telnet have error, if i try to reset this card nothing is happening, if reset card 5 in not make booting from card 6 ??!!!!!!!
On both switch, secondary cpu have all led on Switching & CPU utilization.

Thank in advance !!1

Dan
  



#
# Asic Info :
# SlotNum|Name    |CardType  |MdaType    |Parts Description
#
# Slot  1   --     0x00000001 0x00000000
# Slot  2 8648GBRS 0x24325130 0x00000000  PRS:RSP=26 F2I=4 F2E=5 FTMUX=17 CC=33281 FOQ=270 DPC=7 BMC=264 PIM=1024 MAC=1112670281 Mirror=304 Loopback=274
# Slot  3 8648GTRS 0x24228130 0x00000000  PRS:RSP=26 F2I=4 F2E=5 FTMUX=17 CC=33281 FOQ=270 DPC=7 BMC=264 PIM=768 MAC=1112670281 Mirror=304 Loopback=274
# Slot  4 8612XLRS 0x2433410c 0x00000000  PRS:RSP=26 F2I=4 F2E=5 FTMUX=17 CC=33281 FOQ=270 DPC=7 BMC=264 PIM=1024 MAC=303177837 Mirror=304 Loopback=274
# Slot  5 8692SF   0x200e0100 0x00000000  CPU: CPLD=19 MEZZ=2  SFM: OP=3 TMUX=2 SWIP=23 FAD=16 CF=40
# Slot  6 8692SF   0x200e0100 0x00000000  CPU: CPLD=0  MEZZ=2  SFM: OP=3 TMUX=2 SWIP=23 FAD=16 CF=2
# Slot  7 8648GBRS 0x24325130 0x00000000  PRS:RSP=26 F2I=4 F2E=5 FTMUX=17 CC=33281 FOQ=270 DPC=7 BMC=264 PIM=1024 MAC=1112670281 Mirror=304 Loopback=274
# Slot  8 8648GTRS 0x24228130 0x00000000  PRS:RSP=26 F2I=4 F2E=5 FTMUX=17 CC=33281 FOQ=270 DPC=7 BMC=264 PIM=768 MAC=1112670281 Mirror=304 Loopback=274
# Slot  9   --     0x00000001 0x00000000
# Slot 10   --     0x00000001 0x00000000
#
 


Card Info :


        Slot#         FrontType  FrontHw    Oper   Admin  BackType   BackHw
                                 Version  Status  Status            Version
            2   48X1000BaseX-RS       01      up      up     EDPM3       01
            3  48X1000BaseTX-RS       01      up      up     EDPM2       01
            4 12X10000BaseXLRS-XFP       01      up      up     EDPM3       01
            5               CPU       51      up      up      FSFM       51
            6               CPU       51 dormant      up      FSFM       51

            7   48X1000BaseX-RS       01      up      up     EDPM3       01
            8  48X1000BaseTX-RS       01      up      up     EDPM2       01

 CoreA:5# peer telnet
Trying 127.0.0.6 ...
telnet: could not connect to host 127.0.0.6 errno 03c
« Last Edit: September 07, 2010, 05:58:07 AM by dafle_ro »


Offline nightwatch

  • Full Member
  • ***
  • Posts: 58
Re: 8600 CPU dormant mode
« Reply #1 on: September 02, 2010, 10:19:14 AM »
i had this error code before: errno 03c
It was not with a CPU but the error was the same. I had lost contact with the module. Some link oscillations occurring the bus to the module made it crash once in a while. I could reset the module and it would work, however the same thing would happen again some days after or weeks.

The problem never happened again when the hardware was replaced. I was using an old firmware.
« Last Edit: September 02, 2010, 10:46:37 AM by nightwatch »

Offline Michael McNamara

  • Administrator
  • Hero Member
  • *****
  • Posts: 2157
    • Michael McNamara
Re: 8600 CPU dormant mode
« Reply #2 on: September 02, 2010, 10:59:58 AM »
Hi dafle_ro,

You'll need to serial up to the CPU/SF and see where it's hanging when it boots up.

If you can provide that output we might be able to offer additional assistance.

Good Luck!
If you've found this site useful and helpful, please help me spread the word. Link to us in your blog or homepage or Tweet about us! - Thanks!

Offline dafle_ro

  • Rookie
  • **
  • Posts: 6
Re: 8600 CPU dormant mode
« Reply #3 on: September 06, 2010, 07:17:30 AM »
Hi All,

This situation with CPU in dormant mode appears on both switch 8600 on same time ??!!!!!!!, Is impossible to accessing card 6, wend i put console on this cpu card nothing is happening, cannot  see it. I try power off, take off card from switch, cold reset.


Copyright (c) 2009 Nortel, Inc.
CPU Slot 5:    PPC 745 Map B
Version:       5.1.0.0
Creation Time: May  6 2009, 20:26:19
Hardware Time: SEP 06 2010, 09:23:57 UTC
Memory Size:   0x10000000
Start Type:    cold
SMART MODULAR TECH. HYA ATA
open_file:can't open "/pcmcia/pcmboot.cfg" 0x380003
S_dosFsLib_FILE_NOT_FOUND
/flash/  - Volume is OK

Loaded boot configuration from file /flash/boot.cfg
Attaching network interface lo0... done.

Press <Return> to stop auto-boot...
Loading /flash/p80a5100.img ... 12600442 to 43719988 (43719988)
Starting at 0x1000000...

SMART MODULAR TECH. HYA ATA


Ethernet Routing Switch 8600  System Software Release 5.1.0.0
Copyright (c) 1996-2009 Nortel, Inc.

CPU5 [09/06/10 09:24:40] SW INFO Found serial number <00:1c:eb:9d:30:00> in file <license.dat>
CPU5 [09/06/10 09:24:40] SW INFO License Successfully Loaded From <license.dat> License Type -- ADVANCED
CPU5 [09/06/10 09:24:40] SW INFO System boot
CPU5 [09/06/10 09:24:40] SW INFO ERS System Software Release 5.1.0.0
CPU5 [09/06/10 09:24:44] MPLS INFO All MPLS components are up and active
CPU5 [09/06/10 09:24:44] HW INFO Card inserted: Slot=5 Type=8692SF
CPU5 [09/06/10 09:24:44] HW INFO Card inserted: Slot=6 Type=8692SF
CPU5 [09/06/10 09:24:44] SW INFO RS-Module inserted: Slot=2 Type=8648GBRS, waiting to bootup...
CPU5 [09/06/10 09:24:44] SW INFO RS-Module inserted: Slot=4 Type=8612XLRS, waiting to bootup...
CPU5 [09/06/10 09:24:44] SW INFO RS-Module inserted: Slot=7 Type=8648GTRS, waiting to bootup...
CPU5 [09/06/10 09:24:44] HW INFO Initializing 8692SF in slot #5 ...
CPU5 [09/06/10 09:24:46] HW INFO Initializing 8692SF in slot #6 ...
CPU5 [09/06/10 09:24:58] SW INFO Slot  2: Loading /flash/p80k5100.dld
CPU5 [09/06/10 09:24:58] SW INFO Slot  4: Loading /flash/p80k5100.dld
CPU5 [09/06/10 09:24:58] SW INFO Slot  7: Loading /flash/p80k5100.dld
CPU5 [09/06/10 09:25:24] SW INFO slot 7 found NP heartbeat - RS-Module is online
CPU5 [09/06/10 09:25:26] SW INFO slot 2 found NP heartbeat - RS-Module is online
CPU5 [09/06/10 09:25:27] SW INFO slot 4 found NP heartbeat - RS-Module is online
CPU5 [09/06/10 09:25:27] HW INFO Initializing 8648GBRS in slot #2 ...
CPU5 [09/06/10 09:25:28] HW INFO Initializing 8612XLRS in slot #4 ...
CPU5 [09/06/10 09:25:28] HW INFO Initializing 8648GTRS in slot #7 ...


SNMP-v3 VACM configuration is currently using default parameters.
These parameters should be changed for maximum security.

SNMP-v3 USM configuration is currently using default parameters.
These parameters should be changed for maximum security.


SNMP-v3 Having more than one entry in Group-access table for the same group-name with different security levels can cause a security hole

CPU5 [09/06/10 09:25:29] SW INFO Loading configuration from /flash/config.cfg
CPU5 [09/06/10 09:25:29] HW INFO Card taken off-line: Slot=2 Type=8648GBRS
**************************************************

* Copyright (c) 2009 Nortel, Inc.                *

* All Rights Reserved                            *

* Ethernet Routing Switch 8010                   *

* Software Release 5.1.0.0                       *

**************************************************

Login: adminPassword: *********
CoreB:5# CPU5 [09/06/10 09:25:31] SW INFO NTP Enabled
CoreB:5# CPU5 [09/06/10 09:25:32] SW INFO The system is ready
CoreB:5# CPU5 [09/06/10 09:25:32] SNMP INFO Booted with PRIMARY boot image source - /flash/p80a5100.img
CoreB:5# CPU5 [09/06/10 09:25:32] SW INFO PCMCIA card detected in Master CPU "CoreB" slot 5, Chassis S/N SSPNDT04EP
CoreB:5# CPU5 [09/06/10 09:25:32] SNMP INFO Chassis with Power Supply redundancy
CoreB:5# CPU5 [09/06/10 09:25:32] SNMP INFO Fan Up(FanId=1, OperStatus=2)
CoreB:5# CPU5 [09/06/10 09:25:32] SNMP INFO Fan Up(FanId=2, OperStatus=2)
CoreB:5# sh sys      tech

Sys Info:
---------------


General Info :

   SysDescr     : ERS-8610 (5.1.0.0)
   SysName      : CoreB
   SysUpTime    : 0 day(s), 00:01:33
   

Chassis Info :

   Chassis     : 8010
   Serial#     : SSPNDT04EP
   HwRev       : 50
   H/W Config         : 02
   NumSlots           : 10
   NumPorts           : 108
   GlobalFilter       : enable
   VlanBySrcMac       : disable

ForceTopologyIpFlag: false
   Ecn-Compatib       : enable
   WsmDirectMode      : disable
   max-vlan-resource-reservation : (disable) -> (disable)
   multicast-resource-reservation : (2048) -> (2048)
   BaseMacAddr        : 00:1c:eb:9d:30:00
   MacAddrCapacity : 4096
   Temperature : 26 C
   MgmtMacAddr : 00:1c:eb:9d:33:f4
   System MTU  : 1950
   clock_sync_time : 60


Card Info :


Slot#         FrontType  FrontHw    Oper   Admin  BackType   BackHw
                            Version  Status  Status            Version
       2   48X1000BaseX-RS       01    down    down     EDPM3       01
       4 12X10000BaseXLRS-XFP       01      up      up     EDPM3       01
       5               CPU       51      up      up      FSFM       51
       6               CPU       51 dormant      up      FSFM       51
       7  48X1000BaseTX-RS       01      up      up     EDPM2       01

MezzCard Info :

   Slot#5: MezzCard is notpresent Admin status: disabled Oper Status: down

System Error Info :

   Send Authentication Trap  : false
   Error Code                : 0
   Error Severity            : 0

Port Lock Info :

   Status       : off
   LockedPorts  :

Topology Status Info :


CoreB:5# CPU5 [09/06/10 09:26:19] SNMP INFO Sending Cold-Start Trap
CoreB:5# peer telnet
Trying 127.0.0.6 ...
telnet: could not connect to host 127.0.0.6 errno 03c
CoreB:5#

 Slot 5 :

   FrontType            : CPU
   FrontDescr           : 8692CPU
   FrontAdminStatus     : up
   FrontOperStatus      : up
   FrontSerialNum       : SSPN7N0MAG
   FrontHwVersion       : 51
   FrontPartNumber      : 215872D08
   FrontDateCode        : 20081121

FrontDeviations      :
   FrontModSerialNum    :
   FrontModPartNumber   :
   FrontModAssemblyDate :
   MezzCard             : notpresent
   MezzAdminStatus      : disabled
   MezzOperStatus       : down

   BackType             : FSFM
   BackDescr            : SF
   BackSerialNum        : SSPN7P0T7B
   BackHwVersion        : 51
   BackPartNumber       : 211861D05
   BackDateCode         : 20081121
   BackDeviations       :
   BackModSerialNum     :
   BackModPartNumber    :
   BackModAssemblyDate  :


  Slot 6 :

   FrontType            : CPU

FrontDescr           : 8692CPU
   FrontAdminStatus     : up
   FrontOperStatus      : dormant
   FrontSerialNum       : SSPN7N0MAD
   FrontHwVersion       : 51
   FrontPartNumber      : 215872D08
   FrontDateCode        : 20081120
   FrontDeviations      :
   FrontModSerialNum    :
   FrontModPartNumber   :
   FrontModAssemblyDate :
   MezzCard             : unsupported

   BackType             : FSFM
   BackDescr            : SF
   BackSerialNum        : SSPN7P0T61
   BackHwVersion        : 51
   BackPartNumber       : 211861D05
   BackDateCode         : 20081120
   BackDeviations       :
   BackModSerialNum     :
   BackModPartNumber    :
   BackModAssemblyDate  :

Offline dafle_ro

  • Rookie
  • **
  • Posts: 6
Re: 8600 CPU dormant mode
« Reply #4 on: September 06, 2010, 07:46:10 AM »
I attach image with front leds on card 6, same on both switch 8600.

Offline Michael McNamara

  • Administrator
  • Hero Member
  • *****
  • Posts: 2157
    • Michael McNamara
Re: 8600 CPU dormant mode
« Reply #5 on: September 06, 2010, 03:24:39 PM »
This situation with CPU in dormant mode appears on both switch 8600 on same time ??!!!!!!!, Is impossible to accessing card 6, wend i put console on this cpu card nothing is happening, cannot  see it. I try power off, take off card from switch, cold reset.

This is normal behavior when not operating in an HA configuration. The CPU is dormant but the SF capacity is doubled by having a second CPU/SF in the chassis.

Quote
Copyright (c) 2009 Nortel, Inc.
CPU Slot 5:    PPC 745 Map B
Version:       5.1.0.0
Creation Time: May  6 2009, 20:26:19
Hardware Time: SEP 06 2010, 09:23:57 UTC
Memory Size:   0x10000000
Start Type:    cold
SMART MODULAR TECH. HYA ATA
open_file:can't open "/pcmcia/pcmboot.cfg" 0x380003
S_dosFsLib_FILE_NOT_FOUND
/flash/  - Volume is OK

Loaded boot configuration from file /flash/boot.cfg
Attaching network interface lo0... done.

Press <Return> to stop auto-boot...
Loading /flash/p80a5100.img ... 12600442 to 43719988 (43719988)
Starting at 0x1000000...

SMART MODULAR TECH. HYA ATA


Ethernet Routing Switch 8600  System Software Release 5.1.0.0
Copyright (c) 1996-2009 Nortel, Inc.

CPU5 [09/06/10 09:24:40] SW INFO Found serial number <00:1c:eb:9d:30:00> in file <license.dat>
CPU5 [09/06/10 09:24:40] SW INFO License Successfully Loaded From <license.dat> License Type -- ADVANCED
CPU5 [09/06/10 09:24:40] SW INFO System boot
CPU5 [09/06/10 09:24:40] SW INFO ERS System Software Release 5.1.0.0
CPU5 [09/06/10 09:24:44] MPLS INFO All MPLS components are up and active
CPU5 [09/06/10 09:24:44] HW INFO Card inserted: Slot=5 Type=8692SF
CPU5 [09/06/10 09:24:44] HW INFO Card inserted: Slot=6 Type=8692SF
CPU5 [09/06/10 09:24:44] SW INFO RS-Module inserted: Slot=2 Type=8648GBRS, waiting to bootup...
CPU5 [09/06/10 09:24:44] SW INFO RS-Module inserted: Slot=4 Type=8612XLRS, waiting to bootup...
CPU5 [09/06/10 09:24:44] SW INFO RS-Module inserted: Slot=7 Type=8648GTRS, waiting to bootup...
CPU5 [09/06/10 09:24:44] HW INFO Initializing 8692SF in slot #5 ...
CPU5 [09/06/10 09:24:46] HW INFO Initializing 8692SF in slot #6 ...
CPU5 [09/06/10 09:24:58] SW INFO Slot  2: Loading /flash/p80k5100.dld
CPU5 [09/06/10 09:24:58] SW INFO Slot  4: Loading /flash/p80k5100.dld
CPU5 [09/06/10 09:24:58] SW INFO Slot  7: Loading /flash/p80k5100.dld
CPU5 [09/06/10 09:25:24] SW INFO slot 7 found NP heartbeat - RS-Module is online
CPU5 [09/06/10 09:25:26] SW INFO slot 2 found NP heartbeat - RS-Module is online
CPU5 [09/06/10 09:25:27] SW INFO slot 4 found NP heartbeat - RS-Module is online
CPU5 [09/06/10 09:25:27] HW INFO Initializing 8648GBRS in slot #2 ...
CPU5 [09/06/10 09:25:28] HW INFO Initializing 8612XLRS in slot #4 ...
CPU5 [09/06/10 09:25:28] HW INFO Initializing 8648GTRS in slot #7 ...
...
...
CoreB:5# CPU5 [09/06/10 09:25:32] SW INFO PCMCIA card detected in Master CPU "CoreB" slot 5, Chassis S/N SSPNDT04EP

So the above looks like it was captured from the CPU/SF in Slot 5. What do you get out of the CPU/SF in Slot 6?

Additionally do you have telnet enabled in the boot config (boot.cfg) options?

Good Luck!
If you've found this site useful and helpful, please help me spread the word. Link to us in your blog or homepage or Tweet about us! - Thanks!

Offline dafle_ro

  • Rookie
  • **
  • Posts: 6
Re: 8600 CPU dormant mode
« Reply #6 on: September 07, 2010, 02:52:48 AM »
Hi Michael,

I worried because before that situation, i can do telnet on secondary cpu, can have access on console for slot 6, when rebooting primary cpu from slot 5, is booting from slot 6 secondary cpu, at this moment rebooting just from primary cpu slot 5.
And front leds from secondary cpu (slot 6) not indicate utilization and cpu full.
I try to capture with console from secondary card, but is not responding ??!!
For me seems is something wrong, configurations is OK, maybe hardware ??

I will try to power off one core switch and change card 5 with card 6, will see what is happening.

Thanks,

Dan Flesaru

Offline dafle_ro

  • Rookie
  • **
  • Posts: 6
Re: <SOLVED > 8600 CPU dormant mode
« Reply #7 on: September 07, 2010, 06:04:06 AM »
Hi All,

I solved, but is better to know for others.

If your fabric & cpu leds are all on, that can indicate that the CPU blade is stuck in the boot monitor (has not fully booted). Make sure you can get CLI access to both CPUs in the chassis and that they have booted correctly. Also make sure the ha-mode flag is false on both CPUs (you should have left flag savetostandby true).

Fly,

Dan Flesaru

Offline Michael McNamara

  • Administrator
  • Hero Member
  • *****
  • Posts: 2157
    • Michael McNamara
Re: <SOLVED > 8600 CPU dormant mode
« Reply #8 on: September 07, 2010, 10:55:24 AM »
I'm happy to hear that you've resolved the issue... so when you connected to Slot 6 you found the CPU/SF at the boot monitor prompt. Just for the audience... that's not the only issue that can cause all the LEDs to light up on the CPU/SF.

No need to respond but was there some change that broke the boot.cfg file? Why was the the CPU/SF sitting at the boot monitor prompt?

Cheers!
If you've found this site useful and helpful, please help me spread the word. Link to us in your blog or homepage or Tweet about us! - Thanks!