Introduction
This document describes how to streamline the upgrade process for Cisco Catalyst 3850 Series Switches and offers disaster recovery techniques in the case of a software or boot failure. The upgrade process is simple, but it can seem difficult if you do not have experience with Cisco IOS® XE software upgrades.
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
TFTP
FTP
Components Used
The information in this document is based on the Cisco Catalyst 3850 Series Switch that runs Cisco IOS-XE Versions 03.03.00 and later. The examples in this document use a stacked solution; however, the same commands can be run on a standalone switch.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Note: In order to download Cisco IOS-XE images from the Cisco website, you must have a valid Cisco Connection Online (CCO) account with entitled credentials. Also, Cisco does not offer a free TFTP/FTP solution, so you must install and configure the TFTP/FTP before you begin.
Install Versus Bundle Mode
When you deploy the Cisco Catalyst 3850 Series Switch, you have two modes of operation to choose from: INSTALL and BUNDLE. There are minor differences between the two modes. Review the configuration guide for more details.
Cisco recommends that the INSTALL mode is used during operation because it allows for a more complete set of features and requires fewer resources upon boot. This document provides a brief overview of each mode for reference.
Note: Software defect CSCuw82216may result in switch inoperability due to flash memory corruption when upgrading from a pre-3.6.3 or pre-3.7.2 software release to 3.6.3/3.7.2 in install mode. This issue is resolved in 3.6.4 or 3.7.3 or later software therefore the issue is not seen when upgrading to these software versions. As a result of this software defect, bundle mode is recommended for upgrades involving the effected software versions
Install Mode
This is the default mode for the switch. The INSTALL mode uses a package-provisioning file named packages.conf in order to boot the switch. In addition, there are a number of .pkg files in the flash. Cisco recommends that you do not alter these files unless directed by a Cisco Technical Assistance Center (TAC) engineer.
Bundle Mode
If you are comfortable with the use of traditional monolithic IOS images in order to boot the switch, then the BUNDLE mode should be familiar. The BUNDLE mode consumes more memory than the INSTALL mode because the packages are extracted from the Bundle and copied to the RAM.
Verify the Mode
In order to verify the mode, enter the show version command:
3850-stack#
show version
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.00SE RELEASE SOFTWARE (fc1) Switch Ports Model SW Version SW Image
Mode
------------------ ---------- ---------- ---- 1 32 WS-C3850-24P 03.03.00SE cat3k_caa-universalk9
INSTALL
* 2 56 WS-C3850-48T 03.03.00SE cat3k_caa-universalk9
INSTALL
Upgrade
In order to begin the upgrade process, you must download the IOS-XE .bin file from the Cisco web site and place it in the flash of your active switch. The process that is used in order to copy the file to the switch is not covered in this document.
When you copy the .bin file to a single switch, the install process replicates the file to the other switches in the stack. Once the file is present, enter this command:
3850-stack#
software install file flash:cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin switch 1-2
Note: There are many options available after each command; however, in this example a basic upgrade is run.
Once you begin the upgrade process, the switch pushes the .bin file to the stack member peers.
Preparing install operation ... [2]: Copying software from active switch 1 to switch 2
After all of the members receive the .bin file, it is automatically expanded to the flash.
[1 2]: Starting install operation [1 2]: Expanding bundle flash: cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin [1 2]: Copying package files [1 2]: Package files copied [1 2]: Finished expanding bundle flash: cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin [1 2]: Verifying and copying expanded package files to flash: [1 2]: Verified and copied expanded package files to flash: [1 2]: Starting compatibility checks [1 2]: Finished compatibility checks [1 2]: Starting application pre-installation processing [1 2]: Finished application pre-installation processing
Next, the switch lists a summary of the files that are marked for both removal and addition to the packages.conf pointer file.
[1]: Old files list: Removed cat3k_caa-base.SPA.03.03.00SE.pkg Removed cat3k_caa-drivers.SPA.03.03.00SE.pkg Removed cat3k_caa-infra.SPA.03.03.00SE.pkg Removed cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg Removed cat3k_caa-platform.SPA.03.03.00SE.pkg Removed cat3k_caa-wcm.SPA.10.1.100.0.pkg [2]: Old files list: Removed cat3k_caa-base.SPA.03.03.00SE.pkg Removed cat3k_caa-drivers.SPA.03.03.00SE.pkg Removed cat3k_caa-infra.SPA.03.03.00SE.pkg Removed cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg Removed cat3k_caa-platform.SPA.03.03.00SE.pkg Removed cat3k_caa-wcm.SPA.10.1.100.0.pkg [1]: New files list: Added cat3k_caa-base.SPA.03.03.01SE.pkg Added cat3k_caa-drivers.SPA.03.03.01SE.pkg Added cat3k_caa-infra.SPA.03.03.01SE.pkg Added cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg Added cat3k_caa-platform.SPA.03.03.01SE.pkg Added cat3k_caa-wcm.SPA.10.1.110.0.pkg [2]: New files list: Added cat3k_caa-base.SPA.03.03.01SE.pkg Added cat3k_caa-drivers.SPA.03.03.01SE.pkg Added cat3k_caa-infra.SPA.03.03.01SE.pkg Added cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg Added cat3k_caa-platform.SPA.03.03.01SE.pkg Added cat3k_caa-wcm.SPA.10.1.110.0.pkg
Lastly, the packges.conf file is updated and committed.
[1 2]: Creating pending provisioning file [1 2]: Finished installing software. New software will load on reboot. [1 2]: Committing provisioning file [1 2]: Do you want to proceed with reload? [yes/no]: yes
Verify that the update process is properly completed upon reload.
3850-stack#
show ver | i INSTALL
1 32 WS-C3850-24P 03.03.01SE cat3k_caa-universalk9 INSTALL * 2 56 WS-C3850-48T 03.03.01SE cat3k_caa-universalk9 INSTALL
Flash Cleanup
Residual files remain in the flash from previous versions. If you want to clean up the residual files, you can enter the software clean command instead of a manual deletion of the files. This purges the files that the switch no longer needs in order to operate from the flash of each stack member.
Note: This command also deletes the .bin file that is used in order to install the new IOS software. It is important to remember that once it is extracted, you no longer need it.
The next two sections provide examples of how the flash appears before and after the software clean command is used.
Before Flash Cleanup
3850-stack#
show flash
-#- --length-- ---------date/time----------- -----------------------path----------------------- 2 2097152 Feb 16 2014 11:38:46.0 +00:00 nvram_config 4 257016048 Jan 28 2014 17:22:12.0 +00:00 cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin 5 4096 Jan 28 2014 17:25:50.0 +00:00 mnt 6 4096 Jan 28 2014 17:25:50.0 +00:00 mnt/images 7 4096 Jan 28 2014 17:25:52.0 +00:00 mnt/images/ap.bak 8 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2.md5 9 11591680 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2 10 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1.md5 11 10444800 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1 12 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2.md5 13 13568000 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2 14 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140.md5 15 10291200 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140 16 11 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/version.info 17 1214 Jan 28 2014 17:25:10.0 +00:00 packages.conf.00- 18 79112096 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-base.SPA.03.03.00SE.pkg 19 6474428 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-drivers.SPA.03.03.00SE.pkg 20 34501468 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-infra.SPA.03.03.00SE.pkg 21 1248 Feb 16 2014 11:27:51.0 +00:00 packages.conf 22 34763952 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg 23 796 Feb 19 2014 11:43:13.0 +00:00 vlan.dat 24 24992476 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-platform.SPA.03.03.00SE.pkg 25 77167308 Jan 28 2014 17:25:06.0 +00:00 cat3k_caa-wcm.SPA.10.1.100.0.pkg 26 1224 Jan 28 2014 16:39:58.0 +00:00 packages.conf.01- 27 6571 Dec 20 2013 08:56:32.0 +00:00 BLANK_CONFIG.cfg 28 257193048 Feb 16 2014 11:19:44.0 +00:00 cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin 30 79113792 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-base.SPA.03.03.01SE.pkg 31 74409080 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-base.SPA.03.02.01.SE.pkg 32 2775728 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-drivers.SPA.03.02.01.SE.pkg 33 6476476 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-drivers.SPA.03.03.01SE.pkg 34 32478052 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-infra.SPA.03.02.01.SE.pkg 35 30389028 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EX1.pkg 36 18313952 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-platform.SPA.03.02.01.SE.pkg 37 63402700 Jan 28 2014 16:39:54.0 +00:00 cat3k_caa-wcm.SPA.10.0.101.0.pkg 38 34503664 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-infra.SPA.03.03.01SE.pkg 39 34788684 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg 40 25009040 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-platform.SPA.03.03.01SE.pkg 41 77296448 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-wcm.SPA.10.1.110.0.pkg
237428736 bytes available (1302147072 bytes used)
After Flash Cleanup
3850-stack#
software clean
Preparing clean operation ... [1 2]: Cleaning up unnecessary package files [1 2]: No path specified, will use booted path flash:packages.conf [1 2]: Cleaning flash: [1]: Preparing packages list to delete ... In use files, will not delete: cat3k_caa-base.SPA.03.03.01SE.pkg cat3k_caa-drivers.SPA.03.03.01SE.pkg cat3k_caa-infra.SPA.03.03.01SE.pkg cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg cat3k_caa-platform.SPA.03.03.01SE.pkg cat3k_caa-wcm.SPA.10.1.110.0.pkg packages.conf [2]: Preparing packages list to delete ... In use files, will not delete: cat3k_caa-base.SPA.03.03.01SE.pkg cat3k_caa-drivers.SPA.03.03.01SE.pkg cat3k_caa-infra.SPA.03.03.01SE.pkg cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg cat3k_caa-platform.SPA.03.03.01SE.pkg cat3k_caa-wcm.SPA.10.1.110.0.pkg packages.conf [1]: Files that will be deleted: cat3k_caa-base.SPA.03.02.01.SE.pkg cat3k_caa-base.SPA.03.03.00SE.pkg cat3k_caa-drivers.SPA.03.02.01.SE.pkg cat3k_caa-drivers.SPA.03.03.00SE.pkg cat3k_caa-infra.SPA.03.02.01.SE.pkg cat3k_caa-infra.SPA.03.03.00SE.pkg cat3k_caa-iosd-universalk9.SPA.150-1.EX1.pkg cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg cat3k_caa-platform.SPA.03.02.01.SE.pkg cat3k_caa-platform.SPA.03.03.00SE.pkg cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin cat3k_caa-wcm.SPA.10.0.101.0.pkg cat3k_caa-wcm.SPA.10.1.100.0.pkg packages.conf.00- packages.conf.01- [2]: Files that will be deleted: cat3k_caa-base.SPA.03.02.01.SE.pkg cat3k_caa-base.SPA.03.03.00SE.pkg cat3k_caa-drivers.SPA.03.02.01.SE.pkg cat3k_caa-drivers.SPA.03.03.00SE.pkg cat3k_caa-infra.SPA.03.02.01.SE.pkg cat3k_caa-infra.SPA.03.03.00SE.pkg cat3k_caa-iosd-universalk9.SPA.150-1.EX1.pkg cat3k_caa-iosd-universalk9.SPA.150-1.EZ.pkg cat3k_caa-platform.SPA.03.02.01.SE.pkg cat3k_caa-platform.SPA.03.03.00SE.pkg cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin cat3k_caa-wcm.SPA.10.0.101.0.pkg cat3k_caa-wcm.SPA.10.1.100.0.pkg packages.conf.00- packages.conf.01- [1 2]: Do you want to proceed with the deletion? [yes/no]:
yes
[1 2]: Clean up completed
Here is the output from the show flash command after the flash cleanup:
3850-stack#
show flash
-#- --length-- ---------date/time----------- -------------------path-------------------- 2 2097152 Feb 16 2014 11:38:46.0 +00:00 nvram_config 4 4096 Jan 28 2014 17:25:50.0 +00:00 mnt 5 4096 Jan 28 2014 17:25:50.0 +00:00 mnt/images 6 4096 Jan 28 2014 17:25:52.0 +00:00 mnt/images/ap.bak 7 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2.md5 8 11591680 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap1g2 9 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1.md5 10 10444800 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g1 11 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2.md5 12 13568000 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/ap3g2 13 40 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140.md5 14 10291200 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/c1140 15 11 Oct 03 2013 05:02:21.0 +00:00 mnt/images/ap.bak/version.info 16 1248 Feb 16 2014 11:27:51.0 +00:00 packages.conf 17 796 Feb 19 2014 11:43:13.0 +00:00 vlan.dat 18 6571 Dec 20 2013 08:56:32.0 +00:00 BLANK_CONFIG.cfg 20 79113792 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-base.SPA.03.03.01SE.pkg 21 6476476 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-drivers.SPA.03.03.01SE.pkg 22 34503664 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-infra.SPA.03.03.01SE.pkg 23 34788684 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg 24 25009040 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-platform.SPA.03.03.01SE.pkg 25 77296448 Feb 16 2014 11:27:46.0 +00:00 cat3k_caa-wcm.SPA.10.1.110.0.pkg
1231515648 bytes available (308060160 bytes used)
Auto-Upgrade Feature for Catalyst 3850 Series Switches
There are many scenarios where you must introduce a new switch into a current stack of Catalyst 3850 Series Switches, such as when a new switch is purchased in order to expand the number of usable ports in the stack. In order to successfully add a new switch to a stack, you must ensure that the same software version is run on the new switch. Prior to IOS-XE Version 3.3.1, the only way to ensure that the versions match is to stage the new switch prior to introduction into the stack.
The Catalyst 3850 Series Switches include a feature called Auto-Upgrade that might be familiar if your migration occurs from the Catalyst 3750 Series platform. The goal of this feature is to ensure that a newly added switch is automatically provisioned by the stack members with the correct IOS-XE version. However, one caveat to note is that prior to Version 3.3.1, this feature does not work (reference Cisco bug ID CSCui69999). Therefore, you must ensure that your current stack runs at least Version 3.3.1 in INSTALL mode.
Note: Auto-Upgrade is disabled by default and is not available in BUNDLE mode.
In order to use the Auto-Upgrade feature, add the software auto-upgrade enable command into the configuration of the current stack. This ensures that any newly added stack members are automatically upgraded.
Configure
Once the switch is stacked and boots up, you see a clear indication that there is a mismatch in versions, and the new member does not fully join the stack. If you watch the SYSLOG as the switch attempts to join, you notice that the Auto-Advise feature alerts you that the newly added switch runs a different software version and mode.
Note: For this example, the new switch runs IOS-XE Version 3.2.2 in BUNDLE mode.
%STACKMGR-1-STACK_LINK_CHANGE: STANDBY: 1 stack-mgr: Stack port 2 on switch 1 is up (3850-Stack-1) %STACKMGR-1-STACK_LINK_CHANGE: 2 stack-mgr: Stack port 1 on switch 2 is up %STACKMGR-6-SWITCH_ADDED: 2 stack-mgr: Switch 3 has been added to the stack. %STACKMGR-6-SWITCH_ADDED: STANDBY:1 stack-mgr: Switch 3 has been added to the stack. (3850-Stack-1) %INSTALLER-6-AUTO_ADVISE_SW_INITIATED: 2 installer: Auto advise initiated for switch 3 %INSTALLER-6-AUTO_ADVISE_SW: 2 installer: Switch 3 running bundled software has been added %INSTALLER-6-AUTO_ADVISE_SW: 2 installer: to the stack that is running installed software. %INSTALLER-6-AUTO_ADVISE_SW: 2 installer: The 'software auto-upgrade' command can be used to %INSTALLER-6-AUTO_ADVISE_SW: 2 installer: convert switch 3 to the installed running mode by %INSTALLER-6-AUTO_ADVISE_SW: 2 installer: installing its running software.
Once the newly joined member is fully booted, you see that a mismatch is detected:
3850-Stack#
show switch
Switch/Stack Mac Address : 0c27.24cf.ab80 - Local Mac Address Mac persistency wait time: Indefinite H/W Current Switch# Role Mac Address Priority Version State ------------------------------------------------------------ *1 Active 0c27.24cf.ab80 14 B0 Ready 2 Standby f41f.c238.a800 13 B0 Ready 3 Member b4e9.b0d3.6600 12 0 V-Mismatch
Enable Auto-Upgrade Feature
In Global Configuration mode, enter the software auto-upgrade enable command. This enables the feature for any new switches that join the stack.
3850-Stack(config)
#
software auto-upgrade enable
3850-Stack(config)
#
end
You must reload the newly added switch only; a full stack reload is not necessary. In this case, the newly added switch is switch 3, so the reload slot 3 command is entered.
Tip: The slot mentioned in these commands designates the switch in the stack (slot 1 = switch 1).
3850-Stack#
reload slot 3
Proceed with reload?
[confirm]
%STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Reload Slot Command %STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down %STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been removed from the stack. %STACKMGR-1-STACK_LINK_CHANGE: STANDBY: 2 stack-mgr: Stack port 1 on switch 2 is down (3850-Stack-2) Starting SWITCH-DELETE sequence, switch 3 SWITCH-DELETE sequence complete, switch 3 %STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack. (3850-Stack-2) Starting SWITCH-DELETE sequence, switch 3 (3850-Stack-2) SWITCH-DELETE sequence complete, switch 3 (3850-Stack-2)
Wait a few minutes while the switch reloads in the background. Then, you see this:
%STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is up 3850-Stack# %STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 1 on switch 2 is up (3850-Stack-2) 3850-Stack# %STACKMGR-6-SWITCH_ADDED: 1 stack-mgr: Switch 3 has been added to the stack. %STACKMGR-6-SWITCH_ADDED: STANDBY:2 stack-mgr: Switch 3 has been added to the stack. (3850-Stack-2)
You now see that the conversion from BUNDLE to INSTALL mode occurs, followed by a reload:
%INSTALLER-6-AUTO_UPGRADE_SW_INITIATED: 1 installer: Auto upgrade initiated for switch 3 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Converting switch 3 to installed mode by %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: installing its running software %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Setting the boot var on switch 3 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Finished installing the running software on switch 3 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Reloading switch 3 to boot in installed mode %STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Auto upgrade %STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down %STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been r 3850-Stack#emoved from the stack. %STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 1 on switch 2 is down (3850-Stack-2) Starting SWITCH-DELETE sequence, switch 3 SWITCH-DELETE sequence complete, switch 3 %STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack. (3850-Stack-2) 3850-Stack# Starting SWITCH-DELETE sequence, switch 3 (3850-Stack-2) SWITCH-DELETE sequence complete, switch 3 (3850-Stack-2)
After the reboot, the upgrade continues:
%INSTALLER-6-AUTO_UPGRADE_SW_INITIATED: 1 installer: Auto upgrade initiated for switch 3 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Searching stack for software to upgrade switch 3 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Found donor switch 1 to auto upgrade switch 3 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Upgrading switch 3 with software from switch 1 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Finished installing software on switch 3 %INSTALLER-6-AUTO_UPGRADE_SW: 1 installer: Reloading switch 3 to complete the auto upgrade %STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr: Received reload request for switch 3, reason Auto upgrade %STACKMGR-1-STACK_LINK_CHANGE: 1 stack-mgr: Stack port 2 on switch 1 is down %STACKMGR-6-SWITCH_REMOVED: 1 stack-mgr: Switch 3 has been removed from the stack. %STACKMGR-1-STACK_LINK_CHANGE: STANDBY:2 stack-mgr: Stack port 3850-Stack#t 1 on switch 2 is down (3850-Stack-2) Starting SWITCH-DELETE sequence, switch 3 SWITCH-DELETE sequence complete, switch 3 %STACKMGR-6-SWITCH_REMOVED: STANDBY:2 stack-mgr: Switch 3 has been removed from the stack. (3850-Stack-2)
Another reload is performed automatically. Once the switch boots up, it successfully joins the stack with the correct IOS-XE version and software mode.
%STACKMGR-6-SWITCH_ADDED: 1 stack-mgr: Switch 3 has been added to the stack. %STACKMGR-6-SWITCH_ADDED: STANDBY:2 stack-mgr: Switch 3 has been added to the stack. (3850-Stack-2) %STACKMGR-6-SWITCH_READY: STANDBY:2 stack-mgr: Switch 3 is ready. (3850-Stack-2) %STACKMGR-6-SWITCH_READY: 1 stack-mgr: Switch 3 is ready. Starting SWITCH-ADD sequence, switch 3 %NGWC_USB_CONSOLE-6-CONFIG_ENABLE: Switch 3: Console media-type changed to default Starting SWITCH-ADD sequence, switch 3 (3850-Stack-2) SWITCH-ADD sequence complete, switch 3 (3850-Stack-2) SWITCH-ADD sequence complete, switch 3
Verify
Use the show switch and show version commands in order to verify that the upgrade process is completed properly:
3850-Stack#
show switch
Switch/Stack Mac Address : 0c27.24cf.ab80 - Local Mac Address Mac persistency wait time: Indefinite H/W Current Switch# Role Mac Address Priority Version State ------------------------------------------------------------ *1 Active 0c27.24cf.ab80 14 B0 Ready 2 Standby f41f.c238.a800 13 B0 Ready 3 Member b4e9.b0d3.6600 12 B0 Ready 3850-Stack#
show version
Switch Ports Model SW Version SW Image Mode ------ ----- ----- ---------- ---------- ---- * 1 56 WS-C3850-48P 03.03.01SE cat3k_caa-universalk9 INSTALL 2 56 WS-C3850-48P 03.03.01SE cat3k_caa-universalk9 INSTALL 3 56 WS-C3850-48P 03.03.01SE cat3k_caa-universalk9 INSTALL
Recover from a 3850 Series Switch Boot Failure
There are multiple reasons for a 3850 Series Switch boot failure, such as a corrupt boot image, a corrupt packages.conf file, or missing files. This section describes possible recovery methods.
Note: Ensure that you have knowledge of the two possible boot modes, INSTALL and BUNDLE, before you continue.
Standard Recovery Methods
This section describes the two standard methods that are used in order to recover from a Catalyst 3850 Series Switch boot failure.
USB Recovery
The 3850 Series Switches have a USB port on the front that is used for console access. This USB port is also used with flash drives for image backup and recovery.
If you become stuck at the switch: prompt with a corrupt image or .conf file, you can easily boot to a file that is stored on the USB drive or copy an image from the USB to internal flash. Complete these steps in order to recover from the boot failure:
Verify that the flash drive is recognized and the .bin file exists:
switch:
dir usbflash0:
Directory of usbflash0:/ 74 -rw- 223734376 cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin
Boot to the USB image:
switch:
boot usbflash0:cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin
Note: This process boots the switch into BUNDLE mode.
Tip: You can also copy the .bin file from usbflash0: to flash:, and point the boot statement towards internal flash.
Corrupt File Recovery
There are instances when the packages.conf calls files that no longer exist in flash. You can manually boot an image from the switch: prompt file; however, upon reload it calls the packages.conf file again and fails to boot. If this occurs, Cisco recommends that you back up the current packages.conf file and rename it or delete it. This process is mandatory, as the next step fails if a .conf file already exists. Once you extract the .bin file, a new packages.conf file is created. Complete these steps in order to recover from a corrupt packages.conf file:
Once booted (in BUNDLE mode), verify the files in the flash:
Switch#
dir flash:
Directory of flash:/ 15500 -rwx 1243 Aug 1 2013 07:04:02 +00:00 packages.conf
Copy or rename the current packages.conf file:
Switch#
cp flash:packages.conf flash:packages.conf.badop
Destination filename [packages.conf.bad]? Copy in progress...C 1243 bytes copied in 0.140 secs (8879 bytes/sec) Switch#
dir flash:
Directory of flash:/ 15500 -rwx 1243 Aug 1 2013 07:04:02 +00:00 packages.conf 15502 -rw- 1243 Aug 1 2013 11:53:51 +00:00 packages.conf.bad Switch#
del flash:packages.conf
Delete filename [packages.conf]? Delete flash:/packages.conf? [confirm]
Expand the bundle in order to create a new packages.conf file:
Switch#
software expand running switch 1 to flash:
Preparing expand operation ... [1]: Expanding the running bundle [1]: Copying package files [1]: Package files copied [1]: Finished expanding the running bundle
Verify the boot:
Switch#
show boot
--------------------------- Switch 1 --------------------------- Current Boot Variables: BOOT variable does not exist Boot Variables on next reload: BOOT variable = flash:packages.conf; Manual Boot = no Enable Break = no
Reload the switch:
Switch#
reload
Reload command is being issued on Active unit, this will reload the whole stack Proceed with reload? [confirm]
Emergency Recovery
If the previous recovery methods fail, the 3850 Series Switches have a trap door method that you can use in order to recover the system. You must have a terminal that is connected to the management port of the switch that runs a TFTP server. Download a valid image file from CCO and store it in the root of the TFTP server.
It is likely that the switch is stuck at the switch: prompt. However, if you are in a boot loop, you can use the Mode button on the front of the switch in order to break the cycle: hold the button for approximately ten seconds, and the switch breaks the cycle and stops at the switch: prompt.
Complete these steps in order to perform an emergency recovery:
Set the switch IP address:
switch:
set IP_ADDR 192.0.2.123/255.255.255.0
Set the default gateway:
switch:
set DEFAULT_ROUTER 192.0.2.1
Ping the terminal that contains the TFTP server in order to test the connectivity:
switch:
ping 192.0.2.1
ping 192.0.2.1 with 32 bytes of data ... Host 192.0.2.1 is alive.
Verify that the emergency files exist in the switch file system:
switch:
dir sda9:
Directory of sda9:/ 2 drwx 1024 . 2 drwx 1024 .. 11 -rwx 18958824 cat3k_caa-recovery.bin 36903936 bytes available (20866048 bytes used)
Run the emergency install feature:
switch:
emergency-install tftp://192.0.2.1/cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin
The bootflash will be erased during install operation, continue (y/n)?Y Starting emergency recovery (tftp://192.0.2.1/cat3k_caa-universalk9. SPA.03.02.02.SE.150-1.EX2.bin)... Reading full image into memory......................done Nova Bundle Image -------------------------------------- Kernel Address : 0x6042f5d8 Kernel Size : 0x317ccc/3243212 Initramfs Address : 0x607472a4 Initramfs Size : 0xdc6546/14443846 Compression Format: .mzip Bootable image at @ ram:0x6042f5d8 Bootable image segment 0 address range [0x81100000, 0x81b80000] is in range [0x80180000, 0x90000000]. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@ File "sda9:cat3k_caa-recovery.bin" uncompressed and installed, entry point: 0x811060f0 Loading Linux kernel with entry point 0x811060f0 ... Bootloader: Done loading app on core_mask: 0xf ### Launching Linux Kernel (flags = 0x5) Initiating Emergency Installation of bundle tftp://192.0.2.1/ cat3k_caa-universalk9.SPA.03.03.00.SE.150-1.EZ.bin Downloading bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin... Validating bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin... Installing bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin... Verifying bundle tftp://192.0.2.1/ cat3k_caa-universalk9. SPA.03.03.00.SE.150-1.EZ.bin... Package cat3k_caa-base.SPA.03.03.00.SE.pkg is Digitally Signed Package cat3k_caa-drivers.SPA.03.03.00.SE.pkg is Digitally Signed Package cat3k_caa-infra.SPA.03.03.00.SE.pkg is Digitally Signed Package cat3k_caa-iosd-universalk9.SPA.150-1.EX2.pkg is Digitally Signed Package cat3k_caa-platform.SPA.03.03.00.SE.pkg is Digitally Signed Package cat3k_caa-wcm.SPA.10.0.111.0.pkg is Digitally Signed Preparing flash... Syncing device... Emergency Install successful... Rebooting Restarting system.