Jump to content
rev.dennis

Upgrade fails

Recommended Posts

So I upgrade from 13.1.1.5 to 13.1.3 and out of 30 LTMs (all vCMP Guests) only about half are successful.  What happens is after the system boots up on the new code it shows the configuration is unable to shown.

First step in troubleshooting is go to the CLI and run

tmsh load sys config

See where it errors out (see below for my example)

[root@txsat1slbco22:/S3-red-P::Offline:Disconnected] config # tmsh load sys config
Loading system configuration...
  /defaults/asm_base.conf
  /defaults/config_base.conf
  /defaults/ipfix_ie_base.conf
  /defaults/ipfix_ie_f5base.conf
  /defaults/low_profile_base.conf
  /defaults/low_security_base.conf
  /defaults/policy_base.conf
  /defaults/wam_base.conf
  /defaults/analytics_base.conf
  /defaults/apm_base.conf
  /defaults/apm_oauth_base.conf
  /defaults/apm_saml_base.conf
  /defaults/app_template_base.conf
  /defaults/classification_base.conf
  /var/libdata/dpi/conf/classification_update.conf
  /defaults/ips_base.conf
  /var/libdata/ips/ips_update.conf
  /defaults/daemon.conf
  /defaults/pem_base.conf
  /defaults/profile_base.conf
  /defaults/sandbox_base.conf
  /defaults/security_base.conf
  /defaults/urldb_base.conf
  /usr/share/monitors/base_monitors.conf
  /defaults/cipher.conf
  /defaults/ilx_base.conf
Loading configuration...
Loading schema version: 13.1.1.4
  /config/bigip_base.conf
  /config/bigip_user.conf
  /config/bigip.conf
Loading schema version: 13.1.3
01071008:3: Provisioning failed with error 1 - 'Disk limit exceeded. 16188 MB are required to provision these modules, but only 15752 MB are available.'
.
Unexpected Error: Loading configuration process failed.
[root@txsat1slbco22:/S3-red-P::Offline:Disconnected] config # 2019 Aug 14 13:05:35 slot3/txsat1slbco22 overdog[3111]: 01140043:0: Ha feature software_update reboot requested.

I have four cores assigned to the vCMP Guest which per F5 that should be plenty for three modules provisioned.

txsat1slbco22_viprion.jpg

txsat1slbco22_viprion_resources.jpg

As a test I removed APM (especially since I don't use it yet so no risk) and then clicked System - Configuration - General - Reboot Guest Chassis to reboot and save the config first. 

Then reboot back to the partition (System - Software Management - Boot Locations and click HD1.1 or whatever the partition is that has the new code)  see if the issue goes away.

SAME issue..  CRAP!

My next attempt is to reduce the amount of images/partitions on the drive to just two versus three.  Run the following command to show you how many images/partitions you have

tmsh show sys software
---------------------------------------------------------
Sys::Software Status
Volume  Slot  Product   Version   Build  Active    Status
---------------------------------------------------------
HD1.1      3   BIG-IP    13.1.3   0.0.6     yes  complete
HD1.2      3   BIG-IP  12.1.3.5  0.0.10      no  complete
HD1.3      3   BIG-IP  13.1.1.4   0.0.4      no  complete

---------------------------
Sys::Software Update Check
---------------------------
  Check Enabled        true
  Phonehome Enabled    true
  Frequency          weekly
  Status               none
  Errors                  0

My goal is to always keep two versions. The working one and the future one...  always have 1 rollback.  So in this case we are running 13.1.1.4 and "trying" to upgrade to 13.1.3 so we don't need HD1.2 so lets remove it in order to free up some space.  The below is the command I ran to remove HD1.2

tmsh delete /sys software volume HD1.2

Then double check by re-running your show sys software command

tmsh show sys software

--------------------------------------------------------
Sys::Software Status
Volume  Slot  Product   Version  Build  Active    Status
--------------------------------------------------------
HD1.1      3   BIG-IP    13.1.3  0.0.6     yes  complete
HD1.3      3   BIG-IP  13.1.1.4  0.0.4      no  complete

---------------------------
Sys::Software Update Check
---------------------------
  Check Enabled        true
  Phonehome Enabled    true
  Frequency          weekly
  Status               none
  Errors                  0

Okay now let's just do a quick verify loading the config

tmsh load sys config verify

Validating system configuration...
  /defaults/asm_base.conf
  /defaults/config_base.conf
  /defaults/ipfix_ie_base.conf
  /defaults/ipfix_ie_f5base.conf
  /defaults/low_profile_base.conf
  /defaults/low_security_base.conf
  /defaults/policy_base.conf
  /defaults/wam_base.conf
  /defaults/analytics_base.conf
  /defaults/apm_base.conf
  /defaults/apm_oauth_base.conf
  /defaults/apm_saml_base.conf
  /defaults/app_template_base.conf
  /defaults/classification_base.conf
  /var/libdata/dpi/conf/classification_update.conf
  /defaults/ips_base.conf
  /var/libdata/ips/ips_update.conf
  /defaults/daemon.conf
  /defaults/pem_base.conf
  /defaults/profile_base.conf
  /defaults/sandbox_base.conf
  /defaults/security_base.conf
  /defaults/urldb_base.conf
  /usr/share/monitors/base_monitors.conf
  /defaults/cipher.conf
  /defaults/ilx_base.conf
Validating configuration...
Loading schema version: 13.1.1.4
  /config/bigip_base.conf
  /config/bigip_user.conf
  /config/bigip.conf
  /config/bigip_script.conf
Loading schema version: 13.1.3
There were warnings:
/Common/f5.bigiq-analytics definition:130: warning: [use curly braces to avoid double substitution][($start_hour]
/Common/f5.bigiq-analytics definition:131: warning: [use curly braces to avoid double substitution][($end_hour]
/Common/f5.bigiq-analytics definition:133: warning: [use curly braces to avoid double substitution][$end_minute]
/Common/f5.bigiq-analytics definition:141: warning: [use curly braces to avoid double substitution][$start_random]
/Common/f5.bigiq-analytics definition:145: warning: [use curly braces to avoid double substitution][round("00.[lindex $start_random 1]"]
/Common/f5.bigiq-analytics definition:2189: warning: [use curly braces to avoid double substitution][$nonpriority]
/Common/f5.bigiq-analytics definition:2192: warning: [use curly braces to avoid double substitution][$nonpriority]
/Common/f5.bigiq-analytics definition:2249: warning: [use curly braces to avoid double substitution][$::time]
/Common/f5.bigiq-analytics definition:2290: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2363: warning: [use curly braces to avoid double substitution][($start_hour]
/Common/f5.bigiq-analytics definition:2363: warning: [use curly braces to avoid double substitution][($current_hour]
/Common/f5.bigiq-analytics definition:2364: warning: [use curly braces to avoid double substitution][$time]
/Common/f5.bigiq-analytics definition:2529: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2531: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2547: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2549: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2560: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2562: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2871: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2917: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2920: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2927: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2929: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:2968: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:3012: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:3016: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:3023: warning: [use curly braces to avoid double substitution][$::uniqueid]
/Common/f5.bigiq-analytics definition:3025: warning: [use curly braces to avoid double substitution][$::uniqueid]

curly braces error isn't critical and technically shouldn't be the reason we can't load the config so let's try and load the config.  We do know based on the verify that it does appear to be making it further than before so I believe removing that partition may have solved the issue

tmsh load /sys config

Command line is looking positive

(txsat1slbco21)(cfg-sync Disconnected)(/S3-green-P::Standby)(/Common)(tmos)

Let's log into the GUI to make sure all is good....

DAMN it!  Have to re-activate license on the Viprion chassis which will affect every vCMP Guest you have on that chassis.

txsat1slbco22_viprion_not-operational.jpg

Share this post


Link to post
Share on other sites

×
×
  • Create New...