Hosting

Overview of Persistence Types Reference: https://my.f5.com/manage/s/article/K26898044 Depending on session type, there are several persistence methods to choose from. These are the supported persistence methods in F5 Networks BIG-IP units:   Cookie persistence   Cookie persistence uses the HTTP cookie header to persist connections across a session.  This technique prevents the issues associated with simple persistence because the session ID is unique.   Destination address affinity persistence   Also known as sticky persistence, destination address affinity persistence supports TCP and UDP protocols, and directs session requests to the same server based solely on the destination IP address of a packet.   Hash persistence   Hash persistence allows you to create a persistence hash based on an existing hash persistence profile. Using hash persistence is the same as using universal persistence, except that with hash persistence, the resulting persistence key is a hash of the data, rather than the data itself. A hash value may be created based on source IP, destination IP, and destination port. While not necessarily unique to every session, this technique results in a more even distribution of load across servers.   You cannot associate hash persistence with a virtual server that is managing Fast L4 traffic; use of hash persistence for Fast L4 traffic is disallowed. Host persistence   Host persistence allows the BIG-IP system to use the HTTP Host header passed in an HTTP request to determine which pool member to choose. You can also activate host persistence from within an iRule.   Microsoft Remote Desktop Protocol persistence Microsoft Remote Desktop Protocol (MSRDP) persistence tracks sessions between clients and servers running the Microsoft Remote Desktop Protocol (RDP) service. SIP persistence SIP persistence is an application-specific type of persistence used for servers that receive Session Initiation Protocol (SIP) messages sent through UDP, SCTP, or TCP. You generally use this persistence technique with stateful applications that depend on the client being connected to the same application instance throughout the life of the session. Source address affinity persistence Also known as simple persistence, source address affinity persistence supports TCP and UDP protocols, and directs session requests to the same server based solely on the source IP address of a packet. SSL persistence Because SSL sessions need to be established and are very much tied to a session between client and server, failing to persist SSL-secured sessions results in renegotiation of the session. BIG-IP system uses the SSL session ID to ensure that a session is properly routed to the application instance to which the session first connected. Even when the client's IP address changes, the BIG-IP system still recognizes the connection as being persistent based on the session ID. Universal persistence Universal persistence uses any piece of data (network, application protocol, payload) to persist a session. This technique requires the BIG-IP system to be able to inspect and ultimately extract any piece of data from a request or response.  With universal persistence, you can write an expression that defines the data that the BIG-IP system will persist on in a packet.     Cookie Persistence Reference: https://my.f5.com/manage/s/article/K6917 When you configure a cookie persistence profile to use the HTTP Cookie Insert or HTTP Cookie Rewrite method, the BIG-IP system inserts a cookie into the HTTP response, which well-behaved clients include in subsequent HTTP requests for the host name until the cookie expires. The cookie, by default, is named BIGipServer<pool_name>. The cookie is set to expire based on the expiration setting configured in the persistence profile. The cookie value contains the encoded IP address and port of the destination server. Reference: https://my.f5.com/manage/s/article/K83419154  

On your non-F5 that you plan on storing the files run this command with no passphrase (this is a RHEL 7 box for me) ssh-keygen -t rsa Now copy the public key to the F5 ssh-copy-id -i ~/.ssh/id_rsa.pub root@bigip.fqdn Create a .txt file with all your F5 devices listed in it using FQDN that is resolvable (only one per line) vi bigip_devices.txt Create a file on the linux box that you will be backing up configs to vi bigip_backup.sh Copy the below and paste in that new file #!/bin/sh ## Shell script created by CowboyDenny @ MyWiseGuys ## PRE_REQ ## ssh-copy-id -i ~/.ssh/id_rsa.pub root@bigip ## TEST-VERIFY: ssh root@bigip <--no password login means success ###### SYNTAX to RUN: ./bigip_backup.sh bigip_devices.txt [daily|weekly] cat $1 | while read REMOTE_BIGIP || [[ -n $REMOTE_BIGIP ]]; do start=$SECONDS echo "STARTING with $REMOTE_BIGIP" DATETIME="`date +%Y%m%d_%H%M`" REMOTE_PATH='/var/tmp' LOCAL_PATH="/home/confback/backups/f5" FILE_UCS="$(echo f5_daily_backup_$REMOTE_BIGIP | cut -d'.' -f1)-${DATETIME}.ucs" FILE_SCF="$(echo f5_daily_backup_$REMOTE_BIGIP | cut -d'.' -f1)-${DATETIME}.scf" FILE_CERT="$(echo f5_daily_backup_$REMOTE_BIGIP | cut -d'.' -f1)-${DATETIME}.cert.tar" start=$SECONDS if [ $# -eq 0 ]; then echo "$0: Missing BIGIP FQDN - Try Running again: .bigip_backup.sh bigip_devices.txt" exit 1 elif [ $# -gt 2 ]; then echo "$0: Too many arguments: $@" exit 1 else echo "==================================================================" echo "filename........: $1" echo "REMOTE_BIGIP....: $REMOTE_BIGIP" echo "DATETIME........: $DATETIME" echo "REMOTE_PATH.....: $REMOTE_PATH" echo "LOCAL_PATH......: $LOCAL_PATH" echo "FILE_UCS........: $FILE_UCS" echo "FILE_SCF........: $FILE_SCF" echo "FILE_CERT.......: $FILE_CERT" echo "==================================================================" echo "Variable are SET" echo "" fi #DAILY echo "Do we have a UCS backup from today? Checking..." echo "" ssh -n $REMOTE_BIGIP find $REMOTE_PATH/f5_daily_backup_*.ucs -mtime -1 -ls > /dev/null if [ $? -eq 0 ]; then echo "$0: UCS exists so let's just download it" else echo "`date +%Y%m%d_%H.%M.%S`: saving config" ssh -n $REMOTE_BIGIP tmsh save /sys config > /dev/null echo "`date +%Y%m%d_%H.%M.%S`: creating UCS backup" ssh -n $REMOTE_BIGIP tmsh save /sys ucs $REMOTE_PATH/$FILE_UCS > /dev/null echo "....done with UCS...." fi # echo "`date +%Y%m%d_%H.%M.%S`: copy UCS backup" # scp -v $REMOTE_BIGIP:$REMOTE_PATH/f5_daily_backup_*.ucs $LOCAL_PATH/ > /dev/null # echo "`date +%Y%m%d_%H.%M.%S`: remove UCS backup to save room" # ssh -n $REMOTE_BIGIP rm -f $REMOTE_PATH/f5_daily_backup_*.ucs # echo "....done with UCS...." echo "" #WEEKLY (roughly 12min per device to backup) #echo "`date +%Y%m%d_%H.%M.%S`: creating SCF file" #ssh -n $REMOTE_BIGIP tmsh save /sys config file $REMOTE_PATH/$FILE_SCF no-passphrase > /dev/null #echo "`date +%Y%m%d_%H.%M.%S`: copying SCF file" #scp $REMOTE_BIGIP:$REMOTE_PATH/f5_daily_backup_*.scf* $LOCAL_PATH/ #echo "`date +%Y%m%d_%H.%M.%S`: remove SCF file(s) to save room" #ssh -n $REMOTE_BIGIP rm -f $REMOTE_PATH/f5_daily_backup_*.scf* #echo "....done with SCF...." #echo "" #echo "`date +%Y%m%d_%H.%M.%S`: compressing SSL CERTs" #ssh -n $REMOTE_BIGIP tar -cf "${REMOTE_PATH}/${FILE_CERT}" /config/ssl #echo "`date +%Y%m%d_%H.%M.%S`: copying CERT compressed file" #scp $REMOTE_BIGIP:$REMOTE_PATH/f5_daily_backup_*.cert.tar $LOCAL_PATH/ #echo "`date +%Y%m%d_%H.%M.%S`: remove CERT file to save room" #ssh -n $REMOTE_BIGIP rm -f $REMOTE_PATH/f5_daily_backup_*.cert* #echo "....done with CERT...." #echo "" #GENERAL echo "get $REMOTE_BIGIP:$REMOTE_PATH" | sftp $REMOTE_BIGIP:$REMOTE_PATH/f5_daily_backup_*.* $LOCAL_PATH <<EOF EOF echo "`date +%Y%m%d_%H.%M.%S`: time to cleanup created files and rpm-tmp files" ssh -n $REMOTE_BIGIP rm -f $REMOTE_PATH/{rpm-tmp.*,f5_daily_backup_*.*} echo "" echo "FINISHED with $REMOTE_BIGIP now exiting" duration=$(( SECONDS - start )) echo "Duration(seconds): $duration" echo "Duration(minutes): $(( $duration / 60))" done echo "Cleaning up any backup files older than 30 days on RHEL storage" /usr/bin/find $LOCAL_PATH -type f -mtime +31 -exec rm -f {} \; echo "`date +%Y%m%d_%H.%M.%S`: FINISHED backups" Since I have a horrible memory I won't remember to run this daily so I'm going to leverage a cronjob on the non-F5 Linux box that I'm storing the files and running the script from.  This is how I setup the cronjob crontab -e Now paste the following (of course alter path to where you are storing your shell script you just created above) if you want this to backup files at 4am everyday 0 4 * * * /home/confback/backups/bigip_backup.sh /home/confback/backups/bigip_devices.txt > /home/confback/backups/bigip_backup_`date +20\%y\%m\%d_\%H\%M\%S`.log After it runs (or even while its running) you can checkout the progress by going to the .log file it creates each day.  Using the above example the .log file would be found /home/confback/backups/ Hope this helps someone out..  I know it worked great for our application.

Having issues trying to pass the password in the copy to remote file part so trying the python way like shown here Create file on your remote server vi f5-backup.py Copy and paste the following #! /usr/bin/env python # -*- coding: utf-8 -*- import os import json import datetime import requests import getpass import optparse import sys import hashlib from urllib3.exceptions import InsecureRequestWarning # Root CA for SSL verification ROOTCA = '' CHECKSUM = '' HOSTNAME = '' STATUS = False # credential Ask for user Active Directory authentication information # with a verification of entered password def credential(): #User name capture user = input('Enter Active Directory Username: ') # start infinite loop while True: # Capture password without echoing pwd1 = getpass.getpass('%s, enter your password: ' % user) pwd2 = getpass.getpass('%s, re-Enter Password: ' % user) # Compare the two entered password to avoid typo error if pwd1 == pwd2: # break infinite loop by returning value return user, pwd1 # get_token() will call F5 Big-ip API with username and password to obtain an authentication # security token def get_token(session): # Build URL URL_AUTH = 'https://%s/mgmt/shared/authn/login' % HOSTNAME # Request user credential username, password = credential() # prepare payload for request payload = {} payload['username'] = username payload['password'] = password payload['loginProviderName'] = 'tmos' # set authentication to username and password to obtain the security authentication token session.auth = (username, password) # send request and handle connectivity error with try/except try: resp = session.post(URL_AUTH, json.dumps(payload)).json() except: print("Error sending request to F5 big-ip. Check your hostname or network connection") exit(1) # filter key in response. if 'code' key present, answer was not a 200 and error message with code is printed. for k in resp.keys(): if k == 'code': print('security authentication token creation failure. Error: %s, Message: %s' % (resp['code'],resp['message'])) exit(1) # Print a successful message log and return the generated token print('Security authentication token for user %s was successfully created' % resp['token']['userName']) return resp['token']['token'] # create_ucs will call F5 Big-ip API with security token authentication to create a timestamps ucs backup # file of the F5 Big-ip device configuration def create_ucs(session): URL_UCS = 'https://%s/mgmt/tm/sys/ucs' % HOSTNAME # generate a timestamp file name ucs_filename = HOSTNAME + '_' + datetime.datetime.now().strftime('%Y-%m-%d-%H%M%S') + '.ucs' # prepare the http request payload payload = {} payload['command'] = 'save' payload['name'] = ucs_filename # send request and handle connectivity error with try/except try: resp = session.post(URL_UCS, json.dumps(payload)).json() except: print("Error sending request to F5 big-ip. Check your hostname or network connection") exit(1) # filter key in response. if 'code' key present, answer was not a 200 and error message with code is printed. for k in resp.keys(): if k == 'code': print('UCS backup creation failure. Error: %s, Message: %s' % (resp['code'],resp['message'])) exit(1) # Print a successful message log print("UCS backup of file %s on host %s successfully completed" % (resp['name'], HOSTNAME)) return ucs_filename, checksum(session, ucs_filename) def checksum(session, filename): URL_BASH = 'https://%s/mgmt/tm/util/bash' % HOSTNAME # prepare the http request payload payload = {} payload['command'] = 'run' payload['utilCmdArgs'] = '''-c "sha256sum /var/local/ucs/%s"''' % filename # send request and handle connectivity error with try/except try: resp = session.post(URL_BASH, json.dumps(payload)).json()['commandResult'] except: print("Error sending request to F5 big-ip. Check your hostname or network connection") exit(1) checksum = resp.split() return checksum[0] # delete_ucs will call F5 Big-ip API with security token authentication to delete the ucs backup # file after local download def delete_ucs(session, ucs_filename): URL_BASH = 'https://%s/mgmt/tm/util/bash' % HOSTNAME # prepare the http request payload payload = {} payload['command'] = 'run' payload['utilCmdArgs'] = '''-c "rm -f /var/local/ucs/%s"''' % ucs_filename # send request and handle connectivity error with try/except try: session.post(URL_BASH, json.dumps(payload)).json() except: print("Error sending request to F5 big-ip. Check your hostname or network connection") exit(1) def ucsDownload(ucs_filename, token): global STATUS # Build request URL URL_DOWNLOAD = 'https://%s/mgmt/shared/file-transfer/ucs-downloads/' % HOSTNAME # Define chunck size for UCS backup file chunk_size = 512 * 1024 # Define specific request headers headers = { 'Content-Type': 'application/octet-stream', 'X-F5-Auth-Token': token } # set filename and uri for request filename = os.path.basename(ucs_filename) uri = '%s%s' % (URL_DOWNLOAD, filename) requests.packages with open(ucs_filename, 'wb') as f: start = 0 end = chunk_size - 1 size = 0 current_bytes = 0 while True: content_range = "%s-%s/%s" % (start, end, size) headers['Content-Range'] = content_range #print headers resp = requests.get(uri, headers=headers, verify=False, stream=True) if resp.status_code == 200: # If the size is zero, then this is the first time through the # loop and we don't want to write data because we haven't yet # figured out the total size of the file. if size > 0: current_bytes += chunk_size for chunk in resp.iter_content(chunk_size): f.write(chunk) # Once we've downloaded the entire file, we can break out of # the loop if end == size: break crange = resp.headers['Content-Range'] # Determine the total number of bytes to read if size == 0: size = int(crange.split('/')[-1]) - 1 # If the file is smaller than the chunk size, BIG-IP will # return an HTTP 400. So adjust the chunk_size down to the # total file size... if chunk_size > size: end = size # ...and pass on the rest of the code continue start += chunk_size if (current_bytes + chunk_size) > size: end = size else: end = start + chunk_size - 1 if sha256_checksum(ucs_filename) == CHECKSUM: STATUS = True def sha256_checksum(filename, block_size=65536): sha256 = hashlib.sha256() with open(filename, 'rb') as f: for block in iter(lambda: f.read(block_size), b''): sha256.update(block) return sha256.hexdigest() def f5Backup(hostname): global STATUS, CHECKSUM,HOSTNAME counter = 0 HOSTNAME = hostname # Disable SSL warning for Insecure request requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning) # create a new https session session = requests.Session() # update session header session.headers.update({'Content-Type': 'application/json'}) # Disable TLS cert verification if ROOTCA == '': session.verify = False else: session.verify = ROOTCA # set default request timeout session.timeout = '30' # get a new authentication security token from F5 print('Start remote backup F5 big-Ip device %s ' % HOSTNAME) token = get_token(session) # disable username, password authentication and replace by security token # authentication in the session header session.auth = None session.headers.update({'X-F5-Auth-Token': token}) # create a new F5 big-ip backup file on the F5 device print('Creation UCS backup file on F5 device %s' % HOSTNAME) ucs_filename, CHECKSUM = create_ucs(session) # locally download the created ucs backup file #download_ucs(session, ucs_filename) while not STATUS: print("Download file %s attempt %s" % (ucs_filename, counter+1)) ucsDownload(ucs_filename, token) counter+=1 if counter >2: print('UCS backup download failure. inconscistent' \ 'checksum between origin and destination') print('program will exit and ucs file will not be deleted from F5 device') exit(1) print('UCS backup checksum verification successful') # delete the ucs file from f5 after local download # to keep f5 disk space clean delete_ucs(session, ucs_filename) if __name__ == "__main__": # Define a new argument parser parser=optparse.OptionParser() # import options parser.add_option('--hostname', help='Pass the F5 Big-ip hostname') # Parse arguments (opts,args) = parser.parse_args() # Check if --hostname argument populated or not if not opts.hostname: print('--hostname argument is required.') exit(1) f5Backup(opts.hostname) Save the file then run it with the following syntax python3 f5-backup.py --hostname <fqdn_f5_appliance>  

Here is a script that may help you backup your F5 to a remote server on a regular basis when you don't want to use the F5 tool BIG-IQ Create file vi /var/tmp/script_backup.sh Make file executable chmod 755 /var/tmp/script_backup.sh Copy and Paste the following to the new file TFTP_SERVER=10.0.0.0 DATETIME="`date +%Y%m%d%H%M`" OUT_DIR='/var/tmp' FILE_UCS="f5_lan_${HOSTNAME}.ucs" FILE_SCF="f5_lan_${HOSTNAME}.scf" FILE_CERT="f5_lan_${HOSTNAME}.cert.tar" cd ${OUT_DIR} tmsh save /sys ucs "${OUT_DIR}/${FILE_UCS}" tmsh save /sys config file "${OUT_DIR}/${FILE_SCF}" no-passphrase tar -cf "${OUT_DIR}/${FILE_CERT}" /config/ssl tftp $TFTP_SERVER <<-END 1>&2 mode binary put ${FILE_UCS} put ${FILE_SCF} put ${FILE_CERT} quit END rm -f "${FILE_UCS}" rm -f "${FILE_SCF}" rm -f "${FILE_CERT}" rm -f "${FILE_SCF}.tar" RTN_CODE=$? exit $RTN_COD Once your script runs successfully go ahead and add it to your crontab so it runs on a regular basis crontab -e 30 0 * * 6 /var/tmp/script_backup.sh Now what I would like to do is..... Have a script on my remote server that would run with a cronjob and this script would: connect to BIG-IP copy script up run script to create files copy down files to server storing files cleanup files Go to next BIG-IP in list

This is very awesome and saved me a ton of time since the GUI you can only export one at a time which exports into XML just like this does. Now the real question is.... Do you have a script to import all the xml files?