HiCHperfpkg Author: Maik Ernst (maik.ernst@hds.com) Date: 2016-04-02 Version: 7.3.1 Consists of: bin/ppsetup.pl v1.2.1 bin/ppcollect.pl v4.3.1 bin/pptransfer.pl v2.2.1 bin/ppmonitor.pl v1.5.0 lib/modules/HiCHpp_RAIDCOM.pm v2.1.0 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Previous versions: Date: 2016-03-27 Version: 7.3.0 Consists of: bin/ppsetup.pl v1.2.0 bin/ppcollect.pl v4.3.0 bin/pptransfer.pl v2.2.0 bin/ppmonitor.pl v1.5.0 lib/modules/HiCHpp_RAIDCOM.pm v2.1.0 Date: 2015-08-30 Version: 7.2.0 bin/ppsetup.pl v1.1.2 bin/ppcollect.pl v4.2.0 bin/pptransfer.pl v2.1.0 bin/ppmonitor.pl v1.4.0 lib/modules/HiCHpp_RAIDCOM.pm v2.0.0 Date: 2015-07-24 Version: 7.1.3 bin/ppsetup.pl v1.1.2 bin/ppcollect.pl v4.1.2 bin/pptransfer.pl v2.1.0 bin/ppmonitor.pl v1.4.0 lib/modules/HiCHpp_RAIDCOM.pm v2.0.0 ############################################################################ Modification history bin/ppsetup.pl # MODIFICATIONS # 2015-06-17 - v1.0.0 - initial script # 2015-07-02 - v1.0.1 - Fixing Gx00 bug (ip instead of svpip in cmd.txt and other order cmd.txt) # 2015-07-02 - v1.0.1 Don't allow "." and " " in script path. Disable raidcom for Gx00 # 2015-07-03 - v1.1.0 - Check for controller IPs on Gx00 (GUM) and allow raidcom. # 2015-07-04 - v1.1.1 - Add " to command.txt outpath # 2015-07-24 - v1.1.2 - Changing default from "ppmonitor_switch = on" to "ppmonitor_switch = off" # 2016-03-25 - v1.2.0 - LOG->autoflush(1), format change, adding CFGFILE pptransfer_proxyoptions # - transfer_https_pxy_user and transfer_https_pxy_pwd check (@ and & not allowed) # - adding ppcollect_keepdays, pptransfer_keepdays, ppmonitor_keepdays # - adding DO_UPGRADE (change config files) # 2016-04-02 - v1.2.1 - Change screenoutput 7.3.0 to 7.3.x bin/ppcollect.pl # MODIFICATIONS # 2013-07-09 - v1.0 - initial script # 2013-07-09 - v1.1 - changed from only tar output file to tar/gzip # 2013-07-10 - v1.2 - Adding VSP/USPV/HUSVM switch for correct java export tool command # Deleting existing outpath before export tool start to avoid issue if the script got aborted (e.g. strg-c) # Adding script version information # Set -w automatically when osname is *Win* # Set script start time to variable SHORTRANGE if getting Shortrange was not possible from logfile # Bugfix sort on modified time of export tool log file # 2013-07-23 - v1.3 - Changing default in command txt files to shortrange of 12hours (shortrange -1200:). # The reason for this is that you get partially invalid values (-4) in LU_dat as the collection period # does not match the avaiable data (max 24 hours) when the collection takes too long. # 2013-09-20 - v1.4 - Adding longrange -600000 to all command.txt # 2013-09-25 - v1.5 - Removing longrange -600000 from all command.txt # 2013-10-03 - v1.6 - Adding error output of EXPORTTOOL_RUN # 2013-12-05 - v1.7 - Change command.txt settings from "PhyPG" to "PhyPG Short" and "PhyLDEV" to "PhyLDEV Short" # The reason for this is to shorten the runtime of the export tool # 2014-01-06 - v1.8 - Changing die-function to be logged in file # 2014-02-07 - v2.0 - raidcom collection added => use HDSPERF_RAIDCOM.pm module # 2014-02-07 - v2.1 - adding FindBin, adding HORCM_DIR absolute path due to issue with starting the instance # 2014-02-17 - v2.2 - adding parsable file (semicolon delimiter) from raidcom output # 2014-02-21 - v2.3 - add raidcom collection of parity groups # 2014-04-07 - v2.4 - add raidcom collection of pool and dp_pool # Adding "short" to all long range groups in all command.txt that only short range gets collected. # The reason for this is to shorten the runtime. # 2014-04-07 - v3.0 - Adding G1000 support # Changing absolute path handling (=> abs_path) + slashes to backslash as path # Adding -m at raidcom collection to get all MUs for raidscan # But only for TC and not for SI as the collection would take too long. # 2014-06-13 - v3.1 - Adding 7zip support for archiving with -z. This is to avoid high memory usage if the server has not enough memory to archive. # (When you get error message: "out of memory") # 2014-12-26 - v3.2 - Adding log file output # 2015-04-02 - v3.3 - Adding -f option to use FC device instead of IP command device # Adding -o option to collect only raidcom data and skip export tool data collection # Adding Sys::Hostname to get hostname of script # 2015-04-30 - v3.4 - Using export tool JSanRmiApiSx instead of JSanRmiApiEx for G1000 # 2015-05-02 - v4.0 - Change from parameter to config file => pm file config::tiny needed # Changing script name from HDSperfcollect.pl to HCHppcollect.pl to meet swiss script standard # 2015-05-05 - Bugfix (missing 3 in in CDM file for G1000 when using FC command device) # 2015-05-10 - v4.1.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group # (tgz only works but only with absolute pahts => do not use it anymore) # Log format changed (LOG_MSG) # Adding new parameter like ppcollect_ccicollectionlevel # 2015-07-02 - v4.1.1 - Allow G200, G400, G600 and G800 as parameter # Adding ip-regexp from cmd.txt for G800/G600/G400/G200 (ip instead of svpip) # Adding GUM ip regexp from cmd.txt for G800/G600/G400/G200 # 2015-07-05 - v4.1.2 - Bugfix regexp to get outpath correctly from cmd.txt # Bugfix unquote outpath # 2015-08-29 - v4.2.0 - Implement G1000 MC v3 export toool changed # (also for G800/G600/G400/G200 as assumption that it will be the same soon) # Change variable to hash for export tool # 2016-03-25 - v4.3.0 - LOG->autoflush(1), fix DELETE_OLD_SAVES DELOD log message # - adding DELETE_OLD_LOGS / log file rotation ppcollect_logkeepdays # 2016-04-02 - v4.3.1 - Change log output PWD = *** bin/pptransfer.pl # MODIFICATIONS # 2013-09-26 - v1.0 - initial script # 2013-10-02 - v1.1 - Adding spool directory equal to transfer directory # Adding -e entry # 2014-01-06 - v1.2 - Changing die-function to be logged in file # Changing ADD_SPOOLLIST_NEW function. # NEW: Adding file to spool list txt file AFTER sucessful scp # 2014-01-06 - v1.3 - Adding -r entry (rotate option) # With this entry the data in directory defined with -d will be deleted after specified days. # 2014-02-20 - v1.4 - Change default ioportal target to ftp.hdsioportal.com # Adding plink.exe command at the beginning of every transfter to make sure not to have keyprint issue # This could be the case if different users are starting the transfer # Fixing file to spool list will only be added if exit code of scp is 0 which means sucessful # 2014-06-17 - v1.5 - Adding multiple suffix support (=> 7zip support) # 2014-12-26 - v1.6 - Adding log file output # 2015-04-05 - v1.7 - Adding "log" to suffix list that "scriptlog" dir can be transfered # Adding Sys::Hostname to get hostname of script # 2015-04-20 - v1.8 - Changing default $REMOTE_HOST to "ftp.hdsioportal.com" # 2015-05-01 - v2.0 - Change from parameter to config file => pm file config::tiny needed # Adding https via curl as transfer method # Changing script name from HDSperftransfer.pl to HCHpptransfer.pl to meet swiss script standard # 2015-05-01 - v2.1.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group # Log format changed (LOG_MSG) # Debuglevel, CURL parameter change # 2016-03-25 - v2.2.0 - adding LOG->autoflush(1) # - adding pptransfer_proxyoptions # - changing CURL syntax to pptransfer_proxyoptions # - addicng check HTTPS_PROXYUSER / HTTPS_PROXYPWD (@ and & not allowed) # - adding DELETE_OLD_LOGS / log file rotation pptransfer_logkeepdays # 2016-04-02 - v2.2.1 - Change log output CURL RUN USER PWD = *** bin/ppmonitor.pl # MODIFICATIONS # 2014-01-03 - v1.0 - initial script # 2014-06-16 - v1.1 - Moving up to folder ppmonitor. # Adding .zip as suffix to check # Adding pptransfer checks (-i). # 2014-04-07 - v1.2 - Adding Sys::Hostname to get hostname of script # 2015-05-02 - v1.3 - Changing script name from ppmonitor.pl to ppmonitor.pl to meet swiss script standard # Adding CURL transfer check # 2015-05-10 - v1.4.0 - Reorganize folder structure for HiCHperfpkg according to Swiss development group # 2016-03-25 - v1.5.0 - LOG->autoflush(1), fix logfile checks # - adding DELETE_OLD_LOGS / log file rotation pptransfer_logkeepdays lib/modules/HiCHpp_RAIDCOM.pm # MODIFICATIONS # 2014-02-07 - v1.0 - initial module # 2014-02-11 - v1.1 - adding -CLI to raidscan # 2014-02-14 - v1.2 - adding CHECK_OLD_INSTANCE # 2014-02-16 - v1.3 - add an output file of raidcom which is parsable with semicolon as delimiter # This is quite difficult as the current delimiter is a space and the values can have spaces also. # Not every output of a values is known therefore there are a lot of guesses if strings,digits,etc. # The following values has been testes and they can have one or more spaces in their output: # meta_resource, pool name, host group name, HMO # For now it looks it is working but troubleshooting within the regex might be needed. # If you experience issue please let me know. # 2014-02-21 - v1.4 - add raidcom collection of parity groups # 2014-04-07 - v1.5 - adding -fx to every collection # - add raidcom get pool, raidcom get dp_pool -key opt # 2014-05-22 - v1.6 - Adding -m at raidcom collection to get all MUs for raidscan. # But only for TC and not for SI as the collection would take too long. # 2015-04-02 - v1.7 - Adding -f at HDSperfcollect to use FC device instead of IP command device # 2015-05-02 - v1.8 - Changing module name from HDSPERF_RAIDCOM.pm to HCHpp_RAIDCOM.pm to meet swiss script standard # 2015-05-05 Bugfix (missing 3 in in CDM file for G1000 when using FC command device) # 2015-05-05 - v1.9 - add raidcom level to disable some collections. # 0 = collect all, 1 = disable raidscan, 2 = disable raidscan and host_grp # change log function # 2015-07-03 - v2.0.0 - adding raidcom support for G800,G600,G400,G200$ (GUM IPs instead of SVP IP for these systems) # 2016-03-25 - v2.1.0 - adding raidcom get dp_pool, fixing CMD_RAIDCOM_GET_POOL_OPT, changing randcom from 200 to 500 for horcm instance # - delete old horcm log dir of horcm instance # - adding 2 / 3 options for CMD_RAIDCOM_GET_POOL_OPT and CMD_RAIDCOM_GET_DPPOOL