Find it...

    Pexip Log Tools

    Follow

    General Introduction and use in Microsoft Windows.

    In attempting to make the offline snapshots more readable, I have put together a repackaged Microsoft Windows application, which includes multiple other free third-party tools (such as 7Zip, Python, OpenSSL, Notepad++, grepWin etc.), a few PowerShell scripts (all signed), and Python scripts from my learned colleagues. During installation, the scripts are placed into the c:\tools\scripts folder, with a shortcut added to the users Send To folder. This allows the administrator to simply right click on a snapshot or downloaded log snippet, and “Send To” the Pexip log tools so create some summary reports.

    The script will then extract snapshot files to C:\Snaps, create summarised call reports of the first 20 support log files that appear in the snapshot plus a system overview, then display these in a Notepad++ tab. If you send a downloaded support log snippet to the Pexip log tools, the script will create a call summary for that file, which gets placed into a .\parsed folder in the current working folder. At the same time, the log files that are processed are translated into more human readable versions, similar to how you see the support logs via the Pexip GUI interface.

    So far I have only built this for Windows platforms, and I am NO developer….. This started life as a script to help me with my day to day job, but as a lifelong Windows user I have been “GUI’fied” (hey you know what you know 😊), so rather than use the CLI with tools such as grep and sed, I created these scripts to make things easier for me. It turns out that it may make your life as a Pexip administrator simpler too.

    Essentially, it helps make the Log files readable and was designed primarily to help me analyse call flows – of course the first part to analysing them is to make them readable, but it won’t necessarily help beyond this ;). If you are interested in troubleshooting within a Pexip Infinity deployment, and how Pexip support personnel use these logs to determine issues, then please sign up for the Troubleshooting and Logreading class at https://www.pexip.com/pexip-academy-and-training-programs. Please ensure you read the description of the course and the pre-requisites to make sure this is for you.

    In the expanded snapshot, there are multiple other files that may need unconcatenating (as each log line is joined with a ^M making the raw file difficult to read). For example, other support log files not included in the initial 20 that are analysed (you can use grepWin (installed during setup) as a GUI grep tool to search through the files for specific Call-IDs, then process all of them at once), or the OSStatus log (that contains 5 minute chunks of TOP, Netstat etc., from each of the nodes). Each of these can be processed the same way, by simply right clicking and sending to the Pexip Log Tools.

    To download, install and use the tools, see:

     

    To Update the underlying scripts (in order to better handle newer Pexip versions):

    • Run the following script file: 
      c:\tools\scripts\Get-Pexscripts.ps1
    • WARNING: this will break the script to shows the call reports (logreader). You need to then run the following to install some additional Python libraries: 
      pip install -I lxml
    • BONUS: If you want to show some additional good stuff in the system overview report (DBsummary) in relation to what certificates are installed an applied to the nodes, please run the following to install some more Python libraries: 
      pip install -I pyopenssl

     

    Known Issues:

    • The applications require local administrative privileges in the Windows system for installation.
    • The PowerShell scripts require a minimum of PowerShell version 4 to run. Windows 7 shipped with PowerShell version 2 (which is positively ancient) and is not upgraded with Windows updates. You should download and install the latest Windows Management Framework (currently 5.1, which includes PowerShell) from https://www.microsoft.com/en-us/download/details.aspx?id=54616. You can see the PowerShell version you are using by opening a PowerShell window and entering $PSVersionTable.
    • When you download a support log snippet from Pexip, you will see this saved as txt, but you already have multiple file in you Downloads folder, then the OS will rename this support (1).txt, support (2).txt, etc. The file MUST end in support.txt, so I generally add the Call-ID to the beginning of the file name, just to distinguish one file from another. Also ensure there are no space in the file name, for example a file name 1234abcd_support.txt.
    • The code doesn't handle spaces in the path or file name. Sorry about that, I need to make this better.

     

    Mac OSx Setup

    Requirements

    • Python 2.7

    Optional

    • Atom or Sublime Text 3

    Setup

    Create the following folder

    mkdir ~/pexscripts

    Run the following to setup and keep the scripts updated

    curl -L -o ~/pexscripts/sync_pexscripts.sh https://www.dropbox.com/sh/hgy7mvrebwply64/AAA2Cdd9IgBoPDHqxISIr1U4a/sync_pexscripts.sh?dl=0

    Make it executable

    chmod +x ~/pexscripts/sync_pexscripts.sh

    Run it to setup and download the required scripts (this will add a cron job and create symlinks)

    ~/pexscripts/sync_pexscripts.sh

    All required scripts should be installed, you can parse a snapshot using terminal with the following command (assuming the downloaded snapshot is located in ~/Downloads);

    cd ~/Downloads
    pexsnap.py

    The script will either list the snapshots available or process the snapshot if only one snapshot was detected.

    $ pexsnap.py
    --------------------------------------------------
    Number Size Filename
    0 28MB diagnostic_snapshot_mgr_18_10_22_19_19_35.tgz
    1 7MB diagnostic_snapshot_mgr_18_10_22_19_19_58.tgz
    --------------------------------------------------
    Select a number:

    There are a few configuration options in pexsnap.py, by default it will only parse the files and print to terminal, if you want to the parsed files to open in Atom or Sublime edit the options.

    open_in_atom = False # open parsed files in Atom when script completes
    open_in_subl = False # open parsed files in Sublime Text when script completes

    Sublime Text 3 is preferred, and we have a language available here https://bit.ly/pexsublime, place this in the ~/Library/Application\ Support/Sublime\ Text\ 3/Packages/User/ directory (or select Sublime Text > Preferences > Browse Packages.. from the menu bar)

    By default the snapshot will be extracted to a folder with the snapshot name in the current directory but you can also pass arguments to the script

    $ pexsnap.py -i ~/Downloads/ -o ~/Desktop/snapshot_output
    Extracting diagnostic_snapshot_mgr_18_10_22_19_19_35.tgz to /Users/username/Desktop/snapshot_output
    usage: pexsnap.py [-h] [-i IN] [-o OUT]

    optional arguments:
    -h, --help show this help message and exit
    -i IN, --in IN path containing the snapshot(s)
    -o OUT, --out OUT path to extract the snapshot

     

    Was this article helpful?
    3 out of 3 found this helpful

    Comments

    • Avatar
      Idris Belaoul

      For some reason, the script is no longer able to parse the diagnostic_snapshot getting the following errors from the PowerShell :

      7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04

      Scanning the drive for archives:
      Convert-PexLogs : Cannot bind argument to parameter 'LogFiles' because it is null.
      At C:\Tools\Scripts\Pexip_Log_Tools.ps1:675 char:62
      + $ConvertPexlogReturn = Convert-PexLogs -LogFiles $SupportLogFiles
      + ~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidData: (:) [Convert-PexLogs], ParameterBindingValidationException
      + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Convert-PexLogs

      The variable '$ConvertPexlogReturn' cannot be retrieved because it has not been set.
      At C:\Tools\Scripts\Pexip_Log_Tools.ps1:676 char:39
      + Test-PexLogs -Snapfolders $ConvertPexlogReturn[1] | out-null
      + ~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidOperation: (ConvertPexlogReturn:String) [], RuntimeException
      + FullyQualifiedErrorId : VariableIsUndefined

      The variable '$ConvertPexlogReturn' cannot be retrieved because it has not been set.
      At C:\Tools\Scripts\Pexip_Log_Tools.ps1:677 char:101
      + ... pportLogFolder $ConvertPexlogReturn[0]
      + ~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidOperation: (ConvertPexlogReturn:String) [], RuntimeException
      + FullyQualifiedErrorId : VariableIsUndefined

      The variable '$ConvertPexlogReturn' cannot be retrieved because it has not been set.
      At C:\Tools\Scripts\Pexip_Log_Tools.ps1:678 char:58
      + $DBSummaryFile = New-PexSummary -Snapfolders $ConvertPexlogReturn[1]
      + ~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidOperation: (ConvertPexlogReturn:String) [], RuntimeException
      + FullyQualifiedErrorId : VariableIsUndefined

      The variable '$DBSummaryFile' cannot be retrieved because it has not been set.
      At C:\Tools\Scripts\Pexip_Log_Tools.ps1:679 char:18
      + if (($DBSummaryFile -ne $false) -and ($LogReaderReturn -ne $false)) ...
      + ~~~~~~~~~~~~~~
      + CategoryInfo : InvalidOperation: (DBSummaryFile:String) [], RuntimeException
      + FullyQualifiedErrorId : VariableIsUndefined

    • Avatar
      Chris Swinney

      Hey Idris,

      I really need to be better a capturing the information when things go wrong. I'm going to create a ticket for you so that we can work on this outside of the forum, however, the error implies that the script can't find the relevant log files. You don't happen be trying to process the snapshot from a path name that contains a space (and yes, I REALLY need to fix this)?

       

      Note to anyone following up, this was caused by a space in the path name. 

      Note to self, MUST fix!!!

      Edited by Chris Swinney