Scan script for nzbget.

GetPw.py

#!/usr/bin/python2.7
##############################################################################
### NZBGET SCAN SCRIPT                                     ###
#
# Scans filename of incoming NZBs for embedded passwords.
#
##############################################################################
### OPTIONS                                                                ###
# The RegEx to match the password in the filename.
#regex=(.*)\{\{(.*)\}\}.nzb

### NZBGET SCAN SCRIPT                                     ###
##############################################################################

import re
import getopt
import sys
import os

nzbfile = os.environ.get('NZBNP_NZBNAME')
regex = os.environ.get('NZBPO_REGEX')
if nzbfile:
        pattern = re.compile(regex)
        match = pattern.search(nzbfile)
        password = ""
        name = nzbfile
        if match:
                name = match.group(1)
                password = match.group(2)
                

        print "[NZB] NZBNAME=" + name
        if password:
                print "[NZB] NZBPR_*Unpack:Password=" + password

Pushbullet.py

#!/usr/bin/env python
#
# Modified E-Mail post-processing script for sending Pushbullet notifications NZBGet. 
# Modified for Pushbullet on 8 October 2014.
# Copyright (C) 2014 JVM <jvmed@users.sourceforge.net>
# Copyright (C) 2014 binreader
# # # Copyright (C) 2013-2014 Andrey Prygunkov <hugbug@users.sourceforge.net>
# # #
# # # This program is free software; you can redistribute it and/or modify
# # # it under the terms of the GNU General Public License as published by
# # # the Free Software Foundation; either version 2 of the License, or
# # # (at your option) any later version.
# # # 
# # # This program is distributed in the hope that it will be useful,
# # # but WITHOUT ANY WARRANTY; without even the implied warranty of
# # # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# # # GNU General Public License for more details.
# # # 
# # # You should have received a copy of the GNU General Public License
# # # along with this program; if not, write to the Free Software
# # # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
# # #
# # # $Revision: 1031 $
# # # $Date: 2014-05-29 00:19:39 +0200 (Thu, 29 May 2014) $
# # #

##############################################################################
### NZBGET SCAN/POST-PROCESSING SCRIPT                                     ###

# Sends Pushbullet notification.
#
# This script sends Pushbullet notification  before nzb-file is added to queue and/or when the job is done.
#
# NOTE: This script requires Python installed on your system. Requires NZBGet v14+.
#
# PP-Script version: 1.1.
#
# Further discussion at: nzbget.net/forum/viewtopic.php?f=8&t=1205
# 
##############################################################################
### OPTIONS                                                                ###

# Your Pushbullet Access Token. 
#
# Find your Access Token at <a href="https://www.pushbullet.com/account">https://www.pushbullet.com/account</a>
#AccessToken=

# Pushbullet device(s) to send to. 
# 
# Leave empty to push to all devices. Can also push to individual devices by device IDs, separate each device ID by a space. 
# Device IDs can be found by using cURL: curl -u your_api_key_here: https://api.pushbullet.com/api/devices
#
# More info at <a href="https://www.pushbullet.com/api">https://www.pushbullet.com/api</a>
#DeviceID=

# Pushbullet push title.
#Title=NZBGet

# Add status to title (yes, no).
# 
# Adds [Received], [Success], [Failure], [Paused] states to the Pushbullet title.
#
#TitleStatus=yes

# Add NZB name to title (yes, no).
# 
# Adds full NZB name to Pushbullet title, may be shortened.
#
#TitleNZB=yes

# Pushbullet push type.
#
# Leave as default. More info at <a href="https://www.pushbullet.com/api">https://www.pushbullet.com/api</a>
#Type=note

# Pushbullet API URL.
#
# Leave as default. More info at <a href="https://www.pushbullet.com/api">https://www.pushbullet.com/api</a>
#URL=https://api.pushbullet.com/api/pushes

# Enable Scan Script notification (yes, no).
# 
# Send Pushbullet notification before nzb-file is added to queue.
#
# NOTE: Following options defined only when ScanScript is enabled.
#
#ScanScript=yes

# Skip certain category.
#
# Specify a category to skip for scan notifications. Pushbullet notification will not be sent when this 
# category is detected. Note that post processing notifications can be set for individual categories on the 
# "CATEGORIES" settings page.
# Can also put multiple categories comma separated.
#
#SkipCategory=

# Append top of queue (yes, no).
#
# Notify if added to top of queue. 
#
#Top=yes

# Append added paused (yes, no).
#
# Notify if added to queue paused. 
#
#Paused=yes

# Append category (yes, no).
#
#Category=yes

# Append priority (yes, no).
#
#Priority=yes

# Append source nzb URL (yes, no).
#
#SourceURL=yes

# Enable Post-Processing Script notification (yes, no).
# 
# Send Pushbullet notification before nzb-file is added to queue.
#
# NOTE: Following options defined only when PPScript is enabled.
#
#PPScript=yes

# Append average download speed (yes, no).
#
#AveDownSpeed=yes

# Append download size (yes, no).
#
#DownloadSize=yes

# Append download time (yes, no).
#
#DownloadTime=yes

# Append par time (yes, no).
#
#ParTime=yes

# Append repair time (yes, no).
#
#RepairTime=yes

# Append unpack time (yes, no).
#
#UnpackTime=yes

# Append total post processing script time (yes, no).
#
#PPTime=yes

# Append total time (yes, no).
#
#TotalTime=yes

# Append list of files to the message (yes, no).
#
# Add the list of downloaded files (the content of destination directory).
#FileList=yes

# Append broken-log to the message (yes, no).
#
# Add the content of file _brokenlog.txt. This file contains the list of damaged
# files and the result of par-check/repair. For successful downloads the broken-log
# is usually deleted by cleanup-script and therefore is not sent.
#BrokenLog=yes

# Append post-processing log to the message (Always, Never, OnFailure).
#
# Add the post-processing log of active job.
#PostProcessLog=OnFailure

### NZBGET SCAN/POST-PROCESSING SCRIPT                                     ###
##############################################################################


import os
import sys
import datetime
import urllib2
import urllib

try:
        from xmlrpclib import ServerProxy # python 2
except ImportError:
        from xmlrpc.client import ServerProxy # python 3

# Exit codes used by NZBGet
POSTPROCESS_SUCCESS=93
POSTPROCESS_ERROR=94

# Check if the script is called from nzbget 11.0 or later
if not 'NZBOP_SCRIPTDIR' in os.environ:
        print('*** NZBGet post-processing script ***')
        print('This script is supposed to be called from nzbget (11.0 or later).')
        sys.exit(POSTPROCESS_ERROR)

print('[DETAIL] Script successfully started')
sys.stdout.flush()

required_options = ( 'NZBPO_SCANSCRIPT', 'NZBPO_PPSCRIPT', 'NZBPO_FILELIST', 'NZBPO_BROKENLOG', 'NZBPO_POSTPROCESSLOG', 
'NZBPO_URL', 'NZBPO_ACCESSTOKEN', 'NZBPO_TYPE', 'NZBPO_TITLE', 'NZBPO_TOP', 'NZBPO_PAUSED', 'NZBPO_CATEGORY', 'NZBPO_PRIORITY', 
'NZBPO_SOURCEURL', 'NZBPO_AVEDOWNSPEED', 'NZBPO_DOWNLOADTIME', 'NZBPO_DOWNLOADSIZE', 'NZBPO_PARTIME', 'NZBPO_REPAIRTIME',
'NZBPO_UNPACKTIME', 'NZBPO_PPTIME', 'NZBPO_TOTALTIME')

for optname in required_options:
        if (not optname in os.environ):
                print('[ERROR] Option %s is missing in configuration file. Please check script settings' % optname[6:])
                sys.exit(POSTPROCESS_ERROR)

# Scan Script Notification
                
if 'NZBNP_DIRECTORY' in os.environ:        
        if os.environ['NZBPO_SCANSCRIPT'] == 'no':
                print('[DETAIL] ScanScript Notification not enabled')
                sys.exit(POSTPROCESS_SUCCESS)
                
        skip_category = os.environ['NZBPO_SKIPCATEGORY'].split(',')
        if len( skip_category ) > 0:
                for cat in skip_category:
                        if os.environ['NZBNP_CATEGORY'] == cat:
                                print('[DETAIL] ScanScript Notification skipped')
                                sys.exit(POSTPROCESS_SUCCESS)                   
                
        elif os.environ['NZBNP_CATEGORY'] == os.environ['NZBPO_SKIPCATEGORY']:
               print('[DETAIL] ScanScript Notification skipped')
               sys.exit(POSTPROCESS_SUCCESS)
               
        print('[DETAIL] ScanScript Notification')
        subject = ''
        if os.environ['NZBPO_TitleStatus'] == 'yes':
                subject = ' [Received]'
        text = 'Received: ' + os.environ['NZBNP_NZBNAME']
        if os.environ['NZBPO_TOP'] == 'yes':
                if os.environ['NZBNP_TOP'] == '1':
                        text += '\nAdded to top of queue.'
                elif os.environ['NZBNP_TOP'] == '0':
                        text += '\nNot added to top of queue.'
        if os.environ['NZBPO_PAUSED'] == 'yes':
                if os.environ['NZBNP_PAUSED'] == '1':
                        text += '\nNZB added as paused.'
                        if os.environ['NZBPO_TitleStatus'] == 'yes':
                                subject = ' [Paused]'
                elif os.environ['NZBNP_PAUSED'] == '0':
                        text += '\nNZB is allowed to begin.'
        if os.environ['NZBPO_CATEGORY'] == 'yes':
                if len(os.environ['NZBNP_CATEGORY']) > 0:
                        text += '\nCategory: ' + os.environ['NZBNP_CATEGORY']
        if os.environ['NZBPO_PRIORITY'] == 'yes':
                if len(os.environ['NZBNP_PRIORITY']) > 0:
                        text += '\nPriority: ' + os.environ['NZBNP_PRIORITY']
        if os.environ['NZBPO_SOURCEURL'] == 'yes':      
                text += '\nURL: ' + os.environ['NZBNP_URL']
        if os.environ['NZBPO_TitleNZB'] == 'yes':
                subject += ' ' + os.environ['NZBNP_NZBNAME']
# Post-Processing Notification
                
if 'NZBPP_DIRECTORY' in os.environ:
        if os.environ['NZBPO_PPSCRIPT'] == 'no':
                print('[DETAIL] PPScript Notification not enabled')
                sys.exit(POSTPROCESS_SUCCESS)
        print('[DETAIL] PPScript Notification')
###### EMail.py
        success = os.environ['NZBPP_TOTALSTATUS'] == 'SUCCESS'
        subject = ''
        if success:
                if os.environ['NZBPO_TitleStatus'] == 'yes':
                        subject = ' [Success]'                 
                text = 'Download of "%s" has successfully completed.' % (os.environ['NZBPP_NZBNAME'])
        else:
                if os.environ['NZBPO_TitleStatus'] == 'yes':
                        subject = ' [Failure]'   
                text = 'Download of "%s" has failed.' % (os.environ['NZBPP_NZBNAME'])

        if os.environ['NZBPO_TitleNZB'] == 'yes':
                subject += ' ' + os.environ['NZBPP_NZBNAME']

        text += '\nStatus: %s' % os.environ['NZBPP_STATUS']

        # # add list of downloaded files
        # if os.environ['NZBPO_FILELIST'] == 'yes':
                # text += '\n\nFiles:'
                # for dirname, dirnames, filenames in os.walk(os.environ['NZBPP_DIRECTORY']):
                        # for filename in filenames:
                                # text += '\n' + os.path.join(dirname, filename)[len(os.environ['NZBPP_DIRECTORY']) + 1:]

        # # add _brokenlog.txt (if exists)
        # if os.environ['NZBPO_BROKENLOG'] == 'yes':
                # brokenlog = '%s/_brokenlog.txt' % os.environ['NZBPP_DIRECTORY']
                # if os.path.exists(brokenlog):
                        # text += '\n\nBrokenlog:\n' + open(brokenlog, 'r').read().strip()

        # add post-processing log
        # if os.environ['NZBPO_POSTPROCESSLOG'] == 'Always' or \
                # (os.environ['NZBPO_POSTPROCESSLOG'] == 'OnFailure' and not success):
        # To get the post-processing log we connect to NZBGet via XML-RPC
        # and call method "postqueue", which returns the list of post-processing job.
        # The first item in the list is current job. This item has a field 'Log',
        # containing an array of log-entries.
        # For more info visit http://nzbget.net/RPC_API_reference
        
        # First we need to know connection info: host, port and password of NZBGet server.
        # NZBGet passes all configuration options to post-processing script as
        # environment variables.
        host = os.environ['NZBOP_CONTROLIP'];
        port = os.environ['NZBOP_CONTROLPORT'];
        username = os.environ['NZBOP_CONTROLUSERNAME'];
        password = os.environ['NZBOP_CONTROLPASSWORD'];
        
        if host == '0.0.0.0': host = '127.0.0.1'
        
        # Build an URL for XML-RPC requests
        rpcUrl = 'http://%s:%s@%s:%s/xmlrpc' % (username, password, host, port);
        
        # Create remote server object
        server = ServerProxy(rpcUrl)
        
###### Non EMail.py
        
        groups = server.listgroups(0)
        
        # Find correct nzb in method listgroups 
        nzbID = int(os.environ['NZBPP_NZBID'])
        for i in xrange(len(groups)):
                nzbGroup = groups[i]
                if nzbGroup['NZBID'] == nzbID:
                        break
        
        # add average download speed
        if os.environ['NZBPO_AVEDOWNSPEED'] == 'yes':
                DownloadedSizeMB = float(nzbGroup['DownloadedSizeMB'])
                DownloadTimeSec = float(nzbGroup['DownloadTimeSec'])
                if DownloadTimeSec > 0: # check x/0 errors
                        avespeed = (DownloadedSizeMB/DownloadTimeSec) # MB/s
                        unit = ' MB/s'
                        if avespeed < 1:
                                avespeed = avespeed * 1024 # KB/s
                                unit = ' KB/s'
                        text += '\nAverage download speed: %.1f' % (avespeed) + unit
        
        # add download size
        if os.environ['NZBPO_DOWNLOADSIZE'] == 'yes':                                                           
                DownloadedSize = float(nzbGroup['DownloadedSizeMB'])
                unit = ' MB'
                if DownloadedSize > 1024:
                        DownloadedSize = DownloadedSize / 1024 # GB
                        unit = ' GB'
                text += '\nDownload size: %.1f' % (DownloadedSize) + unit
        
        # add DownloadTime
        if os.environ['NZBPO_DOWNLOADTIME'] == 'yes':           
                DownloadTimeSec = int(nzbGroup['DownloadTimeSec'])
                Hour = DownloadTimeSec/3600
                Min = (DownloadTimeSec - (DownloadTimeSec/3600)*3600)/60
                Sec = (DownloadTimeSec - (DownloadTimeSec/3600)*3600)%60
                text += '\nDownload time: %d:%02d:%02d' % (Hour,Min,Sec)
                
        # add ParTime
        if os.environ['NZBPO_PARTIME'] == 'yes':
                ParTimeSec = int(nzbGroup['ParTimeSec'])
                Hour = ParTimeSec/3600
                Min = (ParTimeSec - (ParTimeSec/3600)*3600)/60
                Sec = (ParTimeSec - (ParTimeSec/3600)*3600)%60
                text += '\nPar time: %d:%02d:%02d' % (Hour,Min,Sec)
                
        # add RepairTime
        if os.environ['NZBPO_REPAIRTIME'] == 'yes':
                RepairTimeSec = int(nzbGroup['RepairTimeSec'])
                Hour = RepairTimeSec/3600
                Min = (RepairTimeSec - (RepairTimeSec/3600)*3600)/60
                Sec = (RepairTimeSec - (RepairTimeSec/3600)*3600)%60
                text += '\nRepair time: %d:%02d:%02d' % (Hour,Min,Sec)
        
        # add UnpackTime
        if os.environ['NZBPO_UNPACKTIME'] == 'yes':
                UnpackTimeSec = int(nzbGroup['UnpackTimeSec'])
                Hour = UnpackTimeSec/3600
                Min = (UnpackTimeSec - (UnpackTimeSec/3600)*3600)/60
                Sec = (UnpackTimeSec - (UnpackTimeSec/3600)*3600)%60
                text += '\nUnpack time: %d:%02d:%02d' % (Hour,Min,Sec)
                
        # add PP script, renaming and moving total time
        if os.environ['NZBPO_PPTIME'] == 'yes':
                PostTotalTimeSec = int(nzbGroup['PostTotalTimeSec']) - \
                        int(nzbGroup['ParTimeSec']) - \
                        int(nzbGroup['UnpackTimeSec'])
                if PostTotalTimeSec < 0: # if -ve, script may not be on first run after download
                        PostTotalTimeSec = int(nzbGroup['PostTotalTimeSec'])
                Hour = PostTotalTimeSec/3600
                Min = (PostTotalTimeSec - (PostTotalTimeSec/3600)*3600)/60
                Sec = (PostTotalTimeSec - (PostTotalTimeSec/3600)*3600)%60
                text += '\nPP script, renaming and moving total time: %d:%02d:%02d' % (Hour,Min,Sec)
        
        # add TotalTime
        if os.environ['NZBPO_TOTALTIME'] == 'yes':
                TotalTimeSec = int(nzbGroup['DownloadTimeSec']) + \
                        int(nzbGroup['PostTotalTimeSec'])
                Hour = TotalTimeSec/3600
                Min = (TotalTimeSec - (TotalTimeSec/3600)*3600)/60
                Sec = (TotalTimeSec - (TotalTimeSec/3600)*3600)%60
                text += '\nTotal time: %d:%02d:%02d' % (Hour,Min,Sec)
                
                # print server.listgroups(0)[0]['TotalTimeSec']
                
###### EMail.py
        
        # add list of downloaded files
        if os.environ['NZBPO_FILELIST'] == 'yes':
                text += '\n\nFiles:'
                for dirname, dirnames, filenames in os.walk(os.environ['NZBPP_DIRECTORY']):
                        for filename in filenames:
                                text += '\n' + os.path.join(dirname, filename)[len(os.environ['NZBPP_DIRECTORY']) + 1:]
                                
        # add _brokenlog.txt (if exists)
        if os.environ['NZBPO_BROKENLOG'] == 'yes':
                brokenlog = '%s/_brokenlog.txt' % os.environ['NZBPP_DIRECTORY']
                if os.path.exists(brokenlog):
                        text += '\n\nBrokenlog:\n' + open(brokenlog, 'r').read().strip()
        
        # add post-processing log
        if os.environ['NZBPO_POSTPROCESSLOG'] == 'Always' or \
                (os.environ['NZBPO_POSTPROCESSLOG'] == 'OnFailure' and not success):
                postqueue = server.postqueue(10000)
                        
                # Get field 'Log' from the first post-processing job
                log = postqueue[0]['Log']
                
                # Now iterate through entries and save them to message text
                if len(log) > 0:
                        text += '\n\nPost-processing log:';
                        for entry in log:
                                text += '\n%s\t%s\t%s' % (entry['Kind'], datetime.datetime.fromtimestamp(int(entry['Time'])), entry['Text'])

# Send message

print('[DETAIL] Sending to Pushbullet')
sys.stdout.flush()
newstr = text.replace("(", "") # these brackets cause errors with cURL, don't know why, occurs with CP entries
text = newstr.replace(")", "")

try:
        password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
        password_mgr.add_password(None, os.environ['NZBPO_URL'], os.environ['NZBPO_ACCESSTOKEN'], '')
        handler = urllib2.HTTPBasicAuthHandler(password_mgr)
        opener = urllib2.build_opener(handler)
        opener.open(os.environ['NZBPO_URL'])
        urllib2.install_opener(opener)       
        
        device_id = os.environ['NZBPO_DEVICEID'].split( )
        if len( device_id ) > 0:
                for i in device_id:
                        values = {'device_iden' : i,
                                  'title' : os.environ['NZBPO_TITLE'] + subject,
                                  'body' : text,
                                  'type' : os.environ['NZBPO_TYPE'] }
                        data = urllib.urlencode(values)
                        req = urllib2.Request(os.environ['NZBPO_URL'], data)
                        urllib2.urlopen(req)
        else:
                values = {'title' : os.environ['NZBPO_TITLE'] + subject,
                          'body' : text,
                          'type' : os.environ['NZBPO_TYPE'] }
                data = urllib.urlencode(values)
                req = urllib2.Request(os.environ['NZBPO_URL'], data)
                urllib2.urlopen(req)
        print('[DETAIL] Sent to Pushbullet')
except Exception as err:
        print('[ERROR] %s' % err)
        sys.exit(POSTPROCESS_ERROR)

# All OK, returning exit status 'POSTPROCESS_SUCCESS' (int <93>) to let NZBGet know
# that our script has successfully completed.
sys.exit(POSTPROCESS_SUCCESS)

Hello,

thanks for your great GetPW.py Script. I use it since a long time, and on my PC it is working great. But since a few time, on the Laptop of my Mom, it will not works right. We use both the same System, it is Debian 9 stretch. Today I look in my synaptic and look, which python packages are installed. I look after that on the Laptop of my mother and there is the same. But I have no idea, why now the Script not works again and detect the password in the nzb file in {{password}}

Do you have an idea?

Best regards
Mr-Flummi

Just in case someone wants to make use of a good old plane sh script, one additional example to extract a {{password}} and / or [[category]] from the file name.

Extra: if you are making use of nzbget within a docker container and mount the script in a folder, ensure you add the execute permission with e.g. chmod +x myscript.sh.

#!/bin/sh 

###########################################
### NZBGET SCAN SCRIPT                  ###

# Extract password and category
#
# This a sh shell script takes the password and category from the NZB file name if present.
#
# Example: [[Computer]]Debian8-Final.iso{{12345}}.nzb
#
# It switches the Category to "Computer" and set the password "12345".

### NZBGET SCAN SCRIPT                  ###
###########################################

newname=$(echo "$NZBNP_FILENAME" | sed -e 's/.*\]//g' -e 's/{.*//g' -e 's/.nzb//g')

case "$NZBNP_FILENAME" in
*[[*)
  category=$(echo "$NZBNP_FILENAME"  | sed -e 's/.*\[//g' -e 's/\].*//g')
  echo "[NZB] CATEGORY=$category";
  ;;
*       ) 
  category="" ;;
esac

case "$NZBNP_FILENAME" in
*{{*)
  passwd=$(echo "$NZBNP_FILENAME" | sed -e 's/.*{//g' -e 's/}.*//g')
  echo "[NZB] NZBPR_*Unpack:Password=$passwd";
  ;;
*       )
  passwd="" ;;
esac

echo "[NZB] NZBNAME=$newname";

exit 93