Skip to content

Instantly share code, notes, and snippets.

@montehurd

montehurd/queryphab.sh Secret

Last active August 4, 2022 21:30
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save montehurd/40ab4dd5cc81b66d78aaaa46752b1f5f to your computer and use it in GitHub Desktop.
Save montehurd/40ab4dd5cc81b66d78aaaa46752b1f5f to your computer and use it in GitHub Desktop.
Download ZIP
Example use of 2 Phabricator APIs. Executes a saved task query, extracts portions of each result, modifies them, converts them to HTML and sends them to Safari for instant preview. Approach used below is for purpose of quick prototyping. Tested on macOS, where you can simply paste the *entire* example into Terminal and press return:
Raw
queryphab.sh
#!/bin/bash
# Edit: This code is just horrible, but it was early in my shell scripting journey.
# This script does the following:
#
# Hits the "maniphest.search" API using a saved query's "query key":
# Any query key may be used. Simply construct your query using Phabricator's "Advanced Search" interface:
# https://phabricator.wikimedia.org/maniphest/query/advanced/
# When you have it fetching what you want, use the "Save Query" button, then grab the saved query's key from its URL.
# Presently it's using this query I constructed for testing purposes:
# https://phabricator.wikimedia.org/maniphest/query/8Nu74C_eapwI
#
# Transforms the results a bit:
# Chooses a few fields from each result and adds a little "remarkup" to each result's ticket description remarkup.
#
# Hits the "remarkup.process" API:
# Converts our transformed remarkup to HTML.
#
# Wraps results in some structural HTML:
# Ensures things like links and styles work.
#
# Sends everthing to a browser:
# So you can instantly see the results.
# Settings:
BASE_URL="https://phabricator.wikimedia.org"
API_TOKEN="api-2lfamy3hsjzihfo6jwmnbd6lphn2"
QUERY_KEY="8Nu74C_eapwI"
# Result transformation script:
read -r -d '' RESULT_TRANSFORMATION << '--END'
import sys, json, urllib;
resultDicts = json.load(sys.stdin)['result']['data'];
remarkup = '\n{F31669139 size=full}\n'.join(list(map(lambda item:
'''== T{number} ==
== {name} ==
{description}
'''
.format(
number=item['id'],
name=item['fields']['name'].encode('utf-8'),
priority=item['fields']['priority']['name'].encode('utf-8'),
description=item['fields']['description']['raw'].encode('utf-8')
), resultDicts)));
encodedRemarkup = urllib.quote_plus(remarkup);
print('contents[0]=' + encodedRemarkup);
--END
# Page presentation script:
read -r -d '' PAGE_PRESENTATION << --END
import sys, json;
html = json.load(sys.stdin)['result'][0]['content'];
print('''
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<link rel="stylesheet" type="text/css" href="https://phab.wmfusercontent.org/res/defaultX/phabricator/31c46c5c/core.pkg.css">
<base href="$BASE_URL" target="_blank">
</head>
<body style="padding:20pt">
<div class="phabricator-remarkup">
{html}
</div>
</body>
</html>
'''.format(html=html));
--END
# Use the query key to hit the "maniphest.search" API:
curl -s "$BASE_URL/api/maniphest.search" \
-d "api.token=$API_TOKEN" \
-d "order=priority" \
-d "queryKey=$QUERY_KEY" |
# Pipe the results to Python for processing:
# - Extracts each result
# - Lightly transforms each result to pretty things up by joining each phab ticket number, title ("name") and description (which is remarkup https://secure.phabricator.com/book/phabricator/article/remarkup/) around a remarkup spacer ("F31669139")
# This "RESULT_TRANSFORMATION" script would be the place to extract some sought description remarkup sub-string if we wanted to further mine/reduce tickets in some way.
python -c "$RESULT_TRANSFORMATION" |
# Pipe our remixed remarkup to "remarkup.process" to convert it to HTML:
curl -s "$BASE_URL/api/remarkup.process" \
-d "api.token=$API_TOKEN" \
-d "context=phriction" \
-d @- |
# Extract HTML, add stylesheet and base URL so it looks nice and links work, then send all of it to a browser so we can see it:
python -c "$PAGE_PRESENTATION" > /tmp/phabricator.tmp.html && open -a "Safari" /tmp/phabricator.tmp.html
# To see html as text use line below instead of line above:
# python -c "$PAGE_PRESENTATION" | open -a "Safari" -f
@montehurd
Copy link
Author

montehurd commented Mar 29, 2020 •
edited

A version of the script with functional "Quick options" buttons for Status and Priority, with Comment support.

Screen Shot 2020-03-29 at 3 51 15 PM
This script also moves all fetching and processing into Python.

#!/bin/bash

BASE_URL="https://phabricator.wikimedia.org"
API_TOKEN="api-2lfamy3hsjzihfo6jwmnbd6lphn2"
QUERY_KEY="F1GAHgzz693c"

# The query:
# https://phabricator.wikimedia.org/maniphest/query/F1GAHgzz693c/

# https://stackoverflow.com/a/44828706
read -r -d '' PYTHON_SCRIPT << --END
import sys, json, urllib, urllib2, base64, re;

def fetch(url, values):
  data = urllib.urlencode(values)
  request = urllib2.Request(url, data)
  response = urllib2.urlopen(request)
  page = response.read()
  return json.loads(page.decode('utf-8'))

# Query the API:
searchResultsJSON = fetch("$BASE_URL/api/maniphest.search", {
  'api.token' : '$API_TOKEN',
  'order' : 'priority',
  'queryKey' : '$QUERY_KEY'
})

resultDicts = searchResultsJSON["result"]["data"]

# Dict with ticketID as key and ticketJSON as value.
ticketsData = dict((x["id"], x) for x in resultDicts)

# Extracts and transforms tickets' remarkup (see: https://secure.phabricator.com/book/phabricator/article/remarkup/).
remarkup = ''.join(list(map(lambda item:
'''TICKET_START:{number}:
= T{number} =
== {name} ==

{description}
TICKET_END'''
.format(
  number=item['id'],
  name=item['fields']['name'].encode('utf-8'),
  description=item['fields']['description']['raw'].encode('utf-8')
), resultDicts)))

# Hit "remarkup.process" to convert our remixed remarkup to HTML:
remarkupResultsJSON = fetch("$BASE_URL/api/remarkup.process", {
  'api.token' : '$API_TOKEN',
  'context' : 'phriction',
  'contents[0]' : remarkup
})

# Extract HTML, add menu, stylesheet and base URL so it looks nice and links work, then send all of it to a browser so we can see it:
menusData = [
  {
    'endpoint': 'maniphest.edit',
    'key': 'status',
    'names': ['Open', 'Resolved', 'Declined', 'Stalled', 'Invalid'],
    'values': ['open', 'resolved', 'declined', 'stalled', 'invalid']
  },
  {
    'endpoint': 'maniphest.edit',
    'key': 'priority',
    'names': ['Unbreak Now!', 'Needs Triage', 'High', 'Medium', 'Low', 'Lowest'],
    'values': ['unbreak', 'triage', 'high', 'medium', 'low', 'lowest']
  }
]

def buttonHTML(ticketID, endpoint, key, name, value, isSelected):
  classString = "class='selected'" if isSelected else ""
  return '''\n<button id="{ticketID}:{endpoint}:{key}:{value}" {classString} onclick="buttonAction({ticketID}, '{endpoint}', '{key}', '{value}')" onblur="showButtonActionMessage({ticketID}, '')">{name}</button>'''.format(ticketID=ticketID, endpoint=endpoint, key=key, value=value, classString=classString, name=name)

def allButtonsHTML(ticketID, menuItemJSON, ticketJSON):
  endpoint = menuItemJSON['endpoint']
  key = menuItemJSON['key']
  selectedName = ticketJSON['fields'][key]['name']
  menuItemNameValuePairs = [{'name': name, 'value': value} for name, value in zip(menuItemJSON['names'], menuItemJSON['values'])]
  return ''.join(list(map(lambda nameValuePair: buttonHTML(ticketID, endpoint, key, nameValuePair['name'], nameValuePair['value'], selectedName == nameValuePair['name']), menuItemNameValuePairs)))

def allMenusHTML(ticketID, ticketJSON):
  menus = ''.join(
    list(
      map(lambda menuItemJSON:
        '''
        <div class="menu">
        {key}:
        <br>
        {buttons}
        </div>
        '''
        .format(
          key=menuItemJSON['key'].capitalize(),
          buttons=allButtonsHTML(ticketID, menuItemJSON, ticketJSON)
        )
        , menusData
      )
    )
  )
  return '''
    <div class="menus phui-tag-core phui-tag-color-object">
      <span class=buttonActionMessage id="buttonActionMessage{ticketID}"></span>
      <h2>Quick options</h2>
      {menus}
      Comment: ( recorded with any <strong>Quick options</strong> chosen above )
      <br>
      <textarea id="ticketID{ticketID}" style="height: 70px; width: 100%;"></textarea>
    </div>
  '''.format(ticketID=ticketID, menus=menus)

def addWrapperDivAndMenuToTicketHTML(match):
  ticketID = match.group(1)
  ticketHTML = match.group(2)
  ticketJSON = ticketsData[int(ticketID)]
  return '''
    <div class=ticket id="T{ticketID}">
      <button class=hide onclick="this.closest('div#T{ticketID}').remove()">Hide</button>
      {ticketHTML}
      {menusString}
    </div>
'''.format(ticketID=ticketID, ticketHTML=ticketHTML, menusString=allMenusHTML(ticketID, ticketJSON))

ticketsHTML = remarkupResultsJSON['result'][0]['content'].encode('utf-8')

ticketsHtmlIncludingWrapperDivsAndMenus = re.sub(pattern=r'TICKET_START:(.*?):(.*?)TICKET_END', repl=addWrapperDivAndMenuToTicketHTML, string=ticketsHTML, flags=re.DOTALL)

print('''
<!DOCTYPE html>
 <html>
   <head>
     <meta charset="UTF-8">
     <link rel="stylesheet" type="text/css" href="https://phab.wmfusercontent.org/res/defaultX/phabricator/31c46c5c/core.pkg.css">
     <base href="$BASE_URL" target="_blank">
     <style>
       div.ticket {{
         border: solid 3px #C7CCD9;
         margin: 35px 35px 100px 35px;
         padding: 5px 20px 20px 20px;
         border-radius: 6px;
         min-width: 740px;
       }}
       button.hide {{
         float: right;
         margin-top: 20px;
       }}
       button:not([class=selected]) {{
         background-color: white;
         border: solid 1px #C7CCD9;
         color: #777C89;
       }}
       button, button:hover {{
         background-image: unset;
         margin-right: 4px;
       }}
       div.menus {{
         border-radius: 6px;
         padding: 6pt;
         margin-top: 20px;
       }}
       span.buttonActionMessage {{
         color: gray;
         margin-left: 25px;
         font-size: 1.2em;
         float: right;
       }}
       div.menu {{
         margin: 8px 8px 8px 14px;
       }}
     </style>
     <script>
       const showButtonActionMessage = (ticketID, messageString) => document.querySelector(\`span#buttonActionMessage\${{ticketID}}\`).innerHTML = messageString
       const updateMenuButtonsSelectionStyle = (ticketID, endpoint, key, value) => {{
         const buttonsSelector = \`button[id^='\${{ticketID}}\:\${{endpoint}}:\${{key}}:']\`
         const buttonToBeSelectedID = \`\${{ticketID}}\:\${{endpoint}}:\${{key}}:\${{value}}\`
         document.querySelectorAll(buttonsSelector).forEach(button => button.classList[button.id == buttonToBeSelectedID ? 'add' : 'remove']('selected'))
       }}
       const buttonAction = (ticketID, endpoint, key, value) => {{
         showButtonActionMessage(ticketID, '💾 Saving...')
         const button = event.toElement
         const commentTextArea = document.querySelector(\`textarea#ticketID\${{ticketID}}\`)
         const comment = commentTextArea.value
         const commentParameters = comment ? \`&transactions[1][type]=comment&transactions[1][value]=\${{comment}}\` : ''
         fetch(\`$BASE_URL/api/\${{endpoint}}\`, {{
           method: 'POST',
           body: \`api.token=$API_TOKEN&output=json&transactions[0][type]=\${{key}}&transactions[0][value]=\${{value}}&objectIdentifier=T\${{ticketID}}\${{commentParameters}}\`
         }})
         .then(response => {{
           if (response.ok == false) throw Error()
           return response.json()
         }})
         .then(json => {{
           if (json.error_code != null) throw Error()
           showButtonActionMessage(ticketID, '🎉 Success')
           updateMenuButtonsSelectionStyle(ticketID, endpoint, key, value)
           commentTextArea.value = ''
         }})
         .catch(error => {{
           showButtonActionMessage(ticketID, '💩 Failure')
         }})
       }}
     </script>
   </head>
   <body>
     <div class="phabricator-remarkup">
       {ticketsHtmlIncludingWrapperDivsAndMenus}
     </div>
   </body>
 </html>
'''.format(ticketsHtmlIncludingWrapperDivsAndMenus=ticketsHtmlIncludingWrapperDivsAndMenus));

--END

python -c "$PYTHON_SCRIPT"> /tmp/phabricator.tmp.html && open -a "Google Chrome" /tmp/phabricator.tmp.html --args --disable-web-security --user-data-dir=/tmp/chrome_dev_test

# To see html as text use line below instead of line above:
# python -c "$PYTHON_SCRIPT" | open -a "Safari" -f

This version moves everything to Python3.

#!/usr/local/bin/python3

import sys, json, urllib.parse, urllib.request, base64, re, subprocess;

BASE_URL = "https://phabricator.wikimedia.org"
API_TOKEN = "api-2lfamy3hsjzihfo6jwmnbd6lphn2"
QUERY_KEY = "F1GAHgzz693c"

def fetch(url, values):
  data = urllib.parse.urlencode(values)
  request = urllib.request.Request(url = url, headers = {}, data = data.encode('utf-8'))
  response = urllib.request.urlopen(request)
  page = response.read()
  return json.loads(page.decode('utf-8'))

# Query the API:
searchResultsJSON = fetch(f"{BASE_URL}/api/maniphest.search", {
  'api.token' : API_TOKEN,
  'order' : 'priority',
  'queryKey' : QUERY_KEY
})

resultDicts = searchResultsJSON["result"]["data"]

# Dict with ticketID as key and ticketJSON as value.
ticketsData = dict((x["id"], x) for x in resultDicts)

# Extracts and transforms tickets' remarkup (see: https://secure.phabricator.com/book/phabricator/article/remarkup/).
remarkup = ''.join(list(map(lambda item:
'''TICKET_START:{number}:
= T{number} =
== {name} ==

{description}
TICKET_END'''
.format(
  number=item['id'],
  name=item['fields']['name'],
  description=item['fields']['description']['raw']
), resultDicts)))

# Hit "remarkup.process" to convert our remixed remarkup to HTML:
remarkupResultsJSON = fetch(f"{BASE_URL}/api/remarkup.process", {
  'api.token' : API_TOKEN,
  'context' : 'phriction',
  'contents[0]' : remarkup
})

# Extract HTML, add menu, stylesheet and base URL so it looks nice and links work, then send all of it to a browser so we can see it:
menusData = [
  {
    'endpoint': 'maniphest.edit',
    'key': 'status',
    'names': ['Open', 'Resolved', 'Declined', 'Stalled', 'Invalid'],
    'values': ['open', 'resolved', 'declined', 'stalled', 'invalid']
  },
  {
    'endpoint': 'maniphest.edit',
    'key': 'priority',
    'names': ['Unbreak Now!', 'Needs Triage', 'High', 'Medium', 'Low', 'Lowest'],
    'values': ['unbreak', 'triage', 'high', 'medium', 'low', 'lowest']
  }
]

def buttonHTML(ticketID, endpoint, key, name, value, isSelected):
  classString = "class='selected'" if isSelected else ""
  return '''\n<button id="{ticketID}:{endpoint}:{key}:{value}" {classString} onclick="buttonAction({ticketID}, '{endpoint}', '{key}', '{value}')" onblur="showButtonActionMessage({ticketID}, '')">{name}</button>'''.format(ticketID=ticketID, endpoint=endpoint, key=key, value=value, classString=classString, name=name)

def allButtonsHTML(ticketID, menuItemJSON, ticketJSON):
  endpoint = menuItemJSON['endpoint']
  key = menuItemJSON['key']
  selectedName = ticketJSON['fields'][key]['name']
  menuItemNameValuePairs = [{'name': name, 'value': value} for name, value in zip(menuItemJSON['names'], menuItemJSON['values'])]
  return ''.join(list(map(lambda nameValuePair: buttonHTML(ticketID, endpoint, key, nameValuePair['name'], nameValuePair['value'], selectedName == nameValuePair['name']), menuItemNameValuePairs)))

def allMenusHTML(ticketID, ticketJSON):
  menus = ''.join(
    list(
      map(lambda menuItemJSON:
        '''
        <div class="menu">
        {key}:
        <br>
        {buttons}
        </div>
        '''
        .format(
          key=menuItemJSON['key'].capitalize(),
          buttons=allButtonsHTML(ticketID, menuItemJSON, ticketJSON)
        )
        , menusData
      )
    )
  )
  return '''
    <div class="menus phui-tag-core phui-tag-color-object">
      <span class=buttonActionMessage id="buttonActionMessage{ticketID}"></span>
      <h2>Quick options</h2>
      {menus}
      Comment: ( recorded with any <strong>Quick options</strong> chosen above )
      <br>
      <textarea id="ticketID{ticketID}" style="height: 70px; width: 100%;"></textarea>
    </div>
  '''.format(ticketID=ticketID, menus=menus)

def addWrapperDivAndMenuToTicketHTML(match):
  ticketID = match.group(1)
  ticketHTML = match.group(2)
  ticketJSON = ticketsData[int(ticketID)]
  return '''
    <div class=ticket id="T{ticketID}">
      <button class=hide onclick="this.closest('div#T{ticketID}').remove()">Hide</button>
      {ticketHTML}
      {menusString}
    </div>
'''.format(ticketID=ticketID, ticketHTML=ticketHTML, menusString=allMenusHTML(ticketID, ticketJSON))

ticketsHTML = remarkupResultsJSON['result'][0]['content']

ticketsHtmlIncludingWrapperDivsAndMenus = re.sub(pattern=r'TICKET_START:(.*?):(.*?)TICKET_END', repl=addWrapperDivAndMenuToTicketHTML, string=ticketsHTML, flags=re.DOTALL)

pageHTML = f'''<!DOCTYPE html>
 <html>
   <head>
     <meta charset="UTF-8">
     <link rel="stylesheet" type="text/css" href="https://phab.wmfusercontent.org/res/defaultX/phabricator/31c46c5c/core.pkg.css">
     <base href="{BASE_URL}" target="_blank">
     <style>
       div.ticket {{
         border: solid 3px #C7CCD9;
         margin: 35px 35px 100px 35px;
         padding: 5px 20px 20px 20px;
         border-radius: 6px;
         min-width: 740px;
       }}
       button.hide {{
         float: right;
         margin-top: 20px;
       }}
       button:not([class=selected]) {{
         background-color: white;
         border: solid 1px #C7CCD9;
         color: #777C89;
       }}
       button, button:hover {{
         background-image: unset;
         margin-right: 4px;
       }}
       div.menus {{
         border-radius: 6px;
         padding: 6pt;
         margin-top: 20px;
       }}
       span.buttonActionMessage {{
         color: gray;
         margin-left: 25px;
         font-size: 1.2em;
         float: right;
       }}
       div.menu {{
         margin: 8px 8px 8px 14px;
       }}
     </style>
     <script>
       const showButtonActionMessage = (ticketID, messageString) => document.querySelector(`span#buttonActionMessage${{ticketID}}`).innerHTML = messageString
       const updateMenuButtonsSelectionStyle = (ticketID, endpoint, key, value) => {{
         const buttonsSelector = `button[id^='${{ticketID}}\:${{endpoint}}:${{key}}:']`
         const buttonToBeSelectedID = `${{ticketID}}\:${{endpoint}}:${{key}}:${{value}}`
         document.querySelectorAll(buttonsSelector).forEach(button => button.classList[button.id == buttonToBeSelectedID ? 'add' : 'remove']('selected'))
       }}
       const buttonAction = (ticketID, endpoint, key, value) => {{
         showButtonActionMessage(ticketID, '💾 Saving...')
         const button = event.toElement
         const commentTextArea = document.querySelector(`textarea#ticketID${{ticketID}}`)
         const comment = commentTextArea.value
         const commentParameters = comment ? `&transactions[1][type]=comment&transactions[1][value]=${{comment}}` : ''
         fetch(`{BASE_URL}/api/${{endpoint}}`, {{
           method: 'POST',
           body: `api.token={API_TOKEN}&output=json&transactions[0][type]=${{key}}&transactions[0][value]=${{value}}&objectIdentifier=T${{ticketID}}${{commentParameters}}`
         }})
         .then(response => {{
           if (response.ok == false) throw Error()
           return response.json()
         }})
         .then(json => {{
           if (json.error_code != null) throw Error()
           showButtonActionMessage(ticketID, '🎉 Success')
           updateMenuButtonsSelectionStyle(ticketID, endpoint, key, value)
           commentTextArea.value = ''
         }})
         .catch(error => {{
           showButtonActionMessage(ticketID, '💩 Failure')
         }})
       }}
     </script>
   </head>
   <body>
     <div class="phabricator-remarkup">
       {ticketsHtmlIncludingWrapperDivsAndMenus}
     </div>
   </body>
 </html>
'''

def sendToBrowser(string, extension):
    filePath = '/tmp/chrome.tmp.' + extension
    f = open(filePath, 'wt', encoding='utf-8')
    f.write(string)
    subprocess.run(f'open -a "Google Chrome" {filePath} --args --disable-web-security --user-data-dir=/tmp/chrome_dev_test', shell=True, check=True, text=True)

sendToBrowser(pageHTML, 'html')
# To see html as text use line below instead of line above:
# sendToBrowser(pageHTML, 'txt')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment