Automatic submit a list of barcodes to Music Magpie and Ziffit

readme.md

I had a stack of CDs that I wanted to sell.

I used barcode2file on my Android phone to capture the barcodes as a plain text file.

Services compared

• Music Magpie (http://www.musicmagpie.co.uk/)
• Ziffit (http://www.ziffit.com/)
• WeBuyBooks (http://www.webuybooks.co.uk/)

Of the above, only WeBuyBooks has a bulk upload facility which can take 50 codes at a time. That was easy, even with hundreds of items.

Music Magpie and Ziffit have a public interface that allows barcodes to be entered one at a time.

I made a couple of python scripts that iterate through the long list of barcodes and submit each one individually. They save their output to a tab-separated file that can be opened in Excel for further comparison, as well as logging to a text file for forensic purposes.

These scripts take the form of python unittests, and use Selenium to drive the browser. They were created mostly in the Selenium IDE, so that's why they're a bit weird. Ziffit script runs fast in Firefox, Music Magpie script runs fast in Chromedriver: Go figure.

The setUp() method has a couple of parameters in it, including save_order that tries to save the basket at the end. This is False by default. Put your username/password into users.py or just hardcode them into the scripts themselves.

Of course this is UK based, because that's where I live.

musicmagpie.py

# -*- coding: utf-8 -*-
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import unittest, time, re
from users import MusicMagpieUser as user

class MusicMagpie(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()
        self.driver.implicitly_wait(30)
        self.base_url = "http://www.musicmagpie.co.uk/"
        self.verificationErrors = []
        self.accept_next_alert = True
        self.barcode_filename = 'barcodes.txt'
        self.log_prefix = 'magpie'
        self.save_order = True

    def test_saved(self):
        driver = self.driver
        driver.get(self.base_url + "login/")
        for i in range(60):
            try:
                if self.is_element_present(By.ID, "ContentPlaceHolderDefault_mainContent_ctl03_login_11_txtEmail"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")
        for i in range(60):
            try:
                if self.is_element_present(By.ID, "ContentPlaceHolderDefault_mainContent_ctl03_login_11_txtPassword"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")
        for i in range(60):
            try:
                if self.is_element_present(By.ID, "ContentPlaceHolderDefault_mainContent_ctl03_login_11_btnLogin"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")
        driver.find_element_by_id("ContentPlaceHolderDefault_mainContent_ctl03_login_11_txtEmail").clear()
        driver.find_element_by_id("ContentPlaceHolderDefault_mainContent_ctl03_login_11_txtEmail").send_keys(user.username)
        driver.find_element_by_id("ContentPlaceHolderDefault_mainContent_ctl03_login_11_txtPassword").clear()
        driver.find_element_by_id("ContentPlaceHolderDefault_mainContent_ctl03_login_11_txtPassword").send_keys(user.password)
        driver.find_element_by_id("ContentPlaceHolderDefault_mainContent_ctl03_login_11_btnLogin").click()
        for i in range(60):
            try:
                if self.is_element_present(By.ID, "ContentPlaceHolderDefault_signIn_6_pnlLoggedIn"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")

        # Logged in.
        driver.get(self.base_url + "start-selling/basket-media/?t=t")
        for i in range(60):
            try:
                if self.is_element_present(By.ID, "getValSmall"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")

        # for each
        timestamp = datetime.strftime(datetime.now(), "%Y%m%d-%H%M%S")
        db_filename = '%s_%s.tsv' % (timestamp, self.log_prefix)
        err_filename = '%s_%s_err.txt' % (timestamp, self.log_prefix)

        with open(self.barcode_filename, 'r') as barcodes:
            for code in barcodes.readlines():
                code = code.strip() if code else None
                if code is None or code == '':
                    continue
                print "Code: %s" % code

                for i in range(60):
                    try:
                        el = driver.find_element_by_id("txtBarcode")
                        if el is None:
                            continue
                        el.click()
                        el.clear()
                        el.send_keys(code)
                        el.clear()
                        el.send_keys(code)
                        time.sleep(0.1)
                        print "sent %s" % code
                        time.sleep(0.2)
                        driver.find_element_by_css_selector("span.show_Small").click()
                        break
                    except: pass
                    time.sleep(1)

                for i in range(60):
                    # Look for either error lbl
                    # or a new row in the basket
                    db = open(db_filename+'.tsv', 'a')
                    err = open(err_filename+'.txt', 'a')
                    try:
                        lbl = driver.find_element(By.ID, 'lblMessage1')
                        print "Label text: %s" % lbl.text

                        if lbl.text == 'Your item has been added':
                            rows = driver.find_elements_by_class_name('rowDetails_Media')
                            if len(rows):
                                record = rows[0].text
                                print record
                                record = record.replace('\n', '\t')
                                db.write(record + '\n')
                                err.write(record + '\n')
                                err.write("%s - %s\n" % (code, lbl.text))
                                break
                        else:
                            err.write("%s - %s\n" % (code, lbl.text.encode('ascii','ignore')))
                            break
                    except Exception as e:
                        print "Exception %s" % e
                    finally:
                        err.close()
                        db.close()
                    time.sleep(1)

        if self.save_order:

            for i in range(60):
                try:
                    if self.is_element_present(By.ID, "btnSaveOrder"): break
                except: pass
                time.sleep(1)
            else: self.fail("time out")
            driver.find_element_by_id("btnSaveOrder").click()

            for i in range(60):
                try:
                    if self.is_element_present(By.ID, "ContentPlaceHolderDefault_mainContent_myProfileMenu_8_BtnSaveOrderCount"): break
                except: pass
                time.sleep(1)
            else: self.fail("time out")

        print "Finished."

    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException, e: return False
        return True
    
    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException, e: return False
        return True
    
    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True
    
    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()

users.py

class ZiffitUser():
    username = ''
    password = ''

class MusicMagpieUser():
    username = ''
    password = ''

ziffit.py

# -*- coding: utf-8 -*-
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import unittest, time, re
from users import ZiffitUser as user

class Ziffit(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Chrome()
        self.driver.implicitly_wait(30)
        self.base_url = "https://www.ziffit.com/"
        self.verificationErrors = []
        self.accept_next_alert = True
        self.barcode_filename = 'barcodes.txt'
        self.log_prefix = 'ziffit'
        self.save_order = True
    
    def test_ziffit(self):
        driver = self.driver
        driver.get(self.base_url + "/")
        for i in range(60):
            try:
                if self.is_element_present(By.CSS_SELECTOR, "a.login.button > strong"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")
        driver.find_element_by_css_selector("a.login.button > strong").click()
        for i in range(60):
            try:
                if self.is_element_present(By.NAME, "USERNAME"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")
        for i in range(60):
            try:
                if self.is_element_present(By.NAME, "PASSWORD"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")
        for i in range(60):
            try:
                if self.is_element_present(By.CSS_SELECTOR, "input.loginFormButton"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")
        driver.find_element_by_name("USERNAME").clear()
        driver.find_element_by_name("USERNAME").send_keys(user.username)
        driver.find_element_by_name("PASSWORD").clear()
        driver.find_element_by_name("PASSWORD").send_keys(user.password)
        driver.find_element_by_css_selector("input.loginFormButton").click()

        time.sleep(5)
        driver.get(self.base_url+'sell-my-cds')
        for i in range(60):
            try:
                if self.is_element_present(By.CSS_SELECTOR, "#search > #eanSearchForm > fieldset > #ean"): break
            except: pass
            time.sleep(1)
        else: self.fail("time out")

        timestamp = datetime.strftime(datetime.now(), "%Y%m%d-%H%M%S")
        db_filename = '%s_%s.tsv' % (timestamp, self.log_prefix)
        err_filename = '%s_%s_log.txt' % (timestamp, self.log_prefix)

        # do the first one, it's
        driver.find_element_by_css_selector("#search > #eanSearchForm > fieldset > #ean").clear()
        driver.find_element_by_css_selector("#search > #eanSearchForm > fieldset > #ean").send_keys("test")
        driver.find_element_by_css_selector("#search > #eanSearchForm > fieldset > input[type=\"submit\"]").click()

        with open(self.barcode_filename, 'r') as barcodes:
            for code in barcodes.readlines():
                code = code.strip() if code else None
                if code is None or code == '':
                    continue
                print "Code: %s" % code

                old_scan_text = driver.find_element_by_css_selector("p.scan-response-message").text
                # Sending the barcode
                try:
                    ean = driver.find_element_by_id("ean")
                    ean.click()
                    ean.send_keys(code)
                except Exception as e:
                    print "Exception while eaning"
                    print e
                driver.find_element_by_css_selector("#basketEanSearchForm > img").click()
                time.sleep(1)

                for i in range(60):
                    db = open(db_filename+'.tsv', 'a')
                    log = open(err_filename+'.txt', 'a')
                    try:
                        scan_text = driver.find_element_by_css_selector("p.scan-response-message").text
                        if scan_text != old_scan_text:
                            # it's updated
                            if 'added to your trade' in scan_text:
                                trs = driver.find_elements_by_css_selector('#basketTable > tbody tr')
                                if len(trs) < 1:
                                    break
                                tds = trs[0].find_elements_by_tag_name('td')
                                l = "%s\t%s\t%s\n" % (tds[1].text.encode('ascii', 'ignore'),
                                                      tds[0].text.encode('ascii', 'ignore'),
                                                      tds[3].text.encode('ascii', 'ignore'))
                                db.write(l)
                                log.write(l)
                                break
                            else:
                                log.write(scan_text)
                                log.write('\n')
                                break
                        else:
                            break
                    except Exception as e:
                        print "Exception"
                        print e
                    time.sleep(1)
                else: self.fail("time out")


        if self.save_order:
            for i in range(60):
                try:
                    if self.is_element_present(By.CSS_SELECTOR, "#basket-operations-container > a > img"): break
                except: pass
                time.sleep(1)
            else: self.fail("time out")
            driver.find_element_by_css_selector("#basket-operations-container > a > img").click()
            for i in range(60):
                try:
                    if self.is_element_present(By.CSS_SELECTOR, "p"): break
                except: pass
                time.sleep(1)
            else: self.fail("time out")

        print "Finished!"

    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException, e: return False
        return True
    
    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException, e: return False
        return True
    
    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True
    
    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()

Hi getting a problem with musicmagpie.py and ziffit.py
I have previously added brackets around the print commands that were showing errors,
On both commas, it comes up as invalid syntax.

def is_element_present(self, how, what):
    try: self.driver.find_element(by=how, value=what)
    except NoSuchElementException, e: return False
    return True

def is_alert_present(self):
    try: self.driver.switch_to_alert()
    except NoAlertPresentException, e: return False
    return True

This is awesome... any idea if it still works in 2021?

It was good when it worked but I would be surprised if it still did tbh, both ziffit and music magpie sites have had a spruce since then, and selenium produced brittle scripts at the best of times.

There's a bit more about it here http://www.euphy.co.uk/2015/05/bulk-upload-for-music-magpie-and-ziffit/ if you're interested in the numbers and the analysis. It wasn't hard to get a rough script going that was just about useful, but it was always clunky! Ziffit and MM have apps now, it's possible they've moved on a bit since then too and might be a bit more helpful about saving batches of barcodes. Good luck!