WordPress XML Splitter

wordpress-xml-splitter.py

'''
Created on May 8, 2010 by @anasimtiaz

Updated on May 28, 2016 by @danielwrobert

This is a "fork" of original script.

Original script URL: http://anasimtiaz.com/?p=51
'''

import Tkinter, tkFileDialog, sys, os;
from Tkinter import *;

procInfo={};
fileFlag = False;
dirFlag = False;

def getFileName(srcFileDisplay):
    fileName = tkFileDialog.askopenfilename();
    srcFileDisplay.delete(0, END);
    srcFileDisplay.insert(0, fileName);
    procInfo['filename']=fileName;
 #   fileFlag = True;

def getOutDir(outDirDisplay):
    outputDir = tkFileDialog.askdirectory();
    outDirDisplay.delete(0, END);
    outDirDisplay.insert(0, outputDir);
    procInfo['outputdir']=outputDir;
#    dirFlag = True;

def writeHeader(currentFile):
    header = '''
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
    xmlns:excerpt="http://wordpress.org/export/1.2/excerpt/"
    xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:wfw="http://wellformedweb.org/CommentAPI/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:wp="http://wordpress.org/export/1.2/"
>
<channel>
<wp:wxr_version>1.2</wp:wxr_version>'''
    
    currentFile.write(header);

def writeFooter(currentFile):
    footer = '''
</channel>
</rss>'''
    
    currentFile.write(footer);

def sysExit():
    sys.exit(1);
    
def startProc(statusDisplay):
    
    dispIndex = 1;
    
    if 'filename' in procInfo and 'outputdir' in procInfo:
    
        filePath,fileName = os.path.split(procInfo['filename']);
        fileNameTxt = os.path.splitext(fileName)[0];
        fileNameExt = os.path.splitext(fileName)[1];
        outDir   = procInfo['outputdir'];
        
        statusDisplay.insert(str(dispIndex)+'.0', 'Reading file ' + fileName + '\n');
        dispIndex += 1;
        xmlFileObj = open(os.path.join(filePath,fileName), "r");
        xmlFile = xmlFileObj.read();
        totalCount = len(xmlFile);
        iteration = 0;
        currentCount = 0;
        maxInc = 2000000;
        EOF = False;
    
    else:

        if 'filename' not in procInfo:
            statusDisplay.insert(str(dispIndex)+'.0', 'Source file not selected\n');
            #statusDisplay.insert('1.0', 'ERROR: Source file not selected\n');
            dispIndex += 1;
        
        if 'outputdir' not in procInfo:
            statusDisplay.insert(str(dispIndex)+'.0', 'Output Directory not selected\n');
            #statusDisplay.insert('2.0', 'ERROR: Output Directory not selected\n');
            dispIndex += 1;
        
        EOF = True;
    
    while(EOF==False):
        currentFileName = fileNameTxt + "_" + str(iteration) + fileNameExt;
        currentFile = open(os.path.join(outDir,currentFileName), 'w');
        statusDisplay.insert(str(dispIndex)+'.0', 'Writing file ' + currentFileName + '\n');
        dispIndex += 1;
        if iteration != 0:
            writeHeader(currentFile);
            
        if (currentCount+maxInc) < totalCount:
            xFile_i = xmlFile[currentCount:currentCount+maxInc];
            incrFile = xFile_i.rfind('</item>') + len('</item>');
            currentFile.write(xFile_i[:incrFile]);
            currentCount += incrFile;
        
        else:
            xFile_i = xmlFile[currentCount:];
            currentFile.write(xFile_i);
            statusDisplay.insert(str(dispIndex)+'.0', 'Finished processing \n');
            dispIndex += 1;
            EOF = True;
            
        if EOF != True:
            writeFooter(currentFile);
            
        iteration += 1;
       
            
            
            
if __name__ == '__main__':
    
    root = Tk();
    root.title('WordPress XML Splitter');
    
    srcFileText = Label(root, text="Source: ").grid(row=0, column=0, sticky=W);
    outDirText  = Label(root, text="Output Dir: ").grid(row=1, column=0, sticky=W);
 
    srcFileDisplay = Entry();
    srcFileDisplay.grid(row=0, column=1, columnspan=11);
    
    outDirDisplay = Entry();
    outDirDisplay.grid(row=1, column=1, columnspan=11);
    
    getAndDispFileName = lambda x=srcFileDisplay : getFileName(x);
    getAndDispOutDir   = lambda x=outDirDisplay  : getOutDir(x);
    
    browseSrcFile = Button(root, text="Browse File", command=getAndDispFileName).grid(row=0, column=12, sticky=W+E);
    browseOutDir  = Button(root, text="Browse Dir.", command=getAndDispOutDir).grid(row=1, column=12, sticky=W+E);
       
    statusDisplay = Text(root, width = 50, height = 10);
    statusDisplay.grid(row=2, column=0, columnspan=13);
    
    startProcPass = lambda x=statusDisplay : startProc(x);
    
    startButton = Button(root, text="Start", command=startProcPass).grid(row=3, column=6, sticky=W+E);
    exitButton = Button(root, text="Exit", command=sysExit).grid(row=3, column=7, sticky=W+E);
    
    
    root.mainloop();

WordPress XML Splitter

Usage

1. Download the above file
2. Open your Terminal application of choice
3. Navigate to the directory where above file has been downloaded to
4. Run the command python wordpress-xml-splitter.py

When you have completed the above steps, you will see a small "WordPress XML Splitter" window open. Click on button "Browse File" to select the XML file you want to split, and then select the folder you want the file to be saved to by clicking on "Browse Dir".

Once you have the export file and output directory selected, click the "Start" button and the processing should start immediately.

After the program completes, you can navigate to the output directory you'd previously selected and you should see a collection of files, not more than 2Mb in size each.

If you wish to adjust the output file size, you can do so by changing the number on line 75. For example, if you want files that are no more than 15Mb in size (the max allowed size for the Import Tool from the WP Admin), you would change the number 2000000 to 15000000.

Credits: Downloaded this file originally from a post on Anas Imtiaz's website, here. I've added in minor updates to the output headers.

Changelog

2016-31-05

• Updated name from mainSplit.py to wordpress-xml-splitter.py to be more reflective of what the script is for.

2016-28-05

• Updated output headers to more closely mirror WordPress export format

Thanks Daniel - very useful.

Thank you for this Daniel!

Thank you Daniel - turns out not all heroes wear capes!

I had to bump maxInc down to 500000; (server defaults didn't like 2mb) but after that it worked like a charm.

Very much appreciated.

Helped me out, thank you!

Thank you all for the comments!

So glad to see that this is still helping folks out years later. When I shared this, I had just put it together as a one-off fork for a project I was working on at the time and chose to keep it shared as a Gist! 😄

Still useful in 2022, thanks for keeping this gist up!

Turned one huge 179MB Wordpress export file into 75 smaller ones, ~2.4MB each. Import via:

for FILE in /home/git/output/*; do wp import $FILE --url=https://blog.domain.com/user1 --authors=mapping.csv; touch $FILE.done; done;

Each file takes about 20-30 minutes to import, will be finished in 30:ish hours 👍 :)

I have changed the script to run on the command-line and removed the graphical interface. Also note the added shebang #!/usr/bin/env python3

https://gist.githubusercontent.com/Patrick-1994-/acd41c084790aa9d07499b9a76245891/raw/add199b1fc8a6a028a5187cfb281d6961294a68c/wordpress-xml-splitter.py

Thanks for this.
I have updated it to work with Python3 here - https://github.com/josylad/WordPress-XML-Splitter

My version must be python3-only since they changed print "x" to print("x") there.

it theres way to split based on an criteria

for exmaple split only nodes based on an critereia of filter