Bulk insert example

Notes:
- From within main() method, I pass in threadCount to control number of threads:  
- DumpFileReader is a thin wrapper around a BufferedReader that controls paging, and CSV splitting on the MySQL dump file
- The number of lines DumpFileReader grabs is also passed in as a main argument. 

The big take-away here is that messing around with threads and line count, I really got an excellent picture of what my test environment was capable of


....
        ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
...
        while ( lines != null && lines.length > 0 ) {                        
            // submit to executor with a copy so we can keep going
            LineInsertRunner task = new LineInsertRunner(Arrays.copyOf(lines, lines.length), dumpFileReader.getLineNumber());
            FutureTask<Integer> future = new FutureTask<Integer>(task);
            executorService.execute(future);
            futures.add(future);
            lines = dumpFileReader.readLines();
        }        
        for (FutureTask<Integer> futureTask : futures) {
            log.info("finished processing {} records",futureTask.get());
        }

...

Then my inner class that implements callable for the future above:

    class LineInsertRunner implements Callable<Integer> {
        private String[] lines;
        private int lineNumber = 0;
        
        LineInsertRunner(String[] lines, int currentLineNumber) {
            this.lines = lines;
            log.info("Reading next {} lines", currentLineNumber);
        }
        
        @Override
        public Integer call() {        
            BatchMutation batchMutation = new BatchMutation();
            // loop and call addInsertion, etc. on batchMutate for some chunk of stuff...
            // ...
            keyspace.batchMutate(batchMutation);
            // ..
            // I return a count of what I did
            return new Integer();
        }
        
    }