Matthew Dowle is working on a fast CSV reader for data.table. Here is test data case generated in R along with some timings:
require(data.table)
n=1e6
DT = data.table( a=sample(1:1000,n,replace=TRUE),
b=sample(1:1000,n,replace=TRUE),
c=rnorm(n),
d=sample(c("foo","bar","baz","qux","quux"),n,replace=TRUE),