-
-
Save pomu0325/1168507 to your computer and use it in GitHub Desktop.
| def time(fn: => Unit) = { | |
| val start = System.currentTimeMillis | |
| fn | |
| println("%d ms".format(System.currentTimeMillis - start)) | |
| } | |
| import dispatch._ | |
| import json._ | |
| import JsHttp._ | |
| time { Http(url("http://search.twitter.com/search.json?q=dispatch") >- {s => Js(s)}) } // 400-500ms NORMAL | |
| time { Http(url("http://search.twitter.com/search.json?q=dispatch") >- {s => Js(s)}) } // 400-500ms STILL NORMAL | |
| time { Http(url("http://search.twitter.com/search.json?q=dispatch") ># {s => s}) } // 16000 ms ># TOO SLOW ! | |
| time { Http(url("http://search.twitter.com/search.json?q=dispatch") >- {s => Js(s)}) } // 16000 ms >- ALSO TOO SLOW! | |
| time { Http(url("http://search.twitter.com/search.json?q=dispatch") >- {s => Js(s)}) } // 500 ms >- BACK TO NORMAL | |
| time { Http(url("http://search.twitter.com/search.json?q=dispatch") ># {s => s}) } // 16000-17000 ms ># SLOW AGAIN | |
| import java.io._ | |
| val js = """{"A":"a","B":[1,2],"C":"c"}""" | |
| time { Js(js) } // 0-1ms | |
| time { Js(new ByteArrayInputStream(js.getBytes)) } // 50-60 ms |
I pulled that commit from your branch into master. Thanks for the extensive debugging!
above patch works in single thread, but might not in multi thread...
Hm... glancing through the inherited traits they don't strike me as thread safe either. It might be better to move the lexical entirely into the apply function so that we have one instance per application.
I should also mention that this JSON parser was not written by me and as you can tell I'm not deeply familiar with its internals. The spray-json library is pretty similar to it and I've been considering deprecating it in favor of binding to spray. Any thoughts on that?
trait scala.util.parsing.combinator.Parsers itself is not thread safe due to "var lastNoSuccess".
it is also mentioned here: http://scala-programming-language.1934581.n4.nabble.com/Scala-Parsers-are-not-thread-safe-td2243477.html
scala.util.parsing.json.JSON has similar implementation (having 2 subclass of Parsers) as dispatch.json's, but it only accept CharArrayReader, so performance issue does not happen.
note scala.util.parsing.json.JSON never returns error detail, so overwriting "var lastNoSuccess" by other thread technically seems to be not a problem.
if you give away the memory usage and change JsonParser.apply(Reader[Char]) to JsonParser.apply(CharSequenceReader), I think it's technically ok (in terms of performance issue).
about spray-json: +1 for it. I've never used spray-json, but I can see it's pretty similar at a glance :)
one thing I worry about is that sjson (and maybe some other library I don't know of) has a dependency to dispatch-json and change of binding breaks the dependency...
seems working: pomu0325/Databinder-Dispatch@66cb919
tests: pomu0325/Databinder-Dispatch@e87a286
patch:
pomu0325/Databinder-Dispatch@1d3b977#json/src/main/scala/Json.scala