Created
May 14, 2011 20:56
-
-
Save anonymous/972625 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[Because of external time constraints I did not have time to | |
polish this message as much as I would normally do. Please | |
"be generous in what you accept" accordingly. --Pm] | |
As a result of the last PDS in January, I sent out an email identifying | |
what Rakudo needs from Parrot in 2011 [1]. This is an interim update | |
to give my views on where Parrot has improved or not improved since | |
that posting. | |
The posting identified five things that Rakudo needed from Parrot: | |
0) anything that makes Parrot/Rakudo faster overall | |
1) improved garbage collection | |
2) fixes to profiling tools and documentation | |
3) serialization | |
4) create .pbc directly from a Parrot program | |
Since that posting, excellent progress has been made on garbage | |
collection (#1 above) -- commendations to all involved, especially | |
bacek++. However, recent tests and benchmarks have shown that the | |
primary motivation for the better GC -- that is, increased speed (#0) -- | |
hasn't materialized on Parrot yet. I discovered this when creating | |
the 2011.04 Rakudo Star release -- based on messages to the mailing | |
lists I was expecting around a 20% performance improvement in Rakudo | |
and Parrot; instead I found a slight degradation in performance on nearly | |
all of the machines I tested. Since Rakudo has been relatively stable | |
over this period of time, most (but certainly not all) of the variation | |
in performance would seem to be attributable to changes in underlying | |
Parrot performance. | |
As a result of the above I've created a Rakudo benchmarking platform [2] | |
that makes it easier to evaluate different Rakudo/Parrot configurations | |
over time. The latest results (from this morning) can be seen at | |
https://github.com/pmichaud/rakbench/blob/master/results/kiwi-x86_64-201105140653.txt | |
https://github.com/pmichaud/rakbench/blob/master/results/plum-x86_64-201105140651.txt | |
https://github.com/pmichaud/rakbench/blob/master/results/orange-x86_64-201105140653.txt | |
What these results seem to show is that the new GC only helps in those | |
places where GC was particularly problematic to begin with, such as | |
the regex tests. In most other cases, released Parrots since January | |
have performed worse (and sometimes *much* worse) than the version released | |
in January of this year. Indeed, it was these bencmarks that pointed | |
out some problems that ultimately led to the fixes that show the | |
performance gains for rakudo-master and rakudo-bleed on some platforms. | |
Bottom line: In spite of the new GC, overall performance has not | |
really improved since January. | |
The new rakbench suite underscores just how important it is that Parrot | |
get some reliable profiling tools in place (#2). The ones that exist now | |
aren't at all reliable, as detailed in my January message. Yesterday | |
I inquired on #parrot about the state of profiling, and was told that | |
since there were no GSOC takers for this task that Parrot will have to | |
formulate a plan to work on them. From Rakudo's perspective, this | |
continues to be an immediate and dire need, so I hope Parrot will do so. | |
Yesterday evening I hacked together some very Rakudo-specific | |
quick-and-dirty PIR-level-sub call count capabilities, and was | |
able to almost instantly identify a 10% overall performance | |
improvement for Rakudo by replacing a frequently-called 1-line | |
method with a much faster 3-line version. Even better improvements | |
should be forthcoming with more investigation. Given that there | |
seems to be little movement on profiling on Parrot, we'll continue | |
to develop the Rakudo-specific version and also likely instrumenting | |
the new NQP with similar capabilities. But Parrot ought to be able | |
to do a far richer form of sub-level profiling than what we can do, | |
and when it's available we'll immediately be able to take advantage | |
of it. | |
Creating .pbc directly from a Parrot program (#4) is being worked on | |
satisfactorily, and although I haven't seen any direct effort towards | |
serialization (#3), I tend to view that a longer-term goal rather than | |
something that is likely to happen immediately. | |
Indeed, it should be restated that all of the items in my January | |
message were being presented with "we'll be glad to see them | |
whenever they can be made available" and no real expectations of | |
delivery dates. That continues to the be the case-- anything Parrot | |
can do to improve the items listed above will be help Rakudo | |
immensely. It also continues to be the case that overall speed | |
is critically important for us, and that thus far Parrot releases | |
have trended in the wrong direction on that front. | |
Lastly, there continues to be a great deal of friction between the | |
Parrot and Rakudo teams regarding stability and deprecation. I don't | |
have any direct answers. I do know that I personally (and several | |
of the other Rakudo team members) are weary of all of the revisiting | |
(I call it "re-litigating") of design, implementation, and process | |
decisions that were supposedly "settled" by discussions long ago | |
and even not-so-long ago. I can provide a list of detailed and | |
recent examples of such "revisits" if we wish to analyze this further. | |
As I've repeatedly stated in the past -- Rakudo developers know that | |
breakages will occur, and that's overall okay with us. What hurts | |
is that when we report the breakages, either after they occur or | |
because we note that a proposed change to Parrot will cause us | |
a serious issue, we end up spending hours and hours of collective | |
discussion time trying to _justify_ to a few Parrot developers why | |
the breakage is painful and unwarranted before we actually get | |
someone in Parrot with sufficient authority/skill to relieve our | |
pain. We don't mind the existence of breakages, we mind the | |
absolutely herculean efforts we have to go through with various | |
Parrot developers to get them to stop denying there's an issue | |
(or that they've already addressed it) so that a mutual solution | |
can be found. Anything that can be done to alleviate this ongoing | |
problem would be a big help. | |
Pm | |
[1] http://lists.parrot.org/pipermail/parrot-dev/2011-January/005410.html | |
[2] http://github.com/pmichaud/rakbench |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment