Parrot Google Summer of Code proposal - Bindings for GMP (www.gmplib.org)

Parrot GSoC proposal - GMP bindings

Personal Information

Bob Kuo
bobjkuo@gmail.com - bubaflub (github, IRC)

Project: Parrot bindings for GMP integer functions

Abstract:

This project aims to provide bindings for all integer functions in the GMP library as 
well as associated documentation and tests.

Benefits:

* All HLLs get access to GMP for free
* Easy stepping stone to the other sets - rational and float - of GMP functions 
* First step in providing a Big Number library
* Allows development of cryptographic and other advanced math libraries

Project Details:

Code will be hosted on github under bubaflub/parrot-gmp or a fork of the
official parrot repo.

Documentation work going on throughout - inline POD, comments, external READMEs.
Besides inline comments, all documentation will be in POD.  Since github
supports POD rendering all docs will be accessible online that way.

The distribution will use distutils and the test suite will be using PIR's
Test::More.  Any examples or code written in an HLL will be tested with that
HLL.  Functions will be considered tested if each possible branch is covered
and there are two or more tests covering each function.

Wrapper classes will be written in PIR and wrap an opaque pointer to the C
GMP integer structure, provide convenience functions for initialization,
stringification and I/O. Further work will be done to provide convenient ways
to do basic math with the wrapper objects - these methods will be slower
but more convenient and clearer for users not as concerned about speed.

April 25th - May 23rd - Community bonding period
  Read docs on NCI, GMP, distutils
  Investigate which versions of GMP should be supported
  Build and test other HLLs on Parrot
  Determine / create a reliable way to measure documentation and test
   coverage of PIR files

May 23rd - Programming start date

May 23rd - 30th - 
  Begin work on definition file for ncidef2pir.pl to generate PIR code
  - Should be able to generate this from GMP headers and documentation

May 30th - June 6th -  
  Finish work on definition file
  Begin work on PMC wrapper classes

June 6th - June 13th
  Begin work on library tests

June 13th - July 4th
  Test coverage to 15-20% of integer functions
  Finish basic work on wrapper classes
    - this should include initialization, stringification, and I/O

July 4th - July 11th
  Test coverage to 40-50%
  Basic examples on how to use functions in PIR
  Basic usage examples in HLLs like Rakudo or Cardinal

July 11th - Midterm - deliverables
  NCI definition file, wrapper classes, tests, docs, and examples
  At this point all initialization (mpz_init* and mpz_clear*), 
  setting (mpz_set*), combined (mpz_init_set*), conversion (mpz_get*),
  basic arithmetic (mpz_add*, mpz_mul*, mpz_sub*), exponentiation
  (mpz_pow*), comparison (mpz_cmp*) and IO functions (mpz_out* and
  mpz_in*) should be completed.
  

July 11th - July 25th
  Test coverage to 100%
  More documentation

August 1st - August 15th
  Advanced examples
     - real world algorithms 
     - HLL usage
  Finish PMC wrapper classes
     - convenience methods for basic math

August 15th - August 22nd
  Overflow week
  Cage cleaning
  
August 22nd - Pencils down - all code, tests, and docs submitted

Deliverables: All GMP integer functions mapped through NCI, convenience 
functions, docs, examples, and 100% test coverage that can be merged
with Parrot when the project is complete

If I finish ahead of time, I'd like to 

* implement the rational and float functions in GMP.
  Time estimate: unknown, but approx. double what it took for
  implementing the integer functions

* wrapper classes in other HLLs - Rakduo and Cardinal especially
  Time estimate: 2 weeks

* start porting some advanced math libraries (like Math::Primality)
for future work.
  Time estimate: 1 week for core module

* provide more advanced examples like a command-line arbitrary precision 
calculator driven through a PCT language and a complete tutorial on how I 
built it.
  Time estimate: unknown, but would likely be a major multi-week project

If I'm running out of time, I'll focus on first the NCI definition file and
the core set of functions listed on the midterm date.  These can be committed
at any step of completion and be continued outside of GSoC.

References:

http://www.gmblib.org
PDD16 for NCI information
Math::GMPz for Perl5 bindings to GMP integer functions

License:

Same as Parrot.

Bio:

I am currently a Masters student living in Champaign-Urbana, IL.  I completed
my undergrad at University of Illinois - Chicago in Math and Computer Science.
I've been a perl programmer for about 7 years and worked through a number of
web development companies and am currently employed at Grant Street Group. The
last two years I've done Google Summer of Code projects - one for Perl and one
through RTEMS. I've been a committer to Parrot for about a year or two.

Eligibility:

I am currently a Masters student and am eligible for the Google Summer of Code.

Can you give an idea about what kinds of infrastructure you are planning to use? What will you be using for your build system (distutils?) and your tests suite? Will your code be on github? If so, will you make use of github pages for documentation?

You say "begin work on library tests" and "more tests". Can you give an idea for what the specific goals are? ("I will have at least X tests total", or "I will completely test feature X", or "I will have code coverage for feature Y at X% or higher", etc).

It looks like you're spending three weeks putting together a definitions file for ncidef2pir, is there any way to automate that work? Is there any way you can parse out the function defs from an existing source, even for the most part?

What are the "wrapper classes" you mention? What is the scope of them, and what language will they be written in? Do you have any specific plans for also writing wrappers for any particular HLLs? Rakudo stands out in my head as a compiler which could benefit from these tools (and which may be a source of additional developers later in the project)

I see a small list of extra ideas at the end, I would love to see these ideas prioritized and given time estimates so you can more easily decide what tasks to move on to when you are finished your primary goals. A small calculator tool would be nice, and could easily borrow from existing language examples. Bindings to the floating-point number functions could be large, because it will require more ncidefs to be written, more tests, more wrapper classes, etc.

Also, are your existing goals prioritized so that if things go more slowly than expected you will be able to remove things from your timeline without failing completely?

What are you going to use for docs? POD?

Whiteknight, I believe I've addressed some of the issues you've raised. Please give it another look and let me know if anything is awry.

This is looking better and better, but I would like to see the suggestions that I mentioned on parrot-dev added.

Updated once again per dukeleto's suggestions on parrot-dev. Still not exactly sure how to do doc and test coverage, perhaps I'll roll my own scripts during the community bonding period.

Test coverage might not be possible to calculate automatically, we don't have any existing tools for that. Either you can do a rough calculation on test coverage (two tests per interface function == 100% for that function), or you can abandon the notion of coverage entirely and just stick with raw test counts.

I would hate to see you wasting any time writing coverage scripts, even during the bonding period. They could be valuable to Parrot in a larger sense, but it's not something you should have to spend your time on.

The week August 15th - August 22nd seems like a bit of a waste to me. Fixing typos in documentation can be an ongoing process, and most small typos aren't too important. Likewise, if you try to follow coding standards from the beginning, you shouldn't need to spend a whole week bring code up to compliance with them. It would be better just to leave that week open as overflow, in case you are running behind. If you are on or ahead of schedule, you can talk to your mentor about new features to add.

the period July 11th - August 15th also looks a little bit light. I suspect that if you meet all your midterm deliverables, you won't have much work to do during this period. Either that, or I don't understand the scope of what the wrapper classes are or what they are expected to do. In the project details, can you give a quick overview of what these wrapper classes will be? Also, can you expand on individual milestones when writing the wrapper classes? The period July 11th - August 15th is too long to not have defined milestones. That's more than a month where you don't have any measurable goals defined.

Keep up the great work! This is looking great.

Thanks again whiteknight! I've clarified a bit of what the PMC classes hold and split the second half of the program into two milestones.

Thanks for the update. This is looking much better now.

I am of the opinion that this functionality should be in Parrot core, i.e., bubaflub should work in a branch of the main parrot.git repo.

Adding all these extra GMP bindings directly in Parrot core? That sounds pretty much counter to the general flow of things recently being moved out of core. Is this just special treatment of GMP, or do we expect bindings for most arbitrary libraries to live in core too?

Currently, all NCI bindings come with Parrot core. That doesn't necessarily have to be the case, but it seems to be the simplest solution that has evolved.

This begs the question: Why are all of our other NCI bindings inside parrot core? Why don't we make a repo for each one? Because it is a pain. All the infrastructure, such as scripts to turn defs into PIR, tests and docs are already in core.

I have nothing against our NCI bindings becoming their own repo or something like that in the future, but for this GSoC project, I think there will be many more eyes on the project if it is in parrot.git. Also, it is my opinion that it is more likely to get merged if it is a branch as opposed to a separate repo.

So I guess what I am saying is: We can keep a close eye on this project if it is done in parrot.git, and after it is polished and shiny, we can move it to a more proper home after GSoC.

plobsing's deep-clone project uses NCI and is external to Parrot core. OpenGL bindings in parrot core have been deprecated and are expected to be moved out into their own project. There are very few other projects in the parrot repo which have the same kind of scope, use NCI, and aren't planned to be moved elsewhere. Parrot does provide a limited number of NCI thunks, but that list can easily be expanded by a plugin project or the project can use a custom NCI engine like libffi which obviates the need for thunks. There would be more eyes on any project if they were all developed in the Parrot repo. That doesn't mean any of them belong there. General feeling has been that the Parrot repo should be reserved for things which have a general usefulness for most users of Parrot. I don't think GMP bindings (or BLAS/LAPACK bindings for that matter) fit that description. We should maybe bring this issue up on parrot-dev?

…

--Andrew Whitworth

On Wed, Mar 30, 2011 at 12:23 PM, leto ***@***.*** wrote: Currently, all NCI bindings come with Parrot core. That doesn't necessarily have to be the case, but it seems to be the simplest solution that has evolved. This begs the question: Why are all of our other NCI bindings inside parrot core? Why don't we make a repo for each one? Because it is a pain. All the infrastructure, such as scripts to turn defs into PIR, tests and docs are already in core. I have nothing against our NCI bindings becoming their own repo or something like that in the future, but for this GSoC project, I think there will be many more eyes on the project if it is in parrot.git. Also, it is my opinion that it is more likely to get merged if it is a branch as opposed to a separate repo. So I guess what I am saying is: We can keep a close eye on this project if it is done in parrot.git, and after it is polished and shiny, we can move it to a more proper home after GSoC. ## Reply to this email directly or view it on GitHub: https://gist.github.com/890669

Howdy, If the deep-clone project is a good example for bubaflub to follow, then that works for me. I would like to see the GMP repo inside the Parrot github organization. Perhaps parrot/gmp ? The name doesn't matter much to me. Duke On Wed, Mar 30, 2011 at 9:32 AM, Whiteknight reply@reply.github.com wrote:

plobsing's deep-clone project uses NCI and is external to Parrot core. OpenGL bindings in parrot core have been deprecated and are expected to be moved out into their own project. There are very few other projects in the parrot repo which have the same kind of scope, use NCI, and aren't planned to be moved elsewhere. Parrot does provide a limited number of NCI thunks, but that list can easily be expanded by a plugin project or the project can use a custom NCI engine like libffi which obviates the need for thunks. There would be more eyes on any project if they were all developed in the Parrot repo. That doesn't mean any of them belong there. General feeling has been that the Parrot repo should be reserved for things which have a general usefulness for most users of Parrot. I don't think GMP bindings (or BLAS/LAPACK bindings for that matter) fit that description. We should maybe bring this issue up on parrot-dev? --Andrew Whitworth On Wed, Mar 30, 2011 at 12:23 PM, leto ***@***.*** wrote: > Currently, all NCI bindings come with Parrot core. That doesn't necessarily have to be the case, but it seems to be the simplest solution that has evolved. > > This begs the question: Why are all of our other NCI bindings inside parrot core? Why don't we make a repo for each one? Because it is a pain. All the infrastructure, such as scripts to turn defs into PIR, tests and docs are already in core. > > I have nothing against our NCI bindings becoming their own repo or something like that in the future, but for this GSoC project, I think there will be many more eyes on the project if it is in parrot.git. Also, it is my opinion that it is more likely to get merged if it is a branch as opposed to a separate repo. > > So I guess what I am saying is: We can keep a close eye on this project if it is done in parrot.git, and after it is polished and shiny, we can move it to a more proper home after GSoC. > ## > > Reply to this email directly or view it on GitHub: > https://gist.github.com/890669 ## Reply to this email directly or view it on GitHub: https://gist.github.com/890669

## Jonathan "Duke" Leto jonathan@leto.net http://leto.net

Yes, it would have a good home inside the Parrot organization. I definitely agree to that.

…

--Andrew Whitworth On Wed, Mar 30, 2011 at 12:51 PM, leto reply@reply.github.com wrote:

Howdy, If the deep-clone project is a good example for bubaflub to follow, then that works for me. I would like to see the GMP repo inside the Parrot github organization. Perhaps parrot/gmp ? The name doesn't matter much to me. Duke On Wed, Mar 30, 2011 at 9:32 AM, Whiteknight ***@***.*** wrote: > plobsing's deep-clone project uses NCI and is external to Parrot core. > OpenGL bindings in parrot core have been deprecated and are expected > to be moved out into their own project. There are very few other > projects in the parrot repo which have the same kind of scope, use > NCI, and aren't planned to be moved elsewhere. > > Parrot does provide a limited number of NCI thunks, but that list can > easily be expanded by a plugin project or the project can use a custom > NCI engine like libffi which obviates the need for thunks. > > There would be more eyes on any project if they were all developed in > the Parrot repo. That doesn't mean any of them belong there. General > feeling has been that the Parrot repo should be reserved for things > which have a general usefulness for most users of Parrot. I don't > think GMP bindings (or BLAS/LAPACK bindings for that matter) fit that > description. > > We should maybe bring this issue up on parrot-dev? > > --Andrew Whitworth > > On Wed, Mar 30, 2011 at 12:23 PM, leto > ***@***.*** > wrote: > > > Currently, all NCI bindings come with Parrot core. That doesn't necessarily have to be the case, but it seems to be the simplest solution that has evolved. > > > > This begs the question: Why are all of our other NCI bindings inside parrot core? Why don't we make a repo for each one? Because it is a pain. All the infrastructure, such as scripts to turn defs into PIR, tests and docs are already in core. > > > > I have nothing against our NCI bindings becoming their own repo or something like that in the future, but for this GSoC project, I think there will be many more eyes on the project if it is in parrot.git. Also, it is my opinion that it is more likely to get merged if it is a branch as opposed to a separate repo. > > > > So I guess what I am saying is: We can keep a close eye on this project if it is done in parrot.git, and after it is polished and shiny, we can move it to a more proper home after GSoC. > > ## > > > > Reply to this email directly or view it on GitHub: > > https://gist.github.com/890669 > ## > > Reply to this email directly or view it on GitHub: > https://gist.github.com/890669 ## Jonathan "Duke" Leto ***@***.*** http://leto.net ## Reply to this email directly or view it on GitHub: https://gist.github.com/890669