Skip to content

Instantly share code, notes, and snippets.

@lucian1900
Created March 28, 2011 22:51
Show Gist options
  • Save lucian1900/891481 to your computer and use it in GitHub Desktop.
Save lucian1900/891481 to your computer and use it in GitHub Desktop.
Proposal for developing a new pynie over GSoC 2011

Personal info:

Lucian Branescu Mihaila

email: lucian dot braneNOSPAMscu at gmail dot com irc: lucian

Project: Python3 on Parrot

Abstract: This project aims to create a Python3 [Python] compiler and object system that works on parrot.

Benefits:

  • (partial) implementation of Python3 on parrot
  • exploration of parrot's suitability for another modern scripting language
  • stress-test of parrot's bootstrapping capabilities, especially with an already self-hosted, mature language

Drawbacks:

  • (initially) dependant on CPython
  • likely slower than other python implementations
  • likely incomplete for a long time

Deliverables:

  • python3-to-parrot compiler written in python3. to be run on CPython
  • python3 object system written in winxed/pir/with 6model. to be run on Parrot
  • testsuite for primitive python implementations, similar to what pynie has now
  • documentation on how to use Python3 on Parrot

Project details:

The project is necessarily made up of two parts: compiler and object system.

I intend to use an existing compiler [CPython compiler] to parse python code and write a parrot backend, likely generating PIR or Winxed. This "stage 0" compiler will run on CPython and generate PIR, to be compiled and run by parrot. Since it depends on the (now) C-only 'ast' module, it won't be boostrapped in the near future even if Python3 on Parrot is complete enough to otherwise compile itself. This isn't a major issue since 1) python is ubiquitous and 2) the 'ast' module is relativel small and other parties may be interested in a pure-python version.

However, python code requires python core types (an object system) to function. This object system necessarily needs to be written in a Parrot HLL, since Python3 on Parrot wouldn't be bootstrapped yet. While it is possible to write a second Python3-subset compiler, I consider it outside the scope of GSoC. I intend to investigate two options:

  • use Winxed (and parrot hashes) to implement python objects. I don't expect this to be particularly efficient, but I do expect it to be relatively low-effort and possible even with the limitations Parrot's current object system has.

  • use 6model. It's full of unknowns and I'm not convinced it can map to Python's object system cleanly and efficiently, in which case it wouldn't be an improvement over parrot's current object system.

The details of the object system can be changed later, ideally after Python3 on Parrot is bootstrapped.

I plan to develop the compiler & object system incrementally, and continuously test them, also against each other. This should improve the quality of the code and provide a better starting point for continuing the project, either by myself or others.

Project schedule:

Week 1

  • experiment with and decide between 6model or a custom object system
  • experiment with and decide between PIR, PBC or Winxed for compiler backend output

Week 2

  • write new tests or adapt old pynie tests for primitive python features. Even the most basic of tests in the official test suite already depend on a full object system, so a more frugal test suite is necessary for a while.
  • make sure all tests pass on good existing Python implementations (CPython, perhaps PyPy)

Week 3

  • initial PIR codegen for 'ast' module, proof of concept
  • int, arithmetic, assignment, print. tests for each
  • should use parrot types (PMCs or otherwise) directly
  • nowhere near python semantics yet

Week 4-6

  • start development on object system
  • object, dict, list, tuple, function, numbers, strings
  • at this point, some tests in the official test suite should start passing

Weeks 7

  • incrementally add features to compiler
  • in parallel, incrementally develop the object system to support compiler features
  • correctly implement scope/context

Week 8

  • more care towards the object system
  • classes, metaclasses, properties, inheritance

Week 9

  • at this point, Python3 on Parrot should start being useful, so some effort should be spent on I/O and other os interaction
  • file, open
  • subset of os module

Week 10-12

  • polish
  • try to make Python3 on Parrot release-ready (tests, user docs, perhaps a [PyPi] package)

Week 13

  • reserved for panic

Optional tasks:

  • investigate ctypes backends: NCI, PyPy's ctypes + libffi
  • if by some miracle Python3 on Parrot passes a majority of the CPython/PyPy test suite, start getting it bootstrapped by rewriting 'ast' module in pure-python

License: python's

References:

  • Python, CPython - python.org
  • CPython compiler - docs.python.org/devguide/compiler.html
  • PyPy - pypy.org
  • PyPi - pypi.python.org/pypi

Likely mentors:

  • allison

Bio:

  • I've done GSoC for the past two years with Sugar Labs, working on Browse. I am now the maintainer of the browser, in my (limited) spare time.
  • I'm currently doing a university module on language design, so I'm relatively familiar with the theory. I don't expect it to be terribly useful though, since I wish to reuse as much existing code as possible
  • I like both Python and Parrot. I am familiar enough with Python that I have a (long) list of things I dislike. I am sufficiently unfamiliar with Parrot that I have an even longer list of things I dislike. I expect the two lists to change over the course of this project.
  • I like snowboarding, but sadly I rarely get the chance to practice it.

Eligibility: I'm graduating from CompSci BSc this year.

@Whiteknight
Copy link

The timeline still seems light on details, but I suspect that this is necessary because of the complexity and multi-pronged nature of this project. Is it possible to break up the timeline into finer-grained chunks?

What testing and documentation tools are you going to use? I assume you're going to use whatever the "standard" solutions are in python world?

Have you read PDD31? Do you have any plans to create a PDD31-compliant compiler object, which could be used in other programs? Would that even be possible with this approach?

@lucian1900
Copy link
Author

I don't really have details until I start playing with it. I'll have a go anyway.

I was going to use whatever CPython uses, which afaik is unittest. Nothing special for docs, just python's docstrings.

PDD31 can be investigated after Python3 on Parrot is boostrapped.

@luben
Copy link

luben commented Mar 29, 2011

If CPython's binary ast lib do not depends heavily on the python interpreter, you could make bindings from parrot directly to python's ast lib using parrot's NCI and make bootstrapping compiler.

@lucian1900
Copy link
Author

I've thought of that, but it uses CPython structures a lot. I don't think it's feasible (certainly harder than rewriting the parser).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment