Skip to content

Instantly share code, notes, and snippets.

@jspahrsummers
Created July 11, 2012 09:33
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jspahrsummers/3089293 to your computer and use it in GitHub Desktop.
Save jspahrsummers/3089293 to your computer and use it in GitHub Desktop.
What programming language or environment features are important to you?

Many Cocoa programmers, myself included, want a more modern programming language than Objective-C, and a more modern programming environment than Cocoa. There's been some talk about what would better fit the bill, and engineers are ready to build that solution, but the question first needs to be answered – what are the most important problems to solve?

I've created the following unordered list of hypothetical goals for improvement to Cocoa/Objective-C, and would like to see how important each one is to the community. If you're interested in sharing your opinion, please fork this gist and order the below list. Feel free to add anything that you think might be missing.

Thanks!

Potential Goals

  • Functional programming, where function application is the main way to invoke code. This does not refer to using objects in a functional style.
  • Referential transparency, also sometimes known as purity or immutability.
  • Easy parallelization, meaning it should be easy to make code run concurrently without ill effect.
  • Algebraic data types, in addition to or instead of Objective-C objects and inheritance.
  • Statically typed, to catch type errors at compile-time. This does not necessarily imply a C-like type system – many modern languages support type inference, among other things.
  • Powerful metaprogramming, to automate code generation or extend the language with itself.
  • Easy to learn (although subjective, it's still interesting to rank it against other goals).
  • Self-hosting, where the toolchain is written in the language/environment itself.
  • Compiled, instead of interpreted. This could mean compilation to an intermediate language (like Objective-C, C, or LLVM IR), not necessarily machine code.
  • Performant – in other words, how important is performance relative to expressiveness and flexibility?
  • Not Cocoa-specific, so that it is easy to port or slightly modify code to run without Cocoa frameworks. This mostly just implies the use of a language that already exists and is used for other purposes.
  • An existing body of open source code, available for use within a Cocoa project.
  • Can be invoked from Cocoa code – not having this implies always invoking Cocoa instead.
  • Can invoke Cocoa code – not having this implies always being invoked from Cocoa instead. Note that invoking Cocoa and being invoked from Cocoa are not mutually exclusive (but one is necessary).
  • Can define Objective-C classes, protocols, categories, etc. in the language itself. This doesn't imply anything about being able to link to custom Objective-C code.
  • Usable for AppKit/UIKit GUIs, in addition to being usable for a model layer (which is a requirement).

Note that this list does not, and is not meant to, capture the advantages and disadvantages of each line item.

@jeremytregunna
Copy link

Requirement should be compiling down to C (preferably, I'll explain why in a second) or LLVM. The goal with compiling down to C would allow us to embed C right in the language itself, allowing to cover corner cases in much the same way that the asm() construct works in C. This code would be inlined right into the compilation result. This would liberate the language syntax from needing to support anything in the same way C does to meet that interfacing with C goal. Plus, we can write objc apps entirely in C if we're masochistic and missing the point, I think it a fair target for computer generated code. Interfacing with C and writing LLVM IR can be a bit much, though that is the flip side of this suggestion.

Remember, not everything need be compiled ahead of time. You can do as much as you can ahead of time, then with the help of a lightweight profiler, you can detect particular hotspots in your program (that may extend over method call bounds) and compile those just in time, leaving the rest of the code on the slow path (the code that isn't hot); this saves time in the JIT. Additionally the JIT can support a monomorphic and polymorphic inline caches on the language side of things, reducing the cost of hitting libobjc (which would be one layer out). If a call site is seeing more than one type, a fixed number of cache lines at the call site can be used to cache object -> selector -> IMP tuples for faster access. Obviously, special care must be taken here in the context of whatever memory management subsystem is used.

On the memory management note, the language grammar should free the developer as much as possible from having to annote things __autoreleasing, __bridge, __block by being smarter when analyzing the graph. We shouldn't have to declare these things in most cases if the compiler were smart enough. I believe this a worthy goal as well.

Finally, encourage less use of global/instance state. There are many ways one can go about this, for instance, full parametric polymorphism outside of objects (in helper functions). However this creates a duality in the system which increases cognitive burden on users of the language, so probably not a good idea. Alternatively, can simply model only behaviour, and not state, and given access control specifiers (public, private, ...) automatically generate ivars or whatnot for the user, but not allow them direct access to it. There must be a mechanism to subvert these system protections if the user needs, for instance, a custom setter; but this can be achieved with my first suggestion trivially. Merely naming rules have to be given to the generated code.

@jspahrsummers
Copy link
Author

@jeremytregunna Some interesting ideas. I think a JIT, in particular, is mostly a non-starter, as it won't work for the iOS or Mac App Store. The latter couple paragraphs seem to be referring to improving ARC and other artifacts of imperative programming – is that correct? Do you have opinions on the goals presented in the list above?

@jeremytregunna
Copy link

Sure. Some of the aspects of a JIT would easily fly on the AppStore. The inline caches for instance, will be fine, and for many types of code, helpful.

With respect to functional programming. It's been my experience with functional languages that they can help for about 75 ~ 80% of the work we do. That said, don't forget, we're operating with inherently imperative code that loves its conditionals, and other such constructs you don't find in pure functional languages. Let's not loose sight of the fact that you won't be able to build a pure functional language and hit UIKit/AppKit (even Foundation for that matter) in the above stated ways.

Your definition of referential transparency is wrong for what I know it to be in my mind, and even according to the wikipedia article you link it to. That said, the stated desire I think is a good goal.

ADTs would be awesome to have available, and you can actually turn on support for type inference in the objc frontend right now with one line of code. literally changing an if(C++11) to an if(C++11 || ObjC). There are apparently reasons it wasn't done inside Apple from what I'm told, though I'm not privy to those details (I'd assume them to be good reasons).

I forget who said it, but "Any language worth its weight in double density floppy disks is self hosting." Probably narrows down who could have said it based on the content of the quote. I'm very much in favour of this goal as well.

With respect to performance, I think such a language needs to be close to if not as fast as Objective-C standing on its own. And shouldn't be much if any slower when talking to the frameworks.

@jspahrsummers
Copy link
Author

@jeremytregunna Sorry, didn't realize that the Mac App Store (apparently?) has no guideline against JITed code. The point about iOS remains, though. Thanks for your feedback!

@jeremytregunna
Copy link

The thing about inline caches is they don't recompile any code on the fly. Your message sending mechanism (the thing that hands off to objc_msgSend) would maintain a list of cache lines which hold a type, a selector and an IMP (the IMP being the cached IMP the selector represents on the particular type).

Before you send, you check the finite set of cache lines for a match, if your type is in it, and if it is check if the selector you want to send is in it as well. If both those predicates hold true, you fire the cached IMP. It's a short circuit, nothing more.

The problem with inline caching is with the bookkeeping. They're expensive if you go megamorphic. So in general, you want to limit the number of cache lines to something small, and when you run out, default to the slow path, or if your profiler is seeing a new type over and over again on the slow path, evict the least used member out of the polymorphic cache and add this new type; optimize the call sites inline cache. This is only really useful in things like loops, otherwise this optimization would rarely if ever be done.

I've used this technique in some iOS apps myself where profiling dictated, but having the runtime do it for me would be awesome.

@joshaber
Copy link

  1. Can invoke Cocoa code
  2. Functional programming
  3. Powerful metaprogramming
  4. Referential transparency
  5. Easy parallelization
  6. Algebraic data types
  7. Statically typed

I honestly don't care about the others.

@joshaber
Copy link

To expand on that, the ability to invoke Objective-C solves a lot of the other problems.

Performance problem? Drop down to Objective-C.
No 3rd party lib in NEW_LANG? Use an Objective-C one.
Need to define a new class, protocol, etc? Drop down to Objective-C.
UI code? Use Objective-C.

All those things would be nice but I'd say they're not required.

@jonsterling
Copy link

Yeah. One of the (few) great strengths of Objective-C is the ability to drop down to C when necessary. I'd like for it to be pretty effortless to integrate with new and existing Objective-C code.

@plukevdh
Copy link

I second Josh's first comment. Also adding two way communication between cocoa and newlang as important.

@jspahrsummers
Copy link
Author

@joshaber @jonsterling @plukevdh Thanks for the feedback!

@plukevdh
Copy link

Out of curiosity and more to understand what you're thinking: i feel like metaprogramming and compiled dont mix very well. Most metaprogramming i've seen requires having knowledge of the program based on information built at runtime. Maybe i just havent seen enough compiled-meta. It would certainly be interesting to see.

@jspahrsummers
Copy link
Author

@plukevdh Sounds like you're talking specifically about reflection. Objective-C already has pretty powerful facilities for that, but it often lacks the power to automate code generation or extend the language itself (for instance, by adding new operators). Those kinds of things can be done pretty well at compile-time, though this would be in newlang and not Objective-C.

@plukevdh
Copy link

Okay, I have not seen introspection in objc. I was more interested in dynamically changing or adding methods to live code. Have not seen that in compiled code before.

@jeremytregunna
Copy link

One thing I just realized is missing from that list is a modular design. Libraries, libraries, libraries! Will make building tooling so much easier.

@jspahrsummers
Copy link
Author

@jeremytregunna Do you mean a modular design for the toolchain or support for modular design using newlang?

@jeremytregunna
Copy link

A modular design for the toolchain naturally. The latter goes without saying, but the former is important for good tooling.

@jonathanpenn
Copy link

It sounds like this project is trying to solve the same problems. Might be worth a look: http://eerolanguage.org

@jeremytregunna
Copy link

It's not trying to solve them the way outlined above. For instance, [] represents a mutable array in eero, and '' a mutable string. That's a far cry from what's outlined above as a (seemingly) core tenant of newlang.

@jspahrsummers
Copy link
Author

@jonathanpenn @jeremytregunna I think maybe the larger point is that eero seems primarily focused on syntactical improvements (though there are some semantic ones as well), keeping everything squarely within the traditional Objective-C paradigm. The goal of this project would be to offer up a completely new paradigm and environment which can still remain compatible.

@jonathanpenn
Copy link

@jeremytregunna @jspahrsummers Ah, I see what you're going for. Well, maybe it would be a good project to learn from for Xcode integration. I'm excited to see where this gist leads!

@jspahrsummers
Copy link
Author

@jonathanpenn Thanks!

@jspahrsummers
Copy link
Author

For what it's worth, here's my list (with things I don't care about omitted):

  1. Can be invoked from Cocoa code
  2. Functional programming
  3. Statically typed
  4. Algebraic data types
  5. Referential transparency
  6. Powerful metaprogramming
  7. Easy parallelization
  8. Can invoke Cocoa code
  9. Can define Objective-C classes, protocols, categories, etc.
  10. Usable for AppKit/UIKit GUIs

@jonsterling
Copy link

jonsterling commented Jul 13, 2012 via email

@jspahrsummers
Copy link
Author

@jonsterling Well, this is why I personally prioritize referential transparency over invoking Cocoa. I favor the approach that @joshaber was proposing, whereby functional code can be embedded in an Objective-C program on a "call-into" basis (perhaps with some bridging of immutable Objective-C objects). Then there are no conflicts between the two models.

EDIT: Also, referential transparency is less important to me than static typing, which Clojure definitely does not have (to any serviceable extent).

@jonsterling
Copy link

jonsterling commented Jul 13, 2012 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment