Skip to content

Instantly share code, notes, and snippets.

@bronson
Created June 8, 2015 21:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bronson/e2238725b8e47aa248e8 to your computer and use it in GitHub Desktop.
Save bronson/e2238725b8e47aa248e8 to your computer and use it in GitHub Desktop.
bobrock [11:11 AM]
I assume Atom grammars can be written in JSON interchangeably with CSON?
lee-dohm [12:25 PM]
I believe so, but I’ve never tried it
thomasjo [12:26 PM]
Yes.
synchronous [12:48 PM]
joined #atom
bobrock [1:02 PM]
Indeed, it seems to be so. I suppose Atom is smart enough to use JS/CS and JSON/CSON in all the cases I’ve run into.
bobrock [1:04 PM]
Does Atom use the exact same syntax parsing rules as TextMate, or just the format of the grammar definitions? TextMate has a limit in that its parser only looks one line at a time and it gets _way_ complex to handle very intricate language structures.
thomasjo [1:41 PM]
It's similar, but not same. https://github.com/atom/first-mate
GitHub
atom/first-mate
first-mate - TextMate helpers
bobrock [2:49 PM]
First Mate is a core plugin for grammar parsing? Are it’s rules documented separately, or just in the code?
bobrock [2:50 PM]
Hmm. Maybe not a plugin, but part of Atom’s core?
bobrock [2:52 PM]
And not that, either! It’s a node module within the Atom app package.
lee-dohm [2:59 PM]
It is part of Atom’s core as an npm module, yes
lee-dohm [3:00 PM]
First Mate is based on the grammar system from Text Mate (which is also used by Sublime), it is almost identical … though there are bugs that are not shared on each side
lee-dohm [3:01 PM]
That is why you can convert a Text Mate grammar for use in Atom
lee-dohm [3:02 PM]
There isn’t much documentation for First Mate … but you can use the Text Mate documentation
bobrock [3:02 PM]
That’s handy because I’ve found this to be an awesome reference and tutorial on TextMate grammar creation: http://www.apeth.com/nonblog/stories/textmatebundle.html
lee-dohm [3:02 PM]
Yeah, I’ve used that reference before :simple_smile:
bobrock [3:11 PM]
And RegEx makes my head hurt. :disappointed:
lee-dohm [3:11 PM]
Hehe
lee-dohm [3:12 PM]
I’m decent with regular expressions … until you start including lookahead, then I fall down go boom
bobrock [3:12 PM]
:simple_smile: I may have ask for some advice as I get going then.
lee-dohm [3:13 PM]
I’d be happy to help
bobrock [3:14 PM]
Thanks. I _think_ I’d like to play with some improvements to the JavaScript grammar.
lee-dohm [3:15 PM]
It would be really helpful if you could resolve the issues in the bug I linked yesterday
bobrock [3:15 PM]
Which was that?
lee-dohm [3:15 PM]
Let me find it
lee-dohm [3:15 PM]
https://github.com/atom/language-javascript/issues/137
GitHub
Many symbols without scopes · Issue #137 · atom/language-javascript · GitHub
The support for js is pretty bad. Most of the issues are in the way of parsing variable names. Load this example into atom. When I say something is not tagged I mean its scope is only .source.js. import {x, y} as p from 'point'; // ...
bobrock [3:18 PM]
Oh, yeah. I commented on that. I guess I thought of it as more of an improvement than a bug fix; but needed, nonetheless.
bobrock [3:19 PM]
This can be handy for RegEx newbies: http://www.myezapp.com/apps/dev/regexp/show.ws
An online utility that helps analyzing regular expression structure.
bobrock [3:22 PM]
And here’s a concept to damage your brain: ZeroWidthNegativeLookahead!
lee-dohm [3:30 PM]
Hehe yeah
bobrock [3:32 PM]
I’m wondering if, because TextMate has been around so long and has the same poor JavaScript grammar (Atom’s came from that one looks like verbatim), if TextMate’s parser is just not up to the language? Is Sublime’s JavaScript grammar “good?"
lee-dohm [3:33 PM]
I don’t know … I didn’t use JavaScript when I was using Sublime
bobrock [3:34 PM]
I’m going to look - I just started up v3 and see it has had 4 updates in May. Maybe he feels Atom’s footsteps behind him?
lee-dohm [3:34 PM]
I suspect that part of the problem is that because JavaScript is such a free-form language that there often isn’t much context to make some grammatic determinations
thomasjo [3:34 PM]
You can't detect if something is a variable with regex; that requires a real parser. Essentially you need an AST.
lee-dohm [3:35 PM]
Yep
thomasjo [3:35 PM]
That's way outside the scope of the syntax highlighter.
bobrock [3:35 PM]
Dang! Sublime looks more like Atom (or the opposite) than I remember!
lee-dohm [3:35 PM]
Hehe
bobrock [3:36 PM]
@thomasjo: Maybe that’s why Brackets’ JavaScript parsing seems better? CodeMirror uses a real parser, I believe.
bobrock [3:38 PM]
Indeed, Sublime’s is no better than Atom’s. :disappointed:
lee-dohm [3:38 PM]
Brackets uses tern.js which yes … does have an actual JavaScript parser, to my recollection
bobrock [3:38 PM]
Maybe Atom just won’t be capable of a good JavaScript grammar.
thomasjo [3:39 PM]
You're confusing things.
lee-dohm [3:39 PM]
:laughing:
bobrock [3:39 PM]
But I swore Brackets used CodeMirror.
thomasjo [3:39 PM]
CodeMirror uses tern.
lee-dohm [3:39 PM]
bobrock, given enough time Atom will be able to have anything and everything that other tools have
lee-dohm [3:40 PM]
We could plug in a native-compiled AST engine and do whatever we like, given enough time
thomasjo [3:41 PM]
Aye
bobrock [3:41 PM]
I suppose Atom will have to evolve to a parser based syntax system as Brackets and PhpStorm have. But I think that writing a JavaScript parser is orders of magnitude more difficult than the simple syntax rules.
bobrock [3:41 PM]
“simple"
thomasjo [3:41 PM]
Have been thinking about experimenting with something like AC+, but for syntax highlighting. For anything running on llvm it would be almost trivial to provide a proper AST.
thomasjo [3:42 PM]
For JS we have tern, which is pretty decent.
lee-dohm [3:42 PM]
Actually, the issue is not a “writing a X parser"
lee-dohm [3:42 PM]
The issue is “writing a re-entrant X parser"
lee-dohm [3:42 PM]
Most parsers are designed to puke at the first error
thomasjo [3:42 PM]
And we don't need to write one; tern.js already exists.
thomasjo [3:43 PM]
We just need to find a good (, extensible) way of leveraging it.
lee-dohm [3:43 PM]
A syntax highlighter solves a different problem …
lee-dohm [3:43 PM]
It needs to highlight code correctly even if there is an error in the middle of things
thomasjo [3:43 PM]
Indeed
lee-dohm [3:44 PM]
Tern.js solves this problem … but _only_ for JS
thomasjo [3:44 PM]
llvm solves it for its "gang".
lee-dohm [3:44 PM]
What we need is an engine that we can write parsing rules for _any_ language … that is also highly performant :grinning:
lee-dohm [3:45 PM]
Oh … and the rules have to be re-entrant, so they recover at the lowest possible level in all cases
thomasjo [3:45 PM]
Agreed.
lee-dohm [3:45 PM]
I would love to quit my job and just work on something like that full-time … but nobody has offered me enough money yet :simple_smile:
bobrock [3:46 PM]
CodeMirror has lots and lots of language “modes.” Why do you say tern.js is limited?
thomasjo [3:46 PM]
Basically if we were to go the custom route, we should write it in C++ IMHO. That way we'd be able to tune the perf to the ridiculous level where we take advantage of the CPUs prefetcher etc.
thomasjo [3:47 PM]
Nobody has said that.
thomasjo [3:47 PM]
tern.js only supports JavaScript. But tern is only one addon used by CodeMirror.
bobrock [3:47 PM]
@lee-dohm: I thought you said tern.js only helps for JS? No?
lee-dohm [3:48 PM]
From the tern.js home page at http://ternjs.net:
lee-dohm [3:48 PM]
> Tern is a stand-alone code-analysis engine **for JavaScript**. It is intended to be used with a code editor plugin to enhance the editor's support for intelligent JavaScript editing.
bobrock [3:48 PM]
Oh, the rabbit hole goes deeper!
lee-dohm [3:48 PM]
Emphasis mine
lee-dohm [3:48 PM]
So yes, tern.js only supports JS …
lee-dohm [3:49 PM]
AND yes … CodeMirror supports other languages
lee-dohm [3:49 PM]
That doesn’t mean that CodeMirror uses tern.js to support those other languages
bobrock [3:49 PM]
Yes, the rabbit hole...
bobrock [3:50 PM]
Since Atom has it’s own home-written editor, I guess it does get complex.
lee-dohm [3:51 PM]
Yeah, some of these things are tough graduate-level problems
bobrock [3:52 PM]
:simple_smile:
steelbrain [6:39 PM]
Docs at https://atom.io/docs/api/v0.204.0/ViewRegistry#instance-addViewProvider suggest using
```atom.views.addViewProvider
modelConstructor: TextEditor
viewConstructor: TextEditorElement
```
to add a view provider, however I get a deprecation warning which says 'atom.views.addViewProvider now takes 2 arguments: a model constructor and a createView function. See docs for details.'
Atom
A hackable text editor for the 21st Century
At GitHub, we’re building the text editor we’ve always wanted: hackable to the core, but approachable on the first day without ever touching a config file. We can’t wait to see what you build with it. (32KB)
(edited)
lee-dohm [6:41 PM]
Yes, the example code is out of date
steelbrain [6:41 PM]
How do I fix it?
lee-dohm [6:42 PM]
Fix the example code? Go into the Atom source and change the documentation comments
steelbrain [6:42 PM]
Fix my package code
steelbrain [6:42 PM]
``` @Subscriptions.add atom.views.addViewProvider
modelConstructor: Panel
viewConstructor: PanelView
```
lee-dohm [6:42 PM]
If you look at the documentation you linked … there is a `.createView()` parameter you can pass
steelbrain [6:43 PM]
But it'll still be the part of the first argument
steelbrain [6:43 PM]
The warning states that there's a second argument, not a third property to the first argument
lee-dohm [6:43 PM]
It’s still the same kind of function though
lee-dohm [6:43 PM]
Just pass it as a second parameter
steelbrain [6:44 PM]
Found something here, which is completely different https://github.com/atom/autocomplete-plus/blob/6f62361e9bf1b80e005d72f5c5e96158b95de43a/lib/autocomplete-manager.coffee#L42
GitHub
atom/autocomplete-plus
autocomplete-plus - View and insert possible completions in the editor while typing
lee-dohm [6:44 PM]
Yep … that’s the right way to do it
steelbrain [6:45 PM]
Wow :smile:
----- May 31st, 2015 -----
lixquid [2:41 AM]
joined #atom. Also, @muhasturk joined.
muhasturk [5:22 AM]
this channel official?
thomasjo [5:23 AM]
Yes
bobrock [5:23 AM]
I guess. I see lots of core developers here.
thomasjo [5:23 AM]
No need to guess.
lee-dohm [5:25 AM]
It was created by the Atom Core team specifically for the purpose of communicating more easily with the community
muhasturk [5:49 AM]
I just wonder, Do developer core team think about to use TypeScript in the future instead of CoffeScript?
lee-dohm [5:53 AM]
I think ES6 JavaScript is more what they would be thinking than TypeScript
lee-dohm [5:53 AM]
If you’re talking about for writing Atom Core or default-installed packages
lee-dohm [5:54 AM]
But, to my knowledge … there are no plans to change Atom Core away from CoffeeScript
lee-dohm [5:55 AM]
They do want ES6 JavaScript to be fully supported including documentation
muhasturk [5:58 AM]
thank you 4 info
lee-dohm [5:59 AM]
You’re welcome
gimiki [7:11 AM]
joined #atom
bobrock [9:03 AM]
I’ve been playing with Brackets a little this morning. I’ll have to give Atom a lot of credit for being a more mature product even before v1 than Brackets is at v1.3.
bobrock
[9:07 AM]
But its JavaScript syntax coloring sure it pretty!
457KB JPEG in #atom • 2 comments • Open original
muhasturk [9:16 AM]
what theme is this?
muhasturk [9:16 AM]
there must be similar one for Atom
bobrock [9:22 AM]
It’s not so much the theme as the parser that scopes the tokens to be colored by the theme. Atom’s JavaScript grammar can’t handle the language well.
stanzilla [9:40 AM]
that's weird considering it's the one it's built in :stuck_out_tongue:
bobrock [9:53 AM]
@stanzilla: Yes, weird and somewhat sad.
thomasjo [9:58 AM]
Nope, not weird at all. @lee-dohm and myself explained the reasons yesterday. A syntax highlighter which operates based on merely parsing syntax can never handle what you want. And as we told you, Brackets uses tern.js to get an AST for JavaScript rendered in the editor, and via the AST they are naturally able to provide richer syntax highlighting since they have more data.
bobrock [10:01 AM]
@thomasjo: No, not so much weird as in “I don’t understand why that is,” but weird as in “It seems to be an odd design decision to use a syntax system that does not support the language Atom’s written in."
thomasjo [10:01 AM]
There's nothing preventing anyone from beginning work on a richer highlighting service, ala autocomplete-plus, but this is not a concern for the core team as of right now. The current system is more than sufficient. It's exactly the same concept being used in Sublime, TextMate, Vim, etc.
thomasjo [10:01 AM]
It supports JavaScript perfectly well... what makes you think it doesn't?
bobrock [10:02 AM]
“Supports” has many meanings, I suppose. By support, I mean “can pick out the language tokens."
bobrock [10:03 AM]
Support well, might be more the idea.
bobrock [10:03 AM]
And it’s not an impediment to using Atom, just a slight inconvenience.
feross [10:04 AM]
joined #atom
thomasjo [10:04 AM]
Brackets started as an IDE for developing web apps/sites. Atom is a general purpose text editor. Their foundations are as such wildly different.
thomasjo [10:05 AM]
It makes perfect sense that the Adobe team put tons of effort into adding IDE features, but it would make zero sense for the Atom team to do the same, seeing as they were initially trying to solve different use cases.
thomasjo [10:05 AM]
Brackets has since evolved of course.
bobrock [10:06 AM]
True enough, and Atom seems to have evolved further. As I said, it seems much more mature than a pre-release product.
eowo [10:42 AM]
joined #atom. Also, @impresario joined.
azakur4 [11:53 AM]
I was a user of brackets but i have to say that brackets is a editor focus in frontend work. I think maybe if for that they focus in a good support for javascript and other stuffs but almost to me the support for php for example was very poor, some members of the brackets community work a lot to try get better but i think atom is very superior in that part. maybe this is because both teams have different focus
azakur4 [11:54 AM]
actually observe a lot of the new release in sublime text and in the last versions they are implementing a new engine for syntax highlighting
azakur4 [11:56 AM]
and that was expected because now i think people can add more syntax highlight adding some "tmLanguage" files with definitions (Almost until i see they work with regex too).
azakur4 [11:56 AM]
im waiting to see how it works maybe we can import they syntax highlight like we made with textmate
azakur4 [11:57 AM]
if they get a better support
shashank [12:24 PM]
joined #atom
bobrock [12:54 PM]
@azakur4: Do you know what extra capabilities the new Sublime syntax coloring engine might bring?
mrchrisadams [1:04 PM]
are there any atom plugins that do anything like reduce contrast on panes that don't have focus when editing?
mrchrisadams [1:04 PM]
or provide any further clue than the blinking cursor?
bobrock [1:07 PM]
What sort of panes are you thinking of?
lixquid [1:16 PM]
If you're thinking of windows, windows that don't have focus have the `is-blurred` class applied to the `body` tag
lixquid [1:16 PM]
You could style that in your custom stylesheet
azakur4 [1:21 PM]
@bobrock: i don't know what extra capabilities have but i think we can make a little of reverse engineering
thomasjo [1:22 PM]
@mrchrisadams: This likely has some issues, but add something like this to your `~/.atom/styles.less` file:
thomasjo [1:22 PM]
```atom-pane:not(.active) {
opacity: 0.5;
}
```
(edited)
azakur4 [1:22 PM]
starting from the files "sublime-syntax" (sorry before i mention tmLanguage but i see the sublime site and he talk about ".sublime-syntax" file)
lixquid [1:29 PM]
@mrchrisadams: @thomasjo: This'll blur on a pane without focus, as well as the entire window being unfocused:
```body.is-blurred atom-pane,
atom-pane:not(.active) {
opacity: 0.5;
}
```
lee-dohm [1:31 PM]
Sublime has always had `sublime-syntax` files that are based on TextMate `tmLanguage` grammars
clandestine [1:32 PM]
joined #atom
lixquid [1:32 PM]
If I recall correctly, Sublime can parse both tmLanguage and sublime-syntax files, but uses different parsing for each (edited)
lee-dohm [1:34 PM]
The sublime-syntax files were just the same content as tmLanguage but stored as JSON, I believe
lee-dohm [1:34 PM]1:34
Instead of XML PList files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment