TOC:
If I miss anything in this document, it's because I've had a hard time tracing other Ruby magic ✨.
The Gitlab CE project is built with with Ruby on Rails framework. An (opinionated) MVC web framework that aims to create modern web applications "easier and more fun".
The comments/discussions is handled by the Note model. The displayed note is the result of the note text parsed in Gitlab Markdown Flavor. The note model has a "mentionable" which implements cross-referencing; keeping track of all (mentionable) references to other users, issues, merge requests, commits or snippets.
A lot of classes and modules are involved. There are extractors, parsers, pipelines and workers that make up the whole mentionable design work.
In the project, discussions and/or comments are represented by the model Note.
The model definition is located at app/models/note.rb.
The schema definition in app/db/schema.rb.
When a note is created, the controller responsible for this is app/controllers/concerns/notes_actions.rb.
To briefly explain what happens here, a function in the controller is mapped to an html request method following RESTful patterns: index for GET, create for POST, update for PATCH, destroy for DELETE. More on rails routing here.
Events that happen in create:
- Trigger create service - triggers build service - creates and validates the Notes model
- Render the note - performs a
cacheless_renderinitially - Returns a response of note formatted in both json and html
Events that happen in index:
- Prepare notes for rendering - performs render of notes -
cacheless_render - Returns the rendered json of notes
Events that happen in update:
- Trigger update service
- Updates attributes
- Triggers todo service - creates todo for mentioned and directly_addressed_users
Since displaying notes are common in many places, the view is in app/views/shared/notes/_note.html.haml
In here, the actual note text's context reference is redacted_note_html and there are a lot of these in the template itself.
Until this point, the view part, the rendering of notes were using the cacheless_render. It seems a bit off knowing the note display, redacted_note_html, is used in a lot of places. Let's go back to the notes
model and see where is this variable defined.
The redacted_note_html is defined as attr_accessor getter/setter instance variable. This is used as a reference for the Banzai::ObjectRenderer later as described here. The attr_mentionable is set to be the note text. The Mentionable separate concern makes use of these
variables too. The Mentionable concern # Contains functionality related to objects that can mention Users, Issues, MergeRequests, Commits or Snippets by # GFM references.
A worker for new notes performs a note post process service which executes create_cross_references of the note. This function # Create a cross-reference Note for each GFM reference to another Mentionable found in the +mentionable_attrs+.
In this part a value is set to redacted_note_html and is sent to the note instance
making it available from the note instance itself. ✨ Magic.
Gitlab Flavored Markdown (GFM)
The stack-trace can bubble up very quickly until we get to this point - all_references. This is where an extractor caches and analyzes the text from the mentionable_attrs.
The all_references function is then used to get mentioned_users and directly_addressed_users.
Inspecting further the Banzai::ObjectRenderer which is responsible for producing redacted
notes, the render calls the post_process_documents leading to documents being generated via Banzai::Pipeline that leads to GfmPipeline. Here, the filters convert Gitlab Flavored Markdown to HTML. The GFM-to-HTML
then should have a handler in the javascript. Each model should have a reference_pattern so that filters are able to convert to html.
An example User reference pattern would be
%r{
(?<!\w)@(?<user>((?:[a-zA-Z0-9_\.][a-zA-Z0-9_\-\.]*[a-zA-Z0-9_\-]|[a-zA-Z0-9_])/)*/(?:(?:[a-zA-Z0-9_\.][a-zA-Z0-9_\-\.]*[a-zA-Z0-9_\-]|[a-zA-Z0-9_])))
}x
# stripped-down. For full code see:
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/models/user.rb#L379-385
# https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/path_regex.rb#L149the pattern matches "@userhandle or @username but not user@example.com".
These matched patterns are mapped with the UserReferenceFilter which replaces user or group references with links. Example output:
<a href="/u/Lodovice" data-user="8" data-reference-filter="UserReferenceFilter" class="gfm gfm-project_member">@Lodovice</a>The extractor extracts possible Gitlab Flavored Markdown (GFM) texts_and_contexts and html_documents. The Gitlab Extractor defines methods in itself from REFERABLES. One of these referables include 'user'.
This is actually tricky and not very straightforward design. Nonetheless, it sets
users method assigned as the resulting references('user') call.
The references function processes html_documents that calls cache_collection_render but is now cached from the options.
The next time Banzai::NoteRenderer.render([@note], @project, current_user) is called from the notes_actions View, it retrieves the cached note's texts_and_contexts + html_documents.
The reference extractor class extends from ReferenceExtractor. This plays a part of the references('user') call. The result is a process call which
returns a list of HTML documents and returns an Array containing all the references. In short, the use of parser here is just to get the references and process their html equivalent.