- MetaBrainz flavored Markdown
- Markdown flavor for use in MetaBrainz projects
It aims to replace existing markup languages in *Brainz platforms with the following goals in mind:
- Ease text editing and copying/pasting
- Improve text markup capabilities
- Simplify code development
It is based on GitHub Flavored Markdown (GFM), that is, CommonMark with the following extensions: autolink table, strikethrough, disallowed raw HTML. It additionally features automatic link conversion of *Brainz references such as MetaBrainz ticket, MusicBrainz documentation, and so on.
Any reference to a ticket will be rendered as a link to that ticket. (Actually, it is an existing feature of BrainzBot as a plugin.)
OTHER-77 https://tickets.metabrainz.org/browse/OTHER-77
will both be rendered as:
<a href="https://tickets.metabrainz.org/browse/OTHER-77" class="improvement-issue" title="OTHER-77: Unify all markup syntax to simple markdown syntax">OTHER-77</a>
which looks like:
OTHER-77
- Pros:
- Automatically link to the issue.
- Issue title is displayed on link rollover.
- Displayed text is still short.
class
attribute can be used to display issue type icon too.- Gracefully degrade on platforms not supporting this feature.
- No con.
Implementation note: It requires to call JIRA’s REST API, already done in BrainzBot’s plugin: https://github.com/metabrainz/brainzbot-plugins/blob/master/botbot_plugins/plugins/jira.py
Any raw (unbracketed) URL of MusicBrainz edit will be rendered as a link with title and short content.
https://musicbrainz.org/edit/123456
will be rendered as
<a href="https://musicbrainz.org/edit/123456" title="Edit #123456">#123456</a>
which looks like:
#123456
Beyond host musicbrainz.org
, beta.musicbrainz.org
(and current host for MusicBrainz server) will be recognized too.
Rendered host will be the original one (or current host for MusicBrainz server).
Optionally, rendered link title
attribute can be localized.
To escape this special rendering, URL can be angle bracketed <https://musicbrainz.org/edit/123456>
.
- Pros:
- Rendered link content (hash and edit number) is shorter than the original link.
- Rendered link is still self-explanatory with its title displayed on rollover.
- Rendered link content integrates almost correctly into sentences of any language.
- Cons:
- Rendered link content cannot be localized, unless locale of the rest of the text is known.
Any absolute URL to a MusicBrinz edit will be displayed as a link titled with an hash and edit number:
https://beta.musicbrainz.org/edit/12345670#note-12345670-1
will be rendered as:
<a href="https://beta.musicbrainz.org/edit/12345670#note-12345670-1" title="Note #1 to edit #123456">#123456 (note #1)</a>
which looks like:
#123456 (note #1)
Optionally, rendered link title
attribute and parenthesis content can be localized.
To escape this special rendering, URL can be angle bracketed <https://musicbrainz.org/edit/123456#note-12345670-1>
.
- Pros:
- Rendered link content is shorter than the original link.
- Rendered link is still self-explanatory with parenthesis content and the title displayed on rollover.
- Rendered link content integrates approximately into sentences of any language.
- Cons:
- Rendered link content cannot be completely localized, unless locale of the rest of the text is known.
https://tickets.metabrainz.org/browse/MBS-764
It would be very interesting to have automatic link with entity name as title.
https://musicbrainz.org/artist/6a0b0138-dc06-4d5c-87b3-fab64f0fd326
will be rendered as:
<a href="https://musicbrainz.org/artist/6a0b0138-dc06-4d5c-87b3-fab64f0fd326" class="artist group-artist" title="Group: MusicBrainz Sound Team">MusicBrainz Sound Team</a>
which looks like:
MusicBrainz Sound Team
To escape this special rendering, URL can be angle bracketed <https://musicbrainz.org/edit/123456#note-12345670-1>
.
- Pros:
- Automatically link to the entity with entity name.
- Entity type is displayed on link rollover.
class
attribute can be used to display entity type icon too.- Gracefully degrade on platforms not supporting this feature.
- Cons:
- Rendered link content cannot be completely localized.
- MBID is hidden.
Implementation note: Discourse MusicBrainz Onebox plugin already retrieves such information: https://github.com/phw/discourse-musicbrainz-onebox
It is the only *Brainz reference that users might be typing by hand. It makes sense to have shortcuts and autocompletion for it.
Reference can be made using URI with doc
scheme.
doc:WikiDocs doc:Release_Group
It currently is case-sensitive, because of MediaWiki limitation.
It should be case-insensitive, once we move the documentation to Brainzed Markdown files.
Case-insensitive search as possible, of every WikiDocs page: https://wiki.musicbrainz.org/Category:WikiDocs_Page
doc:Release_Group
will be rendered as:
<a href="https://musicbrainz.org/doc/Release_Group" title="Documentation: Release Group">Release Group</a>
which looks likes:
Release Group
-
Discourse implementation of Markdown supports CommonMark and many GFM extensions (autolink table, strikethrough).
It is based on markdown-js which is extensible.
Discourse supports plugins too.
It seems feasible and useful to add support for automatic link conversion of *Brainz references.
TODO: Check Discourse (plugin?) support for disallowed raw HTML.
-
JIRA brings its own markup language for ticket description and comment. https://jira.atlassian.com/secure/WikiRendererHelpAction.jspa?section=all
There is currently no viable plugin to support CommonMark/GFM/… instead.
Reviewed plugins:
-
Markin - Markdown for Jira Cloud/Server https://fulstech.wordpress.com/markin-markdown-editor-for-jira/
Pros:
- It is based on markdown-it (JavaScript).
- It additionally supports a limited subset of JIRA’s markup language, that theoretically helps switching the renderer to Markin.
- It prevents code injection since version 10.0.2 released on 2018-03-14, this is close to disallowed raw HTML.
- It is actively maintained, see https://marketplace.atlassian.com/plugins/com.fulstech.jira-gherkin-custom-field/versions
Cons:
- Conflicting syntaxes are badly handled,
especially JIRA’s ordered list items
#
,##
, … which are rendered as Markdown ATX headers. - It comes with a text editor quite unfriendly compared to the stock JIRA’s wiki text editor. This issue has been reported at https://github.com/fulstech/markin/issues/48
- It doesn’t support rendering in activity stream This issue has been reported at https://github.com/fulstech/markin/issues/49
- It is not open source (github is only used an issue tracker), which means we cannot improve/modify it to fit our needs.
After a test period of ten days in MBS and OTHER projects, we uninstalled it, mainly because of its editor UI.
-
Markdown for Jira https://www.atlassian.com/blog/archives/markdown-support-for-jira https://marketplace.atlassian.com/archive/atlassian-jira-markdown-plugin
It is not compatible with the latest version of JIRA, and archived since 2013.
-
Markdown for Confluence https://marketplace.atlassian.com/plugins/org.swift.confluence.markdown/server/overview
It is not for JIRA, but it is based on the former Markdown for JIRA plug-in, and actively maintained.
It is now based on
flexmark
(Java). https://github.com/vsch/flexmark-javaOur only hope is that it gets forked back for JIRA.
In either case, it is probably neither feasible nor wanted to add support for automatic link conversion of *Brainz references.
-
-
MediaWiki brings its own markup language.
There seems to be few interest for supporting Markdown through extensions which are not developed anymore (one is archived, another is unmaintained, third one is still beta and has no recent commit) https://www.mediawiki.org/wiki/Extension:MarkdownSyntax https://www.mediawiki.org/wiki/Category:Markdown_extensions
Anyway, there is a plan to move most of MusicBrainz documentation out of the wiki.
It should then be fairly easy to approximtely convert it to Brainzed Markdown using Pandoc. https://github.com/philipashlock/mediawiki-to-markdown
AFAIK, it doesn’t support any markup language. Annotation, editor biography, and revision note are unformatted plain text fields.
- Review already supports CommonMark syntax, using Python-Markdown. It doesn’t support GFM extensions though. Some features could probably be useful: autolink and automatic link conversion of MusicBrainz entity reference. On the contrary, table and strikethrough seem to be worthless here.
-
Annotation comes with its own limited wiki formatting https://musicbrainz.org/doc/Annotation#Wiki_formatting
Each feature has an equivalent in CommonMark:
- italics
''…''
, bold'''…'''
, and bold italics'''''…'''''
can be respectively replaced with emphasis*…*
, strong**…**
, and strong emphasis***…***
- horizontal rule
----
is compatible with CommonMark - title three levels
= … =
,== … ==
, and=== … ===
can be respectively replaced with ATX headers# …
,## …
, and### …
- bulleted list
*
(with four leading spaces) can be replaced with bulleted list*
(without any leading space) - code block (with height leading spaces) is compatible with CommonMark
- link:
- autolink
url
is compatible with GFM’s autolink extension to CommonMark, with small differences:- trailing punctuation, parenthesis, and other special characters will be better handled. https://github.github.com/gfm/#extended-autolink-path-validation
- long URLs won’t be shortened anymore.
- untitled bracketed link
[url]
can be replaced with angle bracketed link<url>
with same small differences than above - undocumented wiki link
[CamelCase]
can be replaced with descriptive link[CamelCase](https://wiki.musicbrainz.org/CamelCase]
Note: there are many wiki links that contain spaces, e.g.[Fictitious Artist]
. Only wiki links that actually link to the wiki shall be replaced. - descriptive link
[url|description]
can be replaced with[description](url)
- undocumented double-bracketed descriptive wiki link
[[CamelCase|description]]
can be replaced with[description](https://wiki.musicbrainz.org/CamelCase)
- autolink
- square bracket HTML characters entity references
[
and]
are compatible with CommonMark, but they are not needed to write special entity name such as[unknown]
in CommonMark
Migration path (of about 280K latest annotations and 229K previous annotations):
-
Display annotations prior to the syntax change with wiki markup.
-
Gradually enter edits with ModBot to convert latest annotations only.
-
Drop support for wiki markup rendering of annotations prior to the change.
-
Pros:
- Keep track of changes, thus allowing editors to manually fix incorrect conversion by comparing with the original version.
- No need to maintain wiki markup rendering code.
-
Cons:
- A lot of annotation edits (should these not be notified to subscribers?).
- Previous annotations won’t be correctly displayed anymore.
- Heterogeneous data.
- italics
-
Edit note syntax is close to the one for annotation with few differences: https://musicbrainz.org/doc/Edit_Note#Edit_note_syntax
Basic features italics, bold, and autolink are the same as for annotation and have an equivalent in CommonMark, see above section.
Specific features are:
-
Linking to an edit can be replaced with plain edit URLs.
More specifically,
edit #123456 edit:123456 edit 123456
can all be replaced with:
edit https://musicbrainz.org/edit/123456
See the section “MusicBrainz edit URL” for rendering.
-
Linking to WikiDocs page has two forms:
- URI
doc:Release_Group
which is still supported, with autocompletion eventually. - WikiLink
[Release_Group]
which can be replaced by the abovedoc:Release_Group
form.
- URI
Migration path (of about 29M edit notes):
-
Create a table
old_edit_note
to backup converted edit notes. -
Gradually convert every edit note that requires any change. (The most reliable conversion is HTML rendering since limited HTML tags are allowed in Markdown.)
-
Pros:
- Keep track of changes, thus allowing coders to eventually fix incorrect conversion by comparing with the original version.
- No need to maintain wiki markup rendering code.
- Homogeneous data.
-
Cons:
- Potential conversion bug might damage edit notes, until reported by editor and fixed by coder.
- Massive updates of the table
edit_note
. - Additional legacy table
old_edit_note
.
-
-
Editor biography syntax is close to the one for annotation too. (It is undocumented.)
Migration path (of about 750K editor biographies, mostly spam) based on editor last updated date:
-
Display biography of outdated editor with wiki markup rendering.
-
When editor visits the website:
- Make the conversion
- Send both the previous version and the new one by email
-
After six months, drop support for wiki markup.
- Optionally, every outdated editor biography may be converted without notification then.
-
Pros:
- Keep track of changes, thus allowing editors to manually fix incorrect conversion by comparing with the original version.
- Only one notification email sent to active editors only.
- No need to maintain wiki markup rendering code.
-
Cons:
- Potential conversion bug might break editor biography presentation, until noticed and fixed by editor.
- Inactive editors won’t be notified of this change.
-
-
Event setlist comes with its own limited syntax: https://musicbrainz.org/doc/Event/Setlist#Syntax
The syntax is mostly about the constrained structure of setlist. It is a loose legacy from setlist.fm which doesn’t matter anymore. We can use CommonMark instead and implement a constraints validator.
Since event setlist is constrained, it makes it very similar to tracklist. It would make more sense to have a dedicated visual editor instead of fiddling around a text editor. It would still be possible to have a setlist parser to copy/paste setlist. It would even make more sense to store setlist in dedicated tables (with artist credit, durations, bells and whistles) rather than in text field, but it would require far more work.
Although this migration is an opportunity to modify the structure of event setlist at the same time, the following is only about 1:1 equivalents as a subset of CommonMark.
-
@
for artists: It can be replaced with CommonMark’s third level ATX header###
. If necessary, “Artist:” label can be displayed through additional postprocessing. -
*
for works/songs: It can be either dropped (just using a simple line) which has similar rendering, or kept as CommonMark’s bulletted list*
with a better rendering. -
#
for additional info: It can be replaced either with parenthesis(…)
around the line with a different (but convenient?) rendering, or replaced with blockquotes>
. -
links
[MusicBrainz ID|name to be displayed]
It can be easily replaced to CommonMark’s link with description, depending on the context. Examples:@ [e1f1e33e-2e4c-4d43-b91b-7064068d3283|KISS]
would be converted into[KISS](https://musicbrainz.org/artist/e1f1e33e-2e4c-4d43-b91b-7064068d3283)
* [8dddc197-9ef4-4b5a-9908-9310230c1237|What’s It Gonna Take?]
would be converted into[What’s It Gonna Take?](https://musicbrainz.org/work/8dddc197-9ef4-4b5a-9908-9310230c1237)
What are implications:
- Syntax is a bit more verbose. However, it is easier to copy/paste an URL rather than an MBID.
- Any link of the form
(((scheme)?host)?/)?entity_type/mbid
shall be accepted and replaced and with a proper link tohttps://musicbrainz.org/…
before storing the event setlist. - Links point to
musicbrainz.org
rather than to the current host. A postprocessing replacement can be added to render current scheme/host rather than https/musicbrainz.org
Migration path (of about 10K event setlists) based on event last updated date:
-
Display event setlists prior to the syntax change with previous renderer.
-
Gradually enter edits with ModBot to convert every event setlist.
- Optionally, these edits can be ignored for notifications.
-
Pros:
- Keep track of changes, thus allowing editors to eventually fix incorrect conversion by comparing with the original version.
- No need to maintain old event setlist markup rendering code.
- Homogeneous data.
-
Cons:
- Potential conversion bug might damage event setlist, until reported and fixed by editor.
-
It is already supported by Discourse for Community forums.
There is a similar feature in Tickets issue tracker with JIRA syntax [~username]
.
There is a feature request for this in MusicBrainz. https://tickets.metabrainz.org/browse/MBS-9240
However, it has implications far beyond text formatting, more specifically about username normalization and notifications policy.
In MusicBrainz:
- For event setlist:
- Syntax will be completely replaced, and extended with additional features such as linking to a recording.
- For annotation, edit note, and editor biography:
- Italics and bold have different syntax.
- Linking to an edit won’t be supported anymore, but edit URLs will be displayed with the same result.
- Linking to WikiDocs
[CamelCase]
won’t be supported anymore, butdoc:CamelCase
will. - Descriptive link `[url|description] won’t be supported anymore,
- Wiki links and entity links won’t be supported anymore, but
For existing content, see migration paths in above section.
For ticket links, it might be nice if that included additional markup based on the ticket state; for example, add the ticket-type icon in front, and use strikethrough for closed tickets (a bit like what you get for issue links in Jira itself). Priority and status are other potential items to include.
While I like the doc: and edit: prefixes, I do agree that things should not be MusicBrainz-centric. If the plan is to use the same terms everywhere (although it's already fairly clear from BB that this is not the case), it could just be
edit:mb:xxx
anddoc:mb:Foo
. Otherwise, I don't see a problem withmb:doc:Foo
/mbdoc:Foo
/mbedit:1234
. I'd perhaps also expect this to be special URL syntax only (so just shorthand inside [], not just anywhere).