Instantly share code, notes, and snippets.

Embed
What would you like to do?
Hyperlinks in Terminal Emulators
@evaryont

This comment has been minimized.

Copy link

evaryont commented Apr 25, 2017

This isn't made explicit, and I didn't see any concrete consensus in the discussions, how are arbitrary/non-standard URL schemes handled? From the looks of things, it's not the terminal's responsibility, they will just pass the URL to the OS's own file open routines. So if I wanted to generate a link to say, an IRC chatroom:

echo -e '\e]8;;irc://irc.freenode.net/#lobby\e\\#lobby channel\e]8;;\e\\'

All I will need to do is make sure my IRC client is registered in the system to handle irc:// links. Is that correct?

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Apr 25, 2017

@evaryont You're right, we don't intend to go into such details. It's up to the terminal emulator to decide which schemes it supports and what action it takes on them, probably delegating this decision to some lower level OS routine.

Most terminal emulators already autodetect standard URLs appearing on the screen. On one hand, it's reasonable for them to take the same action, regardless whether e.g. "irc://..." is autodetected as a visible string, or specified explicitly via this new escape sequence. On the other hand, the new one is potentially more easily configurable by the users since they won't have to handcraft complicated regexes, so perhaps some terminal emulators will add a configuration option for this.

It might also make sense for terminal emulators to offer the possibility to override the OS-wide action because the OS-wide registry would probably prefer to open a graphical app, whereas if coming from a terminal emulator, you might want to prefer to open a terminal-based IRC client in a new window/tab of the same terminal emulator, or similarly, prefer a lynx-style browser for http/https rather than a graphical one.

This is an open ended question, leaving room for terminal emulators to come up with nice ideas.

@ssbarnea

This comment has been minimized.

Copy link

ssbarnea commented Jun 21, 2017

This is really cool! I found this after looking for the same thing myself (and wasting some time with the reddit thread).
Is there any ANSI "coloring" python library already implementing this? I would love to start giving hyperlinks as output. The option to display them raw is not ok.

@Foxboron

This comment has been minimized.

Copy link

Foxboron commented Jul 8, 2017

Wouldn't this allow rather ugly attack vectors where the user doesn't understand what he/she is clicking?

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Jul 12, 2017

@ssbarnea Not that I'm aware of. Thanks for spreading the news of this feature and asking others to add support!

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Jul 12, 2017

@Foxboron I'm fairly sure there's no risk here at all. The situation shouldn't be any worse than clicking on a link on an untrusted webpage. In fact, webpages are worse. First, they can leak data via the Referer field which isn't filled out if you click from the terminal emulator. Second, they can do nasty JS tricks to alter the target just when you click on the link (sometimes this is actually used for good purposes, like to inject a redirector which hides the referring page). In the terminal emulator a malicious app can still probably overwrite the link target at any time, but it has no clue when you're about to click.

Any webpage that does harmful things (e.g. delete data stored on a server) by just visiting a link suffer from CSRF bug anyway which can be exploited from browsers (malicious websites) too.

@towolf

This comment has been minimized.

Copy link

towolf commented Jul 19, 2017

@stefanhaustein

This comment has been minimized.

Copy link

stefanhaustein commented Sep 3, 2017

What happens to "data:" urls or pure local (#) links? Any chance to get anything to STDIN or back to a terminal based app that doesn't involve registering a link handler with the window manager? Use case: local link handling inside a "pure" command-line app with hyperlink support.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Sep 3, 2017

@stefanhaustein Interpretation of different schemes (or lack thereof) is up to the terminal emulator. E.g. with "data:" URLs they might show a popup with the content, or store in a temporary file and open that. Note that with the current limit of 2083 bytes for the URL in both VTE and iTerm2, the "data:" scheme isn't quite useful. I wouldn't want to significantly increase the limit (let alone remove it altogether) because then a malicious utility could cause the terminal emulator to eat tons of memory.

Handling the link inside the existing terminal emulator is alas again something that would be truly problematic. It would raise tons of questions like what to do if an app is already running or if you're ssh'd to a remote host, and probably a whole lot more. Not to mention that if we did anything like this, that would probably introduce serious security concerns.

The closest you can do with the current design is to specify a handler that opens a new terminal window and a specific app inside that, e.g. "lynx" for "http(s)" scheme. I'm not sure if iTerm2 allows you to configure this. GNOME Terminal takes the handler app from the global GNOME-wide configuration, hence it cannot differ from what's used for graphical apps. This is something we might improve on if there's a high demand – or other VTE-based apps might implement this.

@flarn2006

This comment has been minimized.

Copy link

flarn2006 commented Sep 13, 2017

@egmontkob: Malicious "utilities" aren't worth worrying about. If you're executing a malicious binary, you've got much bigger problems than a terminal emulator eating up memory. However, catting a text file could be a problem, as people expect text files to be safe to display. This is actually similar to an issue MS-DOS had with the ANSI.SYS driver, only that was much worse, as that allowed random text files to change what the keys on your keyboard do—for instance, making a common letter instead type a command that formats your hard drive and press Enter.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Sep 13, 2017

@flarn2006 Thanks for the clarification, I indeed did not clearly distinguish these two. A malicious binary can also be a compromised binary on a remote system that you ssh into, in which case it's like locally cating a maliciously constructed text file (plus all the damage being done over there on the compromised system). As far as the terminal emulator is concerned, it's only the incoming byte stream that matters.

@hdon

This comment has been minimized.

Copy link

hdon commented Oct 10, 2017

A note to any maintainers of terminal emulators out there: please include a prominent option to scale back this feature. I've been using terminal emulators for 20 years and one behavior I find really annoying is clicking even what is obviously a URL and then having it come up in my browser. Sometimes I'm just trying to select text...

(Oh, and rectangle select is a nice feature, too, guys!)

@mofux

This comment has been minimized.

Copy link

mofux commented Nov 28, 2017

For reference, xterm.js is tracking this feature request here: xtermjs/xterm.js#1134

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Nov 28, 2017

@mofux Thanks, added.

@devkev

This comment has been minimized.

Copy link

devkev commented Nov 30, 2017

To follow up on @Foxboron's earlier comment, it would be great if this document included a requirement for terminals to prominently inform the user of the actual URL before they click on it, eg. on hover (similar to web browsers). iTerm2 does this, but I'm not sure about other terminals.

This isn't just a convenience, it's important to prevent attacks like '\e]8;;http://malicious.com/\e\\http://safe.com/\e]8;;\e\\' (ie. the user thinks they're clicking a link to safe.com, but they're actually going to malicious.com), which is why I think it should be mandatory.

@aisamanra

This comment has been minimized.

Copy link

aisamanra commented Nov 30, 2017

Given that our Matterhorn project was linked to up above, I figure I should note here that our newest Matterhorn release—version 40400.0.0—produces hyperlink escape sequences for all links which appear in the chat. (Matterhorn is built on top of the vty and brick libraries, which are medium- and high-level abstractions over terminals, respectively, and both of these libraries added support for hyperlink escape sequences in October.)

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Dec 1, 2017

@aisamanra: Thanks, added.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Dec 1, 2017

@devkev: gnome-terminal shows the target as a tooltip (appears about half a second after you stop moving the mouse). Not sure about tilix and I cannot test it right now.

I'm not sure I fully share your concerns. It's a quite complex scenario and I'm wondering if the terminal emulator is the right way to address this, I'm honestly not sure.

Certain utilities use this feature for linkifying what they autodetect as links anyways, just to keep it working across linebreaks or so. As far as I understand, the aforementioned Matterhorn is such an example: it cannot be tricked by a remote user to show URLs where the actual target differs from the visible text. Other utilities, like ls also always produce trustable output.

Another use case is where the visible URL and the target are significantly different. Let's say a site literally called malicious.com looks totally like Facebook, phishing for your data hoping that you'd enter your credentials there. In this case this malicious URL will also be visible in your browser's URL bar, and I'm not entirely sure how much the terminal emulator also showing this URL would help you notice what's going on.

Yet another use case is where the actual URL looks damn similar to the one it's phishing, let's say faceboook.com (with three o's) or some Unicode ideographs. In this case again the terminal emulator showing the target probably wouldn't help you too much in catching the phishing attempt. Also note that this does not only hold for such OSC 8 explicit hyperlinks, the current feature of autodetecting URLs in pretty much every graphical terminal emulator also suffers from this issue, it isn't addressed there and I'm not aware of anyone ever having complained about it.

To tell a malicious site, it's probably a better approach to have some centrally maintained database of known such sites, and check the URL against that. And that's what modern browsers do when you open the URL. The same functionality could be repeated elsewhere too: In utilities that present links to the users (let's say, terminal based e-mail clients); in the URL-opening framework of popular graphical desktops (e.g. whatever happens behind the scenes after a gtk_show_uri()); or indeed in terminal emulators. I'm not sure what the best place for this is.

Given what browsers already do, terminal emulators here could be another line of defense, but would definitely not be the only one, nor the strongest one. As such, I wouldn't make it a requirement. I'd say terminal emulators are recommended to show the target URL, but it's also up to the users to choose an emulator accordingly (or file feature requests, PRs) if this feature is important for them. Obviously there's also room for further improvements, e.g. some terminal emulators might want download some malware list and check the URL against it, or might highlight the domain name (as browsers do in the URL bar) or might want to warn the user in case non-ASCII letters are present in the domain name, etc.

So I'd suggest that we bring these issues to the attention of terminal emulator developers, and recommend them to at least somehow conveniently let users figure out the URL before they activate it. Does this sound reasonable to you?

@jtdaugherty

This comment has been minimized.

Copy link

jtdaugherty commented Dec 2, 2017

I'll also note that soon Vty will not do hyperlinking by default because of garbage output with older versions of VTE. An API call will be required to opt in and enable it. For more info: jtdaugherty/vty#137

@devkev

This comment has been minimized.

Copy link

devkev commented Dec 4, 2017

@egmontkob, I think you may have missed the point. This isn't about trying to detect malicious sites, or spoofing attempts. It's about users being able to make an informed choice (about clicking on a link). Without the ability to discover the actual url, arbitrary-text links introduce an ambiguity that prevents users from being able to trust any links presented to them. This makes the linking feature effectively useless, and a security risk.

Suppose you have a terminal which:

  • auto links urls
  • displays arbitrary-text links to urls (ie. this spec)
  • does not have any way for users to discover the actual url of these links

Now you (the user) see the linkified text https://www.google.com/ somewhere on your screen. If you click on this, will the Google homepage be loaded in your browser? You have absolutely no way of knowing. Clicking the link could open literally any url at all, including urls that you (the user) would never have chosen to click if you had been properly informed.

This is why it's vital for users to be able to discover the actual url of any link. It's the same reason that web browsers and email clients show users the actual url. It applies to any software that presents arbitrary text hyperlinks.

@devkev

This comment has been minimized.

Copy link

devkev commented Dec 4, 2017

Also, terminology's ticket tracking this feature is https://phab.enlightenment.org/T6329

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Dec 4, 2017

@devkev I understand your usability concerns here, but I don't understand how it's a security or privacy concern, and as such, I'd leave it up to authors of terminal emulators, and leave it up to users to choose a terminal emulator meeting their needs.

As I've already mentioned in this comment, this problem is also present and is more serious on webpages. E.g. search for something in Google and hover over the results. In the bottom corner of the browser, you'll see that the link points let's say to Wikipedia. The moment you click on it (before the browser begins to follow it) the link target is changed to some Google-owned redirector. This is a proof that it can be done on webpages, and hence probably can be done for malicious purposes as well. At least in terminal emulators, the target cannot be altered just when you click on it.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Dec 4, 2017

@devkev I've added the Terminology link, thanks. (Honestly, I don't expect any terminal emulator that refused true color support to implement this one, and hence I myself didn't bother filing feature requests against them.)

@vapier

This comment has been minimized.

Copy link

vapier commented Feb 26, 2018

i've implemented this in hterm (commit) and released it in hterm-1.76. couldn't see any way of updating the gist or sending an update request directly.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Feb 28, 2018

@vapier Thanks a lot, it's great to hear this feature is gaining popularity. I've updated the page. (I've no clue either how to do collaborative stuff on gists.)

@1Hyena

This comment has been minimized.

Copy link

1Hyena commented Mar 11, 2018

Just learned about this! This is awesome, it has enormous implications to text-based gaming (MUDs). Is it possible to send a predefined string to stdin when user clicks on a link? That would make Text User Interfaces s so much more convenient to use.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Mar 13, 2018

@1Hyena It would probably be a quite straightforward change to e.g. gnome-terminal to handle special URIs like send-text:foobar. There's no standard for such protocol (scheme) though, and more importantly, this feature wouldn't be accepted mainstream due to the serious security implications.

@modest

This comment has been minimized.

Copy link

modest commented Apr 7, 2018

It seems critical to include some spec guidance on schema whitelists and security. The reality on most platforms is that many apps register insecure URI schema handlers for private usage. (My Mac has over 140 of these protocols registered.) These apps did not consider the security or privacy implications of the schema being abused by a third party. A malicious URL can cause these apps to take actions without user confirmation, crash the process, execute arbitrary code, login as a different user, or repeatedly call the police.

For this reason, all major web browsers have whitelists for URL schemas that are allowed without prompting the user (usually just http:, https:, and ftp:). For all other URL schemas, the user is prompted after clicking the link and must explicitly allow the schema handler to open. This prevents users from being tricked into clicking on links that abuse poorly-secured schema handlers from installed apps.

Example of a security prompt in Chrome when clicking a skype: link
screen shot 2018-04-06 at 22 19 44

@egmontkob:

I'm fairly sure there's no risk here at all. The situation shouldn't be any worse than clicking on a link on an untrusted webpage.

It is true that this feature is no more dangerous than a web browser, but only if implemented with a schema whitelist like a web browser. Otherwise, this exposes terminals to a dangerous and widespread category of vulnerabilities.

@gsemet

This comment has been minimized.

Copy link

gsemet commented Apr 14, 2018

Guake has been updated and should support this now :)

@nickl-

This comment has been minimized.

Copy link

nickl- commented May 1, 2018

Testing the waters to see if the community has an interest to explore and define standards for currently off-spec but terminal hypertext link related/useful uri schemes. URI schemes and OS specific scheme handlers for terminal hypertext links functionality directed back towards the shell session.

See nickl-/term-hyperlinks and create an issue with your comments and suggestions.

@haya14busa

This comment has been minimized.

Copy link

haya14busa commented May 4, 2018

FWIW, I write a shell script which support tmux too. https://github.com/haya14busa/hyperlink

@PerBothner

This comment has been minimized.

Copy link

PerBothner commented May 10, 2018

DomTerm version 1.0.2 now supports this escape sequence.

DomTerm also has special handling of file: URLs of the form file:/filename#position=line or file:/filename#position=line:column. Clicking on such a link brings up an editor at the specified position. See here and here. These links can be created explicitly (using the \e]8 escape sequence or DomTerm's more general sequence to embed HTML), or implicit (based on matching the pattern FILENAME:LINE: or FILENAME:LINE:COLUMN: - as produced by compiler error messages). This may worth considering as an extension of the specification.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented May 10, 2018

@modest Thanks a lot, I believe you raised an important problem.

In an ideal world, I guess we could just sit back and blame those URL handler apps that do harmful stuff upon clicking on their URLs, couldn't we? We're far from this, and as such, sure it would be desirable for terminal emulators to ask for confirmation.

I don't know if certain operating systems or desktop environments provide any support (framework) for this, or it needs to be implemented from scratch in terminal emulators. Ideally, I guess anyone who provides a method to open a URL (e.g. GTK+'s gtk_show_uri()) should give us some help here. In practice I'm afraid they don't.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented May 10, 2018

@PerBothner Thanks for the heads-up!

I actually looked around a while ago and was surprised to see that there wasn't a common notation for jumping to a certain position. It would sure be nice to see a standard emerging. Compiler errors, as you mention, is a perfect use case. Another one could be the filename as printed by grep in front of every matching line.

(As far as gnome-terminal is concerned, there's nothing we could do in gnome-terminal itself though, handling the position should go into GTK+.)

I can see a couple of issues, though.

A local file, denoted by the file: scheme, could still have a MIME type, probably autodetected based on extension or file contents magic. And that MIME type could be e.g. text/html or any other format that does have the concept of internal positions to jump to. So, we have a conflict since these two features don't have a separate namespace. Should a file:/foo.html#position=5 jump to the fifth line, or to the HTML tag having the ID of "position=5"? (But let's take one step back: Who decides whether to open this file with a plain text viewer as raw text, or with a HTML viewer as rendered HTML? Anyone adding that "#position=5" probably expects that it'll be opened as a text file. So shouldn't we also add the possibility to force opening with a plain text viewer; e.g. by specifying a forced MIME type?)

I find the choice of the word "position=" quite unfortunate, as it's not clearly extendable to other use cases, e.g. byte offset in a text file, page number within a PDF, time offset within a video etc. They're all positions, taking a unit of a different dimension as their argument. Probably "line=" and "column=" would be better, although I'm not sure what should be the separator between them (is another '#' okay?).

For plain text files let's assume that "line" is a clear story (let's for now put aside all those confusions that CR vs. LF vs. CR-LF can cause). "column" is more problematic. What's the encoding of the text file? Do we count bytes, or characters, or width in terminal (including double-wide characters), or Unicode codepoints, or what exactly? How to handle TAB? What about BiDi, is it logical or visual column?

Would be nice to have some standard (maybe even an RFC) about this. I'm afraid DomTerm isn't a strong enough reference everyone would agree to follow.

@PerBothner

This comment has been minimized.

Copy link

PerBothner commented May 29, 2018

"Should a file:/foo.html#position=5 jump to the fifth line, or to the HTML tag having the ID of "position=5"?"

The character '=' is not valid for the id attribute in either XML and HTML 4. It is allowed by HTML 5, but discouraged. Some user agents may be able to go to check if there is an ID with value "position=5", but if they can't it is reasonable to ignore that possibility.

"Probably "line=" and "column=" would be better,"

That doesn't handle ranges: START_LINE:START_COLUMN to END_LINE:END_COLUMN. For an error messages or grep output it can be useful to not only navigate to a position, but also to high-light a range.

"although I'm not sure what should be the separator between them (is another '#' okay?)"

I don't believe so. Using '&' seems reasonable - it matches the convention for "query string" (starting with '?'). Basically, the server handles the '?' part; the browser handles the '#' part.

"So shouldn't we also add the possibility to force opening with a plain text viewer; e.g. by specifying a forced MIME type?"

The format can be extended. For example foo.ext#position=10:12&mime-type=text/plain

"find the choice of the word "position=" quite unfortunate"

I'm not disagreeing. Just keep in mind that the encoding should support ranges.

""column" is more problematic. What's the encoding of the text file? Do we count bytes, or characters, or width in terminal (including double-wide characters), or Unicode codepoints, or what exactly? How to handle TAB? What about BiDi, is it logical or visual column?"

The Language Server Protocol counts 16-bit characters. I.e. Unicode characters not in the BMP (Basic Multilingual Plane) count 2 as they use a surrogate pair. Nice and simple for Java or JavaScript but not very semantic and somewhat error-prone (since people will often skip dealing with it.)

In LSP both line and column numbers are zero-based, not one-based. (DomTerm #position values are one-based, though.)

LSP does specify 1 for with TAB and double-wide characters (in the BMP), which I think is a reasonable choice.

Note it is possible to pick one convention - and allow future extensions. A URL could possible encode positions in more than one way:

foo.java#position=3:10&lsp-position=2:12

Some user-agents could handle #position; some could handle #lsp-position; some could handle both. Some producers (compilers etc) could emit one or the other or both. If a handler sees #lsp-position but can only handle #position it can convert to the former by adding 1. That might not be 100% right but it would usually be good enough.

An RFC could be nice, I agree.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented May 31, 2018

The character '=' is not valid for the id attribute

Still, it's a pretty ugly design if the semantics of a keyword depends on whether the value contains such a sign or not. But it might still be the best we can come up with.

That doesn't handle ranges: START_LINE:START_COLUMN to END_LINE:END_COLUMN. For an error messages or grep output it can be useful to not only navigate to a position, but also to high-light a range.

Is this really needed, or is it an overkill? If really needed, don't we need multiple ranges too? E.g. grep can find multiple matches within a row. I'd rather say it's an overkill for now, but the chosen design should be forward compatible with such possible improvements.

Language Server Protocol

Never heard of that; will need to take a closer look.

counts 16-bit characters. I.e. Unicode characters not in the BMP (Basic Multilingual Plane) count 2 as they use a surrogate pair [...] not very semantic and somewhat error-prone

They use a surrogate pair in UTF-16, the most brain-damaged encoding of Unicode, combining the disadvantages of other representation formats (UCS-4 and UTF-8). This encoding should have died out a long time ago. Luckily AFAICT it's mostly used internally by some legacy stuff that originated in the very early days Unicode, like Java. (Or is it still a common thing in Windows?) Especially in the Unix world where data exchange happens primarily in UTF-8, I can't imagine any valid reason for non-BMP to be handled differently from BMP.

I'd say it's a TERRIBLE thing, freaking error-prone, and gets an absolutely "no go" vote from me.

LSP does specify 1 for with TAB and double-wide characters (in the BMP), which I think is a reasonable choice.

Agree. We need to encode logical positions within the text file, rather than visual positions. The editor that opens the file will know the tab width it chooses and will place the cursor in its corresponding visual position, meaning that the indicated column number will probably be quite different, it's its internal business. Tools like gcc -ftabstop=... will need to take care to generate different column numbers for the URL than for the visual column number they print based on their assumption of tab width.

Note it is possible to pick one convention - and allow future extensions.

Future extensions: yes. Future alternatives: no.

If a handler sees #lsp-position but can only handle #position it can convert to the former by adding 1.

(s/former/latter I guess.)

If a handler recognizes the keyword #lsp-position and knows to add 1, then it already supports it (at least to some basic degree), doesn't it? What if it doesn't recognize this keyword at all?

There has to be a minimum base that's commonly used everywhere, and there could go extensions on top of that, that's okay. It's NOT okay to have competing alternatives, i.e. if an emitter may emit either format X or Y, and a consumer might understand only one of X and Y.

@sigalrm

This comment has been minimized.

Copy link

sigalrm commented Jun 2, 2018

Hello, TermySequence is my new terminal multiplexer which supports this escape sequence from the initial release. Thanks for doing this specification work.

@stuaxo

This comment has been minimized.

Copy link

stuaxo commented Jun 4, 2018

This is great, I've been wanting something like this forever.

We need a way that the shell can be aware a hyperlink has been clicked - it would be good if there was an easy way that clicking a directory, bash could cd into it if properly setup.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Jun 4, 2018

@stuaxo This – or doing anything inside the same terminal, e.g. opening a text-based browser or e-mail client for an http(s) or mailto link – would need an enormous amount of design and work, and is outside of the goals of this project. There would either need to be a separate communication channel between the terminal emulator and the shell solely for the purposes of such new features, or the terminal emulator would need to completely understand what's happening inside (something like iTerm2's shell integration, but probably even more thorough), would need to know whether the shell is sitting idle at the prompt and no partial command has been entered (which probably requires explicit support from the shell), and in that case it could feed the necessary faked keystrokes to type that cd command for you. And then make sure it's free of security issues, and race conditions due to the full duplex asynchronous nature of the communication channel between the shell and the terminal (probably the synthesized cd command would need to be enclosed in special symbols similarly to bracketed paste). It's unlikely for anything like this to happen. Explicit hyperlinks are useful for opening certain resources externally to the current terminal emulator (either in a separate graphical app, or in a newly launched terminal emulator).

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Jun 4, 2018

@stuaxo Thinking about it a bit more, something along the lines of how bracketed paste technically works could work here too. If a certain mode is enabled then clicked hyperlinks would be sent to the app enclosed between some special marks. (If the mode is disabled, which would be the default, then nothing would be sent.) It would be the shell deciding what to do (e.g. cd if it's a file: URL with the local hostname), rather than the terminal emulator deciding to fake type a cd command. It could even change the directory when the command line is not empty (similarly to e.g. how ^X^V is handled in bash). The shell could even be configured to launch certain apps on clicking some types of URLs – the problem being though that it'd only work whenever the shell doesn't have a foreground job running.

In order to move forward, I guess we'd need buy-in from at least one popular shell, plus ideally a few other (non-shell) use cases in mind as well. I can't really think of the latter kind; non-shell apps probably aren't interested in URLs back in the scrollback, and if a fullscreen app is interested about URLs emitted by itself then it can more easily do its own mouse handling. So I'm still afraid it'd be a marginal feature which may not be worth the efforts.

@ssbarnea

This comment has been minimized.

Copy link

ssbarnea commented Jun 5, 2018

@egmontkob See tartley/colorama#134 for adding hyperlink support in colorama, the most popular ANSI python library. Implementing this there will open it to lots of tools. Some support would be needed to get the buy in, I already made a pull request.

PS. I think that it could be very useful for the the "ANSI" cause to have a proper gitlab project (maybe even org) that tracks/documents improvements and support of different extensions on various applications. Using a personal gist does not send the best signal (may give the impression that there is only one supporter behind).

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Jun 5, 2018

@ssbarnea If anyone's willing to create and maintain a documentation of all the terminal emulator extensions out there, it would be highly appreciated. It won't be me.

@stuaxo

This comment has been minimized.

Copy link

stuaxo commented Jun 5, 2018

@egmontkob maybe it's a matter of getting some shells to understand file:/// for local directories - zsh already can change to a directory if you enter a path without cd

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Jun 6, 2018

@stuaxo Having to parse a file:/// URI is not the problematic part here at all.

zsh already can change to a directory if you enter a path without cd

bash too. And it's absolutely irrelevant here.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Sep 29, 2018

I've modified the page to prefer the ECMA standard ST (\e\\) over the nonstandard BEL (\a) terminator. I took the liberty of editing others' comments too to use the preferred form.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Oct 15, 2018

(I don't normally do this, but after long hesitation I decided to remove a few comments from here: a discussion that got way too long and hardly contributed anything to this topic.

The poster insisted that this whole feature is conceptually broken: the terminal emulator should itself match its onscreen contents against configurable regexes and combine them with other externally gathered information (such as looking around in /proc) to recognize filenames and decide what action to take. The poster ignored all the problems and limitations that this approach had which were pretty much the reasons for designing the explicit hyperlink feature.

In my opinion, its only valuable contribution was to point out that when using containers, chroots etc., filenames produced by tools running inside these containers might become broken when trying to open them from the emulator. This is indeed true, however, I find it a way less typical use case than the ones OSC 8 fixes. Moreover, if the terminal emulator is able to somehow figure out the context of the utilities running inside, e.g. how the filenames need to be mapped (as the poster claimed it should), then it can also perform this mapping on the OSC 8 hyperlink targets just as it would do them for regex matches. So, in my opinion, no feature is lost or degraded by introducing explicit hyperlinks, not even with containers in the game.

How a terminal emulator should detect the app's container, chroot etc. and how it should decide the desired mapping of filenames is outside of the scope of this feature and this document.

Should anyone be interested, the deleted discussion was archived by the well known Internet Archive Wayback Machine.)

@sindresorhus

This comment has been minimized.

Copy link

sindresorhus commented Nov 13, 2018

@egmontkob Could you consider moving this to a GitHub/GitLab repo instead? That way people could do PRs to update/improve it. And we would get notifications on new comments.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Nov 13, 2018

@sindresorhus I also feel the pain from these limitations. In maybe a week or two I'll post a brand new design document for terminal emulators (unrelated to this one), and I have already decided to have a proper repo. I'm yet to evaluate whether to go with public GitHub, public GitLab, or GNOME's GitLab. Moving this one too is a good idea!

@drudru

This comment has been minimized.

Copy link

drudru commented Dec 3, 2018

Hi - I was recommended that I check this out. This is a good idea. Thx for working on specing this out. I am implementing this feature right now.

I have a question about the encodings. Do you want to allow space characters and other punctuation in the Params section?

URIs are easier since that is standardized nicely. (No whitespace, etc.)

Also, it would be great if you defined a regex to match this particular OSC code.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Dec 3, 2018

@drudru According to ECMA-48 § 8.3.39, the valid characters within OSC are 0x08..0x0D and 0x20..0x7E. I'm tempted to say that the valid ones for parameters should be the printable ones, that is, 0x20..0x7E, except for the defined separators 0x3A : and 0x3B ;. I don't see a reason for deliberately further excluding let's say the space character from parameters. Does this sound reasonable?

If really required, a formal specification should use ABNF or something similar, not a regex. I don't wish to create one, but if someone does, I'll be happy to add to the page.

A regex can define whether a string matches or not, but – as opposed to ABNF – cannot tell how to break it down to components if it matches. Regexes depend on all various circumstances, like the exact engine interpreting them, or the current locale affecting intervals or equivalence classes.

I cannot see how a robust and fast implementation could use regexes to match against a stream of incoming data, rather than some state machine. For example, it's pretty easy to create a (faulty) regex where the match depends on the timing of the arriving data, which is unacceptable, and I'm not aware of a generic protection against this problem. If I were to write a parser for a terminal emulator, I sure wouldn't risk accidentally having such a regex here and there. Also, the time required gets multipiled by the length of the match if the input arrives byte by byte, e.g. if the regex matches in linear time O(n) then continuously matching against the stream will consume O(n^2) of CPU, etc. Of course it's also multiplied by the number of regexes you define.

Implementations are quietly assumed to already support OSC in general, for setting the window title (where space is definitely allowed) and such, and are expected to hook up recognizing this new numeric entry and handling the subparameters within that already existing framework. I find it unlikely (although sure possible) that a terminal emulator (or other utility) would newly implement parsing OSC for the sake of this feature. I don't want to go into this business as part of this spec, so let's just leave it here in the comments that catching and parsing the entire OSC using a regex is really not a great approach. (If you catch the entire OSC by other means and then parse its parameters using a regex, that's probably fine.)

@drudru

This comment has been minimized.

Copy link

drudru commented Dec 3, 2018

@egmontkob - wrt to 'space', that is very reasonable. I was thinking about it this morning over coffee before I read this thread and came to the same conclusion. It is basically the same pattern for other sequences (CSI for example).

Also, agreed on the use of regex vs. ABNF.

tldr - thanks, it all makes sense, and I have what I need to move forward. If you want to learn more about my implementation and its needs, feel free to read further if you have time.

Just to provide more information on my use case, I am the author of the Ansi-Up javascript library. I am tracking the implementation with this issue: Ansi-Up Issue #51.

In my implementation, my goal is to process the stream and immediately return some HTML that best represents the stream at that point. I only process CSI SGR sequences. This is ideal for viewing the live streaming of log output in a web page. For example, the result of a compiler or a daemon process.

The major difference between my implementation and the use-cases mentioned above is that I don't expect to go back and patch the DOM when I receive new input. This is quite different from the use cases mentioned above. When I implemented 'linkification' in the past, I was relying on developers having enough luck to avoid receiving a partial 'packet'. I became painfully aware of the edge-cases and ended up dropping the feature until I was sent this spec.

In order to implement this feature, I have two choices. I could buffer or not buffer the stream until I receive a hyperlink terminator. I am currently leaning towards buffering even though it is more complex.

The hard part of this implementation is that I need to know whether the pattern matches, could match, or is an illegal pattern. If I can do all of this in a regex, then I get a performance win in the browser. Also, by properly constraining the pattern (no spaces, whitespace here, etc.), I can avoid the horrible degenerate case of someone making a mistake with the pattern and creating a screen-full, single URL. This last piece is important to me.

One other thing to note is that I will be whitelisting URI prefixes. This will eliminate 'javascript:' and 'data:' prefixes.

Also, I would also vote for moving this to a github project.

Thanks again for your spec.

@drudru

This comment has been minimized.

Copy link

drudru commented Dec 5, 2018

@egmontkob - quick note - I decided in my implementation to forbid newlines in the entire sequence. I felt that it prevents any degenerate sequences from preventing my library from working.

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Dec 5, 2018

When I implemented 'linkification' in the past, I was relying on developers having enough luck to avoid receiving a partial 'packet'. I became painfully aware of the edge-cases and ended up dropping the feature until I was sent this spec.

Yeah, on-the-fly autodetected linkification when the data can stall at any time is incompatible with the requirement of flushing the data as soon as it arrives.

I could buffer or not buffer the stream until I receive a hyperlink terminator.

Are you talking about the OSC 8 terminator \e\\ or \a, pretty much mapping to HTML's > (there are two of them in a complete hyperlink), or about the hyperlink terminator \e]8;;\e\\ corresponding to HTML's </a>? (I do get confused by them a lot.)

OSC 8's syntax pretty straightforwardly maps to HTML. The moment you see a \e]8; you know it's related to links. You can ignore the IDs. The moment you see the first character of a URL, you can tell it's an opening tag. Since you'd blacklist (or whitelist) some protocols, you should probably buffer up until the first colon, that is, to see the protocol (and potentially bail out if the protocol is longer than a small threshold). Once you encounter a http:, you can already emit <a href="http: and switch to a mode where any printable input is just copied to your HTML output (taking care to escape <, &, ', " to be sure). On the OSC 8 terminator \e\\ or \a you'd just finish the opening tag by outputting ">.

Or you can of course batch up until the entire opening tag is seen, that probably simplifies a lot, and doesn't degrade the user experience at all (a partially printed opening HTML tag doesn't help users see the data any faster). I wouldn't do any batching with the visible link string, the one between the link's opening entire OSC 8 sequence and the closing entire OSC 8.

@stuaxo

This comment has been minimized.

Copy link

stuaxo commented Dec 12, 2018

Quick RFC for positioning, build on githubs idea of an L line number anchor.

This uses L to denote lines and optionally C for columns.
Lines and columns are 1 based.

It has the advantage of being short, which may look less weird in the context of ANSI.

Examples:

Open at line 10
file:///home/stu/document.txt#L10

Open, positioning cursor at line 20, column 20:
file:///home/stu/document.txt#L10C20

Ranges should open the editor with a range selected, if supported

Select line 10 to 20
#L10-L20

Select from line 10, five characters in, to line 12, 20 characters
#L10C5-L12C20

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Dec 13, 2018

#L10 is probably the most common practice. I've expressed earlier my slight worries about a clash with the HTML tag with the ID of L10 in case the file is detected to be an HTML one. I've also posted dilemmas how to exactly interpret the column. But these are minor issues, and we can live with some imperfection here.

There's a major problem here, though, namely: how would terminal emulators implement such a specification? I'm not familiar with macOS (e.g. iTerm2), so I'll take Linux, GNOME and gnome-terminal as an example.

What happens currently is: gnome-terminal just simply passes the URI to gtk_show_uri() and that's all that it cares about it. The rest of the story is exactly the same as for other URLs opened from anywhere else in GNOME. The MIME type is somehow guessed, then ~/.config/mimeapps.list is consulted (falling back to I-don't-know-what if there's no such file) to locate a .desktop file, then that desktop file is opened, a command line is contsructed by using the given URI (if %u is present) or by converting it to a local file (if %f) and then it's launched. Chances are that I've missed a step or two.

I definitely do not intend to bypass all this infrastructure and come up with something custom one-off solution (disobeying the user's desktop-wide preferences) in gnome-terminal.

Where would the line number be parsed and interpreted? Should all text editors be required to recognize the file: scheme and the #L10 syntax, and as part of that, perform URI-decoding on their command line parameters? E.g. should Vim be modified not to copy the file first, as it does, but to open it at place? What about Emacs, Joe, and presumably a whole lot more that do not recognize file: and just treats it as a local path?

Or shall the desktop spec be extended by new macro(s), let's say %l for the line and something for the column (note: %c is taken), and then whoever is responsible for substituting these macros should interpret the #L10 URI fragment and convert accordingly? Probably a resonable approach.

All this needs cooperation from plenty of parties, probably beginning with a discussion on freedesktop mailing list, followed by a new version of the desktop file specs, followed by implementation, and editors starting to ship corresponding desktop files using these macros or interpreting the URL fragments themselves.

I'm not entirely certain what the best design would be. I'd love to see this happening, I'm happy to cime in in discussions if someone drives it, but I don't have capacity to drive this. And if I just added #L10 to this specification, it would take us nowhere, it presumably wouldn't be sufficient to push all these changes through, and I couldn't implement it in gnome-terminal either.

A much higher level agreement is required on the URI syntax for local files at a given line; that is, an agreement across OSes and desktops and kinds of products (not just terminal emulators). This needs to be followed by implementation in various desktops (such as GNOME). And once it's done, there'll be nothing left to do by terminal emulators, it'll just work.

@langpavel

This comment has been minimized.

Copy link

langpavel commented Dec 27, 2018

For node users there is terminal-link package. And there is discussion on chalk

@koconder

This comment has been minimized.

Copy link

koconder commented Jan 14, 2019

Just tested on iTerm 3.2.6 and not working

@mintty

This comment has been minimized.

Copy link

mintty commented Jan 22, 2019

Mintty now supports hyperlink attributes (just uploaded, hovering to follow).

@mintty

This comment has been minimized.

Copy link

mintty commented Jan 23, 2019

Mintty hyperlink attribute support is now complete.

@lmorg

This comment has been minimized.

Copy link

lmorg commented Feb 6, 2019

I'm a little concerned about how genuine security concerns (such as the following: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda#gistcomment-2404153) have been completely ignored.

I've written some thoughts on this in a related discussion (kovidgoyal/kitty#68 (comment)) but essentially I seriously question just how much convenience this brings beyond a few niche use cases that cannot already be served with other workflows (such as right or shift clicking a URL) that still preserve the WYSIWYG nature of terminal output.

We already suffer from a whole plethora of CSFR, phishing attacks and so on and so forth inside web browsers - let alone a bloat of dubious URL schemas - so the last thing we need to do is make one of our most sensitive and important administrative tools subject of doubt when working with references such as URLs.

That all said, I'm not saying hyperlinks have no place in terminals per se - just that we need to seriously think about how to do this properly rather than jumping in feet first because it seems hip and cool.

@lmorg

This comment has been minimized.

Copy link

lmorg commented Feb 6, 2019

Taken from another discussion, but posting here because the author of that project didn't want the discussion on his issue tracker (kovidgoyal/kitty#68 (comment))

Security aspects have been discussed in the comments section of the feature's gist, I recommend to you guys to go through them.

I already had done (that's how I found this issue) and those concerns weren't addressed at all. In fact I even replied on there stating exactly that.

The only pending concern I find valid is that some OSes allow various apps to register (unsafe) handler apps for various protocols. A solution could be to limit support in Kitty for a few well-known protocols, e.g. http, https, ftp and file.

Just because you don't find other security concerns valid that doesn't mean you should simply ignore them when others do raise legitimate points.

Equally so from a browser. If a malicious HTTP(S) request can be crafted, your website is buggy as it doesn't do proper authentication, CSRF etc. protection.

That's just an acknowledgement that there is a problem. You're not addressing a solution to the problem with your hyperlinks proposal.

It's not strictly speaking a "need", it's a new convenience feature. Sure you can live without it, just as much you can live without curly and colored underlines, raw keyboard reporting, image support and all those other new features newly supported by Kitty or whichever other terminal.

Again, you're not actually addressing what problems this "convenience feature" solves. I've exampled other solutions that offer the same level of convenience but with additional protections against abuse, whereas you're just sweeping any concerns people raise under the carpet (and by your own admission, I'm not the only person to raise these points). Quite frankly, that's not an acceptable way to hold this discussions. Please meet us half way and talk about why you want this. THEN we might be able to find a common middle ground that works for all parties.

First, Kitty might offer a config option to disable hyperlinks. Then you can just entirely disable this feature for yourself. Second, apps (e.g. mutt), if they implement explicit hyperlinks, might also offer a way to disable them. This way you can disable them for apps that handle untrusted data.

That's a reasonable compromise however I'd argue it should be disabled by default. That way you can want individuals about the dangers of using it when they come to enable it.

I use this feature quite frequently. 5 seconds indeed sounds like a good estimate on average (counting for cases where copy-pasting the filename doesn't work due to space characters in it; tab completion stopping at unexpected places where I have to locate the next letter of the filename to be typed; and cases when I mix up the utility's name and type "evince" instead of "geeqie" or the other way around which does happen to me quite frequently). Then multiply this 5 seconds by maybe 10-20 times a day, for me.

The problems you're describing there can easily (and I mean easily) be solved by better command line tooling. Better auto-completion scripts, or a better $SHELL. You could also solve that problem with the terminal emulator detecting spaces in it's clipboard and auto escape it. Or even solve it just be typing a quotation mark before pasting your output. The issue you're describing here is really more of a problem with the user not learning to use their command line environment effectively. Now if they cannot be trusted to a quotation key before pasting random data in, then I certainly wouldn't trust them to click random hyperlinks without doing due diligence first!

In gnome-terminal, a single click doesn't follow the link, it selects text. You need to Ctrl+click, or right-click and choose the corresponding menu entry to open the link.

Given you can open a URL in quite a few terminals - even those that don't support OSC 8 - by doing exactly the same thing, I have to question what the bloody point of OSC 8 is. Just put file: schema in front of the file name and be done with it. You could even write a wrapper script around ls to do this for you. Then you have all parties happy with both security and usability concerns covered.

The feature's specification lists quite a few current and possible future use cases.

I'm sure it does but that doesn't help our discussion here and now where I've listed some use cases and the problems they create.

What's more, you're the one opening issues in every terminal emulators issue tracker requesting this feature so it might be nice if you then didn't expect everyone else to do the donkey work researching the background to this afterwards (nor does it help how you repeatedly sidestep every security-related point people bring up just because you happen not to care about it).

I really don't like how this is being pushed through - if feels like a pet project of yours that you're urging everyone should adopt without any security review. And your responses on here whenever anyone does audit the potential ramifications is essentially just "that doesn't concern me so I'll ignore it". I'm really not happy with how you're going about this and the last thing I want is my terminal to become yet another platform vulnerable to social engineering just because a small minority of hackers things "oooh this will be cool. I could save myself 5 seconds of time". So I urge you to actually investigate:

  1. what problems are you trying to solve?
  2. can those problems already be solved using existing tooling?
  3. what protections can you bring to mitigate any security concerns?

Thus far you're not answering any of those questions.

@lmorg

This comment has been minimized.

Copy link

lmorg commented Feb 6, 2019

reply to kovidgoyal/kitty#68 (comment)

kitty currently requires one or more configurable keys to be pressed while clicking a URL for it to be opened.

So then why do we need dedicated hyperlink support when Kitty already open URLs?

Well kitty supports Linux and macOS and at least on macOS this concern doesn't apply because there are no Macs with touch screens. I'm not sure about Linux as I have never tried to use a touch screen with Linux before.

Linux does support touch screens. But the point is also valid for other terminal emulators as well

I'm pretty sure kitty doesn't work with a screen reader as it uses OpenGL to render stuff. Other terminals that explicitly support screen readers will probably work much better for this sort of thing.

So the same argument but for a different terminal which @egmontkob has also requested this feature on (he's been contacting a great number of developers asking this to be incorporated).

@egmontkob

This comment has been minimized.

Copy link
Owner Author

egmontkob commented Feb 6, 2019

I'm a little concerned about how genuine security concerns (such as the following: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda#gistcomment-2404153) have been completely ignored.

I wouldn't say it's been completely ignored. I responded in a followup comment. Indeed I should've added this to the spec itself, as something implementations should watch out for. It's on my mental todo list.

I also wouldn't say "such as", I believe this is the only security point so far that we've found valid.

make one of our most sensitive and important administrative tools subject of doubt

If you're worried about opening a URL having unexpected, bad consequences, e.g. due to lack of proper CSRF prevention, then it's those websites that should be subject to doubt, not the terminal.

[CSRF] That's just an acknowledgement that there is a problem. You're not addressing a solution to the problem with your hyperlinks proposal.

No. That's the acknowledgement that the problem is elsewhere. Seeing a problem already existing somewhere else, on broken websites, can't be a valid reason for us not to move forward.

We have put a lot of thinking into the security aspects of this issue. One of the principles was: if something can already happen from a browser, and the very same thing can happen from now on from a terminal, that just can't be a bug or a security issue. A security issue would be something that we open up newly and is not yet seen while browsing the web.

If you question this principle, if you're worried about this feature exposing more surface for social engineering attacks, you should just not use this feature, it's that simple. If, after almost 2 years of its design and implementation is some terminals you want to revert all this work, it's clearly not going to happen, we're just wasting our time arguing about it.

If you find a security bug that's a new one according to the principle I've just mentioned, sure let us know.

Please meet us half way and talk about why you want this.

This document mentions plenty of places where this feature can be useful, convenient, potentially speed up one's work.

The world of terminal emulators continuously sees improvements made here and there. Sure you can go back to to a hardware monochrome terminal of decades ago and get your work done from that. Or you can, for each and every single improvement, question "why" that happened. Because it made terminals better for some people. Maybe not for you, but that alone can't be a reason for you to question the feature's usefulness.

The problems you're describing there can easily (and I mean easily) be solved by better command line tooling. Better auto-completion scripts, or a better $SHELL.

Show us please how you can do it better!

Given you can open a URL in quite a few terminals - even those that don't support OSC 8 - by doing exactly the same thing, I have to question what the bloody point of OSC 8 is.

This is a solid proof to me that you have not familiarized yourself enough with our work, and the possible use cases we (and others) have come up with.

I really don't like how this is being pushed through - if feels like a pet project of yours that you're urging everyone should adopt without any security review.

What I really don't like is how casually you're just waving the security flag, without any concrete issue you could come up with (except pointing to that single one so far that someone else found, and I've responded to). Seems to me we've already put a whole lot more into evaluating the security aspects than you have.

the last thing I want is my terminal to become yet another platform vulnerable to social engineering

It's not the terminal that's vulnerable, it's your careless websites. And those websites are already vulnerable, and any other untrusted site you visit can get you into exploring those vulnerabilities. There's nothing new here.

Since it's not the terminal emulator actually fetching the remote site but it delegates this task to a browser, you also get the phishing prevention and stuff that generally a browser provides.

Of course you're always free to use whichever terminal emulator of your choice, which doesn't support this feature or allows you to disable.

That all said, I'm not saying hyperlinks have no place in terminals per se - just that we need to seriously think about how to do this properly rather than jumping in feet first because it seems hip and cool.

Care to actually help, rather than blaming what we've done so far? Because in the latter one I'm not interested at all.

@lmorg

This comment has been minimized.

Copy link

lmorg commented Feb 6, 2019

If you're worried about opening a URL having unexpected, bad consequences, e.g. due to lack of proper CSRF prevention, then it's those websites that should be subject to doubt, not the terminal.

You cannot expect every website ever written to be secure. This is why I and others have repeatedly quoted the follow: https://en.wikipedia.org/wiki/Defense_in_depth_%28computing%29

No. That's the acknowledgement that the problem is elsewhere. Seeing a problem already existing somewhere else, on broken websites, can't be a valid reason for us not to move forward.

But if the problem exists elsewhere that implements your proposal than ergo the problem also exists with your proposed implementation.

We have put a lot of thinking into the security aspects of this issue.

I completely disagree. You're responses demonstrate this. Here's another example of how little thought you've put into this: you haven't even standardised what URL schema's should be supported nor dictated any rules on other non-standard extensions to existing URL schema's (such as line numbers) thus even what little you have outlined could differ from one hyperlink supporting terminal to another. The whole thing feels like a pet project someone rushed through rather than something that's been through a proper review. Which is fine if you're also building your own terminal emulator to support this. But when you're spamming every terminal developer out there to support your made up standard, things get a little more serious.

Now I'm not saying we shouldn't have hyperlinks per se but if we are going to agree on a standard then lets get this designed properly.

If you find a security bug that's a new one according to the principle I've just mentioned, sure let us know.

I have done. You've even acknowledged it's a problem on the web. ;)

This document mentions plenty of places where this feature can be useful, convenient, potentially speed up one's work.

Yes, I've addressed them. Debunked them. Found security issues with some of them. But ultimately that list is just "oooh this would be cool" rather than trying to solve actual real world problems. Which is fine if it wasn't for the fact that you're introducing new security concerns for everyone just for the sake of building yourself a new toy.

The world of terminal emulators continuously sees improvements made here and there. Sure you can go back to to a hardware monochrome terminal of decades ago and get your work done from that. Or you can, for each and every single improvement, question "why" that happened. Because it made terminals better for some people. Maybe not for you, but that alone can't be a reason for you to question the feature's usefulness.

I'm actually all for progress. I've written tools to render images and web pages to the terminal (agnostic of which terminal emulator you use too). I'm a heavy user of colours. I've even written my own readline API and $SHELL because I thought "these tools are good, but I could write something that better serves modern computer usage". However one fundamental principle of the terminal is WYSIWYG - what you see is what you get. Hyperlinks make sense on the web but they're an anti-pattern for how terminals work. If you want or need that kind of UI then we have one already (several in fact).

Show us please how you can do it better!

I already did and you edited that part out of the quote! Another idea is maybe use an interactive ls-like tool where you can select the file (terminals do have mouse support too it doesn't have to be a keyboard driven tool) the file and that tool opens $EDITOR for you.

I've written my own UNIX shell and have hotkeys where if you have a file in the command line (eg some-command -config /etc/command.conf and hit [ctrl]+e it opens that file in $EDITOR. My shell also passes filenames around as JSON arrays to get around problems with white space and other characters. The drawback is I've not followed POSIX standards for my shell so it's not like you're just using Bash or similar. However Bash, Zsh and other shells have pretty sophisticated auto-completion engines. You could even use hotkeys to do partial matches of parameters instead of having to type the first few chars and hit [TAB] repeatedly. Not to mention that you could just put quotation marks around a file name.

This is a solid proof to me that you have not familiarized yourself enough with our work, and the possible use cases we (and others) have come up with.

It's funny that because every time I've asked you to expand on your work you refuse to - simply telling me to read the docs (which I clearly have). If you feel I'm missed some fundamental part of your proposal then you're making it impossibly hard to get educated on it but ignoring my every request for clarification.

What I really don't like is how casually you're just waving the security flag, without any concrete issue you could come up with (except pointing to that single one so far that someone else found, and I've responded to). Seems to me we've already put a whole lot more into evaluating the security aspects than you have.

Are you serious!?! I've listed a multitude of concerns that you've not even addressed (and keep avoiding answering):

  • Hyperlinks break the WYSIWYG nature of terminals - actual address isn't clear to the user without additional user intervention which then undermines the convenience factor of hyperlinks
  • They will cause issues for accessibility users
  • There is no formal spec on how terminals will handle URLs - which means even you can have incompatibilities between two terminals that both technically support this "spec"
  • There isn't even a guideline on which protocol schema's you should support

Isn't that enough problems to be getting started with?

It's not the terminal that's vulnerable, it's your careless websites. And those websites are already vulnerable, and any other untrusted site you visit can get you into exploring those vulnerabilities. There's nothing new here.

...or your PC if a terminal doesn't whitelist URL schema's. Or your terminal itself if that happens to have it's own URL schema. You're awfully confident for someone that has a lot of undefined behaviour in the proposal. :)

Also this: https://en.wikipedia.org/wiki/Defense_in_depth_%28computing%29 (you cannot fix every website but you can mitigate it by giving users the tools to evaluate which links they open before they open it)

Of course you're always free to use whichever terminal emulator of your choice, which doesn't support this feature or allows you to disable.

That argument works both ways: you're free to fork any terminal emulator and add this feature for yourself rather than badgering every terminal project out there to implement this on behalf of everyone else - most of whom never required it in the first place.

However I'm actually trying to be a bit more reasonable and saying "lets find a common middle ground that fixes the problems you're trying to solve without introducing anti-patterns into the terminal". Thus far you're just repeatedly ignoring me, saying "you don't understand - go read the docs" instead of actually talking to me. This is VERY frustrating.

Care to actually help, rather than blaming what we've done so far? Because in the latter one I'm not interested at all.

I'm trying! I keep asking you what problems you're trying to solve and you keep "stonewalling" me.

@drudru

This comment has been minimized.

Copy link

drudru commented Feb 6, 2019

Hi @egmontbob

2 things:

  1. If the text for the anchor is blank, should we just use the URL?
  2. Can we move this to a github project? That way we can use the issue tracker.
@lmorg

This comment has been minimized.

Copy link

lmorg commented Feb 13, 2019

...and here lies the problem. @egmontkob has this idea they want the community to adopt in everyone else's code base yet is completely unwilling to work with the community in any way apart from his own. Including the repeated requests for the "specification" (if you can call it that) to be moved into a proper repository (it's insane to manage this in a gist) and any requests for further discussions that others have made when they've raised potential areas of exploit. Then @egmontkob wonders why people like myself comment about how this feels more like a pet project than a serious specification that's had community input and rigorous thought put in it. For example @egmontkob been submitting feature requests to a dozen terminal emulators before half the details have been decided upon which has meant @egmontkob has then had to raise follow up requests to the early adopters to fix the support they had literally just added.

A concept like this could work really well but I'm honestly a little scared at how badly - inconsistently and insecurely - this will be implemented while it's under their control.

I know this probably comes off a little hard / negative and I'm sorry for that. If this were just one developer hacking their own terminal emulator / CLI tools then I'd be totally fine with it. However this is someone actively going out and requesting the default behaviour of everyone else's tools be changed. I'm not cool with that if it hasn't followed any due diligence - which this hasn't.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment