Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
True Colour (16 million colours) support in various terminal applications and terminals

Colours in terminal

It's a common confusion about terminal colours... Actually we have this:

  • plain ascii
  • ansi escape codes (16 colour codes with bold/italic and background)
  • 256 colour palette (216 colours + 16 ansi + 24 gray) (colors are 24bit)
  • 24bit true colour ("888" colours (aka 16 milion))
printf "\x1b[${bg};2;${red};${green};${blue}m\n"

The 256 colour palete is configured at start, and it's a 666 cube of colours, each of them defined as a 24bit (888 rgb) colour.

This means that current support can only display 256 different colours in the terminal, while truecolour means that you can display 16 milion different colours at the same time.

Truecolour escape codes doesn't use a colour palette. They just specify the colour itself.

Here's a test case:

printf "\x1b[38;2;255;100;0mTRUECOLOR\x1b[0m\n"
awk 'BEGIN{
    s="/\\/\\/\\/\\/\\"; s=s s s s s s s s;
    for (colnum = 0; colnum<77; colnum++) {
        r = 255-(colnum*255/76);
        g = (colnum*510/76);
        b = (colnum*255/76);
        if (g>255) g = 510-g;
        printf "\033[48;2;%d;%d;%dm", r,g,b;
        printf "\033[38;2;%d;%d;%dm", 255-r,255-g,255-b;
        printf "%s\033[0m", substr(s,colnum+1,1);
    }
    printf "\n";
}'

Keep in mind that it is possible to use both ';' and ':' as parameters delimiter.

According to Wikipedia[1], this is only supported by xterm and konsole.

[1] https://en.wikipedia.org/wiki/ANSI_color

Currently, there is no support for the 24-bit colour descriptions in the terminfo/termcap database and utilites. See the discussion thread here: https://lists.gnu.org/archive/html/bug-ncurses/2013-10/msg00007.html

Detection

There's no reliable way, and ncurses/terminfo's maintainer expressed he has no intent on introducing support. S-Lang author added a check for $COLORTERM containing either "truecolor" or "24bit" (case sensitive). In turn, VTE, Konsole and iTerm2 set this variable to "truecolor" (it's been there in VTE for a while, it's relatively new and maybe still git-only in Konsole and iTerm2).

This is obviously not a reliable method, and is not forwarded via sudo, ssh etc. However, whenever it errs, it errs on the safe side: does not advertise support whereas it's actually supported. App developers can freely choose to check for this same variable, or introduce their own method (e.g. an option in their config file), whichever matches better the overall design of the given app. Checking $COLORTERM is recommended though, since that would lead to a more unique desktop experience where the user has to set one variable only and it takes effect across all the apps, rather than something separately for each app.

Here are terminals discussions:

Now supporting truecolour

But there are bunch of libvte-based terminals for GTK2 so they are listed in the another section.

Also, while this one is not exactly a terminal, but a terminal replayer, it still worth mentioning:

Improper support for true colors

Parsing ANSI colour sequences, but approximating them to 256 palette

Note about colour differences: a) RGB axes are not orthogonal, so you cannot use sqrt(R^2+G^2+B^2) formula, b) for colour differences there is more correct (but much more complex) CIEDE2000 formula (which may easily blow up performance if used blindly) [2].

[2] https://github.com/neovim/neovim/issues/793#issuecomment-48106948

Terminal multiplexers

NOT supporting truecolour

[3] You can download patched version here https://github.com/rdebath/PuTTY

[4] You can download patched version here https://github.com/halcy/PuTTY

Here are another console programs discussions:

Supporting True Colour:

Not supporting True Colour:

radare commented Feb 10, 2014

OSX default Terminal also supports truecolor

mintty does not support true color

esn89 commented Jul 6, 2014

Add this one to the list?

https://github.com/thestinger/termite

Owner

XVilka commented Jul 6, 2014

@PhilipDaniels @esn89 thx, added

sheerun commented Jul 6, 2014

Colors in used scheme changes with 256 color palette used by terminal. This means the same color scheme can appear differently on different terminals. It always annoyed me.

Isn't it safer to always use true color? (with fallback to 256 colors)

donatj commented Jul 6, 2014

@radare I don't know what you're talking about but mine certainly doesn't. See: http://jdon.at/Cq6g

Mine doesnt either @radare/@donatj

OS X 10.9.4 Terminal.app does not [1] respond correctly to either the semicolon or the colon-delimeted syntaxes of the 38:2 RGB truecolor syntax [2]. The RGB, 256-color, and 16-color colon-delimited syntaxes are all ignored. The RGB semicolon syntax is ignored, exposing the bug warned of in [3], while 256-color and 16-color semicolon syntax work fine.

[1] https://raw.githubusercontent.com/robertknight/konsole/master/tests/color-spaces.pl

[2] gnachman/iTerm2@5294130#diff-af68b28e420158354052a9c17d2b2aa4R3696

[3] https://groups.google.com/d/msg/iterm2-issues/NXLGvwuzjrc/35gVs4pfjLYJ

Listing all libvte-based terminals and claiming that they all support truecolor is incorrect. Many of those apps rely on GTK+ version 2, and, in turn, can only use VTE version up to 0.28. Those apps do not support true colors. The ones relying on GTK+ version 3 can use a recent enough VTE (0.36 or newer) to have true colors.

DrewRWx commented Aug 22, 2014

http://qterminal.sourceforge.net/ worked for me if semicolons are used as the separators.

Owner

XVilka commented Aug 30, 2014

@egmontkob yep, i'll see the sources and mark them properly (will make a list of an old vte-based terminals)

P.S. Fixed - only 2 terminals doesn't have modern libvte support - i've marked them as unsupported.

I think it might be worth mentioning in brackets that iTerm2 only supports RGB sequences in the nightlies.

Owner

XVilka commented Oct 23, 2014

@andyherbert Thx, added in this gist!

@XVilka, thanks for putting this gist together!!

rkh commented Nov 18, 2014

Anyone figured out how to detect 24bit support to gracefully fall back to 256 colors?

@radare, Mac OSX 10.10.1 Terminal.app does not support truecolor. Test it by running either of these scripts in Terminal.app: color-spaces.pl or 24-bit-color.sh. You'll see it produces only a tiny subset of the correct spectra.

tehmaze commented Feb 9, 2015

@cwensley is using a different sequence: http://picoe.ca/2014/03/07/24-bit-ansi/ (not endorsing the home brew sequence here).

ZyX-I commented Mar 20, 2015

ConEmu (Windows) supports true colors as well. Don’t know about the delimiter though, but at least semicolon should be supported.

You should Putty to the list of terminals that does NOT support truecolor.

quasarj commented Apr 15, 2015

What is this "tmux_escape" workaround? There does not seem to be an option in tmux with this name (even searched the source code) and the only escape-related option is escape-time, which seems rather unrelated.

Owner

XVilka commented Apr 24, 2015

Added link to the tmux truecolor patch https://gist.github.com/JohnMorales/0579990993f6dec19e83

rdebath commented Apr 25, 2015

@scottchiefbaker A patch exists for PuTTY to do truecolour just like XTerm, is that close enough ?

rr- commented Apr 27, 2015

I have a patch for MinTTY but it glitches as soon as the window is scrolled. Is anyone interested?

ZyX-I commented May 3, 2015

@quasarj It is possible to force tmux to send random sequences to the backing terminal unchanged. This adds truecolor “support”, but has problems: sequences will not be resent when scrolling, so you will end up with unhighlighted text. Not sure how this will behave with two panes when both are actively displaying something.

Owner

XVilka commented May 5, 2015

@rr I'm interested in, since I've started patching mintty as well (not finished though) https://github.com/XVilka/mintty/commits/master

jerch commented May 12, 2015

I work on a browser based JS terminal emulator (still alpha), which has truecolor support now - https://github.com/netzkolchose/jquery.browserterminal

rr- commented May 18, 2015

I have revived the plea for adding true color support to URxvt but I got a strongly negative reception from URxvt's maintainer. To cut things short, he states that being able to render pretty images with libcaca is just not enough to add this feature upstream. That, and that it's not widely supported nor standardized, so that people wishing to use this feature would need to hardcode escape codes rather than asking terminal whether it supports this feature.
When I checked the source code myself, it seemed like URxvt is really not ready for storing RGB data. Then again I haven't been analyzing the code for too long.
@XVilka I left you a note in your commits. Basically we did almost everything the same (meaning we went through the same places in the code).

xfce4-terminal is another GTK2 libvte-based one (ie. no truecolor)

Owner

XVilka commented May 20, 2015

@unhammer Thx, added. Also noted, that this (hopefully) will be solved automatically, since Xfce slowly migrating to the GTK+3.

kuntau commented Jun 10, 2015

Anyone know why I can't have true color with mosh on remote machine? SSH working great.

halcy commented Jun 11, 2015

I made a patched-up PuTTY that DOES support real truecolor, I will try to make a proper patch (without a billion whitespace changes and based on baseline PuTTY) when I have the time, but until then, https://github.com/halcy/PuTTY real TrueColor PuTTY.

Owner

XVilka commented Jun 15, 2015

@halcy, thanks! referenced your patched version in the gist. I hope you'll find a time to send your patch to the mainline.

Owner

XVilka commented Jun 22, 2015

mintty now has official support since commit mintty/mintty@43f0ed8

Owner

XVilka commented Jun 22, 2015

So, for now only rxvt-unicode and Terminology users can't enjoy the 16 million of colours.

@XVilka thanks for maintaining this list. Since Tmux moved to Github, the new link to the TrueColor issue is here: tmux/tmux#34

Owner

XVilka commented Jul 2, 2015

@jfelchner: thx, updated

mavey80 commented Jul 13, 2015

@rr: I ignore newbie discussions as a rule, but since this could help an otherwise worthwhile piece of SW to get better (i.e. radare), I'm gonna steer you towards fixing this in a proper and portable manner.

You're completely misrepresenting what Marc Lehmann has written. Your ignorance goes so deep, you're actually saying the exact opposite. Before you or anybody else attempts basic terminal manipulation and programming, do yourself a favor and read the following man pages, which are all already installed on your computers:
$ man terminfo
$ man toe
$ man infocmp
$ man tic

Then you can proceed to either use this information directly or follow up with reading on the ncurses documentation for higher level terminal programming API. Apart from its API, this decades existing and proven framework, which by the way is known to every UNIX programmer aspiring to at least competency-level craft, also contains a huge database of all known terminals and terminal emulators, with their respective properties and control sequences of all their features (not just ANSI). And yes, all this is also already installed on your computer.

That's what you use to build a portable application like the rest of us. You NEVER HARDCODE terminal control sequences into the application. Term. emulators set the TERM env. variable for a reason. You use it directly, or more traditionally via the curses API, to lookup an entry appropriate for the active terminal and then retrieve the required control sequences for whatever function you desire.

This is how you get an overview of the database:
$ toe -a

This is how you compare features of two different terminals (in case you needed persuasion about rxvt-unicode being technically eons in advance of all the other terminal emulators you've listed):
$ infocmp -L1 rxvt-unicode-256color vte-256color

If you absolutely must build an application independent of the entire terminfo database, e.g. a single-terminal support embedded kind of thing, this will get you the C source def's:
$ infocmp -E

Going in the reverse, to compile a terminfo/termcap definition into the binary table:
$ tic

And to get you up and running faster, you'll be looking at these capabilities:
set_a_background (setab)
set_a_foreground (setaf)
set_background (setb)
set_foreground (setf)
set_color_pair (scp)
initialize_color (initc)
orig_colors (oc)
initialize_pair (initp)
orig_pair (op)

Regards,
Dave

@ghost

ghost commented Jul 20, 2015

Hi all,

I have patched urxvt to support 24-bit colors:
https://github.com/spudowiar/rxvt-unicode/tree/24bit
https://aur4.archlinux.org/packages/rxvt-unicode-24bit/

Also, please see README.configure and the note for --enable-24-bit-color about the $TERM environment variable

This switch should break termcap/terminfo compatibility to
"TERM=rxvt-unicode-256color", and consequently set "TERM" to
"rxvt-unicode-24bit" by default but there is no termcap/terminfo
for 24-bit color support

I don't think I can change the number of colours from 256 to 16 million as I have seen programs checking specifically for 256. Anyone can help?

Owner

XVilka commented Jul 22, 2015

@spudowiar Thanks! Added your work in the gist.

@ghost

ghost commented Jul 23, 2015

At the moment I keep TERM set to rxvt-unicode-256color. Any help on termcap/terminfo for 24-bit color?

Owner

XVilka commented Jul 23, 2015

@spudowiar - currently there is no way to indicate 24 bit color support via the mainstream termcap/terminfo. See the corresponding thread/messages from the ncurses maintainer https://lists.gnu.org/archive/html/bug-ncurses/2013-10/msg00007.html
The only way is to fork terminfo/termcap or ask somehow to push the extension.

@ghost

ghost commented Jul 24, 2015

Thought so. Thanks anyway!

zchee commented Aug 6, 2015

@XVilka
The solution for tmux.
Support tmux HEAD.

https://gist.github.com/zchee/9f6f2ca17acf49e04088

screen shot 2015-08-06 at 9 26 46 am

@zchee can you confirm this?

curl -s https://raw.githubusercontent.com/JohnMorales/dotfiles/master/colors/24-bit-color.sh | bash

screen shot 2015-08-07 at 8 06 59 am

zchee commented Aug 7, 2015

@JohnMorales Perfect.

screen shot 2015-08-08 at 2 36 02 am

grawity commented Aug 19, 2015

The Linux console now understands the sequences, though it maps everything to the nearest color in the same old 16-color pallette.

Owner

XVilka commented Aug 19, 2015

@zchee Thx, added in the pad and mentioned in the corresponding tmux issue.

drydenp commented Aug 23, 2015

When you use this inside a screen session, screen will interpret the ANSI, not understand it, and print just regular grey text. It has these abilities to change the colour/boldness/highlight of text depending on rules it sets, so it's not strange that it would also mangle these RGB codes. Actually mailing them at uni-erlangen.de if they have some thoughts about it.

Owner

XVilka commented Sep 3, 2015

Updated the gist, s-lang library now has support of the 24bit color http://lists.jedsoft.org/lists/slang-users/2015/0000020.html

zyga commented Sep 6, 2015

Hey. I just wanted to chip in. I'm working on a similar problem from the other end. As an application, what does the terminal actually support? I keep track of this here: zyga/guacamole#15

zchee commented Sep 13, 2015

This is something to satisfy the personal desire. It is not wrote in order to lead to confusion.
Right?

zchee commented Sep 13, 2015

I hope Truecolor may be realized in the early regular way.

dequis commented Sep 18, 2015

Someone enlighten me: what is the point of using colons in the delimiters instead of semicolons? Was the difference more significant back when this was less widely supported?

jerch commented Sep 21, 2015

@dequis
IMHO the colon notation makes it easier to handle false ANSI codes. Since all SGR commands are stackable within one introducer like this \x1b[0;38;2;0;0;0;7m (--> 'reset', 'rgb(0,0,0)', 'reverse') the colon thing would make it possible to parse the rgb-part as one sub-SGR rule. The semicolon variant can have side effects on any following SGR rule since it is parsed on the higher level and the parser can only guess where the rgb rule ends. Also the colon notation avoids the double meaning of the params (plain 0 as 'reset' vs. 0 after 38;2; as color value).
But the colon notation is not compatible to the original DEC syntax, a colon is not allowed within the PARAMS state there. Although the colon seems to make life easier for any parser I would stick with the semicolons. The 256-coloring also uses the semicolon with all the problems listed above and is widely adopted. Therefore the need for just another sub specification is questionable.

rdebath commented Oct 16, 2015

The original aim of the colon notation was to allow support for floating point number sequences. An ECMA-48 conforming parser is supposed to treat the ':' as another digit or a decimal point or something like that and if it doesn't understand the sequence it skips that bit. In the real world none of them do because ECMA-48 is notoriously unclear and "ITU T.416-199303" (the only 'standard' that uses this for it's colour sequences) was never publicly published. Usually an emulator will treat it like '?' or '=' and assume the entire sequence should be ignored, which is perfect in theory but breaks truecolour SGR. (OTOH the character may terminate the sequence if the parser is a little substandard, which makes SGR with just colons problematic.)

This means that the ':' vs ';' interpretations that XTerm and all the application that clone it or use it do things in slightly different ways and IMO the old sane thing an emulator can now do is to treat ':' and ';' as aliases.

This means that from the user point of view you stick with ';' as @jerch has suggested.

rdebath commented Oct 24, 2015

Okay, I've updated my PuTTY patches to include the ability to store truecolour too. I've assumed it should react the same as the XTerm 256 colour sequences for "saved cursor" and the "alternate screen". The scrollback memory cost is minimal (with a small number of colour changes). Interestingly it actually works quite well with a 256 color Windows screen.

Compiles and runs for Windows, Linux GTK font and Linux Pango.

Owner

XVilka commented Oct 26, 2015

@rdebath: have you tried to send them upstream?

Truecolor support in iTerm2 is now available in the beta builds.

rdebath commented Nov 8, 2015

@XVilka I've just asked.

Meanwhile, for this gist, it seems that the Linux Console supports the 24bit colour sequences ... and converts the colours all the way down to the 16 colours it understands!!

And thirdly, here's another nice 24bit colour tester, it shows up the difference with XTerm's fake 24bit colour quite well, it looks very 'steppy' there and smooth on truecolor.

awk 'BEGIN{
    s="/\\/\\/\\/\\/\\"; s=s s s s s s s s;
    for (colnum = 0; colnum<77; colnum++) {
        r = 255-(colnum*255/76);
        g = (colnum*510/76);
        b = (colnum*255/76);
        if (g>255) g = 510-g;
        printf "\033[48;2;%d;%d;%dm", r,g,b;
        printf "\033[38;2;%d;%d;%dm", 255-r,255-g,255-b;
        printf "%s\033[0m", substr(s,colnum+1,1);
    }
    printf "\n";
}'

PS: It's better in rainbow order.

rdebath commented Nov 9, 2015

I wonder how many applications support 4352 colours ?
Ncurses seems to, as does vim.

colors#4352,
setab=\E[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e%p1%{256}%<%t48;5;%p1%d%e48;2;%p1%{256}%-%{256}%/%{17}%*%d;%p1%{256}%-%{16}%/%{15}%&%{17}%*%d;%p1%{15}%&%{17}%*%d%;m,
setaf=\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e%p1%{256}%<%t38;5;%p1%d%e38;2;%p1%{256}%-%{256}%/%{17}%*%d;%p1%{256}%-%{16}%/%{15}%&%{17}%*%d;%p1%{15}%&%{17}%*%d%;m,

I can't go to higher bits with ncurses because it's limited to 32767 colours and the next one up, 15bpp, is 32768. But I might be able to do 31 levels per colour (29791 colours) or I could add some colour ramps, rainbows etc.

Here's a little colour dump script.

#!/bin/sh
r=0; e=`tput colors`
while [ $r -lt $e ]
do  for c in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
    do  col=$((r+c))
        tput setaf $col
        echo -n " $col"
    done
    tput sgr0
    echo
    r=$((r+16))
done

rdebath commented Dec 13, 2015

Okay, I'll ask this question ...

Does anyone know why the "colour space identifier" is missing ?

ie: from "ITU-T Rec. T.416 (1993 E)" page 41

If the first parameter element has the value 2, 3, or 4, the second
parameter element specifies a colour space identifier referring to a
colour space definition in the document profile.

256 colour palette (216 colours + 16 gray + ansi) (colors are 24bit)

I think it should be (216 colours + 16 ansi + 24 gray). http://bitmote.com/index.php?post/2012/11/19/Using-ANSI-Color-Codes-to-Colorize-Your-Bash-Prompt-on-Linux

dvtm's true color discussion.

Owner

XVilka commented Jan 29, 2016

Updated a bit, added dvtm

rr- commented Feb 16, 2016

@mavey80 I read your comment only now cause you've misspelled my handle...

You're completely misrepresenting what Marc Lehmann has written. Your ignorance goes so deep, you're actually saying the exact opposite.

Please provide specific quotes.

You NEVER HARDCODE terminal control sequences into the application.

Isn't it exactly what I've written:

That, and that it's not widely supported nor standardized, so that people wishing to use this feature would need to hardcode escape codes rather than asking terminal whether it supports this feature.

?

To reword: applications (including ncurses, terminfo) do not universally support 24-bit so if people want to use it, they need to use something that might accidentally work in a certain program that decided to do things in its own way, and the program would need to come up with its own code sequences that might stay in conflict with other programs. Which is bad. Which is why Marc didn't want it, because Marc wants to do things the right way (i.e. only if it gets recognized by the terminfo folk).

↑ correct me if I'm wrong here.

Ultimately, I'm afraid you mistake my role in this particular case: it's not the role of a developer, it's the role of a user. All I want is to see settings-independent RGB colors in my terminal, end of story. I'm perfectly fine with hardcoding terminal control sequences in my stupid script (which will never see light outside of my dotfiles) as well as with having to do ./configure --enable-cryptic-feature. I believe most of people here think likewise. Anything else doesn't really matter to me as the end user.

jmtd commented Feb 18, 2016

Just something else to possibly track, a library for using colours for rust programs, supports the ANSI 14 colour and extended 256 modes currently but not truecolour https://github.com/ogham/rust-ansi-term

kmgrant commented Feb 18, 2016

MacTerm also supports 24-bit color sequences; please add it to your list. (https://github.com/kmgrant/macterm)

cool-retro-term 1.0 (https://github.com/Swordfish90/cool-retro-term) supports True Color (tested with awk script in the gist). I don't know what version added support.

Shougo commented Apr 21, 2016

Vim 7.4.1770 supports true color.
vim/vim@8a633e3

@XVilka: do you happen to have code for 24 bit→256 color reduction done right? I've coded it once using L¹ metric with O(1) complexity (rather than L² with O(256) as xterm does), but I agree that using a proper perceptual metric would be better. I'd want it for another project of mine (https://github.com/kilobyte/termrec) for fallback reasons.

It'd be better to do the thinking once and do it right rather than reinvent the wheel badly many times.

If such code is written, it could also be used in patches for terminals whose authors rejected proper 24 bit because of not wanting to expand their memory usage.

Owner

XVilka commented May 21, 2016 edited

@kilobyte: see this library https://github.com/gfiumara/CIEDE2000 for LAB space and https://github.com/markusn/color-diff this, containing conversion from RGB too.

Owner

XVilka commented May 22, 2016

Looks like XFCE Terminal release with libvte supporting true colors (the one from GTK+ 3) will be available soon: http://blog.alteroot.org/articles/2016-04-12/road-to-xfce-4.14.html

Could someone actually test ConEmu as having 24bit TrueColour?
I am not seeing it.

sickill commented May 25, 2016 edited

Latest version of asciinema-player (https://github.com/asciinema/asciinema-player) also supports true color now: https://asciinema.org/a/46819

kilobyte commented Jun 1, 2016

Windows console as of Windows 10 build 14352 now understands 256/24-bit colour codes: Microsoft/BashOnWindows#76 and approximates them to 16 colour fg/16 bg (just like Linux does 16/8). This limitation is because their native console API and internal view supports only limited attributes.

Owner

XVilka commented Jun 4, 2016

For Terminator users, there is a branch for GTK+3 version, which supports true colors and much more, but it's not yet merged in the master and not yet released as well. Here is the bug for pushing it upstream. So if you want to help this - you can test it more, write patches and ping developers. Looks like this one is almost ready, I see no big stoppers against merging this branch in the master and releasing a new version of Terminator.

@XVilka Black Screen now supports true colour as well.

Owner

XVilka commented Jun 14, 2016

@shockone thanks, added in the list!

hzeller commented Jun 18, 2016

I've made an image viewer that uses the functionality: https://github.com/hzeller/timg
With animated gifs on large terminals it is also a neat performance test for these terminals...

@ghost

ghost commented Jun 23, 2016

@XVilka rxvt-unicode now has an alternative patch in upstream (far more efficient than my patch)

Owner

XVilka commented Jun 28, 2016

@spudowiar corrected the gist, thank you.

Quickly view (satellite) imagery directly in your terminal: https://github.com/daleroberts/tv

Owner

XVilka commented Jul 13, 2016

@daleroberts, @hzeller thanks, added both tools in the gist.

Owner

XVilka commented Jul 29, 2016

Xfce terminal should have support now, since the release 0.6.90. Can anyone please confirm?

Compiled and ran the following:

fornwall commented Sep 4, 2016 edited

@XVilka Termux, a terminal emulator for Android, now supports true color starting from the new 0.40 release!

Xliff commented Sep 7, 2016

Just in case people were wondering what the ${bg} meant (I did), it breaks down like follows:

38 - Extended set forground color
48 - Extended set background color

Just an FYI

@jmtd I added support for truecolor in rust-ansi-term a while back.

Elronnd commented Sep 23, 2016

For the ones that approximate the closes colour, can you mention which one of semicolons and colons they support?

@spudowiar i've compiled rxvt-unicode on ubuntu 14 LTS with --enable-everything --with-x and for whatever reason the color output is wrong. here's a screenshot. the left side is the 24 bit enabled terminal and the right side is the 256 color terminal. the 256 terminal's color cube looks correct.

selection_001

Worth noting: tmux only supports 24bit colours for apps running inside of it, but does not support 24bit colours for itself.

rr- commented Oct 6, 2016

rxvt-unicode only approximates true color and still uses 88 / 256 color pallete underhood.

._.

@rr-: Thanks, I thought I was going insane trying to get this working lol... Guess I'll try to find some other terminal to use.

WHERE IS MY NCMPCPP AND WEECHAT TRUECOLOR‽‽‽ SOMEONE FIX THIS PLS.

Owner

XVilka commented Oct 17, 2016

Thanks, moved rxvt-unicode into a lower section. Shame they still stuck to the ultraconservative approach.

rr- commented Oct 20, 2016

mpv now supports truecolor: mpv-player/mpv@dd02369

khamer commented Oct 22, 2016

Might make sense to mention

akz92 commented Oct 23, 2016

iTerm2 supports True Color since v3 in beta builds, but v3 is now stable so it's not only in beta builds anymore. https://www.iterm2.com/downloads.html

Owner

XVilka commented Nov 8, 2016

Updated the gist, thank you.

Owner

XVilka commented Nov 15, 2016

Finally moved Midnight Commander into the 'support true color' section. See this bug for color themes for mc http://www.midnight-commander.org/ticket/3724

Owner

XVilka commented Nov 15, 2016

For those who want to use Terminator with true color - here you can find an instruction how to build .deb https://bugs.launchpad.net/terminator/+bug/1030562/comments/101

rofl0r commented Nov 16, 2016 edited

when looking at the "true color" midnight commander themes, of which none is using more than a handful of colours, i get the impression what you guys want is really just the ability to freely pick colors with an RGB value from the full range of possible 24bit colous, instead of being restricted to a hardcoded palette - but it is not actually needed to have more than 256 colors in use at the same time in a terminal window. this is already possible since a long time with the "initc" terminfo capability, which looks like

initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,

for example for xterm-256color.
ncurses supports this mode since ages, and i've written a wrapper library to use it in an easy way by just throwing RGB values at it instead of dealing with colorpairs. https://github.com/rofl0r/concol
i've implemented this mode also for netbsd-curses ( sabotage-linux/netbsd-curses@f824eed ) , and i do indeed think that given the ability to select whatever color with RGB values, a restriction to 256 colors at the same time on the terminal makes sense in order to keep the code simple, and resource allocation reasonable.

Windows Insiders build #14931 added true color to the windows console: https://blogs.msdn.microsoft.com/commandline/2016/09/22/24-bit-color-in-the-windows-console/

Hyper.app is a beautiful new cross-platform terminal built on Electron. And it supports true color 👍

Owner

XVilka commented Nov 27, 2016

@flybayer, thanks added!

So, finally only one major terminal doesn't have support of true color mode - PuTTY. Any ideas how to reach its developer?

fpqc commented Dec 1, 2016

Windows 10 bash terminal now has full support

Defman21 commented Dec 5, 2016

pantheon-terminal supports true color as well.

Owner

XVilka commented Dec 6, 2016

Added Patheon Terminal as libvte-based.

Owner

XVilka commented Dec 8, 2016

LilyTerm removed support for GTK3+ :( Tetralet/LilyTerm@62bbd17

xelra commented Dec 16, 2016 edited

So since terminfo doesn't seem to support true colors, is there any way to check a terminal for true color capability? I need to write a script that outputs either tc colors or just 256 colors, based on the capability of the current terminal.

EDIT:
I actually found that in theory one could test the terminal via DECRQSS by sending DCS $ q m ST, but I could only get this to work in xterm and pangoterm (libvterm). It's a basic VT100 escape sequence but seems to be unsupported by libvte, rxvt and Konsole. At least those are the ones I tested. It's a shame so many terminal emulators identify themselves as xterm, but don't actually support xterm. This solution would be really nice to have, because it doesn't look like terminfo/termcap will support truecolor within the next decade. If you're the maintainer/dev of a terminal emulator and reading this, please consider supporting DECRQSS. That would help a lot for actually programatically determining the color capability of a terminal. There's a seamingly infinite amount of developers and users on Stack Overflow and its forks that are desperate for a way to do this, with no existing solution if you go above 256 colors.

Owner

XVilka commented Dec 21, 2016 edited

Well, I'm aware there are some unofficial extensions of terminfo http://lists.schmorp.de/pipermail/rxvt-unicode/2016q2/002251.html

Here how you can use it tmux/tmux#34 (comment)

pickfire commented Jan 2, 2017

Seems like radare2 also supports true color from what I saw in @s-gilles's post about 24bit true color support in vis in which he links to radare2. I hope @XVilka can at radare2 to this page.

rofl0r commented Jan 3, 2017 edited

i've published an example C program that shows how one can use the initc terminfo capability to use user-supplied RGB values via ncurses. https://gist.github.com/rofl0r/12c766fc7e72f90cb56daf5d15652fc9 (screenshot: https://0x0.st/pYT.png )
note that this was possible since years with most modern terminals, that support color changing. there's not the true-color terminal escape sequence, most terminals have their own way to use the initc capability. so from that PoV it seems a lot of misinformation is conglomerated here and hyped by people that don't understand the background (which is admittedly rather complex).

also there seems to be a fundamental misunderstanding: that xterm or ncurses supports "only" 256 colors does not mean that it does not support true color. it supports 256 colors that can be displayed at the same time each one with a custom true color 24 bit RGB value.

so it would be much more productive if you told ppl how they can use the initc terminfo cap, and asked upstream developers to support that capability and support a 256 color mode.

Owner

XVilka commented Jan 3, 2017 edited

@pickfire ofc it suported by radare2 - this is where it all started :)

@rofl0r - still, it prevents the direct RGB SGR sequences, which are vital for programs like radare2, neovim, etc.

rofl0r commented Jan 3, 2017

@XVilka: why are RGB SGI sequences "vital" for these programs ?

xelra commented Jan 4, 2017 edited

@rofl0r Check out what I wrote a few comments back. That sequence is vital, because it actually lets you and any app in any programming language test the color capability of the terminal emulator it's running in. And I mean actually test it, without assumptions or additional user input or configuration. No environment variables or termcap/-info stuff needed.

It's documented here.

oconnor663 commented Jan 6, 2017 edited

I think Alacritty has truecolor support (based on this comment): https://github.com/jwilm/alacritty

Owner

XVilka commented Jan 7, 2017

@rofl0r - to be sure that color theme used is always the same among all users, and to eliminate the need to setup palette each time.

@oconnor663 - done, thx!

ttdoda commented Jan 13, 2017

mlterm is not a libvte-based program. It has original terminal emulation engine.
mlterm provides libvte compatible library. Libvte-based terminals can use that library and run with mlterm's engine.

cowwoc commented Jan 18, 2017

@xelra What does the output of DCS $ q m ST look like?

Hey, the midnight commander ticket number is misspelled (the link is okay). It's easy to remember: 37_24_ for 24-bit color skins. (Yes, I did deliberately wait for such a nice ticket number :))

Owner

XVilka commented Feb 1, 2017

@egmontkob thanks, fixed.

For Terminator 1.90, the note "if compiled with GTK+3" is unnecessary. It can only work with GTK+3 and with vte >= 0.38, these are required dependencies.

Also, since it's written in python, it's not "compiled" as the user would think. (It's "compiled" under the hood, not as an explicit step by the user.) So the wording is also debatable. Thanks!

For the "joe" text editor, the feature request is at https://sourceforge.net/p/joe-editor/patches/116/ (yeah I messed it up that I have filed it under "patches", there's no patch yet).

egmontkob commented Feb 5, 2017 edited

@rofl0r Re your multiple comments, but especially Midnight Commander:

In my opinion, the ideal goal would be to completely get rid of the palette approach. This would allow mcview to display in their full colors (and attributes) text files with ANSI escape sequences (mc bug 1849), mcedit to highlight a color string (e.g. #abcdef) in its exact color, and so on.

mc uses either the ncurses library (which doesn't support truecolors, and its author has made it clear that he won't add support) or slang (which, similarly to ncurses, still has a limited, palette-based truecolor support). See my unanswered question about this on the slang-users mailing list.

My idea of dropping both libraries and coming up our own screen drawing was firmly disliked (mc bug 3264). This could have eliminated this bottleneck, but I understand the reasons nobody wanted to go down this path.

So, in order to be able to use significantly more than the sixty-some color pairs you can define in the skin files, first we'd need to get rid of this bottleneck in the underlying library(-ies) and then further change mc to take advantage of no longer having this constraint. This is absolutely unlikely to happen in the foreseeable future.

As such, in its current (I mean forthcoming, with version 4.8.19) form, mc's truecolor support could be implemented in two ways: either by emitting actual truecolor escape sequences (this is what is implemented), or by fiddling with the palette using the OSC 4 (this is what ncurses's init_color() as well as your code does based on the initc capability). From the simple user's point of view, they would be almost the same. They would probably be the same as long as things work as expected... Hang on...

From the implementation point of view, going for the approach of using the truecolor escape sequences was presumably way easier. At least it was very easy.

The real truecolor support (I mean, where truecolor escape sequences are sent to the terminal emulator) has two drawbacks I can think of. First, it's not supported by all terminal emulators. Indeed, but nor is the other approach. The second is: A bit more data is transferred over the network, serial line etc. In the unlikely case that you're worried about this on your particular setup, you should revert to one of the standard skins, or even black and white. I think it's a non-issue.

You say a reason for the 256-color limit could be to keep the resource allocation reasonable. I don't buy this. Terminal emulators that allow large scrollback buffers should indeed be careful. E.g. VTE (the widget behind GNOME Terminal and many others) uses a runlength encoding of attributes (a continuous region of the same attributes is stored once: 64 bits for the fg+bg colors and all other attributes, and another 64 for the offset of the boundary). Then this stream even gets compressed. For fullscreen apps, or rather, libraries underneath (ncurses, slang) they'd similarly need probably 8 bytes per cell for all the color and attribute information, which, on my laptop, considering e.g. my 239x70 fullscreen terminal, would take 130 kBytes (rather than half of it as they do currently) out of the available 8 or so gigabytes. Not even worth talking about.

Returning to that italic "almost" word, from now on I'll talk about the drawbacks of the other, OSC 4 / initc approach.

We could create a nice support matrix for various terminal emulators whether they support true colors and whether they support OSC 4. But the latter should actually be split to 3 parts: if setting the color with OSC 4 is supported, it is still not sure that OSC 104 for resetting the entry is supported, and that OSC 4 can also query the color. As far as I can remember, there are examples for both of these not being supported even where OSC 4 is available. I'm not sure though, and it might have been older versions.

Wherever truecolor support is missing, it's only a matter of adding this feature (that is, developer laziness/willingness/etc.). We'll see an example where adding OSC 4 is theoretically problematic.

So... first, how do you know if the terminal emulator supports OSC 4? Theoretically you might say you see it from the initc capability. In practice, at least konsole defaults to TERM=xterm (or xterm-256color) which advertises this support, although it doesn't actually support them. That is, you'll get faulty colors in konsole. Okay, let's assume it's fixed.

Obviously, apart from setting the new palette, you'll have to be able to restore the previous. (You need this not only when you quit mc, but also when you toggle the panels on/off, or launch an external command.) How do you do that? There are two ways. Either emit the escape sequence OSC 104 which resets them, or on startup you query the actual colors with OSC 4 and the "?" parameter. Which one would you go for?

How do you know if OSC 104 is supported? Maybe there's a terminfo for that, I don't know.

How do you know if OSC 4 with the "?" attribute to query is supported? OSC commands with the "?" flag suck big time. They are asynchronous, they answer as if the user typed the answer from the keyboard. How long to wait for an answer? Let's say, 1 second, okay? So, if the terminal emulator does not support it, you wait the user for 1 second on each mc startup. Not good. If the terminal supports it, over a slow ssh channel sometimes you'll hit that timeout and mc will not start up with the desired colors. Not good again. Or make mc start up with the default colors, and repaint with the desired skin whenever the response arrives? Terribly user-unfriendly flickering, plus extremely complicated handling of input across all the widgets in mc. Not good again.

Let's note that these two approaches, OSC 104 (reset) and OSC 4 ? (query) (even if we suppose it works reliable, despite just having seen that it theoretically can not) are different. That is, they are different if the user has already altered the palette before starting up mc. In that case 104 would revert to the terminal emulator's default (not the desired behavior), while OSC 4 ? (which, again, cannot be implemented reliably) would revert to the values before starting up mc (the desired behavior).

You scroll up in mc. (Yes, in xterm you can drag the scrollbar with the mouse, even when in application mode.) All the colors there are screwed up.

mc crashes for whatever reason. All the subsequent colors (e.g. "ls -l") will be screwed up. Or you somehow manually fix the palette. Then the remains of mc will look awful.

The ssh channel gets stuck just when you launch or when you quit mc or when you toggle the panels on/off. Due to switching to/from the alternate screen and setting/resetting the palette is not a single atomic step, you might see flickering, or getting stuck for a longer time with faulty colors.

These are the kinds of nightmare you have with the palette approach, everything has an influence on everything else that it shouldn't have.

tmux, some panels next to each other. What should happen, how should tmux communicate the change of palette towards the host terminal emulator? No wonder it does not support them at all. It could always set the palette according to the focused pane, resulting in the other panes always having totally off colors, a terrible user experience. The one I had with X Window and FVWM on my 1024x768 monitor with a 1MB video card until I upgraded to a 2MB video card that could do a fixed 5+6+5 bits instead of a palette. This way 20+ years ago. Or it could translate them to truecolors towards the host emulator. Yes, it could really do it, and I wonder why it doesn't do. But it's another proof of truecolors being way better than the palette nightmare.

Graphical systems, desktops etc. have known for 20+ years now that palettes suck big time, they just can't be made right. In my idealistic opinion, only the RGB colors should remain in terminal emulators, and anyone wishes to have a consistent feeling (e.g. mc's blue being the same as ls's blue), it should be achieved by a common library that these utilities all use. This is what happens at the graphical themes. I know the world is not "ideal" and I know I'm not popular with this idealistic opinion. Don't worry, the legacy 16 / 256 palette colors will stay with us. They are okay. Really, they are really not bad at all. IMO they shouldn't exist, but they do, and they are okay.

Apps redefining them is what's not okay and leads to all kinds of inevitable problems and unreasonable behavior that are simply unexplainable to / unacceptable for the users.

I hope it's clear now why it never occurred to me and would have never implemented truecolor support in mc in a way that's based on modifying the palette. Using the real truecolor escape sequences is the only proper, reliable way. The fact that mc still cannot display more colorpairs at a time than the limit of ncurses/slang is irrelevant and not a valid reason whatsoever to consider the other, braindamaged approach instead. The point of truecolor support is not to suddenly have a crazily colorful UI. The point is to be able to pick any color out of the 16M for any role, and doing this without having an impact on the other colors.

are you sure the color palette for the alternative screen is not separate?
anyway, even the worst-case scenario isn't that bad, because you can typically just refrain from modifying the first 16 colors.
almost no application uses the full palette, and those which do a) can reset the palette themselves and b) you can wrap it into a script that does it.

"are you sure the color palette for the alternative screen is not separate?"
xterm: Tried it and yes, the palette is shared between the two screens.
VTE: I know the codebase well enough to tell for sure without trying that yes, it's also shared here.

Owner

XVilka commented Feb 13, 2017

@egmontkob: will fix today. Yes. making a separate table, or updating this one, separating the colors calculation/ability to show and the ability to use direct RGB sequence.

Owner

XVilka commented Feb 13, 2017

And maybe it's time to make it's a repository, to be able to have multiple contributors, accept pull requests and so on.

One more thing, could you please add a section about how to detect if a terminal emulator supports true colors? Something along these lines:

There's no reliable way, and ncurses/terminfo's maintainer expressed he has no intent on introducing support. S-Lang author added a check for $COLORTERM containing either "truecolor" or "24bit" (case sensitive). In turn, VTE, Konsole and iTerm2 set this variable to "truecolor" (it's been there in VTE for a while, it's relatively new and maybe still git-only (I haven't checked) in Konsole and iTerm2).

This is obviously not a reliable method, and is not forwarded via sudo, ssh etc. However, whenever it errs, it errs on the safe side: does not advertise support whereas it's actually supported. I don't think there's a way it could err on the other side, I'm not sure though.

App developers can freely choose to check for this same variable, or introduce their own method (e.g. an option in their config file), whichever matches better the overall design of the given app. I personally recommend checking $COLORTERM since that would lead to a more unique desktop experience where the user has to set one variable only and it takes effect across all the apps, rather than something separately for each app.

Owner

XVilka commented Feb 16, 2017

@egmontkob: Done, will make a repo this weekend.

Hi, emacs has now learned 24bit colors, though it requires custom terminfo
http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=e463e57
Here's the thread. The discussion continues to this month.
https://lists.gnu.org/archive/html/emacs-devel/2016-08/msg00711.html

Owner

XVilka commented Feb 19, 2017

@holomorph: Awesome news! Thanks, added it.

arcanis commented Feb 20, 2017 edited

For what it's worth, terminfo will probably never be updated to support truecolors.

I suggest we stop asking him, his mind is set and apparently some people have harassed him about this (in private, I guess?).

Let's see if we can come up with something reasonable in https://bugzilla.gnome.org/show_bug.cgi?id=778958.

nicm commented Feb 21, 2017 edited

@egmontkob

tmux uses Tc to mark terminfo entries for terminals that support RGB colours. Assuming I understood correctly, entries with leading upper and then lower case are considered extensions. Tc has made it to the terminfo database in the entry for st but you may need luck to get it added to any of the other entries.

FWIW if everyone standardizes on a different, sensible solution, I will use it.

For me, this means it must use terminfo, so not an environment variable.

emacs' setf24/setb24 is a reasonable idea, but I would strongly recommend changing it to accept three separate R,G,B parameters like conventional terminfo capabilities. Since they have done it already, perhaps set24f/set24b for a fixed version instead (which matches setaf/setab anyway).

nicm commented Feb 21, 2017

A few other things it may be worth noting:

  • I expect that we (OpenBSD) would be willing to add entries for RGB colour to our terminfo database, of course assuming they are sensible and OpenBSD users need them. We already have local changes, for example for rxvt-unicode.

  • The main reason I would recommend using 3 parameters rather than 1 is that using a weird capability form is just throwing up unnecessary roadblocks in front of something where there is already considerable resistance from upstream.

  • I personally would (and will) forget about the colon form entirely, it is silly.

  • The reason I chose a flag (Tc) was that it is easier for users to manage than a pair of long, complex sequences. And this is not a feature with historical baggage where here will be terminals that use different sequences to do the same thing, only modern terminals will support this and they would be silly not to use the same escape sequences as everyone else.

chee commented Feb 21, 2017 edited

$COLORTERM is no longer git-only in Konsole

egmontkob commented Feb 22, 2017 edited

@nicm

I'm completely with you on using terminfo (rather than some env var), three separate parameters (I can't see a valid reason for squeezing them into one), and even the set24f/set24b naming (or whatever else that looks kinda sensible -- what does "a" stand for in setaf/setab?).

I don't get the Tc bits, what's that? Is that a boolean terminfo flag whose presence denotes truecolor support? If so, why do we need such a flag rather than looking for the presence/absence of set24f (or whichever we settle with) itself? Or is it instead of set24f/set24b so that every app would have to hardcode the actual sequences? I don't like this idea that much.

The colon form is not silly, it is a great idea for forward compatibility if and only if implemented correctly in terminal emulators -- alas it is not in quite a few. As such I guess we should go for semicolons here (so with that I agree with you).

@XVilka

The page says konsole supports both colon and semicolon. The linked bugreport where konsole handled this issue has long discussions about it. I've tried it though and it doesn't work (konsole version 16.04.3 on Ubuntu 16.10). Not sure if it was never actually implemented, or broke at some point.

egmontkob commented Feb 23, 2017 edited

@dequis @jerch @rdebath Returning to your old question/answers about semicolons and colons:

What I believe is the main reason is forward compatibility. jerch's response was the closest to that, but I don't think ease of implementation should matter (in fact, implementing support for colons was really nontrivial in VTE).

Let's take this example: \e[38;2;3;4;5;7m.

The main question is: How does a terminal emulator that does not (yet) support truecolors know where the subparameters to "38;2" end and "normal" parameters resume? How does it know that it should not start italic (3), underlined (4), blinking (5) mode but it should render in reverse (7)? In order to know it, it has to know at least that "38;2" takes three additional parameters.

Same question for all the emulators that support truecolors: What to do upon encountering \e[38;6;1;2;3;4;5;6;7;8;9m? I don't know, since I don't know what "38;6" will mean at some point in the future.

This can be fixed by having colon as the separator. Receive a \e[38:2:3:4:5;7m and even if the emulator has no clue what "38:2" means and drops it, it can drop it along with the colon-separated parameters, and still interpret the 7 after the semicolon.

Similarly, if an emulator sees a \e[38:6:1:2:3:4:5:6;7;8;9m, it can drop that mysterious future "38:6:1:2:3:4:5:6" that it doesn't know about, and still enable modes 7, 8 and 9.

In order for this to work, emulators strictly MUST NOT treat colons as aliases to semicolons, in that case there would be absolutely no point in introducing another character for the very same purpose. They must implement this two-level parsing with semicolon being the "major" separator and colon being the "minor" one, and must reject the colon in the "major" separator position. E.g. \e[38:2:10:20:30:48:2:40:50:60m must not switch the background to rgb(40,50,60) (and probably shouldn't change the foreground to rgb(10,20,30) either). Sadly, for compatibility reasons, they should also parse if these params are separated by semicolons (of course then it's not possible to drop unknown sequences) and even ones where the first separator is semicolon and the rest are colons, as discussed in KDE #107487.

For VTE this it was implemented in GNOME #685759. As I've tried xterm it also seems to correctly reject if colon is used somwhere else other than between multiple parameters for a color. As far as I can see, these are all the emulators on Linux that support colons. I have no infrastructure to test the behavior of emulators running on Win or Mac, although I would love to see if they support colons properly in this regard.

nicm commented Feb 23, 2017 edited

@egmontkob

The a in setaf/setab is for "ANSI".

Tc is a flag that says "this terminal supports the ISO RGB colour sequences." Yes, we could check for the presence of set24f/set24b/setf24/setb24/whatever instead if people are using them. There is precedent for flags that mean a hardcoded sequence is supported (such as AX and XT), but I agree that strings giving the actual escape sequences to send would be better.

I really don't have any particular attachment to it being a flag, I was just pointing out what tmux currently does :-).

The colon form might seem sensible but it isn't useful in practice and I doubt it will ever be. But in any case, we agree that terminfo should use semicolons, other than that it doesn't matter.

I could support set24f and set24b in tmux, in the form you suggest in the GNOME bug, but without terminfo support users would still have to add them themselves. I'm a bit reluctant to add any code unless there is some indication other people are going to use it too though.

So do we have opinions from other application developers on whether they would support it? I guess vim, emacs and slang are probably the big ones from the list that currently support RGB colour.

nicm commented Feb 23, 2017

Of course the most future proof solution would be for terminals to be self-documenting and able to tell applications their own capabilities. You can get some (fairly useless) flags and stuff with DA/DA2, and xterm does have some experimental support for retrieving terminfo entries with DCS+q but unfortunately it only supports keys. A way to say "give me your whole terminfo entry" would be much better. We would still need tparm() or equivalent to expand the parameters and whatnot, but it would avoid some of the problems of a central database. Perhaps terminfo is too entrenched anyway. And this is not really on the topic in any case.

egmontkob commented Feb 23, 2017 edited

@nicm Could you please clarify: are you a tmux and openbsd developer? :)

The two of us are pretty much on the same page, that's good (hoping we can convince others). I hope slang and emacs developers will be easy to convince. Not sure about vim, I feel it often tends to prefer its own solution (e.g. its own config file option), I'd delay that for a while.

It would be quite important to get buy-in from a few key distributions that would patch their ncurses, IMO at least Debian and Fedora.

I guess the next step is to find the proper way of coming up with a suggestion and pushing this towards people. I mean the comment thread here is far from ideal, at least there should be a dedicated page for this. What do you think would be the right way to go ahead?

Re Tc vs set24f: setaf/setab have a tiny bit of ambiguity: for the first 16 colors it's not obvious whether to use the legacy or the 256-color escape sequences. At least xterm's entry seems to encode an if-branch for 0-7, 8-15, 16-255 intervals, mapping the first 8+8 to the 30..37/40..47 and 90..97/100..107 range. This might be a reason for having a terminfo entry. RGB doesn't have this kind of question (apart from the colon/semicolon issue). So this might be a tiny reason for going with Tc. However, I believe the nicest is if we copy the pattern of setaf/setab as much as possible, so I still tend to vote for set24f/set24b (or whatever name will be chosen) rather than Tc.

As for the name, somewhere (can't remember where) I saw that either "38;3" or "38;4" (can't remember which) should be similar to "38;2" but work with hue/sat/val. I guess that one also takes 3 8-bit parameters. (I don't think any emulator does or will ever implement this, and if still, it'll probably immediately convert to RGB.) As such, I'm wondering whether maybe setrgbf/setrgbb might be a better name than set24f/set24b.

Re your latest post: I agree with you, something like the $TERMCAP environment variable fully describing the behavior and getting forwarded via ssh might be a good solution. As you also said it's way beyond the scope of our current discussion, and I won't have capacity to work on anything like this.

nicm commented Feb 23, 2017

@egmontkob Yes I am tmux and OpenBSD developer, and I maintain the terminfo database for OpenBSD.

I'm happy with terminfo entries called either setrgbf/setrgbb or set24f/set24b or setf24/setb24, it doesn't matter much to me. I think using three arguments is more important.

I think someone who is interested and has an appreciation of the issue needs to talk to some of the application developers. Since emacs have already defined capabilities, perhaps it would be best to first ask them why they chose to use one parameter and if they would consider changing before it gets into much use (or makes it into a release or whatnot). I don't think the name is of huge importance, or their choice of having entries for both colon and semicolon (I expect the colon one will just die off in time). But the use of one argument instead of three is a big question mark and I think it will make it harder to both get application developers to adopt it as well as reducing any hope of getting it into terminfo upstream. If they do change, then it is just a matter of people adding support (it will be a simple change for tmux and probably for most other programs too), which will probably happen over time so long as developers are aware other programs are using these capabilities.

Saying that, I don't particularly want to volunteer to do this :-).

Didn't you do basically the same as this for SGR mouse? ISTR you opened issues for a load of programs which definitely helped to drive it forward.

nicm commented Feb 23, 2017 edited

There must be a reason they chose to use one argument. Perhaps they just used emacs' internal form of RGB colour, or they didn't know that terminfo can support multiple parameters (which seems unlikely). Anyway, terminfo entries can have up to nine parameters.

In fact, there is already a sequence that accepts RGB colours, but for a different purpose -- initc, which has four arguments:

       initialize_color              initc      Ic        initialize color #1
                                                          to (#2,#3,#4)

initc accepts R,G,B scaled out of 1000:

$ tput initc 1 500 500 500|cat -v
^[]4;1;rgb:7F/7F/7F^[\

So there is an argument that a new sequence should use 1000 too. I'm not sure we should necessarily get into that though, I think 255 would be both simpler and better.

nicm commented Feb 23, 2017

This comment in their commit message at http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=e463e57 does imply they would be willing to change if something better comes along, perhaps we could drop an email to the committer?

+Currently there's no standard way to determine whether a terminal
+supports direct color mode. If such standard arises later on, support
+for @samp{setb24} and @samp{setf24} may be removed.

Owner

XVilka commented Feb 25, 2017

@egmontkob - I've imported that gist into the repo and gave you access. If you can think a better name for a repo and organization - please write your suggestions. I think with all latest developments and suggestions this needs to be a repo to have an issues and pull requests, as like as more than one contributor.

pvinis commented Mar 14, 2017

you can use my scripts to test. I gathered a few of these color-printing scripts in one.

https://github.com/pvinis/colortools

or you can install with brew install pvinis/pvinis/colortools

Please add:

DomTerm-based terminals (qtdomterm, ldomterm) [delimeter: colon, semicolon]

@nicm I'm sorry but I've totally lost motivation in this one (at least for now) and moved on to working on another hopefully cool idea. I might get back to the truecolor vs terminfo story later (in several months/years) but I cannot promise. I hope others will solve it by then :)

TerjeBr commented Mar 20, 2017

I see no link to the repo @XVilka referred to in #gistcomment-2010783 but I assume he meant https://github.com/XVilka/term-color

TerjeBr commented Mar 20, 2017

@XVilka May be you should add a link to that repo in this gist?

Terminix has been renamed to Tilix btw due to some legal issues

nicm commented Mar 24, 2017

@egmontkob fair enough, I don't use it so there is a limit to how much time I want to spend on it myself

jerch commented Apr 20, 2017 edited

Kinda offtopic, but this one got me:

Of course the most future proof solution would be for terminals to be self-documenting and able to tell applications their own capabilities.

This idea is really intriguing. Yet it boils down to the same old problem - you need some "authority" that clarifies the semantics for the players, emulators on one side and program (libs) on the other. Thats kinda what terminfo does atm in a very compact and static way (that might be discussable with all pros and cons). Also note that XML kinda failed for that reason, it just introduced more and more technical sub-sub-specifications while missing the ease of use and ppls needs, and ppl started to use technically simpler solutions like JSON. In the end its all about semantics ;)
Maybe use more of the self reporting functions? Idk, maybe some kind of a "handshake protocol" can help, it still needs the authority...

Edit: And to open pandora's box - this also touches very fundamental POSIX things - what is a terminal, what should it be like in the future? What about backward compatibility? Are we stuck to VT100 with some color enhancements forever? :O Just to name a few...

rr- commented Apr 27, 2017 edited

Isn't it time for termcap to start supporting it?

Regarding mlterm: First, ttdoda's comment is absolutely right, it is not based on VTE. Second, as discussed here, it has an internal palette and easily runs out of available indices, in which case colors become corrupted. As such, I'd argue that it does not properly support true colors.

egmontkob commented May 4, 2017 edited

<spam>
Not related to truecolors, but I thought you guys might like this. This is what I was working on recently: Hyperlinks (a.k.a. HTML-like anchors) in terminal emulators.
</spam>

Owner

XVilka commented May 8, 2017

@egmontkob fixed, thanks.

nicm commented May 12, 2017

@jerch - the structure of terminfo is fine, and having a central authority for the meaning of terminfo capabilities is not the issue. All we need is a way for terminals to report some or all of the terminfo entry themselves and a way for programs to deliver that to libtinfo etc. Then terminal authors can maintain it for their terminal, it can match the terminal's current version, etc, instead of requiring a central repository which is very slow to roll out.

Perhaps add mosh and their discussion on truecolor? mobile-shell/mosh#649

B-bar commented May 26, 2017

Why is it hard to true-color-ize xterm-based terminal?

jdebp commented May 27, 2017

@egmontkob and @nicm, in neovim/neovim#6816 I have already (0fab6e275fe2e7ce357614ab1a7a5f535ba60502) changed nvim to share the Ss and Se terminfo extensions pioneered by tmux. I intend to treat RGB support the same way. For the capabilities, I plan to go with Rüdiger Sonderfeld's proposal from 2013. It is sensible, and uses 3 parameters.

The way that nvim handles Ss and Se capabilities will also be the way that it handles setrgbf and setrgbb. If the capability is present, it will be used as-is with no further modification. terminfo is authoritative if present. If it is absent, nvim will determine when to fix up the terminfo record, based upon a rough parsing of terminal types; just as it does for Ss and Se (q.v.).

nicm commented May 31, 2017

@jdebp from the next release (2.6), tmux will look for setrgbf and setrgbb capabilities for RGB colour. If they are not present, it will fill them in itself if the Tc flag is present (so the current behaviour).

nicm commented May 31, 2017

@jdebp if you do something different, let me know please and I will change tmux, it will be OK to change it until the next release (in Sep or Oct).

whydoubt commented Jun 1, 2017

@XVilka: As @fornwall mentioned, Termux (open-source Android terminal app w/ ssh client) does have true-color support. JuiceSSH and Termius (closed-source Android ssh client apps) do not. ConnectBot (open-source Android ssh client app) does not, but I just sent a PR to add it connectbot/connectbot#531.

whydoubt commented Jun 1, 2017

@XVilka: On the link for img.sh, can you just remove the "?h=vte-0-36". The latest version of img.sh fixed an issue I experienced with the vte-0-36 version.

Owner

XVilka commented Jun 8, 2017

@whydoubt, @dennisse - added mentioned entries and fixed the link. Thanks!

sevagh commented Jun 8, 2017 edited

Hi all. I'm having some truecolor trouble on Alacritty, gnome-terminal (tested both with and without Tmux).

I'm using a program that outputs png images in truecolor to the terminal: https://github.com/nabijaczleweli/termimage

After displaying the image, my output gets blocked/blacked out somehow. This is the result of me pressing enter a few times, then pressing ls twice:
screenshot

You can see the command prompt is hidden, and most of the first ls output is truncated. After that, it's fine.

Anybody know what could be causing this behavior? This only occurs on my computer, I asked some people to test it on their computer and they had no issues.

I'm using Fedora 25, i3wm, lightdm, alacritty/gnome-terminal both tested, tmux and without tmux both tested.

edit 1
Here's a friend's output: https://i.imgur.com/QMtrGf8l.png

As you can see his prompt resumes immediately after the program exits (as expected).

edit 2

So when ls prints a color character on my terminal, that's when things return to normal. This led me to the solution, if I print \033[0m after the image displays, everything is OK. Now I'm curious why I need to explicitly print this character.

egmontkob commented Jun 11, 2017 edited

@sevagh Apparently termimage is broken in the sense that when it exits, it leaves the foreground color at an explicit black rather than the default foreground of the terminal emulator. Hence it's black until someone (shell prompt or ls or anything else) changes it. termimage should restore the terminal's default foreground and background color when it quits.

Update: I see you filed this bug against termimage and they've already fixed it.

@XVilka Can you add Eternal Terminal to the list of products that support true color? https://mistertea.github.io/EternalTCP/

Owner

XVilka commented Jun 26, 2017

@MisterTea @sevagh - thank you, added.

Windows 10 bash console

Do you have any source that says it is restricted to the bash console? All terminals on Windows (PowerShell, CMD, WSL Bash) run in the Windows Console Host. I am pretty sure that they added the colour support to the Console Host, which means it's available to all terminals, not just "bash" (I assume you mean WSL Bash, you can also run a MinGW bash on Windows that is bundled with git)

Tenzer commented Jul 4, 2017

I made a command line image viewer with 24-bit support a while back, don't know if you want to add that as well? https://github.com/Tenzer/explosion

Hi, I forked ls to make it with icons that support true colors: https://github.com/sebastiencs/ls-icons

VortexCortex commented Jul 22, 2017 edited

Adding entries to the terminfo database shouldn't require that the ancient and backwards ncurses "color pair" rendering engine be able to use the feature. IMO, terminfo needs a divorce from ncurses because the ncurses maintainer is getting hurt feelings and retarding progress for absolutely no good reason at all.

In fact, no reason is given at all for not wanting to have a standard 24bit color entry except essentially stating that ncurses indexed rendering method sucks too much for it to use 24bit color. The only real reason I can detect is the unsubstantiated claim that people were "harassing" Dickie -- So PERHAPS he has set out to punish everyone for his hurt feelings by ignoring the most widely used terminal emulators support 24bit color -- Note: it doesn't matter if many use vte or not -- what should matter is the number of users affected by the lack of supporting a standardized 24bit color mode. This should be trivial to add to terminfo, and people actually patch their terminfo databases to add the color support for emacs.

Entries existing in the terminfo database should not be predicated on whether ncurses rendering API can use them or not (the terminfo API is fully capable). There are many existing entries in terminfo which ncurses does not use. We all shouldn't have to suffer just because ncurses duplicated the bad design of curses "color pairs" which results in O(n^2) complexity (vs saner methods of compositing individual FG / BG attributes on the fly, i.e., O(n) attribute complexity). In fact, the very purpose of an API that insulates users from the internal representation or display protocol should be to allow migration to better internal code without breaking applications.

Here is a hack using only escape codes to print 24bit rainbow color on Linux "kernel" terminal The ones on [Ctrl+Alt+F1 to F7]. It will work if your kernel terminal is emulating 16 colors atop a frame buffer that actually supports 24bit color (this is the case on most desktop systems since Linux kernel integrated a framebuffer years ago [back on 2.6.something] ).

An application using this method to display 24bit color just has to keep the display from scrolling since the linux terminal will redraw the screen using its current 16 color palette. This is very hacky but demonstrates that even the lowly Linux terminal COULD easily be made to support 24bit color (just track which colors go where, and Bob's your uncle). Perhaps skilled programmers would submit patches to upgrade the projects if maintainers weren't so crusty, arrogant and dismissive of new features they don't personally desire.

TL;DR: A maintainer of terminfo who is not inept or wilfully ignorant should be jumping at the chance to get out ahead of a feature and get it standardized before it becomes a myriad of incompatible formats. Unfortunately ncurses is apparently not maintained by someone who fits this description. Since Dickie is against adopting standardization of new widely available features it seems he does not want to maintain the ncurses / terminfo project(s). He should just resign if that's the case.

kind regards,
random demoscener

p.s. I would like to add that "TRUECOLOR" is a registered trademark of Microsoft and therefore is a string that should / would be avoided by any knowledgeable FLOSS developer on this planet.

Owner

XVilka commented Jul 26, 2017 edited

@Tenzer @sebastiencs - thanks, added them in list.

Since I can't edit Wikipedia from China or through VPN, can someone please update the corresponding Wikipedia article https://en.wikipedia.org/wiki/ANSI_escape_code to reflect the actual support rate of the 24bit RGB colours among terminal emulators and console programs? Thank you.

joshklod commented Aug 5, 2017

hterm has true color support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment