Emacs24bit.md

24 bit color in emacs, under Blink.sh

What's the problem?

The normal way to enable 24 bit color on a recent emacs (27.1) and a recent OS distribution (one which already has the terminal type definition xterm-direct), is just to set TERM=xterm-direct before running emacs.

But this does not work with blink.sh, probably because of some incompatibility in its underlying library, hterm.

What's the solution

The solution is to add a custom terminal type definition, but not the one recommended by emacs, which also will not work with blink.

Instead, run the script at this gist to add a new terminal type definition xterm-emacs-leg and set TERM=xterm-emacs-leg before running emacs. This will give you working 24 bit color in emacs, under the blink shell. I've tested this for emacs 27.1. I believe it should work as far back as emacs 26.1.

Checking if emacs is using 24 bit color

In emacs you can see how many colors emacs thinks it can display by typing the following key sequence: ESC x eval-expression ENTER (tty-display-color-cells) ENTER.

This will print out a number in the mini buffer at the bottom of the screen: 0 if you’re not running in a tty, 256 for 8 bit color, and 16,000,000+ for 24 bit color.

In emacs you can render the displayable colors by typing the following key sequence: ESC x list-display-colors ENTER.

This opens a new buffer named *colors* which lists the color names rendered in the colors themselves.

If emacs think it can display many colors but then you only see black and white, then there’s an incompatibility between your terminal application and the term entry you are using.

Support for emacs 24 bit color in iOS terminal emulators, with xterm-direct

I tested color support with Ubuntu 20.04 and emacs 27.1 using the xterm-direct terminal type definition built into the OS. It works everywhere except blink.

Termius 4.7.6:

• Reported: 16 M colors
• Rendered in color: yes

Secure ShellFish 2021.12:

• Reported: 16 M colors
• Rendered in color: yes

Blink 13.90.3.277:

• Reported: 16 M colors
• Rendered in color: NO

iTerm2 (latest as of 20210406):

• Reported: 16 M colors
• Rendered in color: yes

In other words, for some rason Blink seems to have an unusual problem that when the term is set to xterm-direct, emacs thinks it can render 24 bit color but the Blink app does not show the colors.

Analysis

Discussion on the Blink discord led to the following tentative conclusions:

1. This is likely happening because Blink relies on the underlying HTerm library, unlike other terminal apps considered.
2. If instead of the term xterm-direct, you use the term xcode-emacs-leg defined by my other gist on this topic, emacs renders 24 bit colors and they appear in Blink.
3. This has to do with support for two competing formats for the codes used to communicate colors (roughly, the official colon-based standard, and the widely used legacy format based on semicolons). The xcode-emacs-leg uses the legacy format, and works. The xterm-emacs termianl type (as recommended by emacs docs) and xterm-direct (as shipped with Ubuntu) all use the official syntax, which doesn't quite work.
4. I went poking around in the tmux and mosh source code, and they are still using the legacy format as well. So perhaps terminal emulators have much better support for it.

yeah, this is much better than using xterm-direct!
I've written about this before, let me find it...
but basically, xterm-direct (and xterm-direct256, which supports 256 color mode too) use a system which
uses the same command for palette colors and 24bit colors
so the first few 24bit colors (shades of blue) don't work.
you can actually see this in emacs, because some blue things end up the wrong color
AND there is a typo in the definition of xterm-direct256, so it doesn't even work to begin with!

meanwhile, emacs's nonstandard setf24/setb24 commands avoid this problem entirely
so I would highly recommend that these be adopted as official, because they are so much better.

Anyway, the one change I would recommend is,
you are supposed to put use=... at the end of the terminfo file, because, actually, earlier definitions override later ones
(but in this case it won't make a difference since xterm-256color won't define setf24/setb24)
also: I put #!/usr/bin/tic -c as the first line of my terminfo file, so you can just execute it and it'll install itself automatically.

here is my terminfo file, by the way:
https://github.com/12Me21/config-share/blob/master/xterm-24bit.term

@12Me21 Thanks so much for this comment! These are great tips and I'll experiment with them. I'm curious how you can tell when it's safe to add more use declarations like the use=xterm+tmux in your gist?

Basically, I barely understand how terminals work. I still really have no idea why infocmp doesn't report the setf24 operation once it's defined, why it doesn't show the 'RGB' capability in xterm-direct, or (the original issue) why the Blink.sh shell doesn't work with the ISO-style format codes.

here's the builtin set of terminfo definitions:
(warning: very large page)
https://invisible-island.net/ncurses/terminfo.src.html
ones with + in the name are specific sets of features, which you can use to build your own definitions
so, xterm+sl is the capabilities for setting the "Status Line" (in modern terminals, that's the window title)
the general structure looks like:

name|description,
  <define individual capabilities>
  <include any number of "+" definitions (i.e xterm+sl)>
  <include ONE base definition (i.e. xterm-256color)>

setf24 and RGB are considered "user defined" capabilities, so you need to pass the -x flag when compiling with tic and when decompiling with infocmp

the reason some terminals use semicolons instead of colons is because someone made a mistake implementing them (they didn't read carefully, and assumed that it used semicolons (because everything else did))
most people who create terminals also "barely understand how terminals work" lol, and just copy existing implementations, so the mistake has spread and persisted

Oh, thanks!

FYI @algal list-display-colors should be list-colors-display. Also, others have observed that the output only shows 556 colors when 24-bit colors are supported, even if (tty-display-color-cells) displays 16M.