Skip to content

Instantly share code, notes, and snippets.

@aminophen
Created January 16, 2019 14:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aminophen/154cb16cfe989dd605b0983f3e3bd20b to your computer and use it in GitHub Desktop.
Save aminophen/154cb16cfe989dd605b0983f3e3bd20b to your computer and use it in GitHub Desktop.
Is this control sequence, or JP char, or non-JP char?
\makeatletter
% https://tex.stackexchange.com/questions/21466/test-if-token-is-a-control-sequence
\def\@ifismacro#1{%
\begingroup\escapechar=-1
\edef\x{\endgroup\def\noexpand\first{\string#1}}\x
\begingroup\escapechar=`\\
\edef\x{\endgroup\def\noexpand\second{\string#1}}\x
\ifnum\pdfstrcmp{\first}{\second}=\z@
\expandafter\@secondoftwo % no backslash in front
\else
\expandafter\@firstoftwo % backslash in front
\fi}
% (u)pTeX-specific: is a character token is JP or non-JP?
\ifx\enablecjktoken\@undefined
\def\@ifisjp#1{% pTeX version
\ifnum`#1<255\relax
\expandafter\@secondoftwo
\else
\expandafter\@firstoftwo
\fi}
\else
\def\@ifisjp#1{% upTeX version
\ifnum\kcatcode`#1=15\relax
\expandafter\@secondoftwo
\else
\expandafter\@firstoftwo
\fi}
\fi
%
\def\report#1{\@ifismacro{#1}{\typeout{CS}}
{\@ifisjp{#1}{\typeout{JP}}{\typeout{non-JP}}}}
\makeatother
\typeout{===== pTeX and upTeX common =====}
\report{A}% => non-JP
\report{漢}% => JP
\report{\relax}% => CS
\report{\"}% => CS
\let\pippo=a
\report{\pippo}% => CS
\ifx\enablecjktoken\undefined
\typeout{===== pTeX-specific behavior =====}
% pTeX: each character code range is given a fixed behavior
% of either JP or non-JP.
\report{°}% => JP (euc: "A1EB = 41451, sjis: "818B = 33163)
\report{^^b0}% => non-JP
\report{^^c0}% => non-JP
%
\else
\typeout{===== upTeX-specific behavior =====}
% upTeX: each character code range (except 0--127, which are always non-JP)
% can have both JP and non-JP behavior.
\report{°}% => JP (U+00B0)
\report{^^b0}% => JP ... Common class by Unicode script
\report{^^c0}% => non-JP ... Latin-1 class by Unicode script
\kcatcode"A0=15\relax % move Common class to non-JP
\report{^^b0}% => non-JP
%
\fi
\typeout{=====}
\stop
@aminophen
Copy link
Author

aminophen commented Jan 16, 2019

Related:

Note:

  • The argument of example macro \report should be only one token.
  • \catcode, \kcatcode and ` can be applied only to a character token. So, first test if the token is a control sequence or not.
  • \catcode is not allowed for JP characters. So, we have to test if the character token is JP char or non-JP char. Here we should note the different behavior of \kcatcode between pTeX and upTeX (the latter one allows flexible changing of
    JP / non-JP classification).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment