Token

A token can be of type:

• literal: the value is the character this token represents. This can be a combination of Unicode codepoints, for example for emojis.
• ^ and _: superscript and subscript commands.
• command: a command such as \sin
• { and }: begin and end group (use for arguments of commands and for grouping)

• #: parameter

• esc: start of a special command. Followed by commandliteral tokens.

• backslash: start of a special command. Followed by commandliteral tokens.
• commandliteral: a-zA-Z for special commands (esc sequence, etc...)
• placeholder: a placeholder value meant to be replaced by some actual value

• space: one or more space characters (including tab, etc...)

  See: TeX:289