when TEKEN_CONS25 is configured. Fix this by adding a function to
set the flag that enables the fix and always calling this function
for syscons.
Expand the man page for teken_set_cons25(). This function is not
very useful since it can only set but not clear 1 flag. In practice,
it is only used when TEKEN_CONS25 is configured and all that does is
choose the the default emulation for syscons at compile time.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
There was already a per-vty defaults field, but it was useless since it was
only initialized when propagating the global settings and thus no different
from the current global settings and not per-vty. The global defaults field
was also invariant after boot time, but not quite so useless.
Fix this by adding a second selection bit the the control flags of the
relevant ioctl(). vidcontrol doesn't support this yet. Setting either
default propagates the change to the current setting for the same level
and then to all lower levels.
Improve the 3-way escape sequence used by termcap to control the cursor.
The "normal" (ve) case has always used reset, so the user could set
it to anything, but since the reset is to a global value this is not
very useful, especially since the "very visible" (vs) case doesn't
reset but inconsistently forces to a blinking block. Change vs to
first reset and then XOR the blinking bit so that it is predictably
different from ve.
attribute field is curs_attr. The base field holds user data translated
in a reversible way and is needed because current field holds this in
an irreversible way for efficiency.
Factor out some common code for the reversible translation. This is
slightly simpler now, and much easier to expand.
Translate the magic flags value -1 to a single control flag internally
up front so other flags can be trusted later. This can be used for the
relevant ioctl() too.
Remove CONS_CURSOR_FLAGS which contained all the control flags. It was
unused and not useful. After adding more flags, there will be tests on
a couple at a time but never on them all. This API should have used this
to disallow unknown flags.
was aliased to a vt sequence, causing and fixing various bugs.
For syscons, this restores support for arg 2 which sets blinking block
too forcefully, and restores bugs for arg 0 and 1. Arg 2 is used for
vs in the cons25 entry in termcap, but I've never noticed an application
that uses this. The bugs involve replacing local settings by global
ones and need better handling of defaults to fix.
For vt, this requires moving the aliasing code from teken to vt where
it belongs. This sequences is very important for cons25 compatibility
in vt since it is used by the cons25 termcap entries for ve, vi and
vs. vt can't properly support vs for either cons25 or xterm since it
doesn't support blinking. For xterm, the termcap entry for vs asks
for something different using 12;25h instead of 25h.
Rename C25CURS for this to C25LCT and change its description to be closer
to echoing the old comment about it. CURS is too generic.
Fix missing syscons escape sequence for setting the global cursor shape
(and type). Only support this in syscons since vt can't emulate anything
in it.
It should toggle between 2 states, but it used a cut-down version of
support for a related 3-state syscons escape sequence and inherited
bugs from that. The usual misbehaviour was that hiding and showing
the cursor reset it to a global default.
Support for the 3-state sequence remains broken by aliasing to the 2-state
sequence. This works better but incompatibly for the 2 cases that it
supports.
scteken_init(). Move the internals of scteken_sync() into a local
function to help do this.
scteken_init() reset or adjusted the default attribute and screen
position at least 3 and 5 times, respectively. Warm init shouldn't
do any more than reset the "input" state.
(scterm-sc.c (which still works after minor editing), only resets
the escape state and the saved cursor position, and then does a
nearly-null sync of the current color.)
This mainly broke mode changes, and was most noticeable when the
background color is not teken's default (usually black). Then the
screen gets cleared in the wrong color. vidcontrol restores the
default normal attribute and tries to restore the default reverse
attribute. vidcontrol doesn't clear the screen again after restoring
the attribute(s), and it is too late to do it there without flicker.
Now the default normal attribute is restored before the change affects
the rendering.
When the foreground color is not teken's default, clearing with the
wrong attributes gave strange cursor colors for some cursor types.
The default reverse attribute is not restored since it is unsupported.
2/3 of the clobbering was from 2 resetting window resizing calls. The
second one is needed to restore the size, but must not reset. Window
resizing also sanitizes the cursor position, and after the main reset
resets the window size, the cursor row would often be adjusted from
24 to 23 if it were not already reset to 0. scteken_sync() is good
for restoring the window size and the cursor position in the correct
order, but was unusable at init time since scp->ts is not always
initialized then. Adjust to use its internals.
I didn't notice any problems from the cursor reset. The cursor should
be reset, and a previous fix was to reset it consistently a little
later.
Doing nothing for warm init works almost as well, if not better. It
is not very useful to reset the escape state for mode changes, since
the reset is especially likely to be null then. The escape state is
most likely to be non-initial and corrupted by its most normal uses
-- sloppy non-atomic output where a context switch or just mixing
stdout with stderr splits up escape sequences.
position. Especially the screen size, and potentially everything except
the input state and attributes. Do this by changing the cursor position
setting method to a general syncing method.
Use proper constructors instead of copying to create kernel terminal
contexts. We really want clones and not new instances, but there is
no method for cloning and there is nothing in the active instance that
needs to be cloned exactly.
Add proper destructors for kernel terminal contexts. I doubt that the
destructor code has every been reached, but if it was then it leaked the
memory of the clones.
Remove freeing of statically allocated memory for the non-kernel terminal
context for the same terminal as the kernel. This is in the nearly
unreachable code. This used to not happen because delicate context
swapping made the user context use the dynamic memory and kernel
context the static memory. I didn't restore this swapping since it
would have been unnatural to have all kernel contexts except 1 dynamic.
The constructor for terminal context has bad layering for reasons
related to the bug. It has to return static memory early before
malloc() works. Callers also can't allocate memory until after the
first constructor selects an emulator and tells upper layers the size
of its context. After that, the cloning hack required the cloning
code to allocate the memory, but for all other constructors it would
be better for the terminal layer to allocate and deallocate the
memory in all cases.
Zero the memory when allocating terminal contexts dynamically.
it to a separate state for each CPU.
Terminal "input" is user or kernel output. Its state includes the current
parser state for escape sequences and multi-byte characters, and some
results of previous parsing (mainly attributes), and in teken the cursor
position, but not completed output. This state must be switched for kernel
output since the kernel can preempt anything, including itself, and this
must not affect the preempted state more than necessary. Since vty0 is
shared, it is necessary to affect the frame buffer and cursor position and
history, but escape sequences must not be affected and attributes for
further output must not be affected.
This used to work. The syscons terminal state contained mainly the parser
state for escape sequences and attributes, but not the cursor position,
and was switched. This was first broken by SMP and/or preemptive kernels.
Then there should really be a separate state for each thread, and one more
for ddb, or locking to prevent preemption. Serialization of printf() helps.
But it is arcane that full syscons escape sequences mostly work in kernel
printf(), and I have never seen them used except by me to test this fix.
They worked perfectly except for the races, since "input" from the kernel
was not special in any way.
This was broken to use teken. The general switch was removed, and the
kernel normal attribute was switched specially. The kernel reverse
attribute (config option SC_CONS_REVERSE_ATTR) became unused, and is
still unusable because teken doesn't support default reverse attributes
(it used to only be used via the ANSI escape sequence to set reverse
video).
The only new difficulty for using teken seems to be that the cursor
position is in the "input" state, so it must be updated in the active
input state for each half of the switch. Do this to complete the
restoration.
The per-CPU state is mainly to make per-CPU coloring work cleanly, at
a cost of some space. Each CPU gets its own full set of attribute
(not just the current attribute) maintained in the usual way. This
also reduces races from unserialized printf()s. However, this gives
races for serialized printf()s that otherwise have none. Nothing
prevents the CPU doing the a printf() changing in the middle of an
escape sequence.
for vt. Restore syscons' rendering of background (bg) brightness as
foreground (fg) blinking and vice versa, and add rendering of blinking
as background brightness to vt.
Bright/saturated is conflated with light/white in the implementation
and in this description.
Bright colors were broken in all cases, but appeared to work in the
only case shown by "vidcontrol show". A boldness hack was applied
only in 1 layering-violation place (for some syscons sequences) where
it made some cases seem to work but was undone by clearing bold using
ANSI sequences, and more seriously was not undone when setting
ANSI/xterm dark colors so left them bright. Move this hack to drivers.
The boldness hack is only for fg brightness. Restore/add a similar hack
for bg brightness rendered as fg blinking and vice versa. This works
even better for vt, since vt changes the default text mode to give the
more useful bg brightness instead of fg blinking.
The brightness bit in colors was unnecessarily removed by the boldness
hack. In other cases, it was lost later by teken_256to8(). Use
teken_256to16() to not lose it. teken_256to8() was intended to be
used for bg colors to allow finer or bg-specific control for the more
difficult reduction to 8; however, since 16 bg colors actually work
on VGA except in syscons text mode and the conversion isn't subtle
enough to significantly in that mode, teken_256to8() is not used now.
There are still bugs, especially in vidcontrol, if bright/blinking
background colors are set.
Restore XOR logic for bold/bright fg in syscons (don't change OR
logic for vt). Remove broken ifdef on FG_UNDERLINE and its wrong
or missing bit and restore the correct hard-coded bit. FG_UNDERLINE
is only for mono mode which is not really supported.
Restore XOR logic for blinking/bright bg in syscons (in vt, add
OR logic and render as bright bg). Remove related broken ifdef
on BG_BLINKING and its missing bit and restore the correct
hard-coded bit. The same bit means blinking or bright bg depending
on the mode, and we want to ignore the difference everywhere.
Simplify conversions of attributes in syscons. Don't pretend to
support bold fonts. Don't support unusual encodings of brightness.
It is as good as possible to map 16 VGA colors to 16 xterm-16
colors. E.g., VGA brown -> xterm-16 Olive will be converted back
to VGA brown, so we don't need to convert to xterm-256 Brown. Teken
cons25 compatibility code already does the same, and duplicates some
small tables. This is mostly for the sc -> te direction. The other
direction uses teken_256to16() which is too generic.
user default normal attribute to the current attribute).
This change only fixes a logic error. scterm_clear() used to be
used for terminal reset, but teken uses a general fill function for
that, leaving scterm_clear() only used for initialization and mode
change, when using the user default attribute is correct. It is not
really a terminal function, but needs to sync its changes with the
terminal layer. Syncing of the attribute is currently broken for
terminal reset, but works for initialization and mode change.
some cases of initialization and resetting of the teken cursor position.
(This bad name is consistent with others, but it is too easy to confuse
with scteken_cursor() which goes in the opposite direction.)
The following cases were broken:
- for booting without a syscons console, the teken and sc positions for
ttyv0 were (0, 0), but are supposed to be somewhere in the middle of
the screen (after carefully preserved BIOS and loader messages) (at
least if there is no mode switch that loses the messages).
- after mode switches, the screen is cleared and the cursor is supposed to
be moved to (0, 0), but it was only moved there for sc.
The following case was hacked to work:
- for booting with a syscons console, it was arranged that scteken_init()
for the console could see a nonzero cursor position and adjust, although
this broke the sc seeing it in the non-console case above.
looked like it might handle reverse attributes, but it actually handles
conversion of attributes in the direction indicated by the new name.
Reverse attributes are just broken.
Rename scteken_attr() to scteken_te_to_sc_attr(). scteken_attr() looked
like it might give teken attributes, but it actually gives sc attributes.
Change scteken_te_to_sc_attr() to return int instead of unsigned int.
u_char would be enough, and it promotes to int, and syscons uses int
or u_short for its attributes everywhere else (u_short holds a shifted
form and it promotes to int too).
by the CPU number.
This was originally for debugging near-deadlock conditions where
multiple CPUs either deadlock or scramble each other's output trying
to report the problem, but I found it interesting and sometimes
useful for ordinary kernel messages. Ordinary kernel messages
shouldn't be interleaved, but if they are then the colorization
makes them readable even if the interleaving is for every character
(provided the CPU printing each message doesn't change).
The default colors are 8-15 starting at 15 (bright white on black)
for CPU 0 and repeating every 8 CPUs. This works best with 8 CPUs.
Non-bright colors and nonzero background colors need special
configuration to avoid unreadable and ugly combinations so are not
configured by default. The next bright color after 15 is 8 (bright
black = dark gray) is not very readable but is the only other color
used with 2 CPUs. After that the next bright color is 9 (bright
blue) which is not much brighter than bright black, but is used with
3+ CPUs. Other bright colors are brighter.
Colorization is configured by default so that it gets tested. It can
only be turned off by configuring SC_KERNEL_CONS_ATTR to anything other
than FG_WHITE. After booting, all colors can be changed using the
syscons.kattr sysctl. This is a SYSCTL_OPAQUE, and no utility is
provided to change it (sysctl only displays it).
The default colors work in all VGA modes that I could test. In 2-color
graphics modes, all 8 bright colors are displayed as bright white, so
the colorization has no effect, but anything with a nonzero background
gives white on white unless the foreground is zero. I don't have an
mono or VGA grayscale hardware to test on. Support for mono mode seems
to have never worked right in syscons (I think bright white gives white
underline with either bold or bright), but VGA grayscale should work
better than 2-color graphics.
Introduce a new formatting bit (TF_CJK_RIGHT) that is set when putting a
cell that is the right part of a CJK fullwidth character. This will
allow drivers like vt(9) to support fullwidth characters properly.
emaste@ has a patch to extend vt(9)'s font handling to increase the
number of Unicode -> glyph maps from 2 ({normal,bold)} to 4
({normal,bold} x {left,right}). This will need to use this formatting
bit to determine whether to draw the left or right glyph.
Reviewed by: emaste
on PowerPC support. This was clearly not something syscons was
designed to do (very specific assumptions about the nature of VGA
consoles on PCs), but fortunately others have long since blazed
the way on making it work regardless of that.
Sponsored by: DARPA, AFRL
(mostly with Catalan characters in mind, but it probably
benefits other languages).
The new mappings are as follows:
▮ -> █
ÀÈÍÏÓÒÚ -> AEIIOOU
ŀ / Ŀ -> l / L
Reviewed by: ed
Approved by: kib (mentor)
Right now syscons(4) uses a cons25-style terminal emulator. The
disadvantages of that are:
- Little compatibility with embedded devices with serial interfaces.
- Bad bandwidth efficiency, mainly because of the lack of scrolling
regions.
- A very hard transition path to support for modern character sets like
UTF-8.
Our terminal emulation library, libteken, has been supporting
xterm-style terminal emulation for months, so flip the switch and make
everyone use an xterm-style console driver.
I still have to enable this on i386. Right now pc98 and i386 share the
same /etc/ttys file. I'm not going to switch pc98, because it uses its
own Kanji-capable cons25 emulator.
IMPORTANT: What to do if things go wrong (i.e. graphical artifacts):
- Run the application inside script(1), try to reduce the problem and
send me the log file.
- In the mean time, you can run `vidcontrol -T cons25' and `export
TERM=cons25' so you can run applications the same way you did before.
You can also build your kernel with `options TEKEN_CONS25' to make all
virtual terminals use the cons25 emulator by default.
Discussed on: current@
xterm and cons25 have some incompatibilities when it comes to escape
sequences for special keys, such as F1 to F12, home, end, etc. Add a new
te_fkeystr() that can be used to override the strings.
scterm-sck won't do anything with this, but scterm-teken will use
teken_get_sequences() to obtain the proper sequence.
Right now if applications want to use the mouse on the command line,
they use sysmouse(4) and install a signal handler in the kernel to
deliver signals when mouse events arrive. This conflicts with my plan to
change to TERM=xterm, so implement proper VT200-style mouse input.
Because mouse input is now streamed through the TTY, it means you can
now SSH to another system on the console and use the mouse there as
well. The disadvantage of the VT200 mouse protocol, is that it doesn't
seem to generate events when moving the cursor. Only when pressing and
releasing mouse buttons.
There are different protocols as well, but this one seems to be most
commonly supported.
Reported by: Paul B. Mahol <onemda gmail com>
Tested with: vim(1)
It is quite inconvenient that if an application for xterm uses 256 color
mode, text suddenly starts to blink (because of ;5; in the middle).
We'd better just implement 256 color mode and add a conversion routine
from 256 to 8 color mode, which doesn't seem to be too bad in practice.
Remapping colors is done quite simple. If one of the channels is most
actively represented, primary colors are used. If two channels are most
actively represented, secondary colors are used. If all three channels
are equal (gray), it picks between black and white.
Reported by: Paul B. Mahol <onemda gmail com>
For some vague reason, it may be possible that scp->cursor_pos exceeds
scp->ysize * scp->xsize. This means that teken_set_cursor() may get
called with an invalid position. Just ignore the old cursor position in
this case.
Reported by: Paul B. Mahol <onemda gmail com>
MFC after: 1 month
Right now libteken processes TF_REVERSE internally and returns the
toggled colors to the console driver. This isn't entirely correct. This
means that the bold flag is always processed by the foreground color,
while reversing should be done after the foreground color has been set
to a brighter version by the bold flag.
This is no problem with the syscons driver, because with VGA it only
supports 16 foreground and 8 background colors. My WIP console driver
reconfigures the graphics hardware to disable the blink functionality
and uses 16 foreground and 16 background colors. This means that this
driver will handle the TF_REVERSE flag a little different from what
syscons does right now.
I initially committed libteken to sys/dev/syscons/teken, but now that
I'm working on a console driver myself, I noticed this was not a good
decision. Move it to sys/teken to make it easier for other drivers to
use a terminal emulator.
Also list teken.c in sys/conf/files, instead of listing it in all the
files.arch files separately.
Characters between 0x07 and 0x0d are now also mapped, which means we can
display almost 256 different characters. Also remap certain types of
dashes and quotes, which means we can finally read our manual pages
without red question marks in them.
Submitted by: Christoph Mallon
Add a small Unicode-to-CP437 remapping table to at least demonstrate
that the terminal emulator is perfectly capable of handling UTF-8. This
will of course break if the user loads a different font map, but it at
least allows people to give it a try.
I can now see the box drawing in dialog(1) and the arrows in mutt(1)
correctly.
Because we only support a single argument to tf_param, use 16 bits for
the pitch and 16 bits for the duration. While there, make the argument
unsigned. There isn't a single param call that needs a signed integer.
Submitted by: danfe (modified)
After I imported libteken into the source tree, I noticed syscons didn't
store the cursor position inside the terminal emulator, but inside the
virtual terminal stat. This is not very useful, because when you
implement more complex forms of line wrapping, you need to keep track of
more state than just the cursor position.
Because the kernel messages didn't share the same terminal emulator as
ttyv0, this caused a lot of strange things, like kernel messages being
misplaced and a missing notification to resize the terminal emulator for
kernel messages never to be resized when using vidcontrol.
This patch just removes kernel_console_ts and adds a special parameter
to te_puts to determine whether messages should be printed using regular
colors or the ones for kernel messages.
Reported by: ache
Tested by: nyan, garga (older version)
Some time ago I started working on a library called libteken, which is
terminal emulator. It does not buffer any screen contents, but only
keeps terminal state, such as cursor position, attributes, etc. It
should implement all escape sequences that are implemented by the
cons25 terminal emulator, but also a fair amount of sequences that are
present in VT100 and xterm.
A lot of random notes, which could be of interest to users/developers:
- Even though I'm leaving the terminal type set to `cons25', users can
do experiments with placing `xterm-color' in /etc/ttys. Because we
only implement a subset of features of xterm, this may cause
artifacts. We should consider extending libteken, because in my
opinion xterm is the way to go. Some missing features:
- Keypad application mode (DECKPAM)
- Character sets (SCS)
- libteken is filled with a fair amount of assertions, but unfortunately
we cannot go into the debugger anymore if we fail them. I've done
development of this library almost entirely in userspace. In
sys/dev/syscons/teken there are two applications that can be helpful
when debugging the code:
- teken_demo: a terminal emulator that can be started from a regular
xterm that emulates a terminal using libteken. This application can
be very useful to debug any rendering issues.
- teken_stress: a stress testing application that emulates random
terminal output. libteken has literally survived multiple terabytes
of random input.
- libteken also includes support for UTF-8, but unfortunately our input
layer and font renderer don't support this. If users want to
experiment with UTF-8 support, they can enable `TEKEN_UTF8' in
teken.h. If you recompile your kernel or the teken_demo application,
you can hold some nice experiments.
- I've left PC98 the way it is right now. The PC98 platform has a custom
syscons renderer, which supports some form of localised input. Maybe
we should port PC98 to libteken by the time syscons supports UTF-8?
- I've removed the `dumb' terminal emulator. It has been broken for
years. It hasn't survived the `struct proc' -> `struct thread'
conversion.
- To prevent confusion among people that want to hack on libteken:
unlike syscons, the state machines that parse the escape sequences are
machine generated. This means that if you want to add new escape
sequences, you have to add an entry to the `sequences' file. This will
cause new entries to be added to `teken_state.h'.
- Any rendering artifacts that didn't occur prior to this commit are by
accident. They should be reported to me, so I can fix them.
Discussed on: current@, hackers@
Discussed with: philip (at 25C3)
