Asked  7 Months ago    Answers:  5   Viewed   78 times

I am studying for an exam and I am confused as to how canonical vs. non-canonical input/output works in Unix (e.g., curses). I understand that there is a buffer to which "line disciplines" are applied for canonical input. Does this mean that the buffer is bypassed for non-canonical input, or does it simply mean that no line disciplines are applied? How does this process differ for input and output operations?

In the curses programs I have worked with that demonstrate canonical input, the input typed by a user is automatically entered either after a certain number of characters have been typed or a certain amount of time has passed. Are either of these things considered "line disciplines" or is this something else entirely?



For canonical input — think shell; actually, think good old-fashioned Bourne shell, since Bash and relatives have command-line editing. You type a line of input; if you make a mistake, you use the erase character (default is Backspace, usually; sometimes Delete) to erase the previous character. If you mess up completely, you can cancel the whole line with the line kill character (not completely standardized, often Control-X). On some systems, you get a word erase with Control-W. All this is canonical input. The entire line is gathered and edited up until the end of line character — Return — is pressed. Thereupon, the whole line is made available to waiting programs. Depending on the read() system calls that are outstanding, the whole line will be available to be read (by one or more calls to read()).

For non-canonical input — think vi or vim or whatever — you press a character, and it is immediately available to the program. You aren't held up until you hit return. The system does no editing of the characters; they are made available to the program as soon as they are typed. It is up to the program to interpret things appropriately. Now, vim does do a number of things that look a bit like canonical input. For example, backspace moves backwards, and in input mode erases what was there. But that's because vim chooses to make it behave like that.

Canonical and non-canonical output is a much less serious business. There are a few bits and pieces of difference, related to things like whether to echo carriage-return before line-feed, and whether to do delays (not necessary with electronics; important in the days when the output device might have been a 110-baud teletype). It can also do things like handle case-insensitive output devices — teletypes, again. Lower-case letters are output in caps, and upper-case letters as backslash and caps.

It used to be that if you typed all upper-case letters to the login prompt, then the login program would automatically convert to the mode where all caps were output with a backslash in front of each actual capital. I suspect that this is no longer done on electronic terminals.

In a comment, TitaniumDecoy asked:

So with non-canonical input, is the input buffer bypassed completely? Also, where do line disciplines come in?

With non-canonical input, the input buffer is still used; if there is no program with a read() call waiting for input from the terminal, the characters are held in the input buffer. What doesn't happen is any editing of the input buffer.

Line disciplines are things like the set of manipulations that the input editing does. So, one aspect of the line discipline is that the erase character erases a prior character in canonical input mode. If you have icase (input case-mapping) set, then upper-case characters are mapped to lower-case unless preceded by a backslash; that is a line discipline, I believe, or an aspect of a line discipline.

I forgot to mention that EOF processing (Control-D) is handled in canonical mode; it actually means 'make the accumulated input available to read()'; if there is no accumulated input (if you type Control-D at the beginning of a line), then the read() will return zero bytes, which is then interpreted as EOF by programs. Of course, you can merrily type more characters on the keyboard after that, and programs that ignore EOF (or run in non-canonical mode) will be quite happy.

Of course, in canonical mode, the characters typed at the keyboard are normally echoed to the screen; you can control whether that echoing occurs. However, this is somewhat tangential to canonical input; the normal editing occurs even when echo is off.

Similarly, the interrupt and quit signals are artefacts of canonical mode processing. So too are the job control signals such as Control-Z to suspend the current process and return to the shell. Likewise, flow control (Control-S, Control-Q to stop and start output) is provided by canonical mode.

Chapter 4 of Rochkind's Advanced Unix Programming, 2nd Edn covers terminal I/O and gives much of this information — and a whole lot more. Other UNIX programming books (at least, the good ones) will also cover it.

Tuesday, June 1, 2021
answered 7 Months ago

I would do this by creating separate a thread which calls normal blocking IO functions and pass it a callback function which it would call when it got input. Are you sure you need to do what you said you want to do?

As for outputting information at the same time, what would happen if the user was in the middle of typing some input and you printed something?

Wednesday, June 2, 2021
answered 6 Months ago

Then you're out of luck i'm afraid. There's no real cross-platform version or port of curses/ncurses, there is a "dialogue" port which works, but it's limited in capabilities.

Your best bet is to run CygWin or MinGW32, both are, in "loose terms", a Linux system+terminal emulator which has much of the binaries you need. They can run native Linux/Unix binaries inside the terminal and access your "host" system files at any time, so it's like patching Windows with a kick-ass terminal with all your goodies from the Linux world. You'll still need some basic knowledge of Linux and how the commands etc work, but you'll figure it out.

Screenshot of MinGW and CygWin

Here's a Pyglet GUI example:

import pyglet
from import *

class main (pyglet.window.Window):
    def __init__ (self):
        super(main, self).__init__(800, 600, fullscreen = False)
        self.button_texture = pyglet.image.load('button.png')
        self.button = pyglet.sprite.Sprite(self.button_texture)

        ## --- If you'd like to play sounds:
        #self.sound ='music.mp3')

        self.alive = 1

    def on_draw(self):

    def on_close(self):
        self.alive = 0

    def on_mouse_press(self, x, y, button, modifiers):
        if x > self.button.x and x < (self.button.x + self.button_texture.width):
            if y > self.button.y and y < (self.button.y + self.button_texture.height):
                self.alive = 0

    def on_key_press(self, symbol, modifiers):
        if symbol == 65307: # [ESC]
            self.alive = 0

    def render(self):

    def run(self):
        while self.alive == 1:

            # -----------> This is key <----------
            # This is what replaces
            # but is required for the GUI to not freeze
            event = self.dispatch_events()

x = main()

Here's the output of that code:

enter image description here

Wednesday, June 9, 2021
answered 6 Months ago

I'm guessing you are using a thread to update the upper display while window.getstr() runs in the main thread?

If so, the problem is that the curses terminal state is a shared resource that can't be updated from two different threads simultaneously. You need to put the terminal into non-blocking mode, use window.getch() to build up an input string and have your thread pass the upper window update task to the main thread.

Saturday, November 13, 2021
answered 2 Weeks ago

I don't know of a way to solve this with apache rules alone as it would require some sort of regex matching and reusing the result of the match in a directive, which isn't possible.

However, it's pretty simple if you introduce a php script into the mix:

RewriteEngine On
RewriteCond %{REQUEST_URI} .(jpg|png|pdf)$
RewriteRule (.*) /canonical-header.php?path=$1

Note that this would send requests for all jpg, png and pdf files to the script regardless of the folder name. If you want to include only specific folders, you could add another RewriteCond to accomplish that.

Now the canonical-header.php script:


// Checking for the presence of the path variable in the query string allows us to easily 404 any requests that
// come directly to this script, just to be safe.
if (!empty($_GET['path'])) {
    // Be sure to add any new file types you want to handle here so the correct content-type header will be sent.
    $mimeTypes = array(
        'pdf' => 'application/pdf',
        'jpg' => 'image/jpeg',
        'png' => 'image/png',

    $path         = filter_input(INPUT_GET, 'path', FILTER_SANITIZE_URL);
    $file         = realpath($path);
    $extension    = pathinfo($path, PATHINFO_EXTENSION);
    $canonicalUrl = 'http://' . $_SERVER['HTTP_HOST'] . '/' . dirname($path);
    $type         = $mimeTypes[$extension];

    // Verify that the file exists and is readable, or send 404
    if (is_readable($file)) {
        header('Content-Type: ' . $type);
        header('Link <' . $canonicalUrl . '>; rel="canonical"');
    } else {
        header('HTTP/1.0 404 Not Found');
        echo "File not found";
} else {
    header('HTTP/1.0 404 Not Found');
    echo "File not found";

Please consider this code untested and check that it works as expected across browsers before releasing it to production.

Wednesday, November 24, 2021
answered 6 Days ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :