tok_str (3)

Leading comments

	$NetBSD: editline.3,v 1.92 2016/05/22 23:54:20 christos Exp $

 Copyright (c) 1997-2014 The NetBSD Foundation, Inc.
 All rights reserved.

 This file was contributed to The NetBSD Foundation by Luke Mewburn.

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
 2. Redi...

(The comments found at the beginning of the groff file "man3/tok_str.3".)

NAME

editline el_init el_init_fd el_end el_reset el_gets el_wgets el_getc el_wgetc el_push el_wpush el_parse el_wparse el_set el_wset el_get el_wget el_source el_resize el_cursor el_line el_wline el_insertstr el_winsertstr el_deletestr el_wdeletestr history_init history_winit history_end history_wend history history_w tok_init tok_winit tok_end tok_wend tok_reset tok_wreset tok_line tok_wline tok_str tok_wstr - line editor, history and tokenization functions

LIBRARY

Lb libedit

SYNOPSIS

In histedit.h Ft EditLine * Fn el_init const char *prog FILE *fin FILE *fout FILE *ferr Ft EditLine * Fn el_init_fd const char *prog FILE *fin FILE *fout FILE *ferr int fdin int fdout int fderr Ft void Fn el_end EditLine *e Ft void Fn el_reset EditLine *e Ft const char * Fn el_gets EditLine *e int *count Ft const wchar_t * Fn el_wgets EditLine *e int *count Ft int Fn el_getc EditLine *e char *ch Ft int Fn el_wgetc EditLine *e wchar_t *wc Ft void Fn el_push EditLine *e const char *mbs Ft void Fn el_wpush EditLine *e const wchar_t *wcs Ft int Fn el_parse EditLine *e int argc const char *argv[] Ft int Fn el_wparse EditLine *e int argc const wchar_t *argv[] Ft int Fn el_set EditLine *e int op ... Ft int Fn el_wset EditLine *e int op ... Ft int Fn el_get EditLine *e int op ... Ft int Fn el_wget EditLine *e int op ... Ft int Fn el_source EditLine *e const char *file Ft void Fn el_resize EditLine *e Ft int Fn el_cursor EditLine *e int count Ft const LineInfo * Fn el_line EditLine *e Ft const LineInfoW * Fn el_wline EditLine *e Ft int Fn el_insertstr EditLine *e const char *str Ft int Fn el_winsertstr EditLine *e const wchar_t *str Ft void Fn el_deletestr EditLine *e int count Ft void Fn el_wdeletestr EditLine *e int count Ft History * Fn history_init void Ft HistoryW * Fn history_winit void Ft void Fn history_end History *h Ft void Fn history_wend HistoryW *h Ft int Fn history History *h HistEvent *ev int op ... Ft int Fn history_w HistoryW *h HistEventW *ev int op ... Ft Tokenizer * Fn tok_init const char *IFS Ft TokenizerW * Fn tok_winit const wchar_t *IFS Ft void Fn tok_end Tokenizer *t Ft void Fn tok_wend TokenizerW *t Ft void Fn tok_reset Tokenizer *t Ft void Fn tok_wreset TokenizerW *t Ft int Fn tok_line Tokenizer *t const LineInfo *li int *argc const char **argv[] int *cursorc int *cursoro Ft int Fn tok_wline TokenizerW *t const LineInfoW *li int *argc const wchar_t **argv[] int *cursorc int *cursoro Ft int Fn tok_str Tokenizer *t const char *str int *argc const char **argv[] Ft int Fn tok_wstr TokenizerW *t const wchar_t *str int *argc const wchar_t **argv[]

DESCRIPTION

The library provides generic line editing, history and tokenization functions, similar to those found in sh(1).

These functions are available in the libedit library (which needs the libtermcap library). Programs should be linked with -ledit ltermcap

The library respects the LC_CTYPE locale set by the application program and never uses setlocale(3) to change the locale. The only locales supported are UTF-8 and the default C or POSIX locale. If any other locale is set, behaviour is undefined.

LINE EDITING FUNCTIONS

The line editing functions use a common data structure, Fa EditLine , which is created by Fn el_init or Fn el_init_fd and freed by Fn el_end .

The wide-character functions behave the same way as their narrow counterparts.

The following functions are available:

Fn el_init

Initialize the line editor, and return a data structure to be used by all other line editing functions, or NULL on failure. Fa prog is the name of the invoking program, used when reading the editrc(5) file to determine which settings to use. Fa fin , Fa fout and Fa ferr are the input, output, and error streams (respectively) to use. In this documentation, references to ``the tty'' are actually to this input/output stream combination.

Fn el_init_fd

Like Fn el_init but allows specifying file descriptors for the stdio(3) corresponding streams, in case those were created with funopen(3).

Fn el_end

Clean up and finish with Fa e , assumed to have been created with Fn el_init or Fn el_init_fd .

Fn el_reset

Reset the tty and the parser. This should be called after an error which may have upset the tty's state.

Fn el_gets

Read a line from the tty. Fa count is modified to contain the number of characters read. Returns the line read if successful, or NULL if no characters were read or if an error occurred. If an error occurred, Fa count is set to -1 and errno contains the error code that caused it. The return value may not remain valid across calls to Fn el_gets and must be copied if the data is to be retained.

Fn el_wgetc

Read a wide character from the tty, respecting the current locale, or from the input queue described in editline(7) if that is not empty, and store it in Fa wc . If an invalid or incomplete character is found, it is discarded, errno is set to Er EILSEQ , and the next character is read and stored in Fa wc . Returns 1 if a valid character was read, 0 on end of file, or -1 on read(2) failure. In the latter case, errno is set to indicate the error.

Fn el_getc

Read a wide character as described for Fn el_wgetc and return 0 on end of file or -1 on failure. If the wide character can be represented as a single-byte character, convert it with wctob(3), store the result in Fa ch , and return 1; otherwise, set errno to Er ERANGE and return -1. In the C or POSIX locale, this simply reads a byte, but for any other locale, including UTF-8, this is rarely useful.

Fn el_wpush

Push the wide character string Fa wcs back onto the input queue described in editline(7). If the queue overflows, for example due to a recursive macro, or if an error occurs, for example because Fa wcs is NULL or memory allocation fails, the function beeps at the user, but does not report the problem to the caller.

Fn el_push

Use the current locale to convert the multibyte string Fa mbs to a wide character string, and pass the result to Fn el_wpush .

Fn el_parse

Parses the Fa argv array (which is Fa argc elements in size) to execute builtin commands. If the command is prefixed with ``prog'' then Fn el_parse will only execute the command if ``prog'' matches the Fa prog argument supplied to Fn el_init . The return value is -1 if the command is unknown, 0 if there was no error or ``prog'' didn't match, or 1 if the command returned an error. Refer to editrc(5) for more information.

Fn el_set

Set parameters. Fa op determines which parameter to set, and each operation has its own parameter list. Returns 0 on success, -1 on failure.

The following values for Fa op are supported, along with the required argument list:

EL_PROMPT , Fa char *(*f)(EditLine *)

Define prompt printing function as Fa f , which is to return a string that contains the prompt.

EL_PROMPT_ESC , Fa char *(*f)(EditLine *) , Fa char c

Same as EL_PROMPT but the Fa c argument indicates the start/stop literal prompt character.

If a start/stop literal character is found in the prompt, the character itself is not printed, but characters after it are printed directly to the terminal without affecting the state of the current line. A subsequent second start/stop literal character ends this behavior. This is typically used to embed literal escape sequences that change the color/style of the terminal in the prompt. 0 unsets it.

EL_REFRESH

Re-display the current line on the next terminal line.

EL_RPROMPT , Fa char *(*f)(EditLine *)

Define right side prompt printing function as Fa f , which is to return a string that contains the prompt.

EL_RPROMPT_ESC , Fa char *(*f)(EditLine *) , Fa char c

Define the right prompt printing function but with a literal escape character.

EL_TERMINAL , Fa const char *type

Define terminal type of the tty to be Fa type , or to TERM if Fa type is NULL

EL_EDITOR , Fa const char *mode

Set editing mode to Fa mode , which must be one of ``emacs'' or ``vi''

EL_SIGNAL , Fa int flag

If Fa flag is non-zero, will install its own signal handler for the following signals when reading command input: SIGCONT SIGHUP SIGINT SIGQUIT SIGSTOP SIGTERM SIGTSTP and SIGWINCH Otherwise, the current signal handlers will be used.

EL_BIND , Fa const char * , Fa ... , NULL

Perform the bind builtin command. Refer to editrc(5) for more information.

EL_ECHOTC , Fa const char * , Fa ... , NULL

Perform the echotc builtin command. Refer to editrc(5) for more information.

EL_SETTC , Fa const char * , Fa ... , NULL

Perform the settc builtin command. Refer to editrc(5) for more information.

EL_SETTY , Fa const char * , Fa ... , NULL

Perform the setty builtin command. Refer to editrc(5) for more information.

EL_TELLTC , Fa const char * , Fa ... , NULL

Perform the telltc builtin command. Refer to editrc(5) for more information.

EL_ADDFN , Fa const char *name , Fa const char *help

Fa "unsigned char (*func)(EditLine *e, int ch)" Add a user defined function, Fn func , referred to as Fa name which is invoked when a key which is bound to Fa name is entered. Fa help is a description of Fa name . At invocation time, Fa ch is the key which caused the invocation. The return value of Fn func should be one of:

CC_NORM

Add a normal character.

CC_NEWLINE

End of line was entered.

CC_EOF

EOF was entered.

CC_ARGHACK

Expecting further command input as arguments, do nothing visually.

CC_REFRESH

Refresh display.

CC_REFRESH_BEEP

Refresh display, and beep.

CC_CURSOR

Cursor moved, so update and perform CC_REFRESH

CC_REDISPLAY

Redisplay entire input line. This is useful if a key binding outputs extra information.

CC_ERROR

An error occurred. Beep, and flush tty.

CC_FATAL

Fatal error, reset tty to known state.

EL_HIST , Fa History *(*func)(History *, int op, ...)

Fa "const char *ptr" Defines which history function to use, which is usually Fn history . Fa ptr should be the value returned by Fn history_init .

EL_EDITMODE , Fa int flag

If Fa flag is non-zero, editing is enabled (the default). Note that this is only an indication, and does not affect the operation of . At this time, it is the caller's responsibility to check this (using Fn el_get ) to determine if editing should be enabled or not.

EL_UNBUFFERED , Fa int flag

If Fa flag is zero, unbuffered mode is disabled (the default). In unbuffered mode, Fn el_gets will return immediately after processing a single character.

EL_GETCFN , Fa el_rfunc_t f

Whenever reading a character, use the function

Ft int Fo f Fa EditLine *e Fa wchar_t *wc Fc

which stores the character in Fa wc and returns 1 on success, 0 on end of file, or -1 on I/O or encoding errors. Functions internally using it include Fn el_wgets , Fn el_wgetc , Fn el_gets , and Fn el_getc . Initially, a builtin function is installed, and replacing it is discouraged because writing such a function is very error prone. The builtin function can be restored at any time by passing the special value EL_BUILTIN_GETCFN instead of a function pointer.

EL_CLIENTDATA , Fa void *data

Register Fa data to be associated with this EditLine structure. It can be retrieved with the corresponding Fn el_get call.

EL_SETFP , Fa int fd , Fa FILE *fp

Set the current editline file pointer for ``input'' Fa fd = 0 ``output'' Fa fd = 1 or ``error'' Fa fd = 2 from Fa fp .

Fn el_get

Get parameters. Fa op determines which parameter to retrieve into Fa result . Returns 0 if successful, -1 otherwise.

The following values for Fa op are supported, along with actual type of Fa result :

EL_PROMPT , Fa char *(*f)(EditLine *) , Fa char *c

Set Fa f to a pointer to the function that displays the prompt. If Fa c is not NULL set it to the start/stop literal prompt character.

EL_RPROMPT , Fa char *(*f)(EditLine *) , Fa char *c

Set Fa f to a pointer to the function that displays the prompt. If Fa c is not NULL set it to the start/stop literal prompt character.

EL_EDITOR , Fa const char **n

Set the name of the editor in Fa n , which will be one of ``emacs'' or ``vi''

EL_GETTC , Fa const char *name , Fa void *value

If Fa name is a valid termcap(5) capability set Fa value to the current value of that capability.

EL_SIGNAL , Fa int *s

Set Fa s to non-zero if has installed private signal handlers (see Fn el_get above).

EL_EDITMODE , Fa int *c

Set Fa c to non-zero if editing is enabled.

EL_GETCFN , Fa el_rfunc_t *f

Set Fa f to a pointer to the function that reads characters, or to EL_BUILTIN_GETCFN if the builtin function is in use.

EL_CLIENTDATA , Fa void **data

Set Fa data to the previously registered client data set by an Fn el_set call.

EL_UNBUFFERED , Fa int *c

Set Fa c to non-zero if unbuffered mode is enabled.

EL_GETFP , Fa int fd, Fa FILE **fp

Set Fa fp to the current editline file pointer for ``input'' Fa fd = 0 ``output'' Fa fd = 1 or ``error'' Fa fd = 2

Fn el_source

Initialize by reading the contents of Fa file . Fn el_parse is called for each line in Fa file . If Fa file is NULL try $HOME/.editrc Refer to editrc(5) for details on the format of Fa file . Fn el_source returns 0 on success and -1 on error.

Fn el_resize

Must be called if the terminal size changes. If EL_SIGNAL has been set with Fn el_set , then this is done automatically. Otherwise, it's the responsibility of the application to call Fn el_resize on the appropriate occasions.

Fn el_cursor

Move the cursor to the right (if positive) or to the left (if negative) Fa count characters. Returns the resulting offset of the cursor from the beginning of the line.

Fn el_line

Return the editing information for the current line in a Fa LineInfo structure, which is defined as follows:

typedef struct lineinfo {
    const char *buffer;    /* address of buffer */
    const char *cursor;    /* address of cursor */
    const char *lastchar;  /* address of last character */
} LineInfo;

Fa buffer is not NUL terminated. This function may be called after Fn el_gets to obtain the Fa LineInfo structure pertaining to line returned by that function, and from within user defined functions added with EL_ADDFN

Fn el_insertstr

Insert Fa str into the line at the cursor. Returns -1 if Fa str is empty or won't fit, and 0 otherwise.

Fn el_deletestr

Delete Fa count characters before the cursor.

HISTORY LIST FUNCTIONS

The history functions use a common data structure, Fa History , which is created by Fn history_init and freed by Fn history_end .

The following functions are available:

Fn history_init

Initialize the history list, and return a data structure to be used by all other history list functions, or NULL on failure.

Fn history_end

Clean up and finish with Fa h , assumed to have been created with Fn history_init .

Fn history

Perform operation Fa op on the history list, with optional arguments as needed by the operation. Fa ev is changed accordingly to operation. The following values for Fa op are supported, along with the required argument list:

H_SETSIZE , Fa int size

Set size of history to Fa size elements.

H_GETSIZE

Get number of events currently in history.

H_END

Cleans up and finishes with Fa h , assumed to be created with Fn history_init .

H_CLEAR

Clear the history.

H_FUNC , Fa void *ptr , Fa history_gfun_t first

Fa "history_gfun_t next" , Fa "history_gfun_t last" , Fa "history_gfun_t prev" , Fa "history_gfun_t curr" , Fa "history_sfun_t set" , Fa "history_vfun_t clear" , Fa "history_efun_t enter" , Fa "history_efun_t add" Define functions to perform various history operations. Fa ptr is the argument given to a function when it's invoked.

H_FIRST

Return the first element in the history.

H_LAST

Return the last element in the history.

H_PREV

Return the previous element in the history. It is newer than the current one.

H_NEXT

Return the next element in the history. It is older than the current one.

H_CURR

Return the current element in the history.

H_SET

Set the cursor to point to the requested element.

H_ADD , Fa const char *str

Append Fa str to the current element of the history, or perform the H_ENTER operation with argument Fa str if there is no current element.

H_APPEND , Fa const char *str

Append Fa str to the last new element of the history.

H_ENTER , Fa const char *str

Add Fa str as a new element to the history and, if necessary, removing the oldest entry to keep the list to the created size. If H_SETUNIQUE has been called with a non-zero argument, the element will not be entered into the history if its contents match the ones of the current history element. If the element is entered Fn history returns 1; if it is ignored as a duplicate returns 0. Finally Fn history returns -1 if an error occurred.

H_PREV_STR , Fa const char *str

Return the closest previous event that starts with Fa str .

H_NEXT_STR , Fa const char *str

Return the closest next event that starts with Fa str .

H_PREV_EVENT , Fa int e

Return the previous event numbered Fa e .

H_NEXT_EVENT , Fa int e

Return the next event numbered Fa e .

H_LOAD , Fa const char *file

Load the history list stored in Fa file .

H_SAVE , Fa const char *file

Save the history list to Fa file .

H_SAVE_FP , Fa FILE *fp

Save the history list to the opened Ft FILE pointer Fa fp .

H_SETUNIQUE , Fa int unique

Set flag that adjacent identical event strings should not be entered into the history.

H_GETUNIQUE

Retrieve the current setting if adjacent identical elements should be entered into the history.

H_DEL , Fa int e

Delete the event numbered Fa e . This function is only provided for readline(3) compatibility. The caller is responsible for free'ing the string in the returned Fa HistEvent .

Fn history returns Gt]= 0 if the operation Fa op succeeds. Otherwise, -1 is returned and Fa ev is updated to contain more details about the error.

TOKENIZATION FUNCTIONS

The tokenization functions use a common data structure, Fa Tokenizer , which is created by Fn tok_init and freed by Fn tok_end .

The following functions are available:

Fn tok_init

Initialize the tokenizer, and return a data structure to be used by all other tokenizer functions. Fa IFS contains the Input Field Separators, which defaults to Aq space , Aq tab , and Aq newline if NULL

Fn tok_end

Clean up and finish with Fa t , assumed to have been created with Fn tok_init .

Fn tok_reset

Reset the tokenizer state. Use after a line has been successfully tokenized by Fn tok_line or Fn tok_str and before a new line is to be tokenized.

Fn tok_line

Tokenize Fa li , If successful, modify: Fa argv to contain the words, Fa argc to contain the number of words, Fa cursorc (if not NULL to contain the index of the word containing the cursor, and Fa cursoro (if not NULL to contain the offset within Fa argv[cursorc] of the cursor.

Returns 0 if successful, -1 for an internal error, 1 for an unmatched single quote, 2 for an unmatched double quote, and 3 for a backslash quoted Aq newline . A positive exit code indicates that another line should be read and tokenization attempted again.

Fn tok_str

A simpler form of Fn tok_line ; Fa str is a NUL terminated string to tokenize.

SEE ALSO

sh(1), signal(3), termcap(3), editrc(5), termcap(5), editline(7)

HISTORY

The library first appeared in BSD 4.4 CC_REDISPLAY appeared in Nx 1.3 . CC_REFRESH_BEEP EL_EDITMODE and the readline emulation appeared in Nx 1.4 . EL_RPROMPT appeared in Nx 1.5 .

AUTHORS

An -nosplit The library was written by An Christos Zoulas . An Luke Mewburn wrote this manual and implemented CC_REDISPLAY CC_REFRESH_BEEP EL_EDITMODE and EL_RPROMPT An Jaromir Dolecek implemented the readline emulation. An Johny Mattsson implemented wide-character support.

BUGS

At this time, it is the responsibility of the caller to check the result of the EL_EDITMODE operation of Fn el_get (after an Fn el_source or Fn el_parse ) to determine if should be used for further input. I.e., EL_EDITMODE is purely an indication of the result of the most recent editrc(5) edit command.