KFAIL_POINT_CODE (9)

Leading comments

Copyright (c) 2009 Isilon Inc http://www.isilon.com/

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
   notice(s), this list of conditions and the following disclaimer as
   the first lines of this file unmodified other than the possible
   addition of one or more copyright notices.
2. Redistributions in binary form must reproduce th...

(The comments found at the beginning of the groff file "man9/KFAIL_POINT_CODE.9freebsd".)

NAME

KFAIL_POINT_CODE KFAIL_POINT_CODE_FLAGS KFAIL_POINT_CODE_COND KFAIL_POINT_RETURN KFAIL_POINT_RETURN_VOID KFAIL_POINT_ERROR KFAIL_POINT_GOTO KFAIL_POINT_SLEEP_CALLBACKS fail_point DEBUG_FP - fail points

SYNOPSIS

In sys/fail.h Fn KFAIL_POINT_CODE parent name code Fn KFAIL_POINT_CODE_FLAGS parent name flags code Fn KFAIL_POINT_CODE_COND parent name cond flags code Fn KFAIL_POINT_RETURN parent name Fn KFAIL_POINT_RETURN_VOID parent name Fn KFAIL_POINT_ERROR parent name error_var Fn KFAIL_POINT_GOTO parent name error_var label Fn KFAIL_POINT_SLEEP_CALLBACKS parent name pre_func pre_arg post_func post_arg code

DESCRIPTION

Fail points are used to add code points where errors may be injected in a user controlled fashion. Fail points provide a convenient wrapper around user-provided error injection code, providing a sysctl(9) MIB, and a parser for that MIB that describes how the error injection code should fire.

The base fail point macro is Fn KFAIL_POINT_CODE where Fa parent is a sysctl tree (frequently DEBUG_FP for kernel fail points, but various subsystems may wish to provide their own fail point trees), and Fa name is the name of the MIB in that tree, and Fa code is the error injection code. The Fa code argument does not require braces, but it is considered good style to use braces for any multi-line code arguments. Inside the Fa code argument, the evaluation of RETURN_VALUE is derived from the Fn return value set in the sysctl MIB.

Additionally, Fn KFAIL_POINT_CODE_FLAGS provides a Fa flags argument which controls the fail point's behaviour. This can be used to e.g., mark the fail point's context as non-sleepable, which causes the sleep action to be coerced to a busy wait. The supported flags are:

FAIL_POINT_USE_TIMEOUT_PATH

Rather than sleeping on a Fn sleep call, just fire the post-sleep function after a timeout fires.

FAIL_POINT_NONSLEEPABLE

Mark the fail point as being in a non-sleepable context, which coerces Fn sleep calls to Fn delay calls.

Likewise, Fn KFAIL_POINT_CODE_COND supplies a Fa cond argument, which allows you to set the condition under which the fail point's code may fire. This is equivalent to:

        if (cond)
                KFAIL_POINT_CODE_FLAGS(...);

See Sx SYSCTL VARIABLES below.

The remaining Fn KFAIL_POINT_* macros are wrappers around common error injection paths:

Fn KFAIL_POINT_RETURN parent name

is the equivalent of KFAIL_POINT_CODE(..., return RETURN_VALUE)

Fn KFAIL_POINT_RETURN_VOID parent name

is the equivalent of KFAIL_POINT_CODE(..., return)

Fn KFAIL_POINT_ERROR parent name error_var

is the equivalent of KFAIL_POINT_CODE(..., error_var = RETURN_VALUE)

Fn KFAIL_POINT_GOTO parent name error_var label

is the equivalent of KFAIL_POINT_CODE(..., { error_var = RETURN_VALUE; goto label;})

SYSCTL VARIABLES

The Fn KFAIL_POINT_* macros add sysctl MIBs where specified. Many base kernel MIBs can be found in the debug.fail_point tree (referenced in code by DEBUG_FP )

The sysctl variable may be set in a number of ways:

  [<pct>%][<cnt>*]<type>[(args...)][-><more terms>]

The <type> argument specifies which action to take; it can be one of:

off

Take no action (does not trigger fail point code)

return

Trigger fail point code with specified argument

sleep

Sleep the specified number of milliseconds

panic

Panic

break

Break into the debugger, or trap if there is no debugger support

print

Print that the fail point executed

pause

Threads sleep at the fail point until the fail point is set to off

yield

Thread yields the cpu when the fail point is evaluated

delay

Similar to sleep, but busy waits the cpu. (Useful in non-sleepable contexts.)

The <pct>% and <cnt>* modifiers prior to <type> control when <type> is executed. The <pct>% form (e.g. "1.2%") can be used to specify a probability that <type> will execute. This is a decimal in the range (0, 100] which can specify up to 1/10,000% precision. The <cnt>* form (e.g. "5*") can be used to specify the number of times <type> should be executed before this <term> is disabled. Only the last probability and the last count are used if multiple are specified, i.e. "1.2%2%" is the same as "2%". When both a probability and a count are specified, the probability is evaluated before the count, i.e. "2%5*" means "2% of the time, but only 5 times total".

The operator -> can be used to express cascading terms. If you specify <term1>-><term2>, it means that if <term1> does not `execute' , <term2> is evaluated. For the purpose of this operator, the return() and print() operators are the only types that cascade. A return() term only cascades if the code executes, and a print() term only cascades when passed a non-zero argument. A pid can optionally be specified. The fail point term is only executed when invoked by a process with a matching p_pid.

EXAMPLES

sysctl debug.fail_point.foobar=2.1%return(5)

21/1000ths of the time, execute Fa code with RETURN_VALUE set to 5.

sysctl debug.fail_point.foobar=2%return(5)->5%return(22)

2/100ths of the time, execute Fa code with RETURN_VALUE set to 5. If that does not happen, 5% of the time execute Fa code with RETURN_VALUE set to 22.

sysctl debug.fail_point.foobar=5*return(5)->0.1%return(22)

For 5 times, return 5. After that, 1/1000th of the time, return 22.

sysctl debug.fail_point.foobar=0.1%5*return(5)

Return 5 for 1 in 1000 executions, but only 5 times total.

sysctl debug.fail_point.foobar=1%*sleep(50)

1/100th of the time, sleep 50ms.

sysctl debug.fail_point.foobar=1*return(5)[pid 1234]

Return 5 once, when pid 1234 executes the fail point.

AUTHORS

An -nosplit This manual page was written by

An Matthew Bryan Aq Mt matthew.bryan@isilon.com and

An Zach Loafman Aq Mt zml@FreeBSD.org .

CAVEATS

It is easy to shoot yourself in the foot by setting fail points too aggressively or setting too many in combination. For example, forcing Fn malloc to fail consistently is potentially harmful to uptime.

The Fn sleep sysctl setting may not be appropriate in all situations. Currently, Fn fail_point_eval does not verify whether the context is appropriate for calling Fn msleep . You can force it to evaluate a sleep action as a delay action by specifying the FAIL_POINT_NONSLEEPABLE flag at the point the fail point is declared.