Lintian::Util • man page

Lintian::Util (3)

Leading comments

Automatically generated by Pod::Man 4.09 (Pod::Simple 3.35)

Standard preamble:
========================================================================

(The comments found at the beginning of the groff file "man3/Lintian::Util.3".)

NAME

Lintian::Util - Lintian utility functions

SYNOPSIS

 use Lintian::Util qw(slurp_entire_file normalize_pkg_path);
 
 my $text = slurp_entire_file('some-file');
 if ($text =~ m/regex/) {
    # ...
 }

 my $path = normalize_pkg_path('usr/bin/', '../lib/git-core/git-pull');
 if (defined $path) {
    # ...
 }
 
 my (@paragraphs);
 eval { @paragraphs = read_dpkg_control_utf8('some/debian/ctrl/file'); };
 if ($@) {
    # syntax error etc.
    die "ctrl/file: $@";
 }
 
 foreach my $para (@paragraphs) {
    my $value = $para->{'some-field'};
    if (defined $value) {
        # ...
    }
 }

DESCRIPTION

This module contains a number of utility subs that are nice to have, but on their own did not warrant their own module.

Most subs are imported only on request.

Debian control parsers

At first glance, this module appears to contain several debian control parsers. In practise, there is only one real parser (``visit_dpkg_paragraph'') - the rest are convenience functions around it.

If you have very large files (e.g. Packages_amd64), you almost certainly want ``visit_dpkg_paragraph''. Otherwise, one of the convenience functions are probably what you are looking for.

Use get_deb_info when: You have a .deb (or .udeb) file and you want the control file from it.
Use get_dsc_info when: You have a .dsc (or .changes) file. Alternative, it is also useful if you have a control file and only care about the first paragraph.
Use read_dpkg_control_utf8 or read_dpkg_control when: You have a debian control file (such debian/control) and you want a number of paragraphs from it.
Use parse_dpkg_control when: When you would have used ``read_dpkg_control_utf8'', except you have an open filehandle rather than a file name.

CONSTANTS

The following constants can be passed to the Debian control file parser functions to alter their parsing flag.

DCTRL_DEBCONF_TEMPLATE: The file should be parsed as debconf template. These have slightly syntax rules for whitespace in some cases.
DCTRL_NO_COMMENTS: The file do not allow comments. With this flag, any comment in the file is considered a syntax error.

VARIABLES

$PKGNAME_REGEX: Regular expression that matches valid package names. The expression is not anchored and does not enforce any ``boundary'' characters.
$PKGVERSION_REGEX: Regular expression that matches valid package versions. The expression is not anchored and does not enforce any ``boundary'' characters.

FUNCTIONS

parse_dpkg_control(HANDLE[, FLAGS[, LINES ]])

Reads a debian control file from

HANDLE

and returns a list of paragraphs in it. A paragraph is represented via a hashref, which maps (lower cased) field names to their values.

FLAGS

(if given) is a bitmask of the DCTRL_* constants. Please refer to ``

CONSTANTS''

for the list of constants and their meaning. The default value for

FLAGS

is 0.

LINES

is given, it should be a reference to an empty list. On return,

LINES

will be populated with a hashref for each paragraph (in the same order as the returned list). Each hashref will also have a special key "START-OF-PARAGRAPH" that gives the line number of the first field in that paragraph. These hashrefs will map the field name of the given paragraph to the line number where the field name appeared.

This is a convenience sub around ``visit_dpkg_paragraph'' and can therefore produce the same errors as it. Please see ``visit_dpkg_paragraph'' for the finer semantics of how the control file is parsed.

NB:

parse_dpkg_control does not close the handle for the caller.

visit_dpkg_paragraph ( CODE, HANDLE[, FLAGS ])

Reads a debian control file from

HANDLE

and passes each paragraph to

CODE. A

paragraph is represented via a hashref, which maps (lower cased) field names to their values.

FLAGS

(if given) is a bitmask of the DCTRL_* constants. Please refer to ``

CONSTANTS''

for the list of constants and their meaning. The default value for

FLAGS

is 0.

If the file is empty (i.e. it contains no paragraphs), the method will contain an empty list. The deb822 contents may be inside a signed

PGP

message with a signature.

visit_dpkg_paragraph will require the

PGP

headers to be correct (if present) and require that the entire file is covered by the signature. However, it will not validate the signature (in fact, the contents of the

PGP SIGNATURE

part can be empty). The signature should be validated separately.

visit_dpkg_paragraph will pass paragraphs to

CODE

as they are completed. If

CODE

can process the paragraphs as they are seen, very large control files can be processed without keeping all the paragraphs in memory.

As a consequence of how the file is parsed,

CODE

may be passed a number of (valid) paragraphs before parsing is stopped due to a syntax error.

NB:

visit_dpkg_paragraph does not close the handle for the caller.

CODE

is expected to be a callable reference (e.g. a sub) and will be invoked as the following:

CODE- >( PARA, LINE_NUMBERS )

The first argument,

PARA,

is a hashref to the most recent paragraph parsed. The second argument,

LINE_NUMBERS,

is a hashref mapping each of the field names to the line number where the field name appeared.

LINE_NUMBERS

will also have a special key "START-OF-PARAGRAPH" that gives the line number of the first field in that paragraph.

The return value of

CODE

is ignored.

If the

CODE

invokes die (or similar) the error is propagated to the caller.

On syntax errors, visit_dpkg_paragraph will call die with the following string:

  "syntax error at line %d: %s\n"

Where %d is the line number of the issue and %s is one of:

Duplicate field %s: The field appeared twice in the paragraph.
Continuation line outside a paragraph (maybe line %d should be .): A continuation line appears outside a paragraph - usually caused by an unintended empty line before it.
Whitespace line not allowed (possibly missing a .): An empty continuation line was found. This usually means that a period is missing to denote an ``empty line'' in (e.g.) the long description of a package.
Cannot parse line %s: Generic error containing the text of the line that confused the parser. Note that all non-printables in %s will be replaced by underscores.
Comments are not allowed: A comment line appeared and
FLAGS
contained
DCTRL_NO_COMMENTS.
PGP signature seen before start of signed message: A ``
BEGIN PGP SIGNATURE''
header is seen and a ``
BEGIN PGP MESSAGE''
has not been seen yet.
Two PGP signatures (first one at line %d): Two ``
BEGIN PGP SIGNATURE''
headers are seen in the same file.
Unexpected %s header: A valid
PGP
header appears (e.g. ``
BEGIN PUBLIC KEY BLOCK''
).
Malformed PGP header: An invalid or malformed
PGP
header appears.
Expected at most one signed message (previous at line %d): Two ``
BEGIN PGP MESSAGE''
headers appears in the same message.
End of file but expected an END PGP SIGNATURE header: The file ended after a ``
BEGIN PGP SIGNATURE''
header without being followed by an ``
END PGP SIGNATURE''.
PGP MESSAGE header must be first content if present: The file had content before
PGP MESSAGE.
Data after the PGP SIGNATURE: The file had data after the
PGP SIGNATURE
block ended.
End of file before BEGIN PGP SIGNATURE: The file had a ``
BEGIN PGP MESSAGE''
header, but no signature was present.

read_dpkg_control_utf8(FILE[, FLAGS[, LINES ]])

read_dpkg_control(FILE[, FLAGS[, LINES ]])

This is a convenience function to ease using ``parse_dpkg_control'' with paths to files (rather than open handles). The first argument must be the path to a

FILE,

which should be read as a debian control file. If the file is empty, an empty list is returned.

Otherwise, this behaves like:

 use autodie;
 
 open(my $fd, '<:encoding(UTF-8)', FILE); # or '<'
 my @p = parse_dpkg_control($fd, FLAGS, LINES);
 close($fd);
 return @p;

This goes without saying that may fail with any of the messages that ``parse_dpkg_control(HANDLE[, FLAGS[,

LINES

]])'' do. It can also emit autodie exceptions if open or close fails.

dpkg_deb_has_ctrl_tarfile()

Check if lintian could use dpkg-deb instead of ar and tar

get_deb_info( DEBFILE )

Extracts the control file from

DEBFILE

and returns it as a hashref.

Basically, this is a fancy convenience for setting up an ar + tar pipe and passing said pipe to ``parse_dpkg_control(HANDLE[, FLAGS[,

LINES

]])''.

DEBFILE

must be an ar file containing a ``control.tar.gz'' member, which in turn should contain a ``control'' file. If the ``control'' file is empty this will return an empty list.

Note: the control file is only expected to have a single paragraph and thus only the first is returned (in the unlikely case that there are more than one).

This function may fail with any of the messages that ``parse_dpkg_control'' do. It can also emit:

 "cannot fork to unpack %s: %s\n"

get_dsc_control ( DSCFILE )

Convenience function for reading dsc files. It will read the

DSCFILE

using ``read_dpkg_control(FILE[, FLAGS[,

LINES

]])'' and then return the first paragraph. If the file has no paragraphs, "undef" is returned instead.

Note: the control file is only expected to have a single paragraph and thus only the first is returned (in the unlikely case that there are more than one).

This function may fail with any of the messages that ``read_dpkg_control(FILE[, FLAGS[,

LINES

]])'' do.

slurp_entire_file (FOH[, NOCLOSE ])

Reads the contents of

FOH

into memory and return it as a scalar.

FOH

can be either the path to a file or an open file handle.

If it is a handle, the optional

NOCLOSE

parameter can be used to prevent the sub from closing the handle. The

NOCLOSE

parameter has no effect if

FOH

is not a handle.

drain_pipe( FD )

Reads and discards any remaining contents from

FD,

which is assumed to be a pipe. This is mostly done to avoid having the ``write''-end die with a

SIGPIPE

due to a ``broken pipe'' (which can happen if you just close the pipe).

May cause an exception if there are issues reading from the pipe.

Caveat: This will block until the pipe is closed from the ``write''-end, so only use it with pipes where the ``write''-end will eventually close their end by themselves (or something else will make them close it).

get_file_digest( ALGO, FILE )

Creates an

ALGO

digest object that is seeded with the contents of

FILE.

If you just want the hex digest, please use ``get_file_checksum(

ALGO, FILE

)'' instead.

ALGO

can be 'md5' or shaX, where X is any number supported by Digest::SHA (e.g. 'sha256').

This sub is a convenience wrapper around Digest::{

MD5,SHA

get_file_checksum( ALGO, FILE )

Returns a hexadecimal string of the message digest checksum generated by the algorithm

ALGO

FILE.

ALGO

can be 'md5' or shaX, where X is any number supported by Digest::SHA (e.g. 'sha256').

This sub is a convenience wrapper around Digest::{

MD5,SHA

is_string_utf8_encoded( STRING )

Returns a truth value if

STRING

can be decoded as valid

UTF-8.

file_is_encoded_in_non_utf8 (...)

Undocumented

do_fork()

Overrides fork to reset signal handlers etc. in the child.

system_env ( CMD )

Behaves like system (

CMD

) except that the environment of

CMD

is cleaned (as defined by ``clean_env''(1)).

clean_env ([ CLOC ])

Destructively cleans %ENV - removes all variables %ENV except a selected few whitelisted variables.

The list of whitelisted %ENV variables are:

 PATH
 LC_ALL (*)
 TMPDIR

(*)

LC_ALL

is a special case as clean_env will change its value to either ``C.UTF-8'' or ``C'' (if

CLOC

is given and a truth value).

perm2oct( PERM )

Translates

PERM

to an octal permission.

PERM

should be a string describing the permissions as done by tar t or ls -l. That is, it should be a string like ``-rw-r---r--''.

If the string does not appear to be a valid permission, it will cause a trappable error.

Examples:

 # Good
 perm2oct('-rw-r--r--') == 0644
 perm2oct('-rwxr-xr-x') == 0755

 # Bad
 perm2oct('broken')      # too short to be recognised
 perm2oct('-resurunet')  # contains unknown permissions

delete_dir ( ARGS )

Convenient way of calling rm -fr

ARGS

copy_dir ( ARGS )

Convenient way of calling cp -a

ARGS

gunzip_file ( IN, OUT )

Decompresses contents of the file

and stores the contents in the file

OUT. IN

is not removed by this call. On error, this function will cause a trappable error.

open_gz ( FILE )

Opens a handle that reads from the GZip compressed

FILE.

On failure, this sub emits a trappable error.

Note: The handle may be a pipe from an external processes.

touch_file( FILE )

Updates the ``mtime'' of

FILE.

FILE

does not exist, it will be created.

On failure, this sub will emit a trappable error.

fail (MSG[, ...])

Use to signal an internal error. The argument(s) will used to print a diagnostic message to the user.

If multiple arguments are given, they will be merged into a single string (by join (' ', @_)). If only one argument is given it will be stringified and used directly.

locate_helper_tool( TOOLNAME )

Given the name of a helper tool, returns the path to it. The tool must be available in the ``helpers'' subdir of one of the ``lintian root'' directories used by Lintian.

The tool name should follow the same rules as check names. Particularly, third-party checks should namespace their tools in the same way they namespace their checks. E.g. ``python/some-helper''.

If the tool cannot be found, this sub will cause a trappable error.

strip ([ LINE ])

Strips whitespace from the beginning and the end of

LINE

and returns it. If

LINE

is omitted, $_ will be used instead. Example

 @lines = map { strip } <$fd>;

In void context, the input argument will be modified so it can be used as a replacement for chomp in some cases:

  while ( my $line = <$fd> ) {
    strip ($line);
    # $line no longer has any leading or trailing whitespace
  }

Otherwise, a copy of the string is returned:

  while ( my $orig = <$fd> ) {
    my $stripped = strip ($orig);
    if ($stripped ne $orig) {
        # $orig had leading or/and trailing whitespace
    }
  }

lstrip ([ LINE ])

Like strip but only strip leading whitespace.

rstrip ([ LINE ])

Like strip but only strip trailing whitespace.

check_path ( CMD )

Returns 1 if

CMD

can be found in

PATH

(i.e. $ENV{

PATH

}) and is executable. Otherwise, the function return 0.

dequote_name( STR, REMOVESLASH )

Strip an extra layer quoting in index file names and optionally remove an initial ``./'' if any.

Remove initial ./ by default

signal_number2name( NUM )

Given a number, returns the name of the signal (without leading ``

SIG''

). Example:

    signal_number2name(2) eq 'INT'

normalize_pkg_path( PATH )

Normalize

PATH

by removing superfluous path segments.

PATH

is assumed to be relative the package root. Note that the result will never start nor end with a slash, even if

PATH

does.

As the name suggests, this is a path ``normalization'' rather than a true path resolution (for that use Cwd::realpath). Particularly, it assumes none of the path segments are symlinks.

normalize_pkg_path will return "q{}" (i.e. the empty string) if

PATH

is normalized to the root dir and "undef" if the path cannot be normalized without escaping the package root.

Examples:
  normalize_pkg_path('usr/share/java/../../../usr/share/ant/file')
    eq 'usr/share/ant/file'
  normalize_pkg_path('usr/..') eq q{};

 The following will return C<undef>:
  normalize_pkg_path('usr/bin/../../../../etc/passwd')

normalize_pkg_path( CURDIR, LINK_TARGET )

Normalize the path obtained by following a link with

LINK_TARGET

as its target from

CURDIR

as the current directory.

CURDIR

is assumed to be relative to the package root. Note that the result will never start nor end with a slash, even if

CURDIR

DEST

does.

normalize_pkg_path will return "q{}" (i.e. the empty string) if the target is the root dir and "undef" if the path cannot be normalized without escaping the package root.

CAVEAT

: This function is not always sufficient to test if it is safe to open a given symlink. Use is_ancestor_of for that. If you must use this function, remember to check that the target is not a symlink (or if it is, that it can be resolved safely).

Examples:

  normalize_pkg_path('usr/share/java', '../ant/file') eq 'usr/share/ant/file'
  normalize_pkg_path('usr/share/java', '../../../usr/share/ant/file')
  normalize_pkg_path('usr/share/java', '/usr/share/ant/file')
    eq 'usr/share/ant/file'
  normalize_pkg_path('/usr/share/java', '/') eq q{};
  normalize_pkg_path('/', 'usr/..') eq q{};

 The following will return C<undef>:
  normalize_pkg_path('usr/bin', '../../../../etc/passwd')
  normalize_pkg_path('usr/bin', '/../etc/passwd')

parse_boolean ( STR )

Attempt to parse

STR

as a boolean and return its value. If

STR

is not a valid/recognised boolean, the sub will invoke croak.

The following values recognised (string checks are not case sensitive):

The integer 0 is considered false
Any non-zero integer is considered true
true, y and yes are considered true
false, n and no are considered false

is_ancestor_of( PARENTDIR, PATH )

Returns true if and only if

PATH

PARENTDIR

or a path stored somewhere within

PARENTDIR

(or its subdirs).

This function will resolve the paths; any failure to resolve the path will cause a trappable error.

unix_locale_split( STR )

Read

STR

as a locale code (e.g. en_GB.UTF-8) and return a list of locale codes ordered by preference. As an example, en_GB.UTF-8 might return

  en_GB
  en

Note encoding is ignored as all Lintian files are always encoded in

UTF-8.

Special cases: The ``C'' or ``

POSIX''

locale returns the empty list. Other strings that do not match the expected format causes a trappable error.

pipe_tee( INHANDLE, OUTHANDLES[, OPTS ])

Read bytes from

INHANDLE

and copy them into all of the handles in the listref

OUTHANDLES.

The optional

OPTS

argument is a hashref of options, see below.

The subroutine will continue to read from

INHANDLE

until it is exhausted or an error occurs (either during read or write). In case of errors, a trappable error will be raised. The handles are left open when the subroutine returns, caller must close them afterwards.

Caller should ensure that handles are using ``blocking'' I/O. The subroutine will use sysread and syswrite when reading and writing.

OPTS,

if given, may contain the following key-value pairs:

chunk_size: A suggested buffer size for read/write. If given, it will be to sysread as
LENGTH
argument when reading from
INHANDLE.

load_state_cache( STATE_DIR )

[Reporting tools only] Load the state cache from

STATE_DIR.

save_state_cache( STATE_DIR, STATE )

[Reporting tools only] Save the

STATE

cache to

STATE_DIR.

find_backlog( LINTIAN_VERSION, STATE )

[Reporting tools only] Given the current lintian version and the harness state, return a list of group ids that are part of the backlog. The list is sorted based on what version of Lintian processed the package.

Note the result is by design not deterministic to reduce the risk of all large packages being in the same run (e.g. like gcc-5 + gcc-5-cross + gcc-6 + gcc-6-cross).

untaint( VALUE )

Untaint

VALUE

Lintian::Util • man page

Lintian::Util • man page

Lintian::Util (3)

Leading comments

NAME

SYNOPSIS

DESCRIPTION

Debian control parsers

CONSTANTS

VARIABLES

FUNCTIONS

SEE ALSO

Man Section

extra • Version

extra • Source

extra • Book

References