Lintian::Tutorial::WritingChecks (3)

Leading comments

Automatically generated by Pod::Man 4.09 (Pod::Simple 3.35)

Standard preamble:
========================================================================

(The comments found at the beginning of the groff file "man3/Lintian::Tutorial::WritingChecks.3".)

NAME

Lintian::Tutorial::WritingChecks -- Writing checks for Lintian

SYNOPSIS

This guide will quickly guide you through the basics of writing a Lintian check. Most of the work is in writing the two files:

  checks/<my-check>.pm
  checks/<my-check>.desc

And then either adding a Lintian profile or extending an existing one.

DESCRIPTION

The basics of writing a check are outlined in the Lintian User Manual (X3.3). This tutorial will focus on the act of writing the actual check. In this tutorial, we will assume the name of the check to be written is ``deb/pkg-check''.

The tutorial will work with a ``binary'' and ``udeb'' check. Checking source packages works in a similar fashion.

Create a check .desc file

As mentioned, this tutorial will focus on the writing of a check. Please see the Lintian User Manual (X3.3) for how to do this part.

Create the Perl check module

Start with the template:

 # deb/pkg-check is loaded as Lintian::deb::pkg_check
 # - See Lintian User Manual X3.3 for more info
 package Lintian::deb::pkg_check;

 use strict;
 use warnings;
 
 use Lintian::Tags qw(tag);
 
 sub run {
     my ($pkg, $type, $info, $proc, $group) = @_;
     return;
 }

The snippet above is a simple valid check that does ``nothing at all''. We will extend it in just a moment, but first let us have a look at the arguments at the setup.

The run sub is the entry point of our ``deb/pkg-check'' check; it will be invoked once per package it should process. In our case, that will be once per ``binary'' (.deb) and once per udeb package processed.

It is given 5 arguments (in the future, possibly more), which are:

$pkg - The name of the package being processed.

(Same as $proc->pkg_name)

$type - The type of the package being processed.

At the moment, $type is one of ``binary'' (.deb), ``udeb'', ``source'' (.dsc) or ``changes''. This argument is mostly useful if certain checks do not apply equally to all package types being processed.

Generally it is advisable to check only binaries (``binary'' and ``udeb''), sources or changes in a given check. But in rare cases, it makes sense to lump multiple types together in the same check and this argument helps you do that.

(Current it is always identical to $proc->pkg_type)

$info - Accessor to the data Lintian has extracted

Basically all information you want about a given package comes from the $info object. Sometimes referred to as either the ``info object'' or (an instance of) Lintian::Collect.

This object (together with a properly set Needs-Info in the .desc file) will grant you access to all of the data Lintian has extracted about this package.

Based on the value of the $type argument, it will be one of Lintian::Collect::Binary, Lintian::Collect::Changes or Lintian::Collect::Source.

(Currently it is the same as $proc->info)

$proc - Basic metadata about the package

This is an instance of Lintian::Processable and is useful for trivially obtaining very basic package metadata. Particularly, the name of source package and version of source package are readily available through this object..

$group - Group of processables from the same source

If you want to do a cross-check between different packages built from the same source, $group helps you access those other packages (if they are available).

This is an instance of Lintian::ProcessableGroup.

Now back to the coding.

Emitting a tag

Emitting a tag is fairly simple in itself. Simply invoke the tag sub with one or more arguments in the run sub. The first argument being the name of the tag to emit and the rest being the ``extra'' information (if any).

The full example being:

 # deb/pkg-check is loaded as Lintian::deb::pkg_check
 # - See Lintian User Manual X3.3 for more info
 package Lintian::deb::pkg_check;
 
 use strict;
 use warnings;
 
 use Lintian::Tags qw(tag);
 
 sub run {
     my ($pkg, $type, $info, $proc, $group) = @_;
     tag 'deb-pkg-check-works',
         "I was emitted for $pkg, which was a $type package";
     return;
 }

Assuming there is a tag called ``deb-pkg-check-works'' in your .desc file, the above will faithfully emit said tag for all packages processed by this check.

Emitting a tag is fairly simple; the hard part is emitting exactly when there is an issue and without introducing a security hole in Lintian/your check.

Accessing fields

Let's do a slightly harder example. Assume we wanted to emit a tag for all packages without a (valid) Multi-Arch field. This requires us to A) identify if the package has a Multi-Arch field and B) identify if the content of the field was valid.

Starting from the top. All $info objects have a method called field, which gives you access to a (raw) field from the control file of the package. It returns "undef" if said field is not present or the content of said field otherwise. Note that field names must be given in all lowercase letters (i.e. use ``multi-arch'', not ``Multi-Arch'').

 # deb/pkg-check is loaded as Lintian::deb::pkg_check
 # - See Lintian User Manual X3.3 for more info
 package Lintian::deb::pkg_check;
 
 use strict;
 use warnings;
 
 use Lintian::Tags qw(tag);
 
 sub run {
     my ($pkg, $type, $info, $proc, $group) = @_;
 
     my $multiarch = $info->field('multi-arch');
 
     # Emit a "missing-multi-arch-field" for all packages without the
     # Multi-Arch field
     tag 'missing-multi-arch-field' unless defined($multiarch);
     return;
 }

This was the first half. Let's look at checking the value. Multi-arch fields can (currently) be one of ``no'', ``same'', ``foreign'' or ``allowed''. One way of checking this would be using the regex:

  /^(?:no|same|foreign|allowed)$/

An alternative route is using a table like:

  my %VALID_MULTI_ARCH_VALUES = map { $_ => 1} qw(no same foreign allowed);

We will use the second here:

 # deb/pkg-check is loaded as Lintian::deb::pkg_check
 # - See Lintian User Manual X3.3 for more info
 package Lintian::deb::pkg_check;
 
 use strict;
 use warnings;
 
 use Lintian::Tags qw(tag);
 
 my %VALID_MULTI_ARCH_VALUES = map { $_ => 1} qw(
      no same foreign allowed
 );
 
 sub run {
     my ($pkg, $type, $info, $proc, $group) = @_;
 
     my $multiarch = $info->field('multi-arch');
 
     if (defined($multiarch)) {
        # The field is present, lets check it is valid.
        tag 'invalid-multi-arch-field', $multiarch
            unless exists $VALID_MULTI_ARCH_VALUES{$multiarch};
     } else {
         # Emit a "missing-multi-arch-field" for all packages without the
         # Multi-Arch field
         tag 'missing-multi-arch-field';
     }
     return;
 }

Notice that Lintian automatically strips leading and trailing spaces on the first line in a field. It also strips trailing spaces from all other lines, but leading spaces and the `` .''-continuation markers are kept as is.

Checking dependencies

Lintian can do some checking of dependencies. For most cases it works similar to a normal dependency check, but keep in mind that Lintian uses pure logic to determine if dependencies are satisfied (i.e. it will not look up relations like Provides for you).

Suppose you wanted all packages with a multi-arch ``same'' field to pre-depend on the package ``multiarch-support''. Well, we could use the $info->relation method for this.

$info->relation returns an instance of Lintian::Relation. This object has an ``implies'' method that can be used to check if a package has an explicit dependency. Note that ``implies'' actually checks if one relation ``implies'' another (i.e. if you satisfied relationA then you definitely also satisfied relationB).

As with the ``field''-method, field names have to be given in all lowercase. However ``relation'' will never return "undef" (not even if the field is missing).

For example:

 my $predepends = $info->relation('pre-depends');
 unless ($predepends->implies('multiarch-support')) {
    tag 'missing-pre-depends-on-multiarch-support';
 }

Inserted in the proper place, our check now looks like:

 # deb/pkg-check is loaded as Lintian::deb::pkg_check
 # - See Lintian User Manual X3.3 for more info
 package Lintian::deb::pkg_check;
 
 use strict;
 use warnings;
 
 use Lintian::Tags qw(tag);
 
 my %VALID_MULTI_ARCH_VALUES = map { $_ => 1} qw(
      no same foreign allowed
 );
 
 sub run {
     my ($pkg, $type, $info, $proc, $group) = @_;
 
     my $multiarch = $info->field('multi-arch');
 
     if (defined($multiarch)) {
         # The field is present, lets check it is valid.
         tag 'invalid-multi-arch-field', $multiarch
             unless exists $VALID_MULTI_ARCH_VALUES{$multiarch};
         if ($multiarch eq 'same') {
             my $predepends = $info->relation('pre-depends');
             tag 'missing-pre-depends-on-multiarch-support'
                 unless $predepends->implies('multiarch-support');
         }
     } else {
         # Emit a "missing-multi-arch-field" for all packages without the
         # Multi-Arch field
         tag 'missing-multi-arch-field';
     }
     return;
 }

Using static data files

Currently our check mixes data and code. Namely all the valid values for the Multi-Arch field are currently hard-coded in our check. We can move those out of the check by using a data file.

Lintian natively supports data files that are either ``sets'' or ``tables'' via Lintian::Data (i.e. ``unordered'' collections). As an added bonus, Lintian::Data transparently supports vendor specific data files for us.

First we need to make a data file containing the values. Which could be:

 # A table of all the valid values for the multi-arch field.
 no
 same
 foreign
 allowed

This can then be stored in the data directory as data/deb/pkg-check/multiarch-values.

Now we can load it by using:

 use Lintian::Data;
 
 my $VALID_MULTI_ARCH_VALUES =
     Lintian::Data->new('deb/pkg-check/multiarch-values');

Actually, this is not quite true. Lintian::Data is lazy, so it will not load anything before we force it to do so. Most of the time this is just an added bonus. However, if you ever have to force it to load something immediately, you can do so by invoking its ``known'' method (with an arbitrary defined string and ignore the result).

Data files work with 3 access methods, ``all'', ``known'' and ``value''.

all

``all'' (i.e. $data->all) returns a list of all the entries in the data file (for key/value tables, all returns the keys). The list is not sorted in any order (not even input order).

known

``known'' (i.e. $data->known('item')) returns a truth value if a given item or key is known (present) in the data set or table. For key/pair tables, the value associated with the key can be retrieved with ``value'' (see below).

value

``value'' (i.e. $data->value('key')) returns a value associated with a key for key/value tables. For unknown keys, it returns "undef". If the data file is not a key/value table but just a set, value returns a truth value for known keys.

While we could use both ``value'' and ``known'', we will use the latter for readability (and to remind ourselves that this is a data set and not a data table).

Basically we will be replacing:

  unless exists $VALID_MULTI_ARCH_VALUES{$multiarch};

with

  unless $VALID_MULTI_ARCH_VALUES->known($multiarch);

So the resulting check is:

 # deb/pkg-check is loaded as Lintian::deb::pkg_check
 # - See Lintian User Manual X3.3 for more info
 package Lintian::deb::pkg_check;
 
 use strict;
 use warnings;
 
 use Lintian::Data;
 use Lintian::Tags qw(tag);
 
 my $VALID_MULTI_ARCH_VALUES =
     Lintian::Data->new('deb/pkg-check/multiarch-values');
 
 sub run {
     my ($pkg, $type, $info, $proc, $group) = @_;
 
     my $multiarch = $info->field('multi-arch');
 
     if (defined($multiarch)) {
         # The field is present, lets check it is valid.
         tag 'invalid-multi-arch-field', $multiarch
             unless $VALID_MULTI_ARCH_VALUES->known($multiarch);
         if ($multiarch eq 'same') {
             my $predepends = $info->relation('pre-depends');
             tag 'missing-pre-depends-on-multiarch-support'
                 unless $predepends->implies('multiarch-support');
         }
     } else {
         # Emit a "missing-multi-arch-field" for all packages without the
         # Multi-Arch field
         tag 'missing-multi-arch-field';
     }
     return;
 }

Accessing contents of the package

Another heavily used mechanism is to check for the presence (or absence) of a given file. Generally this is what the $info->index and $info->sorted_index methods are for. The ``index'' method returns instances of Lintian::Path, which has a number of utility methods.

If you want to loop over all files in a package, the sorted_index will do this for you. If you are looking for a specific file (or directory), a call to ``index'' will be much faster. For the contents of a specific directory, you can use something like:

 if (my $dir = $info->index('path/to/dir/')) {
     foreach my $elem ($dir->children) {
         print $elem->name . " is a file" if $elem->is_file;
         # ...
     }
 }

Keep in mind that using the ``index'' or ``sorted_index'' method will require that you put ``unpacked'' in Needs-Info. See ``Keeping Needs-Info up to date''.

There are also a pair of methods for accessing the control files of a binary package. These are $info->control_index and $info->sorted_control_index.

Accessing contents of a file in a package

When you actually want to see the contents of a file, you can use open (or open_gz) on an object returned by e.g. $info->index. These methods will open the underlying file for reading (the latter applying a gzip decompression).

However, please do assert that the file is safe to read by calling is_open_ok first. Generally, it will only be true for files or safely resolvable symlinks pointing to files. Should you attempt to open a path that does not satisfy those criteria, Lintian::Path will raise a trappable error at runtime.

Alternatively, if you access the underlying file object, you can use the fs_path method. Usually, you will want to test either is_open_ok or is_valid_path first to ensure you do not follow unsafe symlinks. The ``is_open_ok'' check will also assert that it is not (e.g.) a named pipe or such.

Should you call fs_path on a symlink that escapes the package root, the method will throw a trappable error at runtime. Once the path is returned, there are no more built-in fail-safes. When you use the returned path, keep things like ``../../../../../etc/passwd''-symlink and ``fifo'' pipes in mind.

In some cases, you may even need to access the file system objects without using Lintian::Path. This is, of course, discouraged and suffers from the same issues above (all checking must be done manually by you). Here you have to use the ``unpacked'', ``debfiles'' or ``control'' methods from Lintian::Collect or its subclasses.

The following snippet may be useful for testing that a given path does not escape the root.

 use Lintian::Util qw(is_ancestor_of);
 
 my $path = ...;
 # The snippet applies equally well to $info->debfiles and
 # $info->control (just remember to subst all occurrences of
 # $info->unpacked).
 my $unpacked_file = $info->unpacked($path);
 if ( -f $unpacked_file && is_ancestor_of($info->unpacked, $unpacked_file)) {
    # a file and contained within the package root.
 } else {
    # not a file or an unsafe path
 }

Keeping Needs-Info up to date

Keeping the ``Needs-Info'' field of your .desc file is a bit of manual work. In the description for the method there will generally be a line looking something like:

  Needs-Info requirements for using methodx: Y

Which means that the methodx requires Y to work. Here Y is a comma separated list and each element of Y basically falls into 3 cases.

*

The element is the word none

In this case, the method has no ``external'' requirements and can be used without any changes to your Needs-Info. The ``field'' method is an example of this.

This only makes sense if it is the only element in the list.

*

The element is a link to a method

In this case, the method uses another method to do its job. An example is the sorted_control_index method, which uses the control_index method. So using sorted_control_index has the same requirements as using control_index.

*

The element is the name of a collection (e.g. ``control_index'').

In this case, the method needs the given collection to be run. So to use (e.g.) control_index, you have to put ``bin-pkg-control'' in your Needs-Info.

Methods can have different requirements based on the type of package! An example of this ``changelog'', which requires ``changelog-file'' in binary packages and ``Same as debfiles'' in source packages.

Avoiding security issues

Over the years a couple of security issues have been discovered in Lintian. The problem is that people can in theory create some really nasty packages. Please keep the following in mind when writing a check:

*

Avoid 2-arg open, system/exec($shellcmd), `$shellcmd` like the plague.

When you get any one of those wrong you introduce ``arbitrary code execution'' vulnerabilities (we learned this the hard way via

CVE-2009-4014

).

Usually 3-arg open and the non-shell variant of system/exec are enough. When you actually need a shell pipeline, consider using Lintian::Command. It also provides a safe_qx command to assist with capturing stdout as an alternative to `$cmd` (or qx/$cmd/).

*

Do not trust field values.

This is especially true if you intend to use the value as part of a file name. Verify that the field contains what you expect before you use it.

*

Use Lintian::Path (or, failing that, is_ancestor_of)

You might be tempted to think that the following code is safe:

 use autodie;
 
 my $filename = 'some/file';
 my $ufile = $info->unpacked($filename);
 if ( ! -l $ufile) {
    # Looks safe, but isn't in general
    open(my $fd, '<', $ufile);
    ...;
 }

This is definitely unsafe if ``$filename'' contains at least one directory segment. So, if in doubt, use is_ancestor_of to verify that the requested file is indeed the file you think it is. A better version of the above would be:

 use autodie,
 use Lintian::Util qw(is_ancestor_of);
 [...]
 my $filename = 'some/file';
 my $ufile = $info->unpacked($filename);
 if ( ! -l $ufile && -f $ufile && is_ancestor_of($info->unpacked, $ufile)) {
    # $ufile is a file and it is contained within the package root.
    open(m $fd, '<', $ufile);
    ...;
 }

In some cases you can even drop the ``! -l $ufile'' part.

Of course, it is much easier to use the Lintian::Path object (whenever possible).

 my $filename = 'some/file';
 my $ufile = $info->index($filename);
 if ( $ufile && $ufile->is_file && $ufile->is_open_ok) {
     my $fd = $ufile->open;
     ...;
 }

Here you can drop the `` && $ufile->is_file'' if you want to permit safe symlinks.

For more information on the is_ancestor_of check, see is_ancestor_of

SEE ALSO

Lintian::Tutorial::WritingTests, Lintian::Tutorial::TestSuite