Sys::Mmap (3)

Leading comments

Automatically generated by Pod::Man 4.09 (Pod::Simple 3.35)

Standard preamble:
========================================================================

(The comments found at the beginning of the groff file "man3/Sys::Mmap.3pm".)

NAME

Sys::Mmap - uses mmap to map in a file as a Perl variable

SYNOPSIS

    use Sys::Mmap;

    new Mmap $str, 8192, 'structtest2.pl' or die $!;
    new Mmap $var, 8192 or die $!;

    mmap($foo, 0, PROT_READ, MAP_SHARED, FILEHANDLE) or die "mmap: $!";
    @tags = $foo =~ /<(.*?)>/g;
    munmap($foo) or die "munmap: $!";
    
    mmap($bar, 8192, PROT_READ|PROT_WRITE, MAP_SHARED, FILEHANDLE);
    substr($bar, 1024, 11) = "Hello world";

    mmap($baz, 8192, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANON, STDOUT);

    $addr = mmap($baz, 8192, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANON, STDOUT);
    Sys::Mmap::hardwire($qux, $addr, 8192);

DESCRIPTION

The Mmap module uses the mmap call to map in a file as a Perl variable. Memory access by mmap may be shared between threads or forked processes, and may be a disc file that has been mapped into memory. Sys::Mmap depends on your operating system supporting or .1b mmap, of course.

Note that PerlIO now defines a ":mmap" tag and presents mmap'd files as regular files, if that is your cup of joe.

Several processes may share one copy of the file or string, saving memory, and concurrently making changes to portions of the file or string. When not used with a file, it is an alternative to SysV shared memory. Unlike SysV shared memory, there are no arbitrary size limits on the shared memory area, and sparce memory usage is handled optimally on most modern

implementations.

Using the "new()" method provides a "tie()"'d interface to "mmap()" that allows you to use the variable as a normal variable. If a filename is provided, the file is opened and mapped in. If the file is smaller than the length provided, the file is grown to that length. If no filename is provided, anonymous shared inheritable memory is used. Assigning to the variable will replace a section in the file corresponding to the length of the variable, leaving the remainder of the file intact and unmodified. Using "substr()" allows you to access the file at an offset, and does not place any requirements on the length argument to substr() or the length of the variable being inserted, provided it does not exceed the length of the memory region. This protects you from the pathological cases involved in using "mmap()" directly, documented below.

When calling "mmap()" or "hardwire()" directly, you need to be careful how you use the variable. Some programming constructs may create copies of a string which, while unimportant for smallish strings, are far less welcome if you're mapping in a file which is a few gigabytes big. If you use

and attempt to write to the file via the variable you need to be even more careful. One of the few ways in which you can safely write to the string in-place is by using "substr()" as an lvalue and ensuring that the part of the string that you replace is exactly the same length. Other functions will allocate other storage for the variable, and it will no longer overlay the mapped in file.

new Mmap

VARIABLE, LENGTH, OPTIONALFILENAME

Maps

LENGTH

bytes of (the contents of)

OPTIONALFILENAME

if

OPTINALFILENAME

is provided, otherwise uses anonymous, shared inheritable memory. This memory region is inherited by any "fork()"ed children.

VARIABLE

will now refer to the contents of that file. Any change to

VARIABLE

will make an identical change to the file. If

LENGTH

is zero and a file is specified, the current length of the file will be used. If

LENGTH

is larger then the file, and

OPTIONALFILENAME

is provided, the file is grown to that length before being mapped. This is the preferred interface, as it requires much less caution in handling the variable.

VARIABLE

will be tied into the ``Mmap'' package, and "mmap()" will be called for you.

Assigning to

VARIABLE

will overwrite the beginning of the file for a length of the value being assigned in. The rest of the file or memory region after that point will be left intact. You may use substr() to assign at a given position:

substr(

VARIABLE, POSITION, LENGTH

) =

NEWVALUE

mmap(

VARIABLE, LENGTH, PROTECTION, FLAGS, FILEHANDLE, OFFSET

)

Maps

LENGTH

bytes of (the underlying contents of)

FILEHANDLE

into your address space, starting at offset

OFFSET

and makes

VARIABLE

refer to that memory. The

OFFSET

argument can be omitted in which case it defaults to zero. The

LENGTH

argument can be zero in which case a stat is done on

FILEHANDLE

and the size of the underlying file is used instead.

The

PROTECTION

argument should be some ORed combination of the constants

PROT_READ, PROT_WRITE

and

PROT_EXEC

or else

PROT_NONE.

The constants

PROT_EXEC

and

PROT_NONE

are unlikely to be useful here but are included for completeness.

The

FLAGS

argument must include either

MAP_SHARED

or

MAP_PRIVATE

(the latter is unlikely to be useful here). If your platform supports it, you may also use

MAP_ANON

or

MAP_ANONYMOUS.

If your platform supplies

MAP_FILE

as a non-zero constant (necessarily non-POSIX) then you should also include that in

FLAGS. POSIX

.1b does not specify

MAP_FILE

as a

FLAG

argument and most if not all versions of Unix have

MAP_FILE

as zero.

mmap returns undef on failure, and the address in memory where the variable was mapped to on success.

munmap(

VARIABLE

)

Unmaps the part of your address space which was previously mapped in with a call to "mmap(VARIABLE, ...)" and makes

VARIABLE

become undefined.

munmap returns 1 on success and undef on failure.

hardwire(

VARIABLE, ADDRESS, LENGTH

)

Specifies the address in memory of a variable, possibly within a region you've "mmap()"ed another variable to. You must use the same percaustions to keep the variable from being reallocated, and use "substr()" with an exact length. If you "munmap()" a region that a "hardwire()"ed variable lives in, the "hardwire()"ed variable will not automatically be "undef"ed. You must do this manually.

Constants

The Mmap module exports the following constants into your namespace
    

MAP_SHARED MAP_PRIVATE MAP_ANON MAP_ANONYMOUS MAP_FILE

    

PROT_EXEC PROT_NONE PROT_READ PROT_WRITE

Of the constants beginning

MAP_,

only

MAP_SHARED

and

MAP_PRIVATE

are defined in

POSIX

.1b and only

MAP_SHARED

is likely to be useful.

BUGS

Scott Walters doesn't know and is just winging it. There must be a better way to tell Perl not to reallocate a variable in memory...

The tie() interface makes writing to a substring of the variable much less efficient. One user cited his application running 10-20 times slower when ``new Mmap'' is used than when mmap() is called directly.

Malcolm Beattie has not reviewed Scott's work and is not responsible for any bugs, errors, omissions, stylistic failings, importabilities, or design flaws in this version of the code.

There should be a tied interface to hardwire() as well.

Scott Walter's spelling is awful.

hardwire() will segfault Perl if the mmap() area it was refering to is munmap()'d out from under it.

munmap() will segfault Perl if the variable was not successfully mmap()'d previously, or if it has since been reallocated by Perl.

AUTHOR

Todd Rinaldo cleaned up code, modernized again, and merged in many fixes, 2010-2011.

Scott Walters updated for Perl 5.6.x, additions, 2002.

Malcolm Beattie, 21 June 1996.