mbpool (9)
Leading comments
Copyright (c) 2003
Fraunhofer Institute for Open Communication Systems (FhG Fokus).
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions an...
NAME
mbpool - buffer pools for network interfacesSYNOPSIS
In sys/types.h In machine/bus.h In sys/mbpool.h Vt struct mbpool ; Ft int Fo mbp_create Fa struct mbpool **mbp const char *name bus_dma_tag_t dmat Fa u_int max_pages size_t page_size size_t chunk_size Fc Ft void Fn mbp_destroy struct mbpool *mbp Ft void * Fn mbp_alloc struct mbpool *mbp bus_addr_t *pa uint32_t *hp Ft void Fn mbp_free struct mbpool *mbp void *p Ft void Fn mbp_ext_free void * void * Ft void Fn mbp_card_free struct mbpool *mbp Ft void Fn mbp_count struct mbpool *mbp u_int *used u_int *card u_int *free Ft void * Fn mbp_get struct mbpool *mbp uint32_t h Ft void * Fn mbp_get_keep struct mbpool *mbp uint32_t h Ft void Fo mbp_sync Fa struct mbpool *mbp uint32_t h bus_addr_t off bus_size_t len Fa u_int op FcFn MODULE_DEPEND your_module libmbpool 1 1 1
options LIBMBPOOL
DESCRIPTION
Mbuf pools are intended to help drivers for interface cards that need huge amounts of receive buffers, and additionally provides a mapping between these buffers and 32-bit handles.An example of these cards are the Fore/Marconi ForeRunnerHE cards. These employ up to 8 receive groups, each with two buffer pools, each of which can contain up to 8192. This gives a total maximum number of more than 100000 buffers. Even with a more moderate configuration the card eats several thousand buffers. Each of these buffers must be mapped for DMA. While for machines without an IOMMU and with lesser than 4GByte memory this is not a problem, for other machines this may quickly eat up all available IOMMU address space and/or bounce buffers. On sparc64, the default I/O page size is 16k, so mapping a simple mbuf wastes 31/32 of the address space.
Another problem with most of these cards is that they support putting a 32-bit handle into the buffer descriptor together with the physical address. This handle is reflected back to the driver when the buffer is filled, and assists the driver in finding the buffer in host memory. For 32-bit machines, the virtual address of the buffer is usually used as the handle. This does not work for 64-bit machines for obvious reasons, so a mapping is needed between these handles and the buffers. This mapping should be possible without searching lists and the like.
An mbuf pool overcomes both problems by allocating DMA-able memory page wise with a per-pool configurable page size. Each page is divided into a number of equally-sized chunks, the last MBPOOL_TRAILER_SIZE of which are used by the pool code (4 bytes). The rest of each chunk is usable as a buffer. There is a per-pool limit on pages that will be allocated.
Additionally, the code manages two flags for each buffer: ``on-card'' and ``used'' A buffer may be in one of three states:
- free
- None of the flags is set.
- on-card
- Both flags are set. The buffer is assumed to be handed over to the card and waiting to be filled.
- used
- The buffer was returned by the card and is now travelling through the system.
A pool is created with Fn mbp_create . This call specifies a DMA tag Fa dmat to be used to create and map the memory pages via bus_dmamem_alloc9. The Fa chunk_size includes the pool overhead. It means that to get buffers for 5 ATM cells (240 bytes), a chunk size of 256 should be specified. This results in 12 unused bytes between the buffer, and the pool overhead of four byte. The total maximum number of buffers in a pool is Fa max_pages * Fa ( page_size / Fa chunk_size ) . The maximum value for Fa max_pages is 2^14-1 (16383) and the maximum of Fa page_size / Fa chunk_size is 2^9 (512). If the call is successful, a pointer to a newly allocated Vt struct mbpool is set into the variable pointed to by Fa mpb .
A pool is destroyed with Fn mbp_destroy . This frees all pages and the pool structure itself. If compiled with DIAGNOSTICS the code checks that all buffers are free. If not, a warning message is issued to the console.
A buffer is allocated with Fn mbp_alloc . This returns the virtual address of the buffer and stores the physical address into the variable pointed to by Fa pa . The handle is stored into the variable pointed to by Fa hp . The two most significant bits and the 7 least significant bits of the handle are unused by the pool code and may be used by the caller. These are automatically stripped when passing a handle to one of the other functions. If a buffer cannot be allocated (either because the maximum number of pages is reached, no memory is available or the memory cannot be mapped), NULL is returned. If a buffer could be allocated, it is in the ``on-card'' state.
When the buffer is returned by the card, the driver calls Fn mbp_get with the handle. This function returns the virtual address of the buffer and clears the ``on-card'' bit. The buffer is now in the ``used'' state. The function Fn mbp_get_keep differs from Fn mbp_get in that it does not clear the ``on-card'' bit. This can be used for buffers that are returned ``partially'' by the card.
A buffer is freed by calling Fn mbp_free with the virtual address of the buffer. This clears the ``used'' bit, and puts the buffer on the free list of the pool. Note that free buffers are NOT returned to the system. The function Fn mbp_ext_free can be given to Fn m_extadd as the free function. The user argument must be the pointer to the pool.
Before using the contents of a buffer returned by the card, the driver must call Fn mbp_sync with the appropriate parameters. This results in a call to bus_dmamap_sync9 for the buffer.
All buffers in the pool that are currently in the ``on-card'' state can be freed with a call to Fn mbp_card_free . This may be called by the driver when it stops the interface. Buffers in the ``used'' state are not freed by this call.
For debugging it is possible to call Fn mbp_count . This returns the number of buffers in the ``used'' and ``on-card'' states and the number of buffers on the free list.