cdb(3)

NAME
cdb - Constant DataBase library

SYNOPSYS
 #include <cdb.h>
 cc ... -lcdb

DESCRIPTION
cdb is a library to create and access Constant DataBase files. File stores (key,value) pairs and used to quickly find a value based on a given key. Cdb files are create-once files, that is, once created, file cannot be updated but recreated from scratch -- this is why database is called constant. Cdb file is optimized for quick access. Format of such file described in cdb(5) manpage. This manual page corresponds to version 0.72 of tinycdb package. Library defines two non-interlaced interfaces: for querying existing cdb file data (read-only mode) and for creating such a file (almost write-only). Strictly speaking, those modes allows very limited set of opposite operation as well (i.e. in query mode, it is possible to update key's value). All routines in this library are thread-safe as no global data used, except of errno variable for error indication. cdb datafiles may be moved between systems safely, since format does not depend on architecture. Library defines one integer type, cdbi_t, to be used as position in a file or length of a data. Variable of this type have at least 32 bits and treated as unsigned integer. It is ok for this type to be defined as machine word even if it's size is greather than 32 bits.

QUERY MODE
There are two query modes available. First uses a structure that represents a cdb database, just like FILE structure in stdio library, and another works with plain filedescriptor. First mode is more sofisticated and flexible, and usually somewhat faster. It uses mmap(2) internally. This mode may look more "natural" or object-oriented compared to second one. The following routines works with any mode:
cdbi_t cdb_unpack(buf)
   const unsigned char buf[4];
    helper routine to convert 32-bit integer from internal representation to machine format. May be used to handle application integers in a portable way. There is no error return.

  Query Mode 1
All query operations in first more deals with common data structure, struct cdb, assotiated with an open file descriptor. This structure is opaque to application except of one published data member, int cdb_fd, that holds a filedescriptor passed to cdb_init() routine. The following routines exists for accessing cdb database:
int cdb_init(cdbp, fd)
   struct cdb *cdbp;
   int fd;
    initializes structure given by cdbp pointer and assotiates it with opened file descriptor fd. Memory allocation for structure itself if needed and file open operation should be done by application. File fd should be opened at least read-only, and should be seekable. Routine returns 0 on success or negative value on error.
void cdb_free(cdbp)
   struct cdb *cdbp;
    frees internal resources held by structure. Note that this routine does not closes a file.
int cdb_fileno(cdbp)
  const struct cdb *cdbp;
    returns filedescriptor assotiated with cdb (as was passed to cdb_init()).
int cdb_find(cdbp, key, klen)
cdbi_t cdb_datapos(cdbp)
cdbi_t cdb_datalen(cdbp)
   struct cdb *cdbp;
   const void *key;
   cdbi_t klen;
    attempts to find a key given by (key,klen) parameters. If key exists in database, routine returns 1 and places position and length of value assotiated with this key to internal fields inside cdbp structure, to be accessible by cdb_datapos() and cdb_datalen(). If key is not in database, routines returns 0. On error, negative value is returned. Note that using cdb_find() it is possible to lookup only first record with a given key.
int cdb_read(cdbp, buf, len, pos)
   const struct cdb *cdbp;
   void *buf;
   unsigned len;
   cdbi_t pos;
    reads a data from cdb file, starting at position pos of length len, placing result to buf. This routine may be used to get actual value found by cdb_find() or other routines that returns position and length of a data. Returns 0 on success or negative value on error.
int cdb_findinit(cdbfp, cdbp, key, klen)
int cdb_findnext(cdbfp)
  struct cdb_find *cdbfp;
  struct cdb *cdbp;
  const void *key;
  cdbi_t klen;
    sequential-find routines that used separate structure. It is possible to have many than one record with the same key in a database, and these routines allows to enumerate all them. cdb_findinit() initializes search structure pointed to by cdbfp. It will return negative value on error or 0 on success. cdb_findnext() attempts to find next matching key, setting value position and length in cdbfp structure. It will return positive value if given key was found, 0 if there is no more such key(s), or negative value on error. To access value position and length after successeful call to cdb_findnext() (when it returned positive result), use cdb_datapos() and cdb_datalen() macros with cdbp pointer. It is error to use cdb_findnext() after it returned 0 or error condition. These routines is a bit slower than cdb_find().

  Query Mode 2
In this mode, one need to open a cdb file using one of standard system calls (such as open(2)) to obtain a filedescriptor, and then pass that filedescriptor to cdb routines. Available methods to query a cdb database using only a filedescriptor include:
int cdb_seek(fd, key, klen, dlenp)
  int fd;
  const void *key;
  cdbi_t klen;
  cdbi_t *dlenp;
    searches a cdb database (as pointed to by fd filedescriptor) for a key given by (key, klen), and positions file pointer to start of data assotiated with that key if found, so that next read operation from this filedescriptor will read that value, and places length of value, in bytes, to variable pointed to by dlenp. Returns positive value if operation was successeful, 0 if key was not found, or negavive value on error. To read the data from a cdb file, cdb_bread() routine below can be used.
int cdb_bread(fd, buf, len)
  int fd;
  void *buf;
  int len;
    reads data from a file (as pointed to by fd filedescriptor) and places len bytes from this file to a buffer pointed to by buf. Returns 0 if exactly len bytes was read, or a negative value in case of error or end-of-file. This routine ignores interrupt errors (EINTR). Sets errno variable to EIO in case of end-of-file condition (when there is less than len bytes available to read).

  Notes
Note that value of any given key may be updated in place by another value of the same size, by writing to file at position found by cdb_find() or cdb_seek(). However one should be very careful when doing so, since write operation may not succed in case of e.g. power failure, thus leaving corrupted data. When database is (re)created, one can guarantee that no incorrect data will be written to database, but not with inplace update. Note also that it is not possible to update any key or to change length of value.

CREATING MODE

  cdb
database file should usually be created in two steps: first, temporary file created and written to disk, and second, that temporary file is renamed to permanent place. Unix rename(2) call is atomic operation, it removes destination file if any AND renaes another file in one step. This way it is guaranteed that readers will not see incomplete database. To prevent multiple simultaneous updates, locking may also be used. All routines used to create cdb database works with struct cdb_make object that is opaque to application. Application may assume that struct cdb_make has at least the same member(s) as published in struct cdb above.
int cdb_make_start(cdbmp, fd)
   struct cdb_make *cdbmp;
   int fd;
    initializes structure to create a database. File fd should be opened read-write and should be seekable. Returns 0 on success or negative value on error.
int cdb_make_add(cdbmp, key, klen, val, vlen)
   struct cdb_make *cdbmp;
   const void *key, *val;
   cdbi_t klen, vlen;
    adds record with key (key,klen) and value (val,vlen) to a database. Returns 0 on success or negative value on error. Note that this routine does not checks if given key already exists, but cdb_find() will not see second record with the same key. It is not possible to continue building a database if cdb_make_add() returned error indicator.
int cdb_make_finish(cdbmp)
   struct cdb_make *cdbmp;
    finalizes database file, constructing all needed indexes, and frees memory structures. It does not closes filedescriptor. Returns 0 on success or negative value on error.
int cdb_make_exists(cdbmp, key, klen)
   struct cdb_make *cdbmp;
   const void *key;
   cdbi_t klen;
    This routine attempts to find given by (key,klen) key in a not-yet-complete database. It may significantly slow down the whole process, and currently it flushes internal buffer to disk on every call with key those hash value already exists in db. Returns 0 if such key doesn't exists, 1 if it is, or negative value on error. Note that database file should be opened read-write (not write-only) to use this routine. If cdb_make_exists() returned error, it may be not possible to continue constructing database.
int cdb_make_put(cdbmp, key, klen, val, vlen, flag)
   struct cdb_make *cdbmp;
   const void *key, *val;
   cdbi_t klen, vlen;
   int flag;
    This is a somewhat combined cdb_make_exists() and cdb_make_add() routines. flag argument controls how repeated (already existing) keys will be treated:

    CDB_PUT_ADD
        no duplicate checking will be performed. This mode is the same as cdb_make_add() routine does.

    CDB_PUT_REPLACE
        If the key already exists, it will be replaced with a new key,value pair. Note that old pair will not be removed from a database, only from index, so it will appear on sequential reads of all keys and values. Routine will return positive integer if it replaced some previously existing key with a new one.

    CDB_PUT_INSERT
        add key,value pair only if such key does not exists in a database. Routine will return positive number and will do nothing if key is already in a database. Note that since query (see query mode above) will find first added record, this mode is somewhat useless (but allows to reduce database size in case of repeated keys).

    CDB_PUT_WARN
        add key,value pair unconditionnaly, but also check if this key already exists, and return positive number it it is. Combines unconditionnaly cdb_make_exists() and cdb_make_add().

    As with cdb_make_exists(), usage of this routine will significantly slow down database creation process.
void cdb_pack(num, buf)
   cdbi_t num;
   unsigned char buf[4];
    helper routine that used internally to convert machine integer n to internal form to be stored in datafile. May be used to handle application data. There is no error return.
cdbi_t cdb_hash(buf, len)
   const void *buf;
   unsigned len;
    helper routine that calculates cdb hash value of given bytes. CDB hash function is
      hash[n] = (hash[n-1] + (hash[n-1] << 5)) ^ buf[n]
    starting with
      hash[-1] = 5381

ERRORS
cdb library may set errno to following on error:

EPROTO
    database file is corrupted in some way

EINVAL
    the same as EPROTO above if system lacks EPROTO constant

EINVAL
    flag argument for cdb_make_put() is invalid

EEXIST
    flag argument for cdb_make_put() is CDB_PUT_INSERT, and key already exists

ENOMEM
    not enouth memory to complete operation (cdb_make_finish and cdb_make_add)

EIO set by cdb_bread and cdb_seek if a cdb file is shorter than expected or corrupted in some other way.

SEE ALSO
cdb(5), cdb(1), dbm(3), db(3), open(2).

AUTHOR
The tinycdb package written by Michael Tokarev <mjt@corpit.ru>, based on ideas and shares file format with original cdb library by Dan Bernstein.

LICENSE
Public domain.