Fetching

A-A-P home page | A-A-P Recipe Executive
--- | ---
Prev |  | Next

# Chapter 15. Fetching

This is the first paragraph of Fetching.

    FETCHING AND UPDATING:

    A convention about using the "update" and "fetch" targets makes it easy for
    users to know how to use a recipe.  The main recipe for a project should be
    able to be used in three ways:

    1. Without specifying a target.
    	This should build the program in the usual way.  Files with an
    	"fetch" attribute are obtained when they are missing.
    2. With the "fetch" target.
    	This should obtain the latest version of all the files for the
    	program, without building the program.
    3. With the "update" target.
    	This should fetch all the files for the program and then build it.
    	It's like the previous two ways combined.

    Here is an example of a recipe that works this way:

    	STATUS = status.txt
    	SOURCE = main.c version.c
    	INCLUDE = common.h
    	TARGET = myprog

    	$TARGET : $SOURCE
    		:cat $STATUS
    		:sys $CC $CFLAGS $LDFLAGS -o $target $source
    	$TARGET : $STATUS

    	# specify where to fetch the files from
    	:attr {fetch = cvs://:pserver:anonymous@cvs.myproject.sf.net:/cvsroot/myproject} $SOURCE $INCLUDE
    	:attr {fetch = ftp://ftp.myproject.org/pub/%file%} $STATUS

    When the "fetch" target is not specified in the recipe or its children, it
    is automatically generated.  Its build commands will fetch all nodes with
    the "fetch" attribute, except ones with a "constant" attribute set
    (non-empty non-zero).  To do the same manually:

    	fetch:
    		:fetch $SOURCE $INCLUDE $STATUS

    NOTE: When any child recipe defines a "fetch" target no automatic fetching
    is done for any of the recipes.  This may not be what you expect.

    When there is no "update" target it is automatically generated.  It will
    invoke the "fetch" target and the first target in the recipe.  To do
    something similar manually:

    	update: fetch $TARGET

    Although the automatically generated "update" target uses one of $TARGET,
    "all" or the first encountered dependency.

    :fetchall
    :fetchall {attr = val}
    		Fetch all the files in the recipe (and child recipes) that
    		have the "fetch" attribute.
    		Extra attributes for fetching can be specified here, they
    		overrule the attributes of the file itself.

    aap fetch		Does a ":fetchall".

    THE FETCH ATTRIBUTE

    The "fetch" attribute is used to specify a list of locations where the file
    can be fetched from.  The word at the start defines the method used to
    fetch the file:
    	ftp		from ftp server
    	http		from http (www) server
    	scp		secure copy
    	file		local file system
    	cvs		from CVS repository
    			For a module that was already checked out the part
    			after "cvs://" may be empty, CVS will then use the
    			same server (CVSROOT) as when the checkout was done.
    	other 		user defined

    These kinds of locations can be used:

    	ftp://ftp.server.name/full/path/file
    	http://www.server.name/path/file
    	scp://host.name/path:path/file
    	cvs://:METHOD:[[USER][:PASSWORD]@]HOSTNAME[:[PORT]]/path/to/repository
    	file:~user/dir/file
    	file:///etc/fstab

    For a local file there are two possibilities: using "file://" or "file:".
    They both have the same meaning.  Note that for an absolute path, relative to
    the root of the file system, you use either one or three slashes, but not two.
    Thus "file:/etc/fstab" and "file:///etc/fstab" are the file "/etc/fstab".  A
    relative path has two or no slashes, but keep in mind that moving the recipe
    will make it invalid.  You can also use "file:~/file" or "file://~/file" for a
    file in your own home directory, and "file:~jan/file" or "file://~jan/file"
    for a file in the home directory of user "jan".

    To add a new method, define a Python function with the name "fetch_method",
    where "method" is the word at the start.  The function will be called with two
    arguments:
    	name		the url with "method://" removed
    	dict		a dictionary with all attributes used of the URL,
    			dict["name"] is the full URL
    	node		a node object.  Useful items:
    			   node.name		short name
    			   node.absname		full name of the file
    			   node.recipe_dir	directory in which node.name
    						is valid
    			   node.attributes 	dictionary with attributes
    The function should return a non-zero number for success, zero for failure.
    Here is an example:

        :python
    	def fetch_foo(from_name, dict, node):
    	    to_name = node.absname
    	    try:
    		foo_the_file(from_name, to_name, dict["foo_option"])
    	    except:
    	    	return 0
    	    return 1

    In the "fetch" attribute the string "%file%" can be used where the path of
    the local target is to be inserted.  This is useful when several files have a
    common directory.  Similarly "%basename%" can be used when the last item in the
    path is to be used.  This removes the path from the local file name, thus can
    be used when the remote directory is called differently and only the file name
    is the same.  Examples:

          :attr {fetch = ftp://ftp.foo.org/pub/foo/%file%} src/include/bar.h

    Gets the file "src/include/bar.h" from
    "ftp://ftp.foo.org/pub/foo/src/include/bar.h".

          :attr {fetch = ftp://ftp.foo.org/pub/foo/src-2.0/include%basename%}
    			  src/include/bar.h

    Gets the file "src/include/bar.h" from
    "ftp://ftp.foo.org/pub/foo/src-2.0/include/bar.h".

    CACHING:

    Remote files are downloaded when used.  This can take quite a bit of time.
    Therefore downloaded files are cached and only downloaded again when outdated.

    The cache can be spread over several directories.  The list is specified
    with the $CACHE variable.

    NOTE: Using a global, writable directory makes it possible to share the cache
    with other users, but only do this when you trust everybody who can login to
    the system!

    A cached file becomes outdated as specified with the "cache_update" attribute
    or $cache_update.  The value is a number and a name.  Possible values for the
    name:
    	day	number specifies days
    	hour	number specifies hours
    	min	number specifies minutes
    	sec	number specifies seconds
    The default is "12 hour".

    When a file becomes outdated, its timestamp is obtained.  When it differs
    from when the file was last downloaded, the file is downloaded again.  When
    the file changes but doesn't get a new timestamp this will not be noticed.

    When fetching files the cached files are not used (but may be updated).

Prev | Contents | Next
Publishing | Up | Version Control