$Id: DEVELOPERS,v 1.4 2002/09/09 07:04:24 kingofgib Exp $

LIBPNET6, A Portable Network Library
Copyright (c) 2002, Peter Bozarov

Contact through: libpnet6(at)bozz(dot)demon(dot)nl

CONTENTS:

 1. Introduction
 2. Conceptual Overview
    2.1 Portability
    2.2 Intuitive API
 3. The API
 4. Working on libpnet6
    4.1 Porting
    4.2 Adding new functionality
    4.3 Coding standards
 5. Suggestions and Improvements



Introduction
============

This file is intended for people that want to help out with libpnet6
development.


Conceptual Overview
===================

libpnet6 is a small library designed to simplify the writing of
networked applications. To implement this, two concepts have guided
libpnet6 design and development from day one: platform independence
and ease-of-use.  Let's look at each in turn.

	Portability
	-----------

libpnet6 is a OS independent library, which means that our goal is to
provide a consistent API that will compile on any system and that
provides identical functionality regardless of the OS on which it is
compiled. If you've ever tried to write portable network code, you
would have found that althought the basic API is fairly identical on
most platforms, once you get to the more advanced stuff, things just
aren't that easy anymore. For example, functions might take sligthly
different arguments, certain structs might be missing, or defined in
different places, or some of their members might be missing or have
other names; constants might not be defined, or defined in different
places.

It is these difficulties that libpnet6 tries to reconcile. The
libpnet6 API is totally OS independent, and provides the same
functionality, regardless of OS platform (as long as there's a way on
that platform to implement the desired functionality in the first
place). You don't need to worry about the numerous little quirks that
a particular platform might have; libpnet6 takes care of that. If you
do want to worry about the quirks, then you should not be using
libpnet6, but working on it! (Which is why you're reading this.)

	Intuitive API
	-------------

I'm probably not the first to notice that the standard sockets API is
not very intuitive. It is also very low-level, which is fine if you
know what you're doing, but is a nightmare for beginners (and even for
more seasoned programmers). On top of this, there are too many
differences between the IPv4 and IPv6 protocols, which adds another
layer of confusion-fraught complexity.

To account for this, and make writing networked applications
significantly easier, libpnet6's API has been designed to provide an
intuitive and simplified version of the standard network API, but,
without sacrifice of functionality or features. libpnet6 lets you do
whatever you know you can do with the network code, only it makes it
simpler and easier to write (not to mention fully portable).

The API
=======

I've identified three main areas of interest to network programmers:

    - The standard client/server paradigm, with all its subfeatures,
      such as multithreading, forking, synchronization, address
      resolution, non-blocking IO, and what may. This is the primary 
      raison d'etre for libpnet6. Code for this is contained in 
      src. 
    - Accessing network interfaces and reading their configuration.
      This is not strictly required, although a lot of servers do need
      to know about the interfaces present on the system. This part
      is (very) OS dependent. The code is in src/if.
    - Direct link-layer access. This part ended up in the library,
      because I wanted to make libpnet6 fairly self-contained. Though
      not often, accessing the link-layer is arguably part of writing
      (advanced) networked applications, and as such, it should be
      provided for. The code is in src/pkt.

Working on libpnet6
===================

Working on libpnet6 can happen on two fronts: (a) porting libpnet6 to
a new and exotic OS, and (b) adding new and exotic functionality to
the existing code base (and making sure it's portable accross the full
plethora of platforms we (intend to) support).

	Porting
	-------

Porting software is a multi-faceted activity. There's three things one
needs to worry about: compiler quirks, make quirks, and of course, all
software related quirks.

	The compiler: using a Make.header file

libpnet6 deals with compiler quirks by employing a complicated make
procedure, in which the actual Makefile consist of a header, body, and
a trailed. The header is the only OS-dependent part. Each directory
contains a Makefile that includes the header and the trailer, and that
defines the body. In general, when porting to a new OS, only an
appropriate make header file should be created. Header files are
contained in the directory files/ of the libpnet6 distribution.

For example, I'm porting libpnet6 to Tru64. First, I need to figure
out what the default (i.e. the one that is invoked by 'cc') compiler
is like on the given machine: i.e. what are the optimization flags,
the warning flags, and in general other funky things that are
non-standard, but might be required in order to compile the code
properly. For example, I have found out that the compiler's name is
cc, and -O2 is the optimization flag, and there's a bunch of other
flags that might be useful in order to compile. In a file called
Make.header.tru64 located in the files/ directory I define the
following:

CL              =  cc -std1 -trapuv -readonly_strings -portable
CCOPTFLAGS      = -O2
CCDEBUGFLAGS    = -g
CCWARNFLAGS     = -warnprotos

CCEXTRAFLAGS    = -DPNET_TRU64

Note that I use CL, not CC.

That last line makes it possible to identify the platfrom from inside
libpnet6 source code (We can use predefined macros such as __OSF__,
but what's the point, we _know_ we're on Tru64 anywhay.)

Then I tell make where all the stuff is going (this part is OS
independent, but needs to be contained in each header file, I guess
due to poor design on my part...):

OUTPUTDIR       = $(DEVROOT)/lib
EXEOUTPUTDIR    = $(DEVROOT)/bin
LIBDIR          = $(DEVROOT)/lib
INCLUDEDIR      = $(DEVROOT)/src

LINKFLAGS       = -L $(LIBDIR)
INCLUDEFLAGS    = -I $(INCLUDEDIR) -I . -I ..

(The variable DEVROOT is defined as the first thing in each Makefile,
and points to the root of the libpnet6 distribution. Normally it has a
value such as .. or ../.. or ../../.. depending on how deep a given
directory is.)

Now, we need to know how to link code (useful for the examples
contained in the tests/ directory):

LINK            = cc -pthread
LINKOUT         = -o ./
LINKOPTS        = $(LLIBS)
STATIC          = -static

and how to build the libary itself:

AR              = ar
AROPTS          = ruv ./
LIBSUFFIX       = a
OBJSUFFIX       = o
EXESUFFIX       = .tru64

Note, that AROPTS and LINKOUT have a ' ./': this makes it possible to
use macros in the trailer file such as

$(LINKOUT)$(EXEOUTPUTDIR)/$(OUT)$(EXESUFFIX)

(which will expand to '-o ./../../bin/somefile.tru64'

and

$(AROPTS)$(OUTPUTDIR)/$(OUT).$(LIBSUFFIX)

(which expands to 'ruv ./../lib/libpnet6.a'

because some braindead linkers require that the output flag be
concatenated on to the output value itself, while other don't like
that.

Finally, we add some system specific flags:

# System specific link libs
PTHREADLIBS     = -lpthreads
NETLIBS         =
SYSLIBS         = -lm $(PTHREADLIBS)

.SUFFIXES:      .e .o

(NETLIBS is empty here, but Solaris for example requires '-lsocket -lnsl'.)

All in all, this is quite ugly, but, by guided by other
Make.header.<os> files contained in the files directory, you should be
able to come up with a good Make.header file for your OS. And, if you
should managed to devise a way to make this easier or simpler, you're
welcome to do it!

	Make quirks: use only basic make commands

Don't rely on make specific flags, and other make add-ons that are in
general non-standard. It's better to have a long, unwieldy Makefile
that works everywhere, than to have a short, compact, PhuNkY Makefile
that only works on a couple of platforms.

	Software related quirks: use tools/config.c

This file contains code to figure stuff out on each system (I guess I
can use autoconf or some such tool, but this was fun to make). By
typing 'make config' it the root of the libpnet6 distribution,
tools/config.c is compiled, exectuted and a config.h file is created
and put into the src directory. The config.h file contains defines
indicating whether certain files, structs, struct members and
functions are present on the given system.

Let's say you need to test for the presence of a file on a system, say
"/usr/include/net/if_dl.h". Find the 'const char * files[] =' list and
add your file to it, without the '/usr/include/' part,
i.e. "net/if_dl.h". Now, when you type 'make config' in the root of
the libpnet6 distribution, config will look for the file and put a 
#define HAVE_NET_IF_DL_H 1 if it should find the file, else it outputs
nothing.  If you need to test for a file that's not
in /usr/include, let's say, /dev/bpf0, you need to use
file_exists(fname,flags). Here, flags can be FLAG_W (tests if file is
writable), FLAG_R (readable), FLAG_X (executable), FLAG_D (directory),
or a combination, such as FLAG_R | FLAG_D (readable and a directory).
If all you need to know is whether a file exists, a flag of 0 will
suffice.

If you need to test for the presence of a struct, such as 'struct
ipv6_mreq', then you make an entry into the structList[] array. This
struct takes an integer, the struct's name (without the
'struct' part), and a maximum of three include files which provide for
this struct. For ipv6_mreq, we enter

{ 0, "ipv6_mreq", { "sys/types.h", "netinet/in.h", NULL } },

The first int, indicates whether config should fail if the given
struct does not exist. 1 means fail, 0 means keep going. In this
case, we keep going, since a missing ipv6_mreq is not fatal for
libpnet6.

Note: you need to terminate the include list by a NULL. This
is true for all such lists described here.

If the struct is found, config defines

# define HAVE_STRUCT_IPV6_MREQ	1

The next thing we can test for is whether a struct has a specific
member, for example, whether 'struct sockaddr_in6' has a
'sin6_scope_id' member. We add to the structMemList[] an entry of the
following form:

{ 0, "sockaddr_in6", "sin6_scope_id",
        { "sys/types.h", "netinet/in.h", NULL                   } },

Here, the leading int has the same function as with structs, that is
fail or keep going when the given member does not exist. The rest is
fairly obvious I hope.

The last thing, checking whether specific functions are present is
kinda tricky. It can happen that a function exists, but is unusable.
To account for that, we need to check not only the function's name,
but also its arguments, as well as a number of include files that are
required in order to properly use the function. A famous example is
the sysctl() function. FreeBSD and others use it to fetch network
interface information. Linux has it too, but we can't use it in the
same way as on BSD. So we put lots of includes that are present on BSD
and are needed for the proper use of 'sysctl()' (which can be much
more than the files required to simply _declare_ the function). So for
sysctl() we do

{ 0, "sysctl", 6,
    { "sys/types.h","sys/socket.h",
      "net/if_types.h", "sys/param.h","net/route.h",
      "net/if_dl.h", "sys/sysctl.h", NULL } },

This tells us, don't fail if not found, the function's name, the
number of arguments it takes, and a list of include files that are
required for its use. 

	Adding new functionality
	------------------------

In general, this boils down to figuring out how you need to implement
something, and then adding appropriate entries in
tools/config.c. Then, you can code it. But consult with me prior to
doing this, so we can make it fit into the general libpnet6 paradigm.

	Coding standards
	----------------

It's important that everyone adheres to the same coding style. Here's
brief description of how to write libpnet6 code.

- use 4 spaces as indent, but keep the tab length set to 8;
- place opening braces on a new line;
- put functions return value on a separate line;

const char *
func( int a, const char * b )
{
    if ( a > 0 )
	return "A";
    return "B";
}

- Read throught the file stdinc.h for details on how to write debug
statements
- Finally, all functions return -1 on failure and 0 on success. If you
need to return a more detailed status, do it via an argument, e.g.

int
func_with_error_status( normal arguments here, int * pErrorStatus )
{
    if ( some_error signaled )
    {
	if ( pErrorStatus )
	    *pErrorStatus = some_error;
	return -1;
    }

    return 0;
}

Of course, for functions that return a pointer, return NULL on error,
and a valid pointer on success.

Suggestions and Improvements
============================

I hacked the general libpnet6 concept and porting mechanism on my own,
without consulting more intelligent and knowlegeable people that
myself. Consequently, although it might work, it's probably not the
best way to do this. So, if you know any better, please share your
views with me.

Good luck, and have fun coding!

Peter

Aug 26, 2002
