Next: Miscellaneous Classes Up: Implementation Previous: News Server Class Library

Subsections

News Database

The news database consists of the active database (ActiveDB class), the overview database (OverviewDB class) and the article database (Article class). Figure 6.3 shows the relation between the database classes and the news server class (NServer ). The figure is slightly different to the one presented in section 5.2, because the functionality of the Newsgroup class has been integrated into the NServer class. However, this might change in a future release.

**Abbildung 6.3:** Relation between news server class and the news database
$\begin{figure} \begin{center} \leavevmode \epsfig {file=eps/server_newsdb.eps} \end{center}\end{figure}$

In section 5.5, we have explained that the news database exists in two forms. One type uses non volatile memory and the other one uses volatile memory to store the database. In the current version, we have only implemented the database for non volatile memory. This will result in a slightly slower performance in cases where the volatile database would be sufficient.

The non volatile classes use the Non Volatile Container Class Library to store their data.

Non Volatile Container Class Library

The Non Volatile Container Class Library is the basic building block for the classes storing the news database. It implicitly stores data structures on external memory (usually a file in the local file system). The external memory is mapped into the process's memory space and all data items will be stored there.

NVcontainer Class

The NVcontainer class provides the basic methods for the classes of the Non Volatile Container Class Library . Each file starts with the header explained in table 6.2 and shown in Figure 6.4. This header is used to identify a file generated by an NVContainer class and to check whether the size of the file has been changed by another process. In the latter case, the whole file is remapped with the new size.

Tabelle 6.2: The header of an NVContainer
Header-Entry Description

uint32 hlen Length of this header in Bytes. This entry is used to identify an NVContainer .

uint32 version Version of the NVContainer Library. This entry is used to identify the version of the NVContainer that created this database.

uint64 size Size of the database. This entry matches the size of the file and is used to check, whether another process resized the container.

uint64 mtime Time of last ``logical'' modification. This time has to be set by the user of the NVContainer. Hence the timestamp does not necessarily report all modification to the database.

uint64 freelist Pointer to the list of free memory blocks. Free memory blocks are currently managed by a linked list.

uint64 bytes_free Number of free bytes in the NVContainer . This entry can be used to calculate the fragmentation of the free memory.

uint64 userdata Pointer to the data provided by the NVContainer 's subclasses (e.g., a pointer to the head of the linked list, a pointer to the hash table, etc.).

**Tabelle 6.2:** The header of an *NVContainer*
Header-Entry	Description
uint32 hlen	Length of this header in Bytes. This entry is used to identify an NVContainer .
uint32 version	Version of the NVContainer Library. This entry is used to identify the version of the NVContainer that created this database.
uint64 size	Size of the database. This entry matches the size of the file and is used to check, whether another process resized the container.
uint64 mtime	Time of last ``logical'' modification. This time has to be set by the user of the NVContainer. Hence the timestamp does not necessarily report all modification to the database.
uint64 freelist	Pointer to the list of free memory blocks. Free memory blocks are currently managed by a linked list.
uint64 bytes_free	Number of free bytes in the NVContainer . This entry can be used to calculate the fragmentation of the free memory.
uint64 userdata	Pointer to the data provided by the NVContainer 's subclasses (e.g., a pointer to the head of the linked list, a pointer to the hash table, etc.).

**Abbildung 6.4:** Format of an *NVContainer* 's header
$\begin{figure} \begin{center} \leavevmode \epsfig {file=eps/NV-header.eps} \end{center}\end{figure}$

The NVcontainer class provides methods to lock and unlock the database. In the current version, only the whole database can be locked. These methods are necessary to guarantee mutual exclusion. Whenever the database is accessed it has to be locked by a shared lock. A shared lock allows other processes to access the database, but does not allow to modify the database. An exclusive lock guarantees the exclusive access to the database and allows to modify its contents.

Other methods are provided to open and close an existing database and to set and get the time of the last modification. Table 6.3 gives an overview of the NVcontainer class's methods.

Tabelle 6.3: NVcontainer 's methods

lock(int type, int block)
Depending on the type parameter the database will be locked (exclusive or shared) or unlocked. The block parameter defines whether the system call should block if the request cannot be fulfilled immediately.
get_lock()
returns the process currently holding the lock an the database. If several processes hold a shared lock on the database only one processes is reported.
open(char *dbname, int flags)
generates a new list, whose contents are mapped from the file dbname. Currently, the flags parameter is ignored. In future releases this flag will indicate, whether a new database shall be created.
is_open()
checks whether a database is associated with this NVContainer .
close()
closes the database opened with open().
setmtime(int tm, int force)
sets the modification time of the list to tm, either if the force parameter is true, or if tm is later than the current modification time.
getmtime(int *tm)
stores the modification time of the list in *tm.

A detailed explanation of the design decisions and the implementation of the Non Volatile Container Class Library can be found in [Gsc97].

NVlist Class

The NVlist class provides an abstract class for list manipulations. It provides methods to add and remove data items to a list. Hence, it is used as a base for the NVList and the NVHash classes.

NVList and NVHash Classes

The NVList class is based on the NVlist class and provides all functions necessary for list manipulations. Figure 6.5 shows how the list is organized in non volatile memory. The pointers are stored as file offsets relative to the beginning of the file. The methods provided by the NVList class are shown in table 6.4.

**Abbildung 6.5:** Organization of the *NVList* class
$\begin{figure} \begin{center} \leavevmode \epsfig {file=eps/NVList.eps} \end{center}\end{figure}$

Tabelle 6.4: NVList 's methods

NVList(char *dbname)
creates an NVList and opens the database dbname.
is_empty()
checks whether the list is empty.
prepend(char *data,int szdata)
prepends the data pointed to by data with the size of szdata bytes to the list.
append(char *data,int szdata)
appends the data pointed to by data with the size of szdata bytes to the list.
remove()
deletes the first data item from the list.
clear()
removes all data stored within the NVList .

The NVHash class is also based on the NVlist class and provides all functions necessary for the manipulation of a hash table. Figure 6.6 shows the organization of the hash table in the external memory. The hash table is externally linked. The UserData pointer points to an array of pointers. Each of these pointers points to a list that is organized in the same way as the list of the NVList class is organized. The methods provided by this class are shown in table 6.5.

**Abbildung 6.6:** Organization of the *NVHash* class
$\begin{figure} \begin{center} \leavevmode \epsfig {file=eps/NVHash.eps} \end{center}\end{figure}$

Tabelle 6.5: NVHash 's methods

NVHash(char *dbname, int hashsz)
opens a new NVHash container stored in dbname. If the container does not exist already, a new one is generated with a hash-table size of hashsz.
gethashsz()
returns the currently selected hash-table size.
sethashsz(int hashsz)
sets the size of the hash-table to hashsz. This method is not implemented currently.
clear()
removes all data stored in this container.
is_empty()
checks whether any data are stored within NVHash .

Active Database

The Active Database holds a short summary of the available newsgroups on the news server. The information about a newsgroup is passed to and returned from this database using class GroupInfo . The GroupInfo class stores:

article numbers of the first and last article,
number of articles available in this newsgroup,
type of this newsgroup (e.g. whether the group is public, moderated or read-only)

The active database is stored by the ActiveDB class. ActiveDB is an abstract class providing the interface for any active database. Currently only the Non Volatile Active Database (NVActiveDB) class exists. Other classes mentioned in section 5.5.2 will be implemented in future releases.

Since we only have implemented the NVActiveDB it is used for the RServer class as well. As a result the performance of the RServer class may degrade slightly.

**Abbildung 6.7:** Interaction of the *ActiveDB* class with other classes
$\begin{figure} \begin{center} \leavevmode \epsfig {file=eps/cs_active.eps, scale=1.2} \end{center}\end{figure}$

Figure 6.7 shows the relation of the ActiveDB class to the other classes and the ActiveDB 's interaction with the CServer and RServer class:

1.: A client requests the active database from the CServer using the CServer 's active method.
2.: The CServer class checks whether the contents cached in the ActiveDB class have timed out. This is done using ActiveDB 's getmtime method.
3.: If the data held in the ActiveDB is valid, the CServer proceeds with step 10.
4.: The data stored in the ActiveDB is invalid and the ActiveDB is locked for an update. After the the database has been locked, the CServer class checks, whether its active database stored within the ActiveDB class has already been updated by another process. For simplicity, this is not shown in the figure above.
5.: The CServer class requests the RServer class to retrieve the active database from the news server.
6.: The RServer class contacts the news server and requests the active database.
7.: The news server sends back the active database.
8.: The RServer class notifies the ActiveDB class to read the data sent back by the news server using ActiveDB 's read-method.
9.: The CServer unlocks the active database.
10.: Finally the CServer returns a pointer to the ActiveDB .

The methods implemented by the ActiveDB class are shown in table 6.6.

Tabelle 6.6: ActiveDB 's methods

add(GroupInfo &group)
adds the group specified by the group-parameter and its associated information to the active database.
set(GroupInfo &group)
updates the information stored for the newsgroup group.
get(char *name, GroupInfo *group)
retrieves the information associated with the newsgroup name stores it in the group parameter.
read(istream &is)
reads an active database from the input stream is and stores it in the database.
write(ostream &os)
writes the active database to the output stream os.

Overview Database

The Overview Database stores information for each article of a newsgroup: the subject and author of the article, the date the article was posted, the article's identifier and size and to which other newsgroups the article was posted.

Currently only the non volatile version of the overview database (NVOverviewDB ) has been implemented. Hence this class will be used for the RServer class as well. As is the case for the active database, this may degrade the RServer class's performance slightly.

**Abbildung 6.8:** Interaction of the *OverviewDB* class with other classes
$\begin{figure} \begin{center} \leavevmode \epsfig {file=eps/cs_overview.eps, scale=1.2} \end{center}\end{figure}$

Figure 6.8 shows the interaction of the overview database (OverDB ) with the other classes.

1.: A client requests the overview database using the CServer class's xover-method.
2.: The CServer class checks, whether the data stored for the currently selected newsgroup have timed out. These data are stored in the GroupInfo class that also stores the number of the first and last article and the total number of articles of the currently selected newsgroup.
3.: If the overview database is still valid, the CServer class proceeds with step 13.
4.: The CServer locks the overview database (OverviewDB ) for an update.
5.: The CServer requests the RServer class to transfer the overview database from the news server.
6.: The RServer class connects to the news server and selects the appropriate newsgroup.
7.: The news server returns some information about the current newsgroup. This information consists of the number of the first, the last article and the total number of articles.
8.: The RServer passes the information received by the news server to the GroupInfo class. The GroupInfo class stores this information and updates its modification time.
9.: The RServer class requests the overview records for newly arrived articles from the news server.
10.: The news server sends back the requested overview records.
11.: The RServer informs the OverviewDB class to read and store the data returned by the news server.
12.: The CServer unlocks the overview database.
13.: Finally the CServer returns a pointer to the overview database.

The NVOverviewDB is based on the NVList class and allows to call the inherited methods lock(), get_lock(), open(), is_open(), close(), setmtime(), getmtime(), clear() and is_empty(). In addition to these methods, the NVOverviewDB provides the methods shown in table 6.7.

**Tabelle 6.7:** *OverviewDB* 's methods
$\begin{table} \begin{center} \leavevmode \begin{description} \item[select(char... ...ord is stored in the overview database.\end{description} \end{center}\end{table}$

Article Database

Currently each article is stored using its own file. We have implemented a class Article that stores a single news article. This class implements the methods necessary to store and read an article to/from the file system. The Article class is implemented using a Text class that allows to allocate strings of unlimited length. We did not use the String class provided by the GNU C++ library, since this class limits a string to approximately 35000 characters.

Next: Miscellaneous Classes Up: Implementation Previous: News Server Class Library

gschwind@infosys.tuwien.ac.at