News Database

  The Non Volatile Container Class Library and the local file system will be used to store the news database. The Non Volatile Container Class Library has been implemented, because no suitable library currently exists, that stores its data structures on non volatile memory. Special care has been taken to design this library as general as possible. The Non Volatile Container Class Library is used to store the active database and the overview database. To store news articles the local filesystem is used.

Non Volatile Container Class Library

The Non Volatile Container Class Library (NVContainer Class Library) provides a set of containers storing their data structures on non volatile (external) memory (e.g., on the hard disk).

Since the NVContainer classes store their data on external memory, they reduce the virtual memory used by a process using this library. In addition this approach allows that the database provided by an NVContainer class can be used by several processes concurrently. Hence an NVContainer class can be used for the provision of ``shared memory'' too.

The use of a memory mapped file has the following advantages and disadvantages:

+

The containers contents need not be written to or read from the file explicitly.

+

The process does not consume virtual memory for the data mapped from the file. Figure 5.9 shows this on a SunOS system for a process having allocated 5 Megabytes in an NVList. The SZ column indicates the total amount of memory used for the data and stack segments in Kilobytes and the RSS column indicates the total amount of main memory used by the process.

+

If a list should be shared by several processes they will use the same file for their memory. The file's contents can be shared by several processes.

-

If the file size changes the memory region has to be unmapped and remapped again, since the size of the mapped region must be provided with the mmap system-call. This might decrease the performance slightly.

  

Abbildung 5.9: The virtual memory usage of a process with an NVList of 5 Megabytes

The NVContainer class consists of an abstract parent class (NVcontainer ) providing the basic functionality necessary in all its subclasses. The NVcontainer manages the memory allocation and deallocation of the non volatile memory and provides functions for locking and unlocking of the external memory. The latter is necessary to ensure mutual exclusion between different processes that access the database provided by an NVContainer class simultaneously.

The current version of the NVContainer Class Library provides a Non Volatile List (NVList) and a Non Volatile Hash (NVHash) class. Both are subclasses of the abstract class NVlist . NVlist is a direct subclass of NVcontainer and implements basic list operations. In future versions of the NVContainer Class Library we will also provide classes storing their data using trees, arrays and other data structures. Each container class comes with an iterator that allows to access the data stored within the class. Figure 5.10 shows the hierarchy of the classes provided by the NVContainer Class Library . Classes being planned for future releases (e.g., NVtree ) are shown using gray color.

  

Abbildung 5.10: Class hierarchy of the Non Volatile Container classes

The NVList Class and its Iterator

This class uses a list to store its data. Whenever an NVList class is instantiated it creates a new file to store its data or if the file already exists it uses the contents stored in this file. The NVListIter class is used to iterate over an NVList class.

The NVHash Class and its Iterator

This class uses a hash table to store its data. Whenever an NVHash class is instantiated a new file to hold the class's data is created. If the file already exists the contents of this file will be used for the hash table. The NVHashIter class is used to iterate over an NVHash class.

Active Database

  The Active Database consists of several classes. Its classes and their relationship are shown in Figure 5.11. The abstract base class ActiveDB defines only the interface to the active database. This is done using virtual methods to allow the exploitation of polymorphism. VActiveDB and NVActiveDB are subclasses of ActiveDB .

  

Abbildung 5.11: Class hierarchy for the active database

VActiveDB implements an active database stored on volatile memory. It is used for the RServer class, because this class does not need to store its databases on external memory.

NVActiveDB implements an active database stored on non volatile memory. To offer this the NVActiveDB inherits the functionality provided by the NVHash class. The class is used for the news server classes that need to maintain a local news database. These are the LServer and CServer classes.

Overview Database

  The overview database holds a short summary for each article. The summary consists of the date, the subject, the author, the article id, the number of lines of the article, and other data depending on the news server.

The overview database has been introduced to allow news readers a faster access to these data. Without this extension the news reader would have to retrieve the article header for each article, which increases the required network bandwidth and the time necessary to prepare the listing of available articles.

The overview database uses the same design as the active database. It consists of an abstract parent class (OverviewDB ) defining the interface and the subclasses VOverviewDB and NVOverviewDB .

  

Abbildung 5.12: Class hierarchy for the overview database

Article Database

  The article database is stored in the same format used by INN [Sal92] and CNews [CS]. Each newsgroup has its own directory named after the group name. The articles of the newsgroup are stored in its directory and each article is stored in its own file. For example, the article 567 in newsgroup linux.dev.kernel will be stored in linux/dev/kernel/a567.

We adopted this scheme for the News Cache, because we did not want to use a relational database to store articles on the one hand. This has been explained in section 4.4.1. And on the other hand the News Cache does not store as many articles as a news server. Nevertheless, the format of the article database may change in future versions of the News Cache.