NEWS-RECOVERY(8)

NAME
makeactive, makehistory, newsrequeue - tools to recover Usenet databases

SYNOPSIS
makeactive [ -m ] [ -o ] makehistory [ -b ] [ -f filename ] [ -i ] [ -n ] [ -o ] [ -r ] [ -s size ] [ -T tmpdir ] [ -u [ -v ] ] newsrequeue [ -a active ] [ -h history ] [ -d days ] [ -l ] [ -n newsfeeds ] [ input ]

DESCRIPTION
Makeactive invokes find(1) to get a list of all directories in the news spool tree, /var/spool/news. It discards directories named lost+found as well as those that have a period in them. It scans all other directories for all-numeric filenames and determines the highest and lowest number. The program's output is a set of active(5) file lines. Because there is no way to know if a group is moderated or disabled, the fourth field of all entries will be y. Also, mid-level directories that aren't newsgroups will also created as newsgroups with no entries (for example, there is a ``comp.sources.unix'' group, but no ``comp.sources'').

If the ``-o'' flag is used, makeactive will read an existing active file for the list of group names and just renumber all groups. It will preserve the fourth field of the active file if one is present. This is analogous to the ctlinnd(8) ``renumber'' command, except that innd(8) should throttled or not running. Do not use this flag with output redirected to the standard active file!

If the ``-m'' flag is given, then makeactive will attempt to adjust the highest and lowest article numbers wherever possible. If articles are found in a newsgroup, the numbers will reflect what what was found. If no articles are found in a newsgroup, the high number from the old file will be kept, and the low number will be set to one more then the high number. This flag may only be used if the ``-o'' flag is used.

Makeactive exits with non-zero status if any problems occurred. A typical way to use the program is with the following /bin/sh commands:
    ctlinnd throttle "Rebuilding active file"
    TEMP=${TMPDIR-/tmp}/act$$
    if [ -f /usr/lib/news/active ] ; then
        if makeactive -o >${TEMP} ; then
            mv ${TEMP} /usr/lib/news/active
        fi
    else
        if makeactive >${TEMP} ; then
            # Edit to restore moderated
            # and aliased groups.
            ...
            mv ${TEMP} /usr/lib/news/active
        fi
    fi
    ctlinnd reload active "New active file"
    ctlinnd go ''

Makehistory rebuilds the history(5) text file and the associated dbz(3) database. The default name of the text file is /usr/lib/news/history; to specify a different name, use the ``-f'' flag. Makehistory scans the active(5) file to determine which newsgroup directories within the spool directory, /var/spool/news, should be scanned. (If a group is removed, but its spool directory still exists, makehistory will ignore it.) The program reads each file found and writes a history line for it. If the ``-b'' flag is used, then makehistory will remove any articles that do not have valid Message-ID headers in them.

After the text file is written, makehistory will build the dbz database. If the ``-f'' flag is used, then the database files are named file.dir and file.pag. If the ``-f'' flag is not used, then a temporary link to the name history.n is made and the database files are written as history.n.pag and history.n.dir. If the ``-o'' flag is used, then the link is not made and any existing history files are overwritten. If the old database exists, makehistory will use it to determine the size of the new database. To ignore the old database use the ``-i'' flag. Using the ``-o'' flag implies the ``-i'' flag. The program will also ignore any old database if the ``-s'' flag is used to specify the approximate number of entries in the new database. Accurately specifying the size is an optimization that will create a more efficient database. (The size should be the estimated eventual size of the file, typically the size of the old file.) For more information, see the discussion of dbzfresh and dbzsize in dbz(3).

If the ``-u'' flag is given, then makehistory assumes that innd is running. It will pause the server while scanning, and then send ``addhist'' commands (see ctlinnd(8)) to the server for any article that is not found in the dbz database. The command ``makehistory -bu'' is useful after a system crash, to delete any mangled articles and bring the article database back into a more consistent state. If the ``-v'' flag is used with the ``-u'' flag, then makehistory will put a copy of all added lines on its standard output.

To scan the spool directory without rebuilding the dbz files, use the ``-n'' flag. If used with ``-u'', the server will not be paused while scanning. To just build the dbz files from an existing text file, use the ``-r'' flag. The ``-i'' or ``-s'' flags can be useful if there are no valid dbz files to use. A typical way to use this program is with the following /bin/sh commands:
    ctlinnd throttle "Rebuilding history file"
    cd /usr/lib/news
    if makehistory -n -f history.n ; then
        :
    else
        echo Error creating history file!
        exit 1
    fi
    # The following line can be used to retain expired history
    # It is not necessary for the history file to be sorted.
    # awk 'NF==2 { print; }' <history >>history.n
    # View history file for mistakes.
    if makehistory -r -s `wc -l <history` -f history.n; then
        mv history.n history
        mv history.n.dir history.dir
        mv history.n.pag history.pag
    fi
    ctlinnd go ''

Makehistory needs to create a temporary file that contains one line for each article it finds, which can become very large. This file is created in the /tmp directory. The ``TMPDIR'' environment variable may be used to specify a different directory. Alternatively, the ``-T'' flag may be used to specify a temporary directory. In addition, the sort(1) that is invoked during the build writes large temporary files (often to /var/tmp but see your system manpages). If the ``-T'' flag is used, then the flag and its value will be passed to sort. On most systems this will change the temporary directory that sort uses. if used, this flag and its value will be passed on to the sort(1) command that is invoked during the build.

Makehistory does not handle symbolic links. If the news spool area is split across multiple partitions, the following commands should probably be run before the database is regenerated:
    cd /var/spool/news
    find . -type l -name '[1-9]*' -print | xargs -t rm
Make sure to run the command on all the appropriate partitions!

Newsrequeue can be used to rewrite batchfiles after a system crash. It operates in two modes. In the first mode, it first reads the active and newsfeeds(5) files to determine where the different newsgroups are to be distributed. To specify alternate locations for these files, use the ``-a'' or ``-n'' flags. It then opens the history database. To specify a different file, use the ``-h'' flag

Once the files are opened, newsrequeue reads from the specified input file, or standard input if no file is specified. Each line should have a single Message-ID, surrounded in angle brackets; any other text on the line is ignored. For example, the history file (or trailing subset of it) is acceptable input to the program operating in this mode. If the ``-d'' flag is used, then only articles that were received within the specified number of days will be processed.

Newsrequeue uses the first two fields of the newsfeed entry — the sitename and the excludes field, and the patterns and distribs field. It ignores all flags in the third field except for the ``N'' field, and also ignores the fourth field altogether.

The second mode is used if the ``-l'' flag is used. In this mode, it reads from the specified input file, or standard input if no file is specified. Each line should look like an innd log entry. It parses entries for accepted articles, looks up the Message-ID in the history database to get the filename, and then scans the list of sites.

In either mode, the output of newsrequeue consists of one line for each article that should be forwarded. Each such line contains the Message-ID, the filename, and the list of sites that should receive the article. The output is suitable for piping into filechan(8).

HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews.

SEE ALSO
active(5), ctlinnd(8), dbz(3), filechan(8), history(5), innd(8), newsfeeds(5).