This is son1036.txt in view mode; [Download] [Up]
INTERNET DRAFT to be NEWS sec. -
News Article Format and Transmission
Henry Spencer
Status of this Memo
This document is intended to become an Internet Draft.
Internet Drafts are working documents of the Internet Engi-
neering Task Force (IETF), its Areas, and its Working
Groups. Note that other groups may also distribute working
documents as Internet Drafts.
Internet Drafts are draft documents valid for a maximum of
six months. Internet Drafts may be updated, replaced, or
obsoleted by other documents at any time. It is not appro-
priate to use Internet Drafts as reference material or to
cite them other than as a "working draft" or "work in
progress".
Please check the I-D abstract listing contained in each
Internet Draft directory to learn the current status of this
or any other Internet Draft. (Actually, this draft is at
too early a stage to even be listed there yet.)
It is hoped that a later version of this Draft will obsolete
RFC 1036 and will become an Internet standard.
References to the "successor to this Draft" refer not to
later versions of this draft, but to a hypothetical future
rewrite of this Draft (in the same way that this Draft is a
rewrite of RFC 1036).
Distribution of this memo is unlimited.
Abstract
This Draft defines the format and procedures for interchange
of network news articles. It is hoped that a later version
of this Draft will obsolete RFC 1036, reflecting more recent
experience and accommodating future directions.
Network news articles resemble mail messages but are broad-
cast to potentially-large audiences, using a flooding algo-
rithm that propagates one copy to each interested host (or
group thereof), typically stores only one copy per host, and
does not require any central administration or systematic
registration of interested users. Network news originated
as the medium of communication for Usenet, circa 1980.
17 Jan 1994 - 1 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. -
Since then Usenet has grown explosively, and many Internet
sites participate in it. In addition, the news technology
is now in widespread use for other purposes, on the Internet
and elsewhere.
This Draft primarily codifies and organizes existing prac-
tice. A few small extensions have been added in an attempt
to solve problems that are considered serious. Major exten-
sions (e.g. cryptographic authentication) that need signifi-
cant development effort are left to be undertaken as inde-
pendent efforts.
Table of Contents
TBW
1. Introduction
Network news articles resemble mail messages but are broad-
cast to potentially-large audiences, using a flooding algo-
rithm that propagates one copy to each interested host (or
groups thereof), typically stores only one copy per host,
and does not require any central administration or system-
atic registration of interested users. Network news origi-
nated as the medium of communication for Usenet, circa 1980.
Since then Usenet has grown explosively, and many Internet
sites participate in it. In addition, the news technology
is now in widespread use for other purposes, on the Internet
and elsewhere.
The earliest news interchange used the so-called "A News"
article format. Shortly thereafter, an article format
vaguely resembling Internet mail was devised and used
briefly. Both of those formats are completely obsolete;
they are documented in appendix A for historical reasons
only. With publication of RFC 850 [rrr] in 1983, news arti-
cles came to closely resemble Internet mail messages, with
some restrictions and some additional headers. RFC 1036
[rrr] in 1987 updated RFC 850 without making major changes.
In the intervening five years, the RFC 1036 article format
has proven quite satisfactory, although minor extensions
appear desirable to match recent developments in areas such
as multi-media mail. RFC 1036 itself has not proven quite
so satisfactory. It is often rather vague and does not
address some issues at all; this has caused significant
interoperability problems at times, and implementations have
diverged somewhat. Worse, although it was intended primar-
ily to document existing practice, it did not precisely
match existing practice even at the time it was published,
and the deviations have grown since.
17 Jan 1994 - 2 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 1
This Draft attempts to specify the format of articles, and
the procedures used to exchange them and process them, in
sufficient detail to allow full interoperability. In addi-
tion, some tentative suggestions are made about directions
for future development, in an attempt to avert unnecessary
divergence and consequent loss of interoperability. Major
extensions (e.g. cryptographic authentication) that need
significant development effort are left to be undertaken as
independent efforts.
NOTE: One question this all may raise is: why is
there no News-Version header, analogous to MIME-
Version, specifying a version number corresponding
to this specification? The answer is: it doesn't
appear to be useful, given news's backward-
compatibility constraints. The major use of a
version number is indicating which of several
INCOMPATIBLE interpretations is relevant. The
impossibility of orchestrating any sort of simul-
taneous change over news's installed base makes it
necessary to avoid such incompatible changes (as
opposed to extensions) entirely. MIME has a ver-
sion number mostly because it introduced incompat-
ible changes to the interpretation of several
"Content-" headers. This Draft attempts no
changes in interpretation and it appears doubtful
that future Drafts will find it feasible to intro-
duce any.
UNRESOLVED ISSUE: Should this be reconsidered?
Only if the header has SPECIFIC IDENTIFIABLE uses
today. Otherwise it's just useless added bulk.
As in this Draft's predecessors, the exact means used to
transmit articles from one host to another is not specified.
NNTP [rrr] is probably the most common transmission method
on the Internet, but a number of others are known to be in
use, including the UUCP protocol [rrr] extensively used in
the early days of Usenet and still much used on its fringes
today.
Several of the mechanisms described in this Draft may seem
somewhat strange or even bizarre at first reading. As with
Internet mail, there is no reasonable possibility of updat-
ing the entire installed base of news software promptly, so
interoperability with old software is crucial and will
remain so. Compatibility with existing practice and robust-
ness in an imperfect world necessarily take priority over
17 Jan 1994 - 3 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 1
elegance.
2. Definitions, Notations, and Conventions
2.1. Textual Notations
Throughout this Draft, "MAIL" is short for "RFC 822 [rrr] as
amended by RFC 1123 [rrr]". (RFC 1123's amendments are
mostly relatively small, but they are not insignificant.)
See also the discussion in section 3 about this Draft's
relationship to MAIL. "MIME" is short for "RFCs 1341 and
1342" (or their updated replacements).
UNRESOLVED ISSUE: Update these numbers.
"ASCII" is short for "the ANSI X3.4 character set" [rrr].
While "ASCII" is often misused to refer to various character
sets somewhat similar to X3.4, in this Draft, "ASCII" means
X3.4 and only X3.4.
NOTE: The name is traditional (to the point where
the ANSI standard sanctions it) even though it is
no longer an acronym for the name of the standard.
NOTE: ASCII, X3.4, contains 128 characters, not
all of them printable. Character sets with more
characters are not ASCII, although they may
include it as a subset.
Certain words used to define the significance of individual
requirements are capitalized. "MUST" means that the item is
an absolute requirement of the specification. "SHOULD"
means that the item is a strong recommendation: there may be
valid reasons to ignore it in unusual circumstances, but
this should be done only after careful study of the full
implications and a firm conclusion that it is necessary,
because there are serious disadvantages to doing so. "MAY"
means that the item is truly optional, and implementors and
users are warned that conformance is possible but not to be
relied on.
The term "compliant", applied to implementations etc., indi-
cates satisfaction of all relevant "MUST" and "SHOULD"
requirements. The term "conditionally compliant" indicates
satisfaction of all relevant "MUST" requirements but viola-
tion of at least one relevant "SHOULD" requirement.
This Draft contains explanatory notes using the following
format. These may be skipped by persons interested solely
in the content of the specification. The purpose of the
notes is to explain why choices were made, to place them in
context, or to suggest possible implementation techniques.
17 Jan 1994 - 4 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 2.1
NOTE: While such explanatory notes may seem super-
fluous in principle, they often help the less-
than-omniscient reader grasp the purpose of the
specification and the constraints involved. Given
the limitations of natural language for descrip-
tive purposes, this improves the probability that
implementors and users will understand the true
intent of the specification in cases where the
wording is not entirely clear.
All numeric values are given in decimal unless otherwise
indicated. Octets are assumed to be unsigned values for
this purpose. Large numbers are written using the North
American convention, in which "," separates groups of three
digits but otherwise has no significance.
2.2. Syntax Notation
Although the mechanisms specified in this Draft are all
described in prose, most are also described formally in the
modified BNF notation of RFC 822. Implementors will need to
be familiar with this notation to fully understand this
specification, and are referred to RFC 822 for a complete
explanation of the modified BNF notation. Here is a brief
illustrative example:
sentence = clause *( punct clause ) "."
punct = ":" / ";"
clause = 1*word [ "(" clause ")" / "," 1*word ]
word = <any English word>
This defines a sentence as some clauses separated by puncts
and ended by a period, a punct as a colon or semicolon, a
clause as at least one <word> optionally followed by either
a parenthesized clause or a comma and at least one more
<word>, and a <word> as (informally) any English word. <>
are used to enclose names when (and only when) distinguish-
ing them from surrounding text is useful. The full form of
the repetition notation is <m>"*"<n><thing>, denoting <m>
through <n> repetitions of <thing>; <m> defaults to zero,
<n> to infinity, and the "*" and <n> can be omitted if <m>
and <n> are equal, so 1*word is one or more words, 1*5word
is one through five words, and 2word is exactly two words.
The character "\" is not special in any way in this nota-
tion.
This Draft is intended to be self-contained; all syntax
rules used in it are defined within it, and a rule with the
same name as one found in MAIL does not necessarily have the
same definition. The lexical layer of MAIL is NOT, repeat
NOT, used in this Draft, and its presence must not be
assumed; notably, this Draft spells out all places where
17 Jan 1994 - 5 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 2.2
white space is permitted/required and all places where con-
structs resembling MAIL comments can occur.
NOTE: News parsers historically have been much
less permissive than MAIL parsers.
2.3. Definitions
The term "character set", wherever it is used in this Draft,
refers to a coded character set, in the sense of ISO charac-
ter set standardization work, and must not be misinterpreted
as meaning merely "a set of characters".
In this Draft, ASCII character 32 is referred to as "blank";
the word "space" has a more generic meaning.
An "article" is the unit of news, analogous to a MAIL "mes-
sage".
A "poster" is a human being (or software equivalent) submit-
ting a possibly-compliant article to be "posted": made
available for reading on all relevant hosts. A "posting
agent" is software that assists posters to prepare articles,
including determining whether the final article is compli-
ant, passing it on to a relayer for posting if so, and
returning it to the poster with an explanation if not. A
"relayer" is software which receives allegedly-compliant
articles from posting agents and/or other relayers, files
copies in a "news database", and possibly passes copies on
to other relayers.
NOTE: While the same software may well function
both as a relayer and as part of a posting agent,
the two functions are distinct and should not be
confused. The posting agent's purpose is (in
part) to validate an article, supply header infor-
mation that can or should be supplied automati-
cally, and generally take reasonable actions in an
attempt to transform the poster's submission into
a compliant article. The relayer's purpose is to
move already-compliant articles around efficiently
without damaging them.
A "reader" is a human being reading news articles. A "read-
ing agent" is software which presents articles to a reader.
NOTE: Informal usage often uses "reader" for both
these meanings, but this introduces considerable
potential for confusion and misunderstanding, so
this Draft takes care to make the distinction.
A "newsgroup" is a single news forum, a logical bulletin
board, having a name and nominally intended for articles on
17 Jan 1994 - 6 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 2.3
a specific topic. An article is "posted to" a single news-
group or several newsgroups. When an article is posted to
more than one newsgroup, it is said to be "cross-posted";
note that this differs from posting the same text as part of
each of several articles, one per newsgroup. A "hierarchy"
is the set of all newsgroups whose names share a first com-
ponent (see the name syntax in section 5.5).
A newsgroup may be "moderated", in which case submissions
are not posted directly, but mailed to a "moderator" for
consideration and possible posting. Moderators are typi-
cally human but may be implemented partially or entirely in
software.
A "followup" is an article containing a response to the con-
tents of an earlier article (the followup's "precursor"). A
"followup agent" is a combination of reading agent and post-
ing agent that aids in the preparation and posting of a fol-
lowup.
Text comparisons are "case-sensitive" if they consider
uppercase letters (e.g. "A") different from lowercase let-
ters (e.g. "a"), and "case-insensitive" if letters differing
only in case (e.g. "A" and "a") are considered identical.
Categories of text are said to be case-(in)sensitive if com-
parisons of such texts to others are case-(in)sensitive.
A "cooperating subnet" is a set of news-exchanging hosts
which is sufficiently well-coordinated (typically via a cen-
tral administration of some sort) that stronger assumptions
can be made about hosts in the set than about news hosts in
general. This is typically used to relax restrictions which
are otherwise required for worst-case interoperability; mem-
bers of a cooperating subnet MAY interchange articles that
do not conform to this Draft's specifications, provided all
members have agreed to this and provided the articles are
not permitted to leak out of the subnet. The word "subnet"
is used to emphasize that a cooperating subnet is typically
not an isolated universe; care must be taken that traffic
leaving the subnet complies with the restrictions of the
larger net, not just those of the cooperating subnet.
A "message ID" is a unique identifier for an article, usu-
ally supplied by the posting agent which posted it. It dis-
tinguishes the article from every other article ever posted
anywhere (in theory). Articles with the same message ID are
treated as identical copies of the same article even if they
are not in fact identical.
A "gateway" is software which receives news articles and
converts them to messages of some other kind (e.g. mail to a
mailing list), or vice-versa; in essence it is a translating
relayer that straddles boundaries between different methods
of message exchange. The most common type of gateway
17 Jan 1994 - 7 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 2.3
connects newsgroup(s) to mailing list(s), either unidirec-
tionally or bidirectionally, but there are also gateways
between news networks using this Draft's news format and
those using other formats.
A "control message" is an article which is marked as con-
taining control information; a relayer receiving such an
article will (subject to permissions etc.) take actions
beyond just filing and passing on the article.
NOTE: "Control article" would be more consistent
terminology, but "control message" is already well
established.
An article's "reply address" is the address to which mailed
replies should be sent. This is the address specified in
the article's From header (see section 5.2), unless it also
has a Reply-To header (see section 6.3).
The notation (e.g.) "(ASCII 17)" following a name means
"this name refers to the ASCII character having value 17".
An "ASCII printable character" is an ASCII character in the
range 33-126. An "ASCII control character" is an ASCII
character in the range 0-31, or the character DEL (ASCII
127). A "non-ASCII character" is a character having a value
exceeding 127.
NOTE: Blank is neither an "ASCII printable charac-
ter" nor an "ASCII control character".
2.4. End Of Line
How the end of a text line is represented depends on the
context and the implementation. For Internet transmission
via protocols such as SMTP [rrr], an end-of-line is a CR
(ASCII 13) followed by an LF (ASCII 10). ISO C [rrr] and
many modern operating systems indicate end-of-line with a
single character, typically ASCII LF (aka "newline"), and
this is the normal convention when news is transmitted via
UUCP. A variety of other methods are in use, including out-
of-band methods in which there is no specific character that
means end-of-line.
This Draft does not constrain how end-of-line is represented
in news, except that characters other than CR and LF MUST
not be usurped for use in end-of-line representations.
Also, obviously, all software dealing with a particular copy
of an article must agree on the convention to be used.
"EOL" is used to mean "whatever end-of-line representation
is appropriate"; it is not necessarily a character or
sequence of characters.
17 Jan 1994 - 8 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 2.4
NOTE: If faced with picking an EOL representation
in the absence of other constraints, use of a sin-
gle character simplifies processing, and the ASCII
standard [rrr] specifies that if one character is
to be used for this purpose, it should be LF
(ASCII 10).
NOTE: Inside MIME encodings, use of the Internet
canonical EOL representation (CR followed by LF)
is mandatory. See [rrr].
2.5. Case-Sensitivity
Text in newsgroup names, header parameters, etc. is case-
sensitive unless stated otherwise.
NOTE: This is at variance with MAIL, which is
case-insensitive unless stated otherwise, but is
consistent with news historical practice and
existing news software. See the comments on back-
ward compatibility in section 1.
2.6. Language
Various constant strings in this Draft, such as header names
and month names, are derived from English words. Despite
their derivation, these words do NOT change when the poster
or reader employing them is interacting in a language other
than English. Posting and reading agents SHOULD translate
as appropriate in their interaction with the poster or
reader, but the forms that actually appear in articles are
always the English-derived ones defined in this Draft.
3. Relation To MAIL (RFC 822 etc.)
The primary intent of this Draft is to completely describe
the news article format as a subset of MAIL's message format
augmented by some new headers. Unless explicitly noted oth-
erwise, the intent throughout is that an article MUST also
be a valid MAIL message.
NOTE: Despite obvious similarities between news
and mail, opinions vary on whether it is possible
or desirable to unify them into a single service.
However, it is unquestionably both possible and
useful to employ some of the same tools for manip-
ulating both mail messages and news articles, so
there is specific advantage to be had in defining
them compatibly. Furthermore, there is no appar-
ent need to re-invent the wheel when slight exten-
sions to an existing definition will suffice.
17 Jan 1994 - 9 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 3
Given that this Draft attempts to be self-contained, it
inevitably contains considerable repetition of information
found in MAIL. This raises the possibility of unintentional
conflicts. Unless specifically noted otherwise, any wording
in this Draft which permits behavior that is not MAIL-
compliant is erroneous and should be followed only to the
extent that the result remains compliant with MAIL.
NOTE: RFC 1036 said "where this standard conflicts
with [RFC 822], RFC-822 should be considered cor-
rect and this standard in error". Taken liter-
ally, this was obviously incorrect, since RFC 1036
imposed a number of restrictions not found in RFC
822. The intent, however, was reasonable: to
indicate that UNINTENTIONAL differences were
errors in RFC 1036.
Implementors and users should note that MAIL is deliberately
an extensible standard, and most extensions devised for mail
are also relevant to (and compatible with) news. Note par-
ticularly MIME [rrr], summarized briefly in appendix B,
which extends MAIL in a number of useful ways that are defi-
nitely relevant to news. Also of note is the work in
progress on reconciling PEM (Privacy Enhanced Mail, which
defines extensions for authentication and security) with
MIME, after which this may also be relevant to news.
UNRESOLVED ISSUE: Update the MIME/PEM information.
Similarly, descriptions here of MIME facilities should be
considered correct only to the extent that they do not
require or legitimize practices that would violate those
RFCs. (Note that this Draft does extend the application of
some MIME facilities, but this is an extension rather than
an alteration.)
4. Basic Format
4.1. Overall Syntax
The overall syntax of a news article is:
17 Jan 1994 - 10 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.1
article = 1*header separator body
header = start-line *continuation
start-line = header-name ":" space [ nonblank-text ] eol
continuation = space nonblank-text eol
header-name = 1*name-character *( "-" 1*name-character )
name-character = letter / digit
letter = <ASCII letter A-Z or a-z>
digit = <ASCII digit 0-9>
separator = eol
body = *( [ nonblank-text / space ] eol )
eol = <EOL>
nonblank-text = [ space ] text-character *( space-or-text )
text-character = <any ASCII character except NUL (ASCII 0),
HT (ASCII 9), LF (ASCII 10), CR (ASCII 13),
or blank (ASCII 32)>
space = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )
space-or-text = space / text-character
An article consists of some headers followed by a body. An
empty line separates the two. The headers contain struc-
tured information about the article and its transmission. A
header begins with a header name identifying it, and can be
continued onto subsequent lines by beginning the continua-
tion line(s) with white space. (Note that section 4.2.3
adds some restrictions to the header syntax indicated here.)
The body is largely-unstructured text significant only to
the poster and the readers.
NOTE: Terminology here follows the current custom
in the news community, rather than the MAIL con-
vention of (sometimes) referring to what is here
called a "header" as a "header field" or "field".
Note that the separator line must be truly empty, not just a
line containing white space. Further empty lines following
it are part of the body, as are empty lines at the end of
the article.
NOTE: Some systems make no distinction between
empty lines and lines consisting entirely of white
space; indeed, some systems cannot represent
entirely empty lines. The grammar's requirement
that header continuation lines contain some print-
able text is meant to ensure that the empty/space
distinction cannot confuse identification of the
separator line.
NOTE: It is tempting to authorize posting agents
to strip empty lines at the beginning and end of
the body, but such empty lines could possibly be
part of a preformatted document.
Implementors are warned that trailing white space, whether
alone on the line or not, MAY be significant in the body,
17 Jan 1994 - 11 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.1
notably in early versions of the "uuencode" encoding for
binary data. Trailing white space MUST be preserved unless
the article is known to have originated within a cooperating
subnet that avoids using significant trailing white space,
and SHOULD be preserved regardless. Posters SHOULD avoid
using conventions or encodings which make trailing white
space significant; for encoding of binary data, MIME's
"base64" encoding is recommended. Implementors are warned
that ISO C implementations are not required to preserve
trailing white space, and special precautions may be neces-
sary in implementations which do not.
NOTE: Unfortunately, the signature-delimiter con-
vention (described in section 4.3.2) does use sig-
nificant trailing white space. It's too late to
fix this; there is work underway on defining an
organized signature convention as part of MIME,
which is a preferable solution in the long run.
Posters are warned that some very old relayer software mis-
behaves when the first non-empty line of an article body
begins with white space.
4.2. Headers
4.2.1. Names and Contents
Despite the restrictions on header-name syntax imposed by
the grammar, relayers and reading agents SHOULD tolerate
header names containing any ASCII printable character other
than colon (":", ASCII 58).
NOTE: MAIL header names can contain any ASCII
printable character (other than colon) in theory,
but in practice, arbitrary header names are known
to cause trouble for some news software. Section
4.1's restriction to alphanumeric sequences sepa-
rated by hyphens is believed to permit all widely-
used header names without causing problems for any
widely-used software. Software is nevertheless
encouraged to cope correctly with the full range
of possibilities, since aberrations are known to
occur.
Relayers MUST disregard headers not described in this Draft
(that is, with header names not mentioned in this Draft),
and pass them on unaltered.
Posters wishing to convey non-standard information in head-
ers SHOULD use header names beginning with "X-". No stan-
dard header name will ever be of this form. Reading agents
SHOULD ignore "X-" headers, or at least treat them with
17 Jan 1994 - 12 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.2.1
great care.
The order of headers in an article is not significant. How-
ever, posting agents are encouraged to put mandatory headers
(see section 5) first, followed by optional headers (see
section 6), followed by headers not defined in this Draft.
NOTE: While relayers and reading agents must be
prepared to handle any order, having the signifi-
cant headers (the precise definition of "signifi-
cant" depends on context) first can noticeably
improve efficiency, especially in memory-limited
environments where it is difficult to buffer up an
arbitrary quantity of headers while searching for
the few that matter.
Header names are case-insensitive. There is a preferred
case convention, which posters and posting agents SHOULD
use: each hyphen-separated "word" has its initial letter (if
any) in uppercase and the rest in lowercase, except that
some abbreviations have all letters uppercase (e.g. "Mes-
sage-ID" and "MIME-Version"). The forms used in this Draft
are the preferred forms for the headers described herein.
Relayers and reading agents are warned that articles might
not obey this convention.
NOTE: Although software must be prepared for the
possibility of random use of case in header names
(and other case-independent text), establishing a
preferred convention reduces pointless diversity,
and may permit optimized software that looks for
the preferred forms before resorting to less-
efficient case-insensitive searches.
In general, a header can consist of several lines, with each
continuation line beginning with white space. The EOLs pre-
ceding continuation lines are ignored when processing such a
header, effectively combining the start-line and the contin-
uations into a single logical line. The logical line, less
the header name, colon, and any white space following the
colon, is the "header content".
4.2.2. Undesirable Headers
A header whose content is empty is said to be an empty
header. Relayers and reading agents SHOULD not consider
presence or absence of an empty header to alter the seman-
tics of an article. Posting agents SHOULD delete empty
headers from articles before posting them.
Headers that merely state defaults explicitly (e.g., a Fol-
lowup-To header with the same content as the Newsgroups
header, or a MIME Content-Type header with contents
17 Jan 1994 - 13 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.2.2
"text/plain; charset=us-ascii") or state information that
reading agents can typically determine easily themselves
(e.g. the length of the body in octets) are redundant, con-
veying no information whatsoever. Headers that state infor-
mation which cannot possibly be of use to a significant num-
ber of relayers, reading agents, or readers (e.g., the name
of the software package used as the posting agent) are use-
less and pointless. Posters and posting agents SHOULD avoid
including redundant or useless headers in articles.
NOTE: Information that someone, somewhere, might
someday find useful is best omitted from headers.
(There's quite enough of it in article bodies.)
Headers should contain information of known util-
ity only. This is not meant to preclude inclusion
of information primarily meant for news-software
debugging, but such information should be included
only if there is real reason, preferably based on
experience, to suspect that it may be genuinely
useful. Articles passing through gateways are the
only obvious case where inclusion of debugging
information appears clearly legitimate. (See sec-
tion 10.1.)
NOTE: A useful rule of thumb for software imple-
mentors is: "if I had to pay a dollar a day for
the transmission of this header, would I still
think it worthwhile?".
4.2.3. White Space and Continuations
The colon following the header name on the start-line MUST
be followed by white space, even if the header is empty. If
the header is not empty, at least some of the content MUST
appear on the start-line. Posting agents MUST enforce these
restrictions, but relayers (etc.) SHOULD accept even arti-
cles that violate them.
NOTE: MAIL does not require white space after the
colon, but it is usual. RFC 1036 required the
white space, even in empty headers, and some
existing software demands it. In MAIL, and
arguably in RFC 1036 (although the wording is
vague), it is technically legitimate for the white
space to be part of a continuation line rather
than the start-line, but not all existing software
will accept this. Deleting empty headers and
placing some content on the start-line avoids this
issue... which is desirable because trailing
blanks, easily deleted by accident, are best not
made significant in headers.
17 Jan 1994 - 14 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.2.3
In general, posters and posting agents SHOULD use blank
(ASCII 32), not tab (ASCII 9), where white space is desired
in headers. Existing software does not consistently accept
tab as synonymous with blank in all contexts. In particu-
lar, RFC 1036 appeared to specify that the character immedi-
ately following the colon after a header name was required
to be a blank, and some news software insists on that, so
this character MUST be a blank. Again, posting agents MUST
enforce these restrictions but relayers SHOULD be more tol-
erant.
Since the white space beginning a continuation line remains
a part of the logical line, headers can be "broken" into
multiple lines only at white space. Posting agents SHOULD
not break headers unnecessarily. Relayers SHOULD preserve
existing header breaks, and SHOULD not introduce new breaks.
Breaking headers SHOULD be a last resort; relayers and read-
ing agents SHOULD handle long header lines gracefully. (See
the discussion of size limits in section 4.6.)
4.3. Body
Although the article body is unstructured for most of the
purposes of this Draft, structure MAY be imposed on it by
other means, notably MIME headers (see appendix B).
4.3.1. Body Format Issues
The body of an article MAY be empty, although posting agents
SHOULD consider this an error condition (meriting returning
the article to the poster for revision). A posting agent
which does not reject such an article SHOULD issue a warning
message to the poster and supply a non-empty body. Note
that the separator line MUST be present even if the body is
empty.
NOTE: An empty body is probably a poster error
except, arguably, for some control messages... and
even they really ought to have a body explaining
the reason for the control message. Some old
reading agents are known to generate empty bodies
for "cancel" control messages, so posting agents
might opt not to reject body-less articles in such
cases (although it would be better to fix the
reading agents to request a body). However, some
existing news software is known to react badly to
body-less articles, hence the request for posting
agents to insert a body in such cases.
NOTE: A possible posting-agent-supplied body text
(already used by one widespread posting agent) is
"This article was probably generated by a buggy
17 Jan 1994 - 15 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.3.1
news reader.". (The use of "reader" to refer to
the reading agent is traditional, although this
Draft uses more precise terminology.)
NOTE: The requirement for the separator line even
in a bodyless article is inherited from MAIL, and
also distinguishes legitimately-bodyless articles
from articles accidentally truncated in the middle
of the headers.
Note that an article body is a sequence of lines terminated
by EOLs, not arbitrary binary data, and in particular it
MUST end with an EOL. However, relayers SHOULD treat the
body of an article as an uninterpreted sequence of octets
(except as mandated by changes of EOL representation and by
control-message processing) and SHOULD avoid imposing con-
straints on it. See also section 4.6.
4.3.2. Body Conventions
Although body lines can in principle be very long (see sec-
tion 4.6 for some discussion of length limits), posters
SHOULD restrict body line lengths to circa 70-75 characters.
On systems where text is conventionally stored with EOLs
only at paragraph breaks and other "hard return" points,
with software breaking lines as appropriate for display or
manipulation, posting agents SHOULD insert EOLs as necessary
so that posted articles comply with this restriction.
NOTE: News originated in environments where line
breaks in plain text files were supplied by the
user, not the software. Be this good or bad, much
reading-agent and posting-agent software assumes
that news articles follow this convention, so it
is often inconvenient to read or respond to arti-
cles which violate it. The "70-75" number comes
from the widespread use of display devices which
are 80 columns wide, and the desire to leave a bit
of margin for quoting etc. (see below).
Reading agents confronted with body lines much longer than
the available output-device width SHOULD break lines as
appropriate. Posters are warned that such breaks may not
occur exactly where the poster intends.
NOTE: "As appropriate" would typically include
breaking lines when supplying the text of an arti-
cle to be quoted in a reply or followup, something
that line-breaking reading agents often neglect to
do now.
Although styles vary widely, for plain text it is usual to
use no left margin, leave the right edge ragged, use a
17 Jan 1994 - 16 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.3.2
single empty line to separate paragraphs, and employ normal
natural-language usage on matters such as upper/lowercase.
(In particular, articles SHOULD not be written entirely in
uppercase. In environments where posters have access only
to uppercase, posting agents SHOULD translate it to lower-
case.)
NOTE: Most people find substantial bodies of text
entirely in uppercase relatively hard to read,
while all-lowercase text merely looks slightly
odd. The common association of uppercase with
strong emphasis adds to this.
Tone of voice does not carry well in written text, and mis-
understandings are common when sarcasm, parody, or exaggera-
tion for humorous effect is attempted without explicit warn-
ing. It has become conventional to use the sequence ":-)",
which (on most output devices) resembles a rotated "smiley
face" symbol, as a marker for text not meant to be taken
literally, especially when humor is intended. This practice
aids communication and averts unintended ill-will; posters
are urged to use it. A variety of analogous sequences are
used with less-standardized meanings [Sanderson].
The order of arrival of news articles at a particular host
depends somewhat on transmission paths, and occasionally
articles are lost for various reasons. When responding to a
previous article, posters SHOULD not assume that all readers
understand the exact context. It is common to quote some of
the previous article to establish context. This SHOULD be
done by prefacing each quoted line (even if it is empty)
with the character ">". This will result in multiple levels
of ">" when quoted context itself contains quoted context.
NOTE: It may seem superfluous to put a prefix on
empty lines, but it simplifies implementation of
functions such as "skip all quoted text" in read-
ing agents.
Readability is enhanced if quoted text and new text are sep-
arated by an empty line.
Posters SHOULD edit quoted context to trim it down to the
minimum necessary. However, posting agents SHOULD not
attempt to enforce this by imposing overly-simplistic rules
like "no more than 50% of the lines should be quotes".
NOTE: While encouraging trimming is desirable, the
50% rule imposed by some old posting agents is
both inadequate and counterproductive. Posters do
not respond to it by being more selective about
quoting; they respond by padding short responses,
or by using different quoting styles to defeat
automatic analysis. The former adds unnecessary
17 Jan 1994 - 17 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.3.2
noise and volume, while the latter also defeats
more useful forms of automatic analysis that read-
ing agents might wish to do.
NOTE: At the very least, if a minimum-unquoted
quota is being set, article bodies shorter than
(say) 20 lines, or perhaps articles which exceed
the quota by only a few lines, should be exempt.
This avoids the ridiculous situation of complain-
ing about a 5-line response to a 6-line quote.
NOTE: A more subtle posting-agent rule, suggested
for experimental use, is to reject articles that
appear to contain quoted signatures (see below).
This is almost certainly the result of a careless
poster not bothering to trim down quoted context.
Also, if a posting agent or followup agent pre-
sents an article template to the poster for edit-
ing, it really should take note of whether the
poster actually made any changes, and refrain from
posting an unmodified template.
Some followup agents supply "attribution" lines for quoted
context, indicating where it first appeared and under whose
name. When multiple levels of quoting are present and
quoted context is edited for brevity, "inner" attribution
lines are not always retained. The editing process is also
somewhat error-prone. Reading agents (and readers) are
warned not to assume that attributions are accurate.
UNRESOLVED ISSUE: Should a standard format for
attribution lines be defined? There is already
considerable diversity... but automatic news anal-
ysis would be substantially aided by a standard
convention.
Early difficulties in inferring return addresses from arti-
cle headers led to "signatures": short closing texts, auto-
matically added to the end of articles by posting agents,
identifying the poster and giving his network addresses etc.
If a poster or posting agent does append a signature to an
article, the signature SHOULD be preceded with a delimiter
line containing (only) two hyphens (ASCII 45) followed by
one blank (ASCII 32). Posting agents SHOULD limit the
length of signatures, since verbose excess bordering on
abuse is common if no restraint is imposed; 4 lines is a
common limit.
NOTE: While signatures are arguably a blemish,
they are a well-understood convention, and convey-
ing the same information in headers exposes it to
mangling and makes it rather less conspicuous. A
standard delimiter line makes it possible for
reading agents to handle signatures specially if
17 Jan 1994 - 18 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.3.2
desired. The choice of delimiter is somewhat
unfortunate, since it relies on preservation of
trailing white space, but it is too well-
established to change. There is work underway to
define a more sophisticated signature scheme as
part of MIME, and this will presumably supersede
the current convention in due time.
NOTE: Four 75-column lines of signature text is
300 characters, which is ample to convey name and
mail-address information in all but the most
bizarre situations.
4.4. Characters And Character Sets
Header and body lines MAY contain any ASCII characters other
than CR (ASCII 13), LF (ASCII 10), and NUL (ASCII 0).
NOTE: CR and LF are excluded because they clash
with common EOL conventions. NUL is excluded
because it clashes with the C end-of-string con-
vention, which is significant to most existing
news software. These three characters are
unlikely to be transmitted successfully.
However, posters SHOULD avoid using ASCII control characters
except for tab (ASCII 9), formfeed (ASCII 12), and backspace
(ASCII 8). Tab signifies sufficient horizontal white space
to reach the next of a set of fixed positions; posters are
warned that there is no standard set of positions, so tabs
should be avoided if precise spacing is essential. Formfeed
signifies a point at which a reading agent SHOULD pause and
await reader interaction before proceeding. Backspace
SHOULD be used only for underlining, done by a sequence of
underscores (ASCII 95), followed by an equal number of
backspaces, signifying that the same number of text charac-
ters following are to be underlined. Posters are warned
that underlining is not available on all output devices and
is best not relied on for essential meaning. Reading agents
SHOULD recognize underlining and translate it to the appro-
priate commands for devices that support it.
NOTE: Interpretation of almost all control charac-
ters is device-specific to some degree, and
devices differ. Tabs and underlining are sup-
ported, to some extent, by most modern devices and
reading agents, hence the cautious exemptions for
them. The underlining method is specified because
the inverse method, text and then underscores, is
tempting to the naive... but if sent unaltered to
a device that shows only the most recent of sev-
eral overstruck characters rather than a compos-
ite, the result can be utterly unreadable.
17 Jan 1994 - 19 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.4
NOTE: A common interpretation of tab is that it is
a request to space forward to the next position
whose number is one more than a multiple of 8,
with positions numbered sequentially starting at
1. (So tab positions are 9, 17, 25, ...) Reading
agents not constrained by existing system conven-
tions might wish to use this interpretation.
NOTE: It will typically be necessary for a reading
agent to catch and interpret formfeed, not just
send it to the output device. The actions per-
formed by typical output devices on receiving a
formfeed are neither adequate for nor appropriate
to the pause-for-interaction meaning.
Cooperating subnets which wish to employ non-ASCII character
sets by using escape sequences (employing, e.g., ESC (ASCII
27), SO (ASCII 14), and SI (ASCII 15)) to alter the meaning
of superficially-ASCII characters MAY do so, but MUST use
MIME headers to alert reading agents to the particular char-
acter set(s) and escape sequences in use. A reading agent
SHOULD not pass such an escape sequence through, unaltered,
to the output device unless the agent confirms that the
sequence is one used to affect character sets and has reason
to believe that the device is capable of interpreting that
particular sequence properly.
NOTE: Cooperating-subnet organizers are warned
that some very old relayers strip certain control
characters out of articles they pass along. ESC
is known to be among the affected characters.
NOTE: There are now standard Internet encodings
for Japanese [rrr] and Vietnamese [rrr] in partic-
ular.
Articles MUST not contain any octet with value exceeding
127, i.e. any octet that is not an ASCII character.
NOTE: This rule, like others, may be relaxed by
unanimous consent of the members of a cooperating
subnet, provided suitable precautions are taken to
ensure that rule-violating articles do not leak
out of the subnet. (This has already been done in
many areas where ASCII is not adequate for the
local language(s).) Beware that articles contain-
ing non-ASCII octets in headers are a violation of
the MAIL specifications and are not valid MAIL
messages. MIME offers a way to encode non-ASCII
characters in ASCII for use in headers; see sec-
tion 4.5.
NOTE: While there is great interest in using 8-bit
character sets, not all software can yet handle
17 Jan 1994 - 20 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.4
them correctly. Hence the restriction to cooper-
ating subnets. MIME encodings can be used to
transmit such characters while remaining within
the octet restriction.
In anticipation of the day when it is possible to use non-
ASCII characters safely anywhere, and to provide for the
(substantial) cooperating subnets that are already using
them, transmission paths SHOULD treat news articles as unin-
terpreted sequences of octets (except perhaps for transfor-
mations between EOL representations) and relayers SHOULD
treat non-ASCII characters in articles as ordinary charac-
ters.
NOTE: 8-bit enthusiasts are warned that not all
software conforms to these recommendations yet.
In particular, standard NNTP [rrr] is a 7-bit pro-
tocol, and there may be implementations which
enforce this rule. Be warned, also, that it will
never be safe to send raw binary data in the body
of news articles, because changes of EOL represen-
tation may (will!) corrupt it.
Except where cooperating subnets permit more direct
approaches, MIME [rrr] headers and encodings SHOULD be used
to transmit non-ASCII content using ASCII characters; see
section 4.5, appendix B, and the MIME RFCs for details. If
article content can be expressed in ASCII, it SHOULD be.
Failing that, the order of preference for character sets is
that described in MIME [rrr].
NOTE: Using the MIME facilities, it is possible to
transmit ANY character set, and ANY form of binary
data, using only ASCII characters. Equally impor-
tant, such articles are self-describing and the
reading agent can tell which octet-to-symbol map-
ping is intended! Designation of some preferred
character sets is intended to minimize the number
of character sets that a reading agent must under-
stand in order to display most articles properly.
Articles containing non-ASCII characters, articles using
ASCII characters (values 0 through 127) to refer to non-
ASCII symbols, and articles using escape sequences to shift
character sets SHOULD include MIME headers indicating which
character set(s) and conventions are being used, and MUST do
so unless such articles are strictly confined to a cooperat-
ing subnet which has its own pre-agreed conventions. MIME
encodings are preferred over all these techniques. If it
comes to a relayer's attention that it is being asked to
pass an article using such techniques outward across what it
knows to be the boundary of such a cooperating subnet, it
MUST report this error to its administrator, and MAY refuse
to pass the article beyond the subnet boundary. If it does
17 Jan 1994 - 21 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.4
pass the article, it MUST re-encode it with MIME encodings
to make it conform to this Draft.
NOTE: Such re-encoding is a non-trivial task, due
to MIME rules such as the prohibition of nested
encodings. It's not just a matter of pouring the
body through a simple filter.
Reading agents SHOULD note MIME headers and attempt to show
the reader the closest possible approximation to the
intended content. They SHOULD not just send the octets of
the article to the output device unaltered, unless there is
reason to believe that the output device will indeed inter-
pret them correctly. Reading agents MUST not pass ASCII
control characters or escape sequences, other than as dis-
cussed above, unaltered to the output device; only by chance
would the result be the desired one, and there is serious
potential for harmful side effects, either accidental or
malicious.
NOTE: Exactly what to do with unwanted control
characters/sequences depends on the philosophy of
the reading agent, but passing them straight to
the output device is almost always wrong. If the
reading agent wants to mark the presence of such a
character/sequence in circumstances where only
ASCII printable characters are available, trans-
lating it to "#" might be a suitable method; "#"
is a conspicuous character seldom used in normal
text.
NOTE: Reading agents should be aware that many old
output devices (or the transmission paths to them)
zero out the top bit of octets sent to them. This
can transform non-ASCII characters into ASCII con-
trol characters.
Followup agents MUST be careful to apply appropriate trans-
formations of representation to the outbound followup as
well as the inbound precursor. A followup to an article
containing non-ASCII material is very likely to contain non-
ASCII material itself.
4.5. Non-ASCII Characters In Headers
All octets found in headers MUST be ASCII characters. How-
ever, it is desirable to have a way of encoding non-ASCII
characters, especially in "human-readable" headers such as
Subject. MIME [rrr] provides a way to do this. Full
details may be found in the MIME specifications; herewith a
quick summary to alert software authors to the issues...
17 Jan 1994 - 22 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.5
encoded-word = "=?" charset "?" encoding "?" codes "?="
charset = 1*tag-char
encoding = 1*tag-char
tag-char = <ASCII printable character except !()<>@,;:\"[]/?=>
codes = 1*code-char
code-char = <ASCII printable character except ?>
An encoded word is a sequence of ASCII printable characters
that specifies the character set, encoding method, and bits
of (potentially) non-ASCII characters. Encoded words are
allowed only in certain positions in certain headers. Spe-
cific headers impose restrictions on the content of encoded
words beyond that specified in this section. Posting agents
MUST ensure that any material resembling an encoded word
(complete with all delimiters), in a context where encoded
words may appear, really is an encoded word.
NOTE: The syntax is a bit ugly, but it was
designed to minimize chances of confusion with
legitimate header contents, and to satisfy diffi-
cult constraints on use within existing headers.
An encoded word MUST not be more than 75 octets long. Each
line of a header containing encoded word(s) MUST be at most
76 octets long, not counting the EOL.
NOTE: These limits are meant to bound the looka-
head needed to determine whether text that begins
"=?" is really an encoded word.
The details of charsets and encodings are defined by MIME
[rrr]; the sequence of preferred character sets is the same
as MIME's. Encoded words SHOULD not be used for content
expressible in ASCII.
When an encoded word is used, other than in a newsgroup name
(see section 5.5), it MUST be separated from any adjacent
non-space characters (including other encoded words) by
white space. Reading agents displaying the contents of
encoded words (as opposed to their encoded form) should
ignore white space adjacent to encoded words.
UNRESOLVED ISSUE: Should this section be deleted
entirely, or made much more terse? The material
is relevant, but too complex to discuss fully.
NOTE: The deletion of intervening white space per-
mits using multiple encoded words, implicitly con-
catenated by the deletion, to encode text that
will not fit within a single 75-character encoded
word.
Reading-agent implementors are warned that although this
Draft completely specifies where encoded words may appear in
17 Jan 1994 - 23 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.5
the headers it defines, there are other headers (e.g. the
MIME Content-Description header) that MAY contain them.
4.6. Size Limits
Implementations SHOULD avoid fixed constraints on the sizes
of lines within an article and on the size of the entire
article.
Relayers SHOULD treat the body of an article as an uninter-
preted sequence of octets (except as mandated by changes of
EOL representation and processing of control messages), not
to be altered or constrained in any way.
If it is absolutely necessary for an implementation to
impose a limit on the length of header lines, body lines, or
header logical lines, that limit shall be at least 1000
octets, including EOL representations. Relayers and trans-
mission paths confronted with lines beyond their internal
limits (if any) MUST not simply inject EOLs at random
places; they MAY break headers (as described in 4.2.3) as a
last resort, and otherwise they MUST either pass the long
lines through unaltered, or refuse to pass the article at
all (see section 9.1 for further discussion).
NOTE: The limit here is essentially the same mini-
mum as that specified for SMTP mail in RFC 821
[rrr]. Implementors are warned that Path (see
section 5.6) and References (see section 6.5)
headers, in particular, often become several hun-
dred characters long, so 1000 is not an overly
generous limit.
All implementations MUST be able to handle an article
totalling at least 65,000 octets, including headers and EOL
representations, gracefully and efficiently. All implemen-
tations SHOULD be able to handle an article totalling at
least 1,000,000 (one million) octets, including headers and
EOL representations, gracefully and efficiently. "Grace-
fully and efficiently" is intended to preclude not only
failures, but also major loss of performance, serious prob-
lems in error recovery, or resource consumption beyond what
is reasonably necessary.
NOTE: The intent here is to prohibit lowering the
existing de-facto limit any further, while
strongly encouraging movement towards a higher
one. Actually, although improvements are desir-
able in some cases, much news software copes rea-
sonably well with very large articles. The same
cannot be said of the communications software and
protocols used to transmit news from one host to
another, especially when slow communications links
17 Jan 1994 - 24 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.6
are involved. Occasional huge articles that
appear now (by accident or through ignorance) typ-
ically leave trails of failing software, system
problems, and irate administrators in their wake.
NOTE: It is intended that the successor to this
Draft will raise the "MUST" limit to 1,000,000 and
the "SHOULD" limit still further.
Posters SHOULD limit posted articles to at most 60,000
octets, including headers and EOL representations, unless
the articles are being posted only within a cooperating sub-
net which is known to be capable of handling larger articles
gracefully. Posting agents presented with a large article
SHOULD warn the poster and request confirmation.
NOTE: The difference between this and the earlier
"MUST" limit is margin for header growth, differ-
ing EOL representations, and transmission over-
heads.
NOTE: Disagreeable though these limits are, it is
a fact that in current networks, an article larger
than 64K (after header growth etc.) simply is not
transmitted reliably. Note also the comments
above on the trauma caused by single extremely-
large articles now; the problems are real and cur-
rent. These problems arguably should be fixed,
but this will not happen network-wide in the imme-
diate future. Hence the restriction of larger
articles to cooperating subnets, for now.
Posters using non-ASCII characters in their text MUST take
into account the overhead involved in MIME encoding, unless
the article's propagation will be entirely limited to a
cooperating subnet which does not use MIME encodings for
non-ASCII characters. For example, MIME base64 encoding
involves growth by a factor of approximately 4/3, so an
article which would likely have to use this encoding should
be at most about 45,000 octets before encoding.
Posters SHOULD use MIME "message/partial" conventions to
facilitate automatic reassembly of a large document split
into smaller pieces for posting. It is recommended that the
content identifier used should be a message ID, generated by
the same means as article message IDs (see section 5.3), and
that all parts should have a See-Also header (see section
6.16) giving the message IDs of at least the previous parts
and preferably all the parts.
NOTE: See-Also is more correct for this purpose
than References, although References is in common
use today (with less-formal reassembly arrange-
ments). MIME reassemblers should probably examine
17 Jan 1994 - 25 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 4.6
articles suggested by References headers if See-
Also headers are not present to indicate the
whereabouts of the other parts of "mes-
sage/partial" articles.
To repeat: implementations SHOULD avoid fixed constraints on
the sizes of lines within an article and on the size of the
entire article.
4.7. Example
Here is a sample article:
From: jerry@eagle.ATT.COM (Jerry Schwarz)
Path: cbosgd!mhuxj!mhuxt!eagle!jerry
Newsgroups: news.announce
Subject: Usenet Etiquette -- Please Read
Message-ID: <642@eagle.ATT.COM>
Date: Mon, 17 Jan 1994 11:14:55 -0500 (EST)
Followup-To: news.misc
Expires: Wed, 19 Jan 1994 00:00:00 -0500
Organization: AT&T Bell Laboratories, Murray Hill
body
body
body
5. Mandatory Headers
An article MUST have one, and only one, of each of the fol-
lowing headers: Date, From, Message-ID, Subject, Newsgroups,
Path.
NOTE: MAIL specifies (if read most carefully) that
there must be exactly one Date header and exactly
one From header, but otherwise does not restrict
multiple appearances of headers. (Notably, it
permits multiple Message-ID headers!) This
appears singularly useless, or even harmful, in
the context of news, and much current news soft-
ware will not tolerate multiple appearances of
mandatory headers.
Note also that there are situations, discussed in the rele-
vant parts of section 6, where References, Sender, or
Approved headers are mandatory.
In the discussions of the individual headers, the content of
each is specified using the syntax notation. The convention
used is that the content of, for example, the Subject header
17 Jan 1994 - 26 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5
is defined as <Subject-content>.
5.1. Date
The Date header contains the date and time when the article
was submitted for transmission:
Date-content = [ weekday "," space ] date space time
weekday = "Mon" / "Tue" / "Wed" / "Thu"
/ "Fri" / "Sat" / "Sun"
date = day space month space year
day = 1*2digit
month = "Jan" / "Feb" / "Mar" / "Apr" / "May" / "Jun"
/ "Jul" / "Aug" / "Sep" / "Oct" / "Nov" / "Dec"
year = 4digit / 2digit
time = hh ":" mm [ ":" ss ] space timezone
timezone = "UT" / "GMT"
/ ( "+" / "-" ) hh mm [ space "(" zone-name ")" ]
hh = 2digit
mm = 2digit
ss = 2digit
zone-name = 1*( <ASCII printable character except ()\> / space )
This is a restricted subset of the MAIL date format.
If a weekday is given, it MUST be consistent with the date.
The modern Gregorian calendar is used, and dates MUST be
consistent with its usual conventions; for example, if the
month is May, the day must be between 1 and 31 inclusive.
The year SHOULD be given as four digits, and posting agents
SHOULD enforce this; however, relayers MUST accept the two-
digit form, and MUST interpret it as having the implicit
prefix "19".
NOTE: Two-digit year numbers can, should, and must
be phased out by 1999.
The time is given on the 24-hour clock, e.g. two hours
before midnight is "22:00" or "22:00:00". The hh must be
between 00 and 23 inclusive, the mm between 0 and 59 inclu-
sive, and the ss between 0 and 61 inclusive.
NOTE: Leap seconds very occasionally result in
minutes that are 60 or 61 seconds long.
The date and time SHOULD be given in the poster's local
timezone, including a specification of that timezone as a
numeric offset (which SHOULD include the timezone name, e.g.
"EST", supplied in parentheses like a MAIL comment). If
not, they MUST be given in Universal Time (abbreviated "UT";
"GMT" is a historical synonym for "UT"). The timezone name
in parentheses, if present, is a comment; software MUST
ignore it, except that reading agents might wish to display
17 Jan 1994 - 27 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.1
it to the reader. Timezone names other than "UT" and "GMT"
MUST appear only in the comment.
NOTE: Attempts to deal with a full set of timezone
names have all foundered on the vast number of
such names in use and the duplications (for exam-
ple, there are at least FOUR different timezones
called "EST" by somebody). Even the limited set
of North American zone names authorized by MAIL is
subject to confusion and misinterpretation. Hence
the flat ban on non-UT timezone names except as
comments.
NOTE: RFC 1036 specified that use of GMT (aka UT,
UTC) was preferred. However, the local time (in
the poster's timezone) is arguably information of
possible interest to the reader, and this requires
some indication of the poster's timezone. Numeric
offsets are an unambiguous way of doing this, and
their use was indeed sanctioned by RFC 1036 (that
is, this is a change of preference only).
NOTE: There is frequent confusion, including
errors in some news software, regarding the sign
of numeric timezones. Zones west of Greenwich
have negative offsets. For example, North Ameri-
can Eastern Standard Time is zone -0500 and North
American Eastern Daylight Time is zone -0400.
NOTE: Implementors are warned that the hh in a
timezone can go up to about 14; it is not limited
to 12. This is because the International Date
Line does not run exactly along the boundary
between zone -1200 and zone +1200.
NOTE: The comments in section 2.6 regarding trans-
lation to other languages are relevant here. The
Date-content format, and the spellings of its com-
ponents, as found in articles themselves, are
always as defined in this Draft, regardless of the
language used to interact with readers and
posters. Reading and posting agents should trans-
late as appropriate. Actually, even English-
language reading and posting agents will probably
want to do some degree of translation on dates, if
only to abbreviate the lengthy format and (per-
haps) translate to and from the reader's timezone.
5.2. From
The From header contains the electronic address, and possi-
bly the full name, of the article's author:
17 Jan 1994 - 28 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.2
From-content = address [ space "(" paren-phrase ")" ]
/ [ plain-phrase space ] "<" address ">"
paren-phrase = 1*( paren-char / space / encoded-word )
paren-char = <ASCII printable character except ()<>\>
plain-phrase = plain-word *( space plain-word )
plain-word = unquoted-word / quoted-word / encoded-word
unquoted-word = 1*unquoted-char
unquoted-char = <ASCII printable character except !()<>@,;:\".[]>
quoted-word = quote 1*( quoted-char / space ) quote
quote = <" (ASCII 34)>
quoted-char = <ASCII printable character except "()<>\>
address = local-part "@" domain
local-part = unquoted-word *( "." unquoted-word )
domain = unquoted-word *( "." unquoted-word )
(Encoded words are described in section 4.5.) The full name
is distinguished from the electronic address either by
enclosing the former in parentheses (making it resemble a
MAIL comment, after the address) or by enclosing the latter
in angle brackets. The second form is preferred. In the
first form, encoded words inside the full name MUST be com-
posed entirely of <paren-char>s. In the second form,
encoded words inside the full name may not contain charac-
ters other than letters (of either case), digits, and the
characters "!", "*", "+", "-", "/", "=", and "_". The local
part is case-sensitive (except that all case counterparts of
"postmaster" are deemed equivalent), the domain is case-
insensitive, and all other parts of the From content are
comments which MUST be ignored by news software (except
insofar as reading agents may wish to display them to the
reader). Posters and posting agents MUST restrict them-
selves to this subset of the MAIL From syntax; relayers MAY
accept a broader subset, but see the discussion in section
9.1.
NOTE: The syntax here is a restricted subset of
the MAIL From syntax, with quoting particularly
restricted, for simple parsing. In particular,
the presence of "<" in the From content indicates
that the second form is being used, otherwise the
first form is being used. The major restrictions
here are those already de-facto imposed by exist-
ing software.
NOTE: Overly-lenient posting agents sometimes per-
mit the second form with a full name containing
"(" or ")", but it is extremely rare for a full
name to contain "<" or ">" even in mail. Accord-
ingly, reading agents wishing to robustly deter-
mine which form is in use in a particular article
should key on the presence or absence of "<", not
the presence or absence of "(".
17 Jan 1994 - 29 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.2
The address SHOULD be a valid and complete Internet domain
address, capable of being successfully mailed to by an
Internet host (possibly via an MX record and a forwarder).
The pseudo-domain ".uucp" MAY be used for hosts registered
in the UUCP maps (e.g. name "xyz.uucp" for registered site
"xyz"), but such hosts SHOULD discontinue this usage (either
by arranging a proper Internet address and forwarder, or by
using the "% hack" (see below)), as soon as possible. Bit-
net hosts SHOULD use Internet addresses, avoiding the obso-
lescent ".bitnet" pseudo-domain. Other forms of address
MUST not be used.
NOTE: "Other forms" specifically include UK-style
"backward" domains ("uk.oxbridge.cs" is in the
Czech Republic, not the UK), pure-UUCP addressing
("knee!shin!foot" instead of
"foot%shin@knee.uucp"), and abbreviated domains
("zebra.zoo" instead of "zebra.zoo.toronto.edu").
If it is necessary to use the local part to specify a rout-
ing relative to the nearest Internet host, this MUST be done
using the "% hack", using "%" as a secondary "@". For exam-
ple, to specify that mail to the address should go to Inter-
net host "foo.bar.edu", then to non-Internet host "ein",
then to non-Internet host "deux", for delivery there to
mailbox "fred", a suitable address would be:
fred%deux%ein@foo.bar.edu
Analogous forms using "!" in the local part MUST not be
used, as they are ambiguous; they should be expressed in the
"%" form.
NOTE: "a!b@c" can be interpreted as either "b%c@a"
or "b%a@c", and there is no consistency in which
choice is made. Such addresses consequently are
unreliable. The "%" form does not suffer from
this problem, and although its use is officially
discouraged, it is a de-facto standard, to the
point that MAIL recognizes it.
Relayers MUST not, repeat MUST not, repeat MUST not, rewrite
From lines, in any way, however minor or innocent-seeming.
Trying to "fix" a non-conforming address has a very high
probability of making things worse. Either pass it along
unchanged, or reject the article.
NOTE: An additional reason for banning the use of
"!" addressing is that it has a much higher proba-
bility of being rewritten into mangled unrecogniz-
ability by old relayers.
Posters and posting agents SHOULD avoid use of the charac-
ters "!" and "@" in full names, as they may trigger unwanted
17 Jan 1994 - 30 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.2
header rewriting by old, simple-minded news software.
NOTE: Also, the characters "." and ",", not infre-
quently found in names (e.g., "John W. Campbell,
Jr."), are NOT, repeat NOT, allowed in an unquoted
word. A From header like the following MUST not
be written without the quotation marks:
From: "John W. Campbell, Jr." <editor@analog.com>
5.3. Message-ID
The Message-ID header contains the message ID of the arti-
cle, a unique identifier distinguishing the article from
every other article:
Message-ID-content = message-id
message-id = "<" local-part "@" domain ">"
As with From addresses, a message ID's local part is case-
sensitive and its domain is case-insensitive. The "<" and
">" are parts of the message ID, not peculiarities of the
Message-ID header.
NOTE: News message IDs are a restricted subset of
MAIL message IDs. In particular, no existing news
software copes properly with MAIL quoting conven-
tions within the local part, so they are forbid-
den. This is unfortunate, particularly for X.400
gateways that often wish to include characters
which are not legal in unquoted message IDs, but
it is impossible to fix net-wide. See the notes
on gatewaying in section 10.
The domain in the message ID SHOULD be the full Internet
domain name of the posting agent's host. Use of the ".uucp"
pseudo-domain (for hosts registered in the UUCP maps) or the
".bitnet" pseudo-domain (for Bitnet hosts) is permissible,
but SHOULD be avoided.
Posters and posting agents MUST generate the local part of a
message ID using an algorithm which obeys the specified syn-
tax (words separated by ".", with certain characters not
permitted) (see section 5.2 for details), and will not
repeat itself (ever). The algorithm SHOULD not generate
message IDs which differ only in case of letters. Note the
specification in section 6.5 of a recommended convention for
indicating subject changes. Otherwise the algorithm is up
to the implementor.
NOTE: The crucial use of message IDs is to distin-
guish circulating articles from each other and
17 Jan 1994 - 31 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.3
from articles circulated recently. They are also
potentially useful as permanent indexing keys,
hence the requirement for permanent uniqueness...
but indexers cannot absolutely rely on this
because the earlier RFCs urged it but did not
demand it. All major implementations have always
generated permanently-unique message IDs by
design, but in some cases this is sensitive to
proper administration, and duplicates may have
occurred by accident.
NOTE: The most popular method of generating local
parts is to use the date and time, plus some way
of distinguishing between simultaneous postings on
the same host (e.g. a process number), and encode
them in a suitably-restricted alphabet. An older
but now less-popular alternative is to use a
sequence number, incremented each time the host
generates a new message ID; this is workable, but
requires careful design to cope properly with
simultaneous posting attempts, and is not as
robust in the presence of crashes and other mal-
functions.
NOTE: Some buggy news software considers message
IDs completely case-insensitive, hence the advice
to avoid relying on case distinctions. The
restrictions placed on the "alphabet" of local
parts and domains in section 5.2 have the useful
side effect of making it unnecessary to parse mes-
sage IDs in complex ways to break them into case-
sensitive and case-insensitive portions.
The local part of a message ID MUST not be "postmaster" or
any other string that would compare equal to "postmaster" in
a case-insensitive comparison. Message IDs MUST be no
longer than 250 octets, including the "<" and ">".
NOTE: "Postmaster" is an irksome exception to
case-sensitivity in local parts, inherited from
MAIL, and simply avoiding it is the best way to
deal with it (not that it's likely, but the issue
needs to be dealt with). The length limit is
undesirable, but is present in widely-used exist-
ing software. The limit is actually 255, but a
small safety margin is wise.
5.4. Subject
The Subject header's content (the "subject" of the article)
is a short phrase describing the topic of the article:
17 Jan 1994 - 32 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.4
Subject-content = [ "Re: " ] nonblank-text
Encoded words MAY appear in this header.
If the article is a followup, the subject SHOULD begin with
"Re: " (a "back reference"). If the article is not a fol-
lowup, the subject MUST not begin with a back reference.
Back references are case-insensitive, although "Re: " is the
preferred form. A followup agent assisting a poster in
preparing a followup SHOULD prepend a back reference, UNLESS
the subject already begins with one. If the poster deter-
mines that the topic of the followup differs significantly
from what is described in the subject, a new, more descrip-
tive, subject SHOULD be substituted (with no back refer-
ence). An article whose subject begins with a back refer-
ence MUST have a References header referencing the precur-
sor.
NOTE: A back reference is FOUR characters, the
fourth being a blank. RFC 1036 was confused about
this. Observe also that only ONE back reference
should be present.
NOTE: There is a semi-standard convention, often
used, in which a subject change is flagged by mak-
ing the new Subject-content of the form:
new topic (was: old topic)
possibly with "old topic" somewhat truncated.
Posters wishing to do something like this are
urged to use this exact form, to simplify auto-
mated analysis.
For historical reasons, the subject MUST not begin with
"cmsg " (note that this sequence ends with a blank).
NOTE: Some old news software takes a subject
beginning with "cmsg " as an indication that the
article is a control message (see sections 6.6 and
7). This mechanism is obsolete and undesirable,
but accidental triggering of it is still possible.
The subject SHOULD be terse. Posters SHOULD avoid trying to
cram their entire article into the headers; even the sim-
plest query usually benefits from a sentence or two of elab-
oration and context, and the details of header display vary
widely among reading agents.
NOTE: All-in-the-subject articles are sometimes
the result of misunderstandings over the interac-
tion protocol of a posting agent. Posting agents
might wish to give special attention to the possi-
bility that a poster specifying a very long
17 Jan 1994 - 33 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.4
subject might have thought he was typing the body
of the article.
5.5. Newsgroups
The Newsgroups header's content specifies which newsgroup(s)
the article is posted to:
Newsgroups-content = newsgroup-name *( ng-delim newsgroup-name )
newsgroup-name = plain-component *( "." component )
component = plain-component / encoded-word
plain-component = component-start *13component-rest
component-start = lowercase / digit
lowercase = <letter a-z>
component-rest = component-start / "+" / "-" / "_"
ng-delim = ","
Encoded words used in newsgroup names MUST not contain char-
acters other than letters, digits, "+", "-", "/", "_", "=",
and "?" (although they may encode them).
A newsgroup name consists of one or more components, which
may be plain components or (except for the first) encoded
words. A plain component MUST contain at least one letter,
MUST begin with a letter or digit, and MUST not be longer
than 14 characters. The first component MUST begin with a
letter; subsequent components SHOULD begin with a letter.
Newsgroup names MUST not contain uppercase letters, except
where required by encodings in encoded words. The sequences
"all" and "ctl" MUST not be used as components.
NOTE: The alphabet and syntax specified encom-
passes all existing names of widespread news-
groups, while avoiding various forms that are
known to cause problems. Important existing soft-
ware uses various non-alphanumeric characters as
punctuation adjacent to newsgroup names. (It
would, in fact, be preferable to ban "+" from
newsgroup names, were it not that several
widespread newsgroups related to the C++ program-
ming language already use it.)
NOTE: Much existing software converts the news-
group name into a directory path and stores the
articles themselves using numeric filenames, so
all-digit name components can be troublesome; the
"Great Renaming" early in the history of Usenet
included revisions of several newsgroup names to
eliminate such components.
NOTE: The same storage technique is the reason for
the 14-character limit. The limit is now largely
historical, since most modern systems have much
17 Jan 1994 - 34 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.5
larger limits on the length of a directory entry's
name, but many old systems are still in use. Sys-
tems with shorter limits also exist, but news
software on such systems has had to deal with the
problem already, since there are several
widespread newsgroups with 14-character components
in their names. Implementors are warned that it
is intended that the successor to this Draft will
increase the 14-character limit, and are urged to
fix their software to handle longer names grace-
fully (if such fixes are necessary, given the
intended domain of application of the particular
software).
NOTE: The requirement that the first character of
a name be a letter accommodates existing software
which assumes it can tell the difference between a
newsgroup name and other possible syntactic enti-
ties by inspecting the first character. Similar
considerations motivate excluding "+", "-", and
"_" from coming first in a component, and the
preference for components that do not begin with
digits. The "all" sequence is used as a wildcard
symbol in much existing software, and the "ctl"
sequence was involved in an obsolete historical
mechanism for marking control messages, so they
are best avoided.
NOTE: Possibly newsgroup names should have been
case-insensitive, but all existing software treats
them as case-sensitive. (RFC 977 [rrr] claims
that they are case-insensitive in NNTP, but exist-
ing implementations are believed to ignore this.)
The simplest solution is just to ban use of upper-
case letters, since no widespread newsgroup name
uses them anyway; this avoids any possibility of
confusion.
NOTE: The syntax has the disadvantage of contain-
ing no white space, making it impossible to con-
tinue a Newsgroups header across several lines.
Implementors of relayers and reading agents are
warned that it is intended that the successor to
this Draft will change the definition of ng-delim
to:
ng-delim = "," [ space ]
and are urged to fix their software to handle
(i.e., ignore) white space following the commas.
Meanwhile, posters must avoid inserting such space
(despite the natural-language convention which
permits it) and posting agents should strip it
out.
17 Jan 1994 - 35 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.5
NOTE: Encoded words as components are somewhat
problematic, but are clearly desirable for use in
non-English-speaking nations. They are not sub-
ject to the 14-character limit, and this (plus the
possibility of "/" within them) may require spe-
cial handling in news software.
Encoded words are allowed in newsgroup names ONLY where non-
ASCII characters are necessary to the name, and must use the
"b" encoding [rrr] and the first suitable character set in
the MIME order of preferred character sets [rrr].
NOTE: Since the newsgroup name is the encoded
form, NOT the underlying non-ASCII form, there is
room for terrible confusion here if the choice of
encoding for a particular name is not fully stan-
dardized.
Posters SHOULD use only the names of existing newsgroups in
the Newsgroups header, because newsgroups are NOT created
simply by being posted to. However, it is legitimate to
cross-post to newsgroup(s) which do not exist on the posting
agent's host, provided that at least one of the newsgroups
DOES exist there, and followup agents MUST accept this
(posting agents MAY accept it, but SHOULD at least alert the
poster to the situation and request confirmation). Relayers
MUST not rewrite Newsgroups headers in any way, even if some
or all of the newsgroups do not exist on the relayer's host.
NOTE: Early experience with news software that
created newsgroups when they were mentioned in a
Newsgroups header was thoroughly negative: posters
frequently mistype newsgroup names.
NOTE: While it is legitimate for some of an arti-
cle's newsgroups not to exist on the host where it
is posted, this IS a rather unusual situation
except in followups (which should go to all news-
groups the precursor was posted to, even if not
all of them reach the site where the followup is
being posted).
NOTE: Rewriting Newsgroups headers to strip
locally-unknown newsgroups is superficially
attractive. However, early experience with
exactly that policy was thoroughly negative: news
propagation is more redundant and much less
orderly than many people imagine, and in particu-
lar it is not unheard-of for the (sometimes)
fastest path between two (say) U of Toronto sites
to pass outside U of Toronto... in which case
newsgroup stripping can cause incomplete propaga-
tion. Having an article's set of newsgroups
change as it propagates can also result in
17 Jan 1994 - 36 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.5
followups not achieving the same propagation as
the original. It's been tried; it's more trouble
than it's worth; don't do it.
NOTE: In particular, newsgroup stripping superfi-
cially looks like a solution to the problem of
duplicate regional newsgroup names. For example,
both University of Toronto and University of Texas
have "ut.general" newsgroups, and material cross-
posted to that name and a global newsgroup appears
in both universities' local newsgroups. However,
the side effects of stripping are sufficiently
unacceptable to disqualify it for this purpose.
Don't do it.
Cross-posting an article to several relevant newsgroups is
far superior to posting separate articles with duplicated
content to each newsgroup, because reading agents can detect
the situation and show the article to a reader only once.
Posters SHOULD cross-post rather than duplicate-post.
NOTE: On the other hand, cross-posting to a large
number of newsgroups usually indicates that the
poster has not thought about his audience; arti-
cles are rarely pertinent to more than (say) half
a dozen newsgroups. Posting agents might wish to
request confirmation when the number of newsgroups
exceeds (say) five in the presence of a Followup-
To header, or (say) two in the absence of such a
header.
NOTE: One problem with cross-postings is what to
do with an article cross-posted to a set of news-
groups including both moderated and unmoderated
ones. Posters tend to expect such an article to
show up immediately in the unmoderated newsgroups,
especially if they do not realize that one or more
of the newsgroups is moderated. However, since it
is not possible for a moderator to retroactively
add an already-posted article to a moderated news-
group, the only correct action is to mail such an
article to one (and only one) of the moderators
for action. It is probably best for the posting
agent to detect this situation and ask the poster
what action is preferred. The acceptable choices
are to alter the newsgroup list or to mail to a
moderator of the poster's choice; the posting
agent should NOT offer duplicate-posting as an
easy-to-request option (if only because many mod-
erators will reject a submission that has already
been posted to unmoderated newsgroups).
NOTE: An article cross-posted to multiple moder-
ated newsgroups really should have approval from
17 Jan 1994 - 37 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.5
all the moderators involved. In practice, the
only straightforward way to do this is to send the
article to one of them and have him consult the
others.
A newsgroup SHOULD not appear more than once in the News-
groups header.
Newsgroup names having only one component are reserved for
newsgroups whose propagation is restricted to a single host
(or the administrative equivalent). It is inadvisable to
name a newsgroup "poster" because that word has special
meaning in the Followup-To header (see section 6.1). The
names "control" and "junk" are frequently used for pseudo-
newsgroups internal to relayer implementations, and hence
are also best avoided.
NOTE: Beware of the duplicate-regional-newsgroup-
names problem mentioned above. In particular,
there are many, many hosts with a newsgroup named
"general", and some surprising things show up in
such newsgroups when people cross-post. It is
probably better to use multi-component names,
which are less likely to be duplicated. Fred's
Widget House should use "fwh.general" rather than
just "general" as its in-house general-topics
newsgroup.
It is conventional to reserve newsgroup names beginning with
"to." for test messages sent on an essentially point-to-
point basis (see also the ihave/sendme protocol described in
section 7.2); newsgroup names beginning with "to." SHOULD
not be used for any other purpose. The second (and possibly
later) components of such a name should, together, comprise
the relayer name (see section 5.6) of a relayer. The news-
group exists only at the named relayer and its neighbors.
The neighbors all pass that newsgroup to the named relayer,
while the named relayer does not pass it to anyone.
The order of newsgroup names in the Newsgroups header is not
significant.
5.6. Path
The Path header's content indicates which relayers the arti-
cle has already visited, so that unnecessary redundant
transmission can be avoided:
17 Jan 1994 - 38 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.6
Path-content = [ path-list path-delimiter ] local-part
path-list = relayer-name *( path-delimiter relayer-name )
relayer-name = 1*rn-char
rn-char = letter / digit / "." / "-" / "_"
path-delimiter = "!"
The Path content is a list of relayer names, separated by
path delimiters, followed (after a final delimiter) by the
local part of a mailing address. Each relayer MUST prepend
its name, and a delimiter, to the Path content in all arti-
cles it processes. A relayer MUST not pass an article to a
neighboring relayer whose name is already mentioned in an
article's path list, unless this is explicitly requested by
the neighbor in some way. The Path content is case-
sensitive.
NOTE: The Path header supplied by a posting agent
should normally contain only the local part. The
relayer that the posting agent passes the article
to for posting will prepend its relayer name to
get the path list started.
NOTE: Observe that the trailing local part is NOT
part of the path list. This Path header:
Path: fee!fie!foe!fum
contains three relayer names: "fee", "fie", and
"foe". A relayer named "fum" is still eligible to
be sent this article.
NOTE: This syntax has the disadvantage of contain-
ing no white space, making it impossible to con-
tinue a Path header across several lines. Imple-
mentors of relayers and reading agents are warned
that it is intended that the successor to this
Draft will change the definition of path delimiter
to:
path-delimiter = "!" [ space ]
and are urged to fix their software to handle
(i.e., ignore) white space following the exclama-
tion points. They are urged to hurry; some ill-
behaved systems reportedly already feel free to
add such white space.
NOTE: RFC 1036 allows considerably more flexibil-
ity in choice of delimiter, in theory, but this
flexibility has never been used and most news
software does not implement it properly. The
grammar reflects the current reality. Note, in
particular, that RFC 1036 treats "_" as a delim-
iter, but in fact it is known to appear in relayer
17 Jan 1994 - 39 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.6
names occasionally.
Because an article will not propagate to a relayer already
mentioned in its path list, the path list MUST not contain
any names other than those of relayers the article has
passed through AS NEWS. This is trivially obvious for nor-
mal news articles, but requires attention from the modera-
tors of moderated newsgroups and the implementors and main-
tainers of gateways.
NOTE: For the same reason, a relayer and its
neighbors need to agree on the choice of relayer
name, and names should not be changed without
notifying neighbors.
Relayer names need to be unique among all relayers which
will ever see the articles using them. A relayer name is
normally either an "official" name for the host the relayer
runs on, or some other "official" name controlled by the
same organization. Except in cooperating subnets that agree
to some other convention, and don't let articles using it
escape beyond the subnet, a relayer name MUST be either a
UUCP name registered in the UUCP maps (without any domain
suffix such as ".UUCP"), or a complete Internet domain name.
Use of a (registered) UUCP name is recommended, where prac-
tical, to keep the length of the path list down.
The use of Internet domain names in the path list presents
one problem: domain names are case-insensitive, but the path
list is case-sensitive. Relayers using domain names as
their relayer names MUST pick a standard form for the name,
and use that form consistently to the exclusion of all oth-
ers. The preferred form for this purpose, which relayers
SHOULD use, is the all-lowercase form.
NOTE: It is arguably unfortunate that the path
list is case-sensitive, but it is much too late to
change this. Most Internet sites do, in any
event, use one standardized form of their name
almost everywhere.
In the ordinary case, where the poster is the author of the
article, the local part following the path list SHOULD be
the local part of the poster's full Internet domain mailing
address.
NOTE: It should be just the local part, not the
full address. The character "@" does not appear
in a Path header.
The Path content somewhat resembles a mailing address, par-
ticularly in the UUCP world with its manual routing and "!"
address syntax. Historically, this resemblance was impor-
tant, and the Path content was often used as a reply
17 Jan 1994 - 40 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 5.6
address. This practice has always been somewhat unreliable,
since news paths are not always mail paths and news relayer
names are not always recognized by mail handlers, and its
reliability has generally worsened in recent times. The
widespread use of and recognition of Internet domain
addresses, even outside the actual Internet, has largely
eliminated the problem. Readers SHOULD not use the Path
content as a reply address. On the other hand, relayer
administrators are urged not to break this usage without
good reason; where practical, paths followed by news SHOULD
be traversable by mail, and mail handlers SHOULD recognize
relayer names as host names.
It will typically be difficult or impractical for gateways
and moderators to supply a Path content that is useful as a
reply address for the author, bearing in mind that the path
list they supply will normally be empty. (To reiterate: the
path list MUST not contain any names other than those of
relayers the article has passed through AS NEWS.) They
SHOULD supply a local part that will result in replies to a
Path-derived address being returned to the sender with a
brief explanation. Software permitting, the local part
"not-for-mail" is recommended.
NOTE: A moderator or gateway administrator who
supplies a local part that delivers such mail to
an administrative mailbox will quickly discover
why it should be bounced automatically! It is
best, however, for the returned message to include
an explanation of what has probably happened,
rather than just a mysterious "undeliverable mail"
complaint, since the sender may not be aware that
his/her software is unwisely using the Path con-
tent as a reply address. Reply software might
wish to question attempts to reply to a Path-
derived address ending in "not-for-mail" (which is
why a specific name is being recommended here).
6. Optional Headers
Many MAIL headers, and many of those specified in present
and future MAIL extensions, are potentially applicable to
news. Headers specific to MAIL's point-to-point transmis-
sion paradigm, e.g. To and Cc, SHOULD not appear in news
articles. (Gateways wishing to preserve such information
for debugging probably SHOULD hide it under different names;
prefixing "X-" to the original headers, resulting in e.g.
"X-To", is suggested.)
The following optional headers are either specific to news
or of particular note in news articles; an article MAY con-
tain some or all of them. (Note that there are some circum-
stances in which some of them are mandatory; these are
17 Jan 1994 - 41 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6
explained under the individual headers.) An article MUST
not contain more than one of any of these headers.
6.1. Followup-To
The Followup-To header contents specify which newsgroup(s)
followups should be posted to:
Followup-To-content = Newsgroups-content / "poster"
The syntax is the same as that of the Newsgroups content,
with the exception that the magic word "poster" means that
followups should be mailed to the article's reply address
rather than posted. In the absence of Followup-To, the
default newsgroup(s) for a followup are those in the News-
groups header.
NOTE: The way to request that followups be mailed
to a specific address other than that in the From
line is to supply "Followup-To: poster" and a
Reply-To header. Putting a mailing address in the
Followup-To line is incorrect; posting agents
should reject or rewrite such headers.
NOTE: There is no syntax for "no followups
allowed" because "Followup-To: poster" accom-
plishes this effect without extra machinery.
Although it is generally desirable to limit followups to the
smallest reasonable set of newsgroups, especially when the
precursor was cross-posted widely, posting agents SHOULD not
supply a Followup-To header except at the poster's explicit
request.
NOTE: In particular, it is incorrect for the post-
ing agent to assume that followups to a cross-
posted article should be directed to the first
newsgroup only. Trimming the list of newsgroups
should be the poster's decision, not the posting
agent's. However, when an article is to be cross-
posted to a considerable number of newsgroups, a
posting agent might wish to SUGGEST to the poster
that followups go to a shorter list.
6.2. Expires
The Expires header content specifies a date and time when
the article is deemed to be no longer useful and should be
removed ("expired"):
Expires-content = Date-content
17 Jan 1994 - 42 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.2
The content syntax is the same as that of the Date content.
In the absence of Expires, the default is decided by the
administrators of each host the article reaches, who MAY
also restrict the extent to which the Expires header is hon-
ored.
The Expires header has two main applications: removing arti-
cles whose utility ends on a specific date (e.g., event
announcements which can be removed once the day of the event
is past) and preserving articles expected to be of prolonged
usefulness (e.g., information aimed at new readers of a
newsgroup). The latter application is sometimes abused.
Since individual hosts have local policies for expiration of
news (depending on available disk space, for instance),
posters SHOULD not provide Expires headers for articles
unless there is a natural expiration date associated with
the topic. Posting agents MUST not provide a default
Expires header. Leave it out and allow local policies to be
used unless there is a good reason not to. Expiry dates are
properly the decision of individual host administrators;
posters and moderators SHOULD set only expiry dates that
most administrators would agree with.
NOTE: A poster preparing an Expires header for an
article whose utility ends on a specific day
should typically specify the NEXT day as the
expiry date. A meeting on July 7th remains of
interest on the 7th.
6.3. Reply-To
The Reply-To header content specifies a reply address dif-
ferent from the author's address given in the From header:
Reply-To-content = From-content
In the absence of Reply-To, the reply address is the address
in the From header.
Use of a Reply-To header is preferable to including a simi-
lar request in the article body, because reply-preparation
software can take account of Reply-To automatically.
6.4. Sender
The Sender header identifies the poster, in the event that
this differs from the author identified in the From header:
Sender-content = From-content
In the absence of Sender, the default poster is the author
(named in the From header).
17 Jan 1994 - 43 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.4
NOTE: The intent is that the Sender header have a
fairly high probability of identifying the person
who really posted the article. The ability to
specify a From header naming someone other than
the poster is useful but can be abused.
If the poster supplies a From header, the posting agent MUST
ensure that a Sender header is present, unless it can verify
that the mailing address in the From header is a valid mail-
ing address for the poster. A poster-supplied Sender header
MAY be used, if its mailing address is verifiably a valid
mailing address for the poster; otherwise the posting agent
MUST supply a Sender header and delete (or rename, e.g. to
X-Unverifiable-Sender) any poster-supplied Sender header.
NOTE: It might be useful to preserve a poster-
supplied Sender header so that the poster can sup-
ply the full-name part of the content. The mail-
ing address, however, must be right. Hence, the
posting agent must generate the Sender header if
it is unable to verify the mailing address of a
poster-supplied one.
NOTE: NNTP implementors, in particular, are urged
to note this requirement (which would eliminate
the need for ad hoc headers like NNTP-Posting-
Host), although there are admittedly some imple-
mentation difficulties. A user name from an RFC
1413 server and a host name from an inverse map-
ping of the address, perhaps with a "full name"
comment noting the origin of the information,
would be at least a first approximation:
Sender: fred@zoo.toronto.edu (RFC-1413@reverse-lookup; not verified)
While this does not completely meet the specs, it
comes a lot closer than not having a Sender header
at all. Even just supplying a placeholder for the
user name:
Sender: somebody@zoo.toronto.edu (user name unknown)
would be better than nothing.
6.5. References
The References header content lists message IDs of precur-
sors:
References-content = message-id *( space message-id )
A followup MUST have a References header, and an article
which is not a followup MUST not have a References header.
17 Jan 1994 - 44 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.5
In a followup, if the precursor had a References header, the
message ID of the precursor is appended to the end of the
precursor's References-content to form the followup's Refer-
ences-content. a References header containing the precur-
sor's message ID. A followup to an article which had a Ref-
erences header MUST have a References header containing the
precursor's References content, plus the precursor's message
ID appended to the end of the list.
NOTE: Use the See-Also header (section 6.16) for
interconnection of articles which are not in a
followup relationship to each other.
NOTE: In retrospect, RFCs 850 and 1036, and the
implementations whose practice they represented,
erred here. The proper MAIL header to use for
references to precursors is In-Reply-To, and the
References header is meant to be used for the pur-
poses here ascribed to See-Also. This incompati-
bility is far too solidly established to be fixed,
unfortunately. The best that can be done is to
provide a clear mapping between the two, and urge
gateways to do the transformation. The news usage
is (now) a deliberate violation of the MAIL speci-
fications; articles containing news References
headers are technically not valid MAIL messages,
although it is unlikely that much MAIL software
will notice because the incompatibility is at a
subtle semantic level that does not affect the
syntax.
UNRESOLVED ISSUE: Would it be better to just give
up and admit that news uses References for both
purposes?
Followup agents SHOULD not shorten References headers. If
it is absolutely necessary to shorten the header, as a des-
perate last resort, a followup agent MAY do this by deleting
some of the message IDs. However, it MUST not delete the
first message ID, the last three message IDs (including that
of the immediate precursor), or any message ID mentioned in
the body of the followup. If it is possible for the fol-
lowup agent to determine the Subject content of the articles
identified in the References header, it MUST not delete the
message ID of any article where the Subject content changed
(other than by prepending of a back reference). The fol-
lowup agent MUST not delete any message ID whose local part
ends with "_-_" (underscore (ASCII 95), hyphen (ASCII 45),
underscore); followup agents are urged to use this form to
mark subject changes, and to avoid using it otherwise.
NOTE: As software capable of exploiting References
chains has grown more common, the random shorten-
ing permitted by RFC 1036 has become increasingly
17 Jan 1994 - 45 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.5
troublesome. ANY shortening is undesirable, and
software should do it only in cases of dire neces-
sity. In such cases, these rules attempt to limit
the damage.
NOTE: The first message ID is very important as
the starting point of the "thread" of discussion,
and absolutely should not be deleted. Keeping the
last three message IDs gives thread-following
software a fighting chance to reconstruct a full
thread even if an article or two is missing.
Keeping message IDs mentioned in the body is obvi-
ously desirable.
NOTE: Subject changes are difficult to determine,
but they are significant as possible beginnings of
new threads. The "_-_" convention is provided so
that posting agents (which have more information
about subjects) can flag articles containing a
subject change in a way that followup agents can
detect without access to the articles themselves.
The sequence is chosen as one that is fairly
unlikely to occur by accident.
NOTE: Is "_-_" really worth having?
When a References header is shortened, at least three blanks
SHOULD be left between adjacent message IDs at each point
where deletions were made. Software preparing new Refer-
ences headers SHOULD preserve multiple blanks in older Ref-
erences content.
NOTE: It's desirable to have some marker of where
deletions occurred, but the restricted syntax of
the header makes this difficult. Extra white
space is not a very good marker, since it may be
deleted by software that ill-advisedly rewrites
headers, but at least it doesn't break existing
software.
To repeat: followup agents SHOULD not shorten References
headers.
NOTE: Unfortunately, reading agents and other
software analyzing References patterns have to be
prepared for the worst anyway. The worst includes
random deletions and the possibility of circular
References chains (when References is misused in
place of See-Also, section 6.16).
17 Jan 1994 - 46 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.6
6.6. Control
The Control header content marks the article as a control
message, and specifies the desired actions (other than the
usual ones of filing and passing on the article):
Control-content = verb *( space argument )
verb = 1*( letter / digit )
argument = 1*<ASCII printable character>
The verb indicates what action should be taken, and the
argument(s) (if any) supply details. In some cases, the
body of the article may also contain details. Section 7
describes the standard verbs. See also the Also-Control
header (section 6.15).
NOTE: Control messages are often processed and
filed rather differently than normal articles.
NOTE: The restriction of verbs to letters and dig-
its is new, but is consistent with existing prac-
tice and potentially simplifies implementation by
avoiding characters significant to command inter-
preters. Beware that the arguments are under no
such restriction in general.
NOTE: Two other conventions for distinguishing
control messages from normal articles were for-
merly in use: a three-component newsgroup name
ending in ".ctl" or a subject beginning with
"cmsg " was considered to imply that the article
was a control message. These conventions are
obsolete. Do not use them.
6.7. Distribution
The Distribution header content specifies geographic or
organizational limits on an article's propagation:
Distribution-content = distribution *( dist-delim distribution )
dist-delim = ","
distribution = plain-component
A distribution is syntactically identical to a one-component
newsgroup name, and must satisfy the same rules and restric-
tions. In the absence of Distribution, the default distri-
bution is "world".
NOTE: This syntax has the disadvantage of contain-
ing no white space, making it impossible to con-
tinue a Distribution header across several lines.
Implementors of relayers and reading agents are
warned that it is intended that the successor to
17 Jan 1994 - 47 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.7
this Draft will change the definition of dist
delimiter to:
dist-delim = "," [ space ]
and are urged to fix their software to handle
(i.e., ignore) white space following the commas.
A relayer MUST not pass an article to another relayer unless
configuration information specifies transmission to that
other relayer of BOTH (a) at least one of the article's
newsgroup(s), and (b) at least one of the article's distri-
bution(s). In effect, the only role of distributions is to
limit propagation, by preventing transmission of articles
that would have been transmitted had the decision been based
solely on newsgroups.
A posting agent might wish to present a menu of possible
distributions, or suggest a default, but normally SHOULD not
supply a default without giving the poster a chance to over-
ride it. A followup agent SHOULD initially supply the same
Distribution header as found in the precursor, although the
poster MAY alter this if appropriate.
Despite the syntactic similarity and some historical confu-
sion, distributions are NOT newsgroup names. The whole
point of putting a distribution on an article is that it is
DIFFERENT from the newsgroup(s). In general, a meaningful
distribution corresponds to some sort of region of propaga-
tion: a geographical area, an organization, or a cooperating
subnet.
NOTE: Distributions have historically suffered
from the completely uncontrolled nature of their
name space, the lack of feedback to posters on
incomplete propagation resulting from use of ran-
dom trash in Distribution headers, and confusion
with newsgroups (arising partly because many
regions and organizations DO have internal news-
groups with names resembling their internal dis-
tributions). This has resulted in much garbage in
Distribution headers, notably the pointless prac-
tice of automatically supplying the first compo-
nent of the newsgroup name as a distribution
(which is MOST unlikely to restrict propagation!).
Many sites have opted to maximize propagation of
such ill-formed articles by essentially ignoring
distributions. This unfortunately interferes with
legitimate uses. The situation is bad enough that
distributions must be considered largely useless
except within cooperating subnets that make an
organized effort to restrain propagation of their
internal distributions.
17 Jan 1994 - 48 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.7
NOTE: The distributions "world" and "local" have
no standard magic meaning (except that the former
is the default distribution if none is given).
Some pieces of software do assign such meanings to
them.
6.8. Keywords
The Keywords header content is one or more phrases intended
to describe some aspect of the content of the article:
Keywords-content = plain-phrase *( "," [ space ] plain-phrase )
Keywords, separated by commas, each follow the <plain-
phrase> syntax defined in section 5.2. Encoded words in
keywords MUST not contain characters other than letters (of
either case), digits, and the characters "!", "*", "+", "-",
"/", "=", and "_".
NOTE: Posters and posting agents are asked to take
note that keywords are separated by commas, not by
white space. The following Keywords header con-
tains only one keyword (a rather unlikely and
improbable one):
Keywords: Thompson Ritchie Multics Linux
and should probably have been written:
Keywords: Thompson, Ritchie, Multics, Linux
This particular error is unfortunately rather
widespread.
NOTE: Reading agents and archivers preparing
indexes of articles should bear in mind that user-
chosen keywords are notoriously poor for indexing
purposes unless the keywords are picked from a
predefined set (which they are not in this case).
Also, some followup agents unwisely propagate the
Keywords header from the precursor into the fol-
lowup by default. At least one news-based experi-
ment has found the contents of Keywords headers to
be completely valueless for indexing.
6.9. Summary
The Summary header content is a short phrase summarizing the
article's content:
Summary-content = nonblank-text
17 Jan 1994 - 49 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.9
As with the subject, no restriction is placed on the content
since it is intended solely for display to humans.
NOTE: Reading agents should be aware that the Sum-
mary header is often used as a sort of secondary
Subject header, and (if present) its contents
should perhaps be displayed when the subject is
displayed.
The summary SHOULD be terse. Posters SHOULD avoid trying to
cram their entire article into the headers; even the sim-
plest query usually benefits from a sentence or two of elab-
oration and context, and not all reading agents display all
headers.
6.10. Approved
The Approved header content indicates the mailing addresses
(and possibly the full names) of the persons or entities
approving the article for posting:
Approved-content = From-content *( "," [ space ] From-content )
An Approved header is required in all postings to moderated
newsgroups; the presence or absence of this header allows a
posting agent to distinguish between articles posted by the
moderator (which are normal articles to be posted normally)
and attempted contributions by others (which should be
mailed to the moderator for approval). An Approved header
is also required in certain control messages, to reduce the
probability of accidental posting of same; see the relevant
parts of section 7.
NOTE: There is, at present, no way to authenticate
Approved headers to ensure that the claimed
approval really was bestowed. Nor is there an
established mechanism for even maintaining a list
of legitimate approvers (such a list would quickly
become out of date if it had to be maintained by
hand). Such mechanisms, presumably relying on
cryptographic authentication, would be a worth-
while extension to this Draft, and experimental
work in this area is encouraged. (The problem is
harder than it sounds because news is used on many
systems which do not have real-time access to key
servers.)
NOTE: Relayer implementors, please note well: it
is the POSTING AGENT that is authorized to distin-
guish between moderator postings and attempted
contributions, and to mail the latter to the mod-
erator. As discussed in section 9.1, relayers
MUST not, repeat MUST not, send such mail; on
17 Jan 1994 - 50 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.10
receipt of an unApproved article in a moderated
newsgroup, they should discard the article, NOT
transform it into a mail message (except perhaps
to a local administrator).
NOTE: RFC 1036 restricted Approved to a single
From-content. However, multiple moderation is no
longer rare, and multi-moderator Approved headers
are already in use.
6.11. Lines
The Lines header content indicates the number of lines in
the body of the article:
Lines-content = 1*digit
The line count includes all body lines, including the signa-
ture if any, including empty lines (if any) at beginning or
end of the body. (The single empty separator line between
the headers and the body is not part of the body.) The
"body" here is the body as found in the posted article,
AFTER all transformations such as MIME encodings.
Reading agents SHOULD not rely on the presence of this
header, since it is optional (and some posting agents do not
supply it). They MUST not rely on it being precise, since
it frequently is not.
NOTE: The average line length in article bodies is
surprisingly consistent at about 40 characters,
and since the line count typically is used only
for approximate judgements ("is this too long to
read quickly?"), dividing the byte count of the
body by 40 gives an estimate of the body line
count that is adequate for normal use. This esti-
mate is NOT adequate if the body has been MIME
encoded... but neither is the Lines header, since
at least one major relayer will supply a Lines
header for an article that lacks one, and will not
consider the possibility of MIME encodings when
computing the line count.
NOTE: It would be better to have a Content-Size
header as part of MIME, so that body parts could
have their own sizes, and so that the units used
could be appropriate to the data type (line count
is not a useful measure of the size of an encoded
image, for example). Doing this is preferable to
trying to fix Lines.
UNRESOLVED ISSUE: Update on Content-Size?
17 Jan 1994 - 51 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.11
Relayers SHOULD discard this header if they find it neces-
sary to re-encode the article in such a way that the origi-
nal Lines header would be rendered incorrect.
6.12. Xref
The Xref header content indicates where an article was filed
by the last relayer to process it:
Xref-content = relayer 1*( space location )
relayer = relayer-name
location = newsgroup-name ":" article-locator
article-locator = 1*<ASCII printable character>
The relayer's name is included so that software can deter-
mine which relayer generated the header (and specifically,
whether it really was the one that filed the copy being
examined). The locations specify what newsgroups the arti-
cle was filed under (which may differ from those in the
Newsgroups header) and where it was filed under them. The
exact form of an article locator is implementation-specific.
NOTE: Reading agents can exploit this information
to avoid presenting the same article to a reader
several times. The information is sometimes
available in system databases, but having it in
the article is convenient. Relayers traditionally
generate an Xref header only if the article is
cross-posted, but this is not mandatory, and there
is at least one new application ("mirroring":
keeping news databases on two hosts identical)
where the header is useful in all articles.
NOTE: The traditional form of an article locator
is a decimal number, with articles in each news-
group numbered consecutively starting from 1.
NNTP [rrr] demands that such a model be provided,
and there may be other software which expects it,
but it seems desirable to permit flexibility for
unorthodox implementations.
A relayer inserting an Xref header into an article MUST
delete any previous Xref header. A relayer which is not
inserting its own Xref header SHOULD delete any previous
Xref header. A relayer MAY delete the Xref header when
passing an article on to another relayer.
NOTE: RFC 1036 specified that the Xref header was
not transmitted when an article was passed to
another relayer, but the major news implementa-
tions have never obeyed this rule, and applica-
tions like mirroring depend on this disobedience.
17 Jan 1994 - 52 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.12
A relayer MUST use the same name in Xref headers as it uses
in Path headers. Reading agents MUST ignore an Xref header
containing a relayer name that differs from the one that
begins the path list.
6.13. Organization
The Organization header content is a short phrase identify-
ing the poster's organization:
Organization-content = nonblank-text
This header is typically supplied by the posting agent. The
Organization content SHOULD mention geographical location
(e.g. city and country) when it is not obvious from the
organization's name.
NOTE: The motive here is that the organization is
often difficult to guess from the mailing address,
is not always supplied in a signature, and can
help identify the poster to the reader.
NOTE: There is no "s" in "Organization".
The Organization content is provided for identification
only, and does not imply that the poster speaks for the
organization or that the article represents organization
policy. Posting agents SHOULD permit the poster to override
a local default Organization header.
6.14. Supersedes
The Supersedes header content specifies articles to be can-
celled on arrival of this one:
Supersedes-content = message-id *( space message-id )
Supersedes is equivalent to Also-Control (section 6.15) with
an implicit verb of "cancel" (section 7.1).
NOTE: Supersedes is normally used where the arti-
cle is an updated version of the one(s) being can-
celled.
NOTE: Although the ability to use multiple message
IDs in Supersedes is highly desirable (see section
7.1), posters are warned that existing implementa-
tions often do not correctly handle more than one.
NOTE: There is no "c" in "Supersedes".
17 Jan 1994 - 53 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.15
6.15. Also-Control
The Also-Control header content marks the article as being a
control message IN ADDITION to being a normal news article,
and specifies the desired actions:
Also-Control-content = Control-content
An article with an Also-Control header is filed and passed
on normally, but the content of the Also-Control header is
processed as if it were found in a Control header.
NOTE: It is sometimes desirable to piggyback con-
trol actions on a normal article, so that the
article will be filed normally but will also be
acted on as a control message. This header is
essentially a generalization of Supersedes.
NOTE: Be warned that some old relayers do not
implement Also-Control.
6.16. See-Also
The See-Also header content lists message IDs of articles
that are related to this one but are not its precursors:
See-Also-content = message-id *( space message-id )
See-Also resembles References, but without the restrictions
imposed on References by the followup rules.
NOTE: See-Also provides a way to group related
articles, such as the parts of a single document
that had to be split across multiple articles due
to its size, or to cross-reference between paral-
lel threads.
NOTE: See the discussion (in section 6.5) on MAIL
compatibility issues of References and See-Also.
NOTE: In the specific case where it is desired to
essentially make another article PART of the cur-
rent one, e.g. for annotation of the other arti-
cle, MIME's "message/external-body" convention can
be used to do so without actual inclusion. "news-
message-ID" was registered as a standard external-
body access method, with a mandatory NAME parame-
ter giving the message ID and an optional SITE
parameter suggesting an NNTP site that might have
the article available (if it is not available
locally), by IANA 22 June 1993.
17 Jan 1994 - 54 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.17
6.17. Article-Names
The Article-Names header content indicates any special sig-
nificance the article may have in particular newsgroups:
Article-Names-content = 1*( name-clause space )
name-clause = newsgroup-name ":" article-name
article-name = letter 1*( letter / digit / "-" )
Each name clause specifies a newsgroup (which SHOULD be
among those in the Newsgroups header) and an article name
local to that newsgroup. Article names MAY be used by
relayers to file the article in special ways, or they MAY
just be noted for possible special attention by reading
agents. Article names are case-sensitive.
NOTE: This header provides a way to mark special
postings, such as introductions, frequently-asked-
question lists, etc., so that reading agents have
a way of finding them automatically. The news-
group name is specified for each article name
because the names may be newsgroup-specific; for
example, many frequently-asked-question lists are
posted to "news.answers" in addition to their
"home" newsgroup, and they would not be known by
the same name(s) in both newsgroups.
The Article-Names header SHOULD be ignored unless the arti-
cle also contains an Approved header.
NOTE: This stipulation is made in anticipation of
the possibility that Approved headers will be
involved in cryptographic authentication.
The presence of an Article-Names header does not necessarily
imply that the article will be retained unusually long
before expiration, or that previous article(s) with similar
Article-Names headers will be cancelled by its arrival.
Posters preparing special postings SHOULD include appropri-
ate other headers, such as Expires and Supersedes, to
request such actions.
Different networks MAY establish different sets of article
names for the special postings they deem significant; it is
preferable for usage to be standardized within networks,
although it might be desirable for individual newsgroups to
have different naming conventions in some situations. Arti-
cle names MUST be 14 characters or less. The following
names are suggested but are not mandatory:
intro Introduction to the newsgroup for newcomers.
charter Charter, rules, organization, moderation poli-
cies, etc.
17 Jan 1994 - 55 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.17
background Biographies of special participants, history of
the newsgroup, notes on related newsgroups, etc.
subgroups Descriptions of sub-newsgroups under this news-
group, e.g. "sci.space.news" under "sci.space".
facts Information relating to the purpose of the news-
group, e.g. an acronym glossary in "sci.space".
references Where to get more information: books, journals,
FTP repositories, etc.
faq Answers to frequently-asked questions.
menu If present, a list of all the other article
names local to this newsgroup, with brief
descriptions of their contents.
Such articles may be divided into subsections using the MIME
"multipart/mixed" conventions. If size considerations make
it necessary to split such articles, names ending in a
hyphen and a part number are suggested; for example, a
three-part frequently-asked-questions list could have arti-
cle names "faq-1", "faq-2", and "faq-3".
NOTE: It is somewhat premature to attempt to stan-
dardize article names, since this is essentially a
new feature with no experience behind it. How-
ever, if reading agents are to attach special sig-
nificance to these names, some attempt at standard
conventions is imperative. This is a first
attempt at providing some.
6.18. Article-Replacing
The Article-Replacing header content indicates what previous
articles this one is deemed (by the poster) to replace:
Article-Replacing-content = message-id *( space message-id )
Each message ID identifies a previous article that this one
is deemed to replace. This MUST not cause the previous
article(s) to be cancelled or otherwise altered, unless this
is implied by other headers (e.g. Supersedes); Article-
Replacing is merely an advisory which MAY be noted for spe-
cial attention by reading agents.
NOTE: This header provides a way to mark articles
which are only minor updates of previous ones,
containing no significant new information and not
worth reading if the previous ones have been read.
17 Jan 1994 - 56 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 6.18
NOTE: If suitable conventions using MIME multipart
bodies and the "message/external-body" body-part
type can be developed, a replacing article might
contain only differences between the old text and
the new text, rather than a complete new copy.
This is the motivation for not making Article-
Replacing also function as Supersedes does: the
replacing article might depend on the continued
presence of the replaced article.
UNRESOLVED ISSUE: Is "replacing" a poor name, in
view of this last? "Superseding" unfortunately
introduces other possibilities for confusion.
7. Control Messages
The following sections document the currently-defined con-
trol messages. "Message" is used herein as a synonym for
"article" unless context indicates otherwise.
Posting agents are warned that since certain control mes-
sages require article bodies in quite specific formats, sig-
natures SHOULD not be appended to such articles, and it may
be wise to take greater care than usual to avoid unintended
(although perhaps well-meaning) alterations to text supplied
by the poster. Relayers MUST assume that control messages
mean what they say; they MAY be obeyed as is or rejected,
but MUST not be reinterpreted.
The execution of the actions requested by control messages
is subject to local administrative restrictions, which MAY
deny requests or refer them to an administrator for
approval. The descriptions below are generally phrased in
terms suggesting mandatory actions, but any or all of these
MAY be subject to local administrative approval (either as a
class or case-by-case). Analogously, where the description
below specifies that a message or portion thereof is to be
ignored, this action MAY include reporting it to an adminis-
trator.
NOTE: The exact choice of local action might
depend on what action the control message
requests, who it claims to come from, etc.
Relayers MUST propagate even control messages they do not
understand.
In the following sections, each type of control message is
defined syntactically by defining its arguments and its
body. For example, "cancel" is defined by defining cancel-
arguments and cancel-body.
17 Jan 1994 - 57 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.1
7.1. cancel
The cancel message requests that one or more previous arti-
cles be "cancelled":
cancel-arguments = message-id *( space message-id )
cancel-body = body
The argument(s) identify the articles to be cancelled, by
message ID. The body is a comment, which software MUST
ignore, and SHOULD contain an indication of why the cancel-
lation was requested. The cancel message SHOULD be posted
to the same newsgroup(s), with the same distribution(s), as
the article(s) it is attempting to cancel.
NOTE: Using the same newsgroups and distributions
maximizes the chances of the cancel message propa-
gating everywhere the target articles went.
NOTE: RFC 1036 permitted only a single message-id
in a cancel message. Support for cancelling mul-
tiple articles is highly desirable, especially for
use with Supersedes (see section 6.14). If sev-
eral revisions of an article appear in fast suc-
cession, each using Supersedes to cancel the pre-
vious one, it is possible for a middle revision to
be destroyed by cancellation before it is propa-
gated onward to cancel its predecessor. Allowing
each article to cancel several predecessors
greatly alleviates this problem. (Posting agents
preparing a cancel of an article which itself can-
cels other articles might wish to add those arti-
cles to the cancel-arguments.) However, posters
should be aware that much old software does not
implement multiple cancellation properly, and
should avoid using it when reliable cancellation
is vitally important.
When an article (the "target article") is to be cancelled,
there are four cases of interest: the article hasn't arrived
yet, it has arrived and been filed and is available for
reading, it has expired and been archived on some less-
accessible storage medium, or it has expired and been
deleted. The next few paragraphs discuss each case in turn
(in reverse order, which is convenient for the explanation).
EXPIRED AND DELETED. Take no action.
EXPIRED AND ARCHIVED. If the article is readily accessible
and can be deleted or made unreadable easily, treat as under
AVAILABLE below. Otherwise treat as under EXPIRED AND
DELETED.
17 Jan 1994 - 58 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.1
NOTE: While it is desirable for archived articles
to be cancellable, this can easily involve rewrit-
ing an entire archive volume just to get rid of
one article, perhaps with manual actions required
to arrange it. It is difficult to envision a sit-
uation so dire as to require such measures from
hundreds or thousands of administrators, or for
that matter one in which widespread compliance
with such a request is likely.
AVAILABLE. Compare the mailing addresses from the From
lines of the cancel message and the target article, bearing
in mind that local parts (except for "postmaster") are case-
sensitive and domains are case-insensitive. If they do not
match, either refer the issue to an administrator for a
case-by-case decision, or treat as if they matched.
NOTE: It is generally trivial to forge articles,
so nothing short of cryptographic authentication
is really adequate to ensure that a cancel came
from the original article's author. Moreover, it
is highly desirable to permit authorities other
than the author to cancel articles, to allow for
cases in which the author is unavailable, uncoop-
erative, or malicious, and in which damage and/or
legal problems may be minimized by prompt cancel-
lation. Reliable authentication that would permit
such administrative cancels would be a worthwhile
extension to this Draft, and experimental work in
this area is encouraged.
NOTE: Meanwhile, a simple check of addresses is
useful accident prevention and catches at least
the most simple-minded forgers. Since the intent
is accident prevention rather than ironclad secu-
rity, use of the From address is appropriate, all
the more so because in the presence of gateways
(especially redundant multiple gateways), the
author may not have full control over Sender head-
ers.
NOTE: The "refer... or treat as if they matched"
rule is intended to specifically forbid quietly
ignoring cancels with mismatched addresses.
If the addresses match, then if technically possible, the
relayer MUST delete the target article completely and imme-
diately. Failing that, it MUST make the target article
unreadable (preferably to everyone, minimally to everyone
but the administrator) and either arrange for it to be
deleted as soon as possible or notify an administrator at
once.
17 Jan 1994 - 59 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.1
NOTE: To allow for events such as criminal
actions, malicious forgeries, and copyright
infringements, where damage and/or legal problems
may be minimized by prompt cancellation, complete
removal is strongly preferred over merely making
the target article unreadable. The potential for
malice is outweighed by the importance of really
getting rid of the target article in some legiti-
mate cases. (In cases of inadvertent copyright
violation in particular, the ability to quickly
remedy the violation is of considerable legal
importance.) Failing that, making it unreadable
is better than nothing.
NOTE: Merely annotating the article so that read-
ers see an indication that the author wanted it
cancelled is not acceptable. Making the article
unreadable is the minimum action.
NOTE: There have been experiments with making can-
celled articles unreadable, so that local news
administrators could reverse cancellations. In
practice, administrators almost never find cause
to do so. Removal appears to be clearly prefer-
able where technically feasible.
NOT ARRIVED YET. If practical, retain the cancel message
until the target article does arrive, or until there is no
further possibility of it arriving and being accepted (see
section 9.2), and then treat as under AVAILABLE. Failing
that, arrange for the target article to be rejected and dis-
carded if it does arrive.
NOTE: It may well be impractical to retain the
control message, given uncertainty about whether
the target article will ever arrive. Existing
practice in such cases is to assume that addresses
would match and arrange the equivalent of dele-
tion. This is often done by making a spurious
entry in a database of already-seen message IDs
(see section 9.3), so that if the article does
arrive, it will be rejected as a duplicate.
The cancel message MUST be propagated onward in the usual
fashion, regardless of which of the four cases applied, so
that the target article will be cancelled everywhere even if
cancellation and target article follow different routes.
NOTE: RFC 1036 appeared to require stopping cancel
propagation in the NOT ARRIVED YET case, although
the wording was somewhat unclear. This appears to
have been an unwise decision; there are known
cases of important cancellations (in situations
of, e.g., inadvertent copyright violation)
17 Jan 1994 - 60 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.1
achieving rather poorer propagation than the tar-
get article. News propagation is often a much
less orderly process than the authors of RFC 1036
apparently envisioned. Modern implementations
generally propagate the cancellation regardless.
Posting agents meant for use by ordinary posters SHOULD
reject an attempt to post a cancel message if the target
article is available and the mailing address in its From
header does not match the one in the cancel message's From
header.
NOTE: This, again, is primarily accident preven-
tion.
7.2. ihave, sendme
The ihave and sendme control messages implement a crude
batched predecessor of the NNTP [rrr] protocol. They are
largely obsolete in the Internet, but still see use in the
UUCP environment, especially for backup feeds that normally
are active only when a primary feed path has failed.
NOTE: The ihave and sendme messages defined here
have ABSOLUTELY NOTHING TO DO WITH NNTP, despite
similarities of terminology.
The two messages share the same syntax:
ihave-arguments = *( message-id space ) relayer-name
sendme-arguments = ihave-arguments
ihave-body = *( message-id eol )
sendme-body = ihave-body
Message IDs MUST appear in either the arguments or the body,
but not both. Relayers SHOULD generate the form putting
message IDs in the body, but the other form MUST be sup-
ported for backward compatibility.
NOTE: RFC 1036 made the relayer name optional, but
difficulties could easily ensue in determining the
origin of the message, and this option is believed
to be unused nowadays. Putting the message IDs in
the body is strongly preferred over putting them
in the arguments because it lends itself much bet-
ter to large numbers of message IDs and avoids the
empty-body problem mentioned in section 4.3.1.
The ihave message states that the named relayer has filed
articles with the specified message IDs, which may be of
interest to the relayer(s) receiving the ihave message. The
sendme message requests that the relayer receiving it send
the articles having the specified message IDs to the named
17 Jan 1994 - 61 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.2
relayer.
These control messages are normally sent essentially as
point-to-point messages, by using "to." newsgroups (see sec-
tion 5.5) that are sent only to the relayer the messages are
intended for. The two relayers MUST be neighbors, exchang-
ing news directly with each other. Each relayer advertises
its new arrivals to the other using ihave messages, and each
uses sendme messages to request the articles it lacks.
NOTE: Arguably these point-to-point control mes-
sages should flow by some other protocol, e.g.
mail, but administrative and interfacing issues
are simplified if the news system doesn't need to
talk to the mail system.
To reduce overhead, ihave and sendme messages SHOULD be sent
relatively infrequently and SHOULD contain substantial num-
bers of message IDs. If ihave and sendme are being used to
implement a backup feed, it may be desirable to insert a
delay between reception of an ihave and generation of a
sendme, so that a slightly slow primary feed will not cause
large numbers of articles to be requested unnecessarily via
sendme.
7.3. newgroup
The newgroup control message requests that a new newsgroup
be created:
newgroup-arguments = newsgroup-name [ space moderation ]
moderation = "moderated" / "unmoderated"
newgroup-body = body
/ [ body ] descriptor [ body ]
descriptor = descriptor-tag eol description-line eol
descriptor-tag = "For your newsgroups file:"
description-line = newsgroup-name space description
description = nonblank-text [ " (Moderated)" ]
The first argument names the newsgroup to be created, and
the second one (if present) indicates whether it is moder-
ated. If there is no second argument, the default is
"unmoderated".
NOTE: Implementors are warned that there is occa-
sional use of other forms in the second argument.
It is suggested that such violations of this
Draft, which are also violations of RFC 1036,
cause the newgroup message to be ignored. RFC
1036 was slightly vague about how second arguments
other than "moderated" were to be treated (specif-
ically, whether they were illegal or just
ignored), but it is thought that all existing
17 Jan 1994 - 62 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.3
major implementations will handle "unmoderated"
correctly, and it appears desirable to tighten up
the specs to make it possible for other forms to
be used in future.
The body is a comment, which software MUST ignore, except
that if it contains a descriptor, the description line is
intended to be suitable for addition to a list of newsgroup
descriptions. The description cannot be continued onto
later lines, but is not constrained to any particular
length. Moderated newsgroups have descriptions that end
with the string " (Moderated)" (note that this string begins
with a blank).
NOTE: It is unfortunate that the description line
is part of the body, rather than being supplied in
a header, but this is established practice. News-
group creators are cautioned that the descriptor
tag must be reproduced exactly as given above,
alone on a line, and is case-sensitive. (To
reduce errors in this regard, posting agents might
wish to question or reject newgroup messages which
do not contain a descriptor.) Given the desire
for short lines, description writers should avoid
content-free phrases like "discussion of" and
"news about", and stick to defining what the news-
group is about.
The remainder of the body SHOULD contain an explanation of
the purpose of the newsgroup and the decision to create it.
NOTE: Criteria for newsgroup creation vary widely
and are outside the scope of this Draft, but if
formal procedures of one kind or another were fol-
lowed in the decision, the body should mention
this. Administrators often look for such informa-
tion when deciding whether to comply with cre-
ation/deletion requests.
A newgroup message which lacks an Approved header MUST be
ignored.
NOTE: It would also be desirable to ignore a new-
group message unless its Approved header names a
person who is authorized (in some sense) to create
such a newsgroup. A cooperating subnet with suf-
ficiently strong coordination to maintain a cor-
rect and current list of authorized creators might
wish to do so for its internal newsgroups. It
also (or alternatively) might wish to ignore a
newgroup message for an internal newsgroup that
was posted (or cross-posted) to a non-internal
newsgroup.
17 Jan 1994 - 63 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.3
NOTE: As mentioned in section 6.10, some form of
(cryptographic?) authentication of Approved head-
ers would be highly desirable, especially for con-
trol messages.
It would be desirable to provide some way of supplying a
moderator's address in a newgroup message for a moderated
newsgroup, but this will cause problems unless effective
authentication is available, so it is left for future work.
NOTE: This leaves news administrators stuck with
the annoying chore of arranging proper mailing of
moderated-newsgroup submissions. On Usenet, this
can be simplified by exploiting a forwarding
facility that some major sites provide: they main-
tain forwarding addresses, each the name of a mod-
erated newsgroup with all periods (".", ASCII 46)
replaced by hyphens ("-", ASCII 45), which forward
mail to the current newsgroup moderators. More
advice on the subject of forwarding to moderators
can be found in the document titled "How to Con-
struct the Mailpaths File", posted regularly to
the Usenet newsgroups news.lists, news.admin.misc,
and news.answers.
A newgroup message naming a newsgroup that already exists is
requesting a change in the moderation status or description
of the newsgroup. The same rules apply.
7.4. rmgroup
The rmgroup message requests that a newsgroup be deleted:
rmgroup-arguments = newsgroup-name
rmgroup-body = body
The sole argument is the newsgroup name. The body is a com-
ment, which software MUST ignore; it SHOULD contain an
explanation of the decision to delete the newsgroup.
NOTE: Criteria for newsgroup deletion vary widely
and are outside the scope of this Draft, but if
formal procedures of one kind or another were fol-
lowed in the decision, the body should mention
this. Administrators often look for such informa-
tion when deciding whether to comply with cre-
ation/deletion requests.
A rmgroup message which lacks an Approved header MUST be
ignored.
NOTE: It would also be desirable to ignore a
rmgroup message unless its Approved header names a
17 Jan 1994 - 64 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.4
person who is authorized (in some sense) to delete
such a newsgroup. A cooperating subnet with suf-
ficiently strong coordination to maintain a cor-
rect and current list of authorized deleters might
wish to do so for its internal newsgroups. It
also (or alternatively) might wish to ignore a
rmgroup message for an internal newsgroup that was
posted (or cross-posted) to a non-internal news-
group.
Unexpected deletion of a newsgroup being a disruptive
action, implementations are strongly advised to refer
rmgroup messages to an administrator by default, unless per-
haps the message can be determined to have originated within
a cooperating subnet whose members are considered trustwor-
thy. Abuses have occurred.
7.5. sendsys, version, whogets
The sendsys message requests that a description of the
relayer's news feeds to other relayers be mailed to the
article's reply address:
sendsys-arguments = [ relayer-name ]
sendsys-body = body
If there is an argument, relayers other than the one named
by the argument MUST not respond. The body is a comment,
which software MUST ignore; it SHOULD contain an explanation
of the reason for the request.
The version message requests that the name and version of
the relayer software be mailed to the reply address:
version-arguments =
version-body = body
There are no arguments. The body is a comment, which soft-
ware MUST ignore; it SHOULD contain an explanation of the
reason for the request.
The whogets message requests that a description of the
relayer and its news feeds to other relayers be mailed to
the article's reply address:
whogets-arguments = newsgroup-name [ space relayer-name ]
whogets-body = body
The first argument is the name of the "target newsgroup",
specifying the newsgroup for which propagation information
is desired. This MUST be a complete newsgroup name, not the
name of a hierarchy or a portion of a newsgroup name that is
not itself the name of a newsgroup. If there is a second
17 Jan 1994 - 65 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.5
argument, only the relayer named by that argument should
respond. The body is a comment, which software MUST ignore;
it SHOULD contain an explanation of the reason for the
request.
NOTE: Whogets is intended as a replacement for
sendsys (and version) with a precisely-specified
reply format. Since the syntax for specifying
what newsgroups get sent to what other relayers
varies widely between different forms of relayer
software, the only practical way to standardize
the reply format is to indicate a specific news-
group and ask where THAT newsgroup propagates.
The requirement that it be a complete newsgroup
name is intended to (largely) avoid the problem of
having to answer "yes and no" in cases where not
all newsgroups in a hierarchy are sent.
Any of these messages lacking an Approved header MUST be
ignored. Response to any of these messages SHOULD be
delayed for at least 24 hours, and no response should be
attempted if the message has been cancelled in that time.
Also, no response SHOULD be attempted unless the local part
of the destination address is "newsmap". News administra-
tors SHOULD arrange for mail to "newsmap" on their systems
to be discarded (without reply) unless legitimate use is in
progress.
NOTE: Because these messages can cause many, many
relayers to send mail to one person, such mes-
sages, specifying mailing to an innocent person's
mailbox, have been forged as a half-witted practi-
cal joke. A delay gives administrators time to
notice a fraudulent message and act (by cancelling
the message, preparing to divert the flood of mail
into the bit bucket, or both). Restriction of the
destination address to "newsmap" reduces the
appeal of fraud by making it impossible to use it
to harass a normal user. (A site which does NOT
discard mail to "newsmap", but rather bounces it
back, may incur higher communications costs than
if the mail had been accepted into a user's mail-
box... but a malicious forger could accomplish
this anyway, by using an address whose local part
is very unlikely to be a legitimate mailbox name.)
NOTE: RFC 1036 did not require the Approved header
for these control messages. This has been added
because of the possibility that cryptographic
authentication of Approved headers will become
available.
The body of the reply to a sendsys message SHOULD be of the
form:
17 Jan 1994 - 66 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.5
sendsys-reply = responder 1*sys-line
responder = "Responding-System:" space domain eol
sys-line = relayer-name ":" newsgroup-patterns [ ":" text ] eol
newsgroup-patterns = newsgroup-name *( "," newsgroup-name )
The first line identifies the responding system, using a
syntax resembling a header (but note that it is part of the
BODY). Remaining lines indicate what newsgroups are sent to
what other systems. The syntax of newsgroup patterns is not
well standardized; the form described is common (often with
newsgroup names only partially given, denoting all names
starting with a particular set of components) but not uni-
versal. The whogets message provides a better-defined
alternative.
The reply to a version message is of somewhat ill-defined
form, with a body normally consisting of a single line of
text that somehow describes the version of the relayer soft-
ware. The whogets message provides a better-defined alter-
native.
The body of the reply to a whogets message MUST be of the
form:
whogets-reply = responder-domain responder-relayer response-date
responding-to arrived-via responder-version
whogets-delimiter *pass-line
responder-domain = "Responding-System:" space domain eol
responder-relayer = "Responding-Relayer:" space relayer-name eol
response-date = "Response-Date:" space date eol
responding-to = "Responding-To:" space message-id eol
arrived-via = "Arrived-Via:" path-list eol
responder-version = "Responding-Version:" space nonblank-text eol
whogets-delimiter = eol
pass-line = relayer-name [ space domain ] eol
The first six lines identify the responding relayer by its
Internet domain name (use of the ".uucp" and ".bitnet"
pseudo-domains is permissible, for registered hosts in them,
but discouraged) and its relayer name, specify the date when
the reply was generated and the message ID of the whogets
message being replied to, give the path list (from the Path
header) of the whogets message (which MAY, if absolutely
necessary, be truncated to a convenient length, but MUST
contain at least the leading three relayer names), and indi-
cate the version of relayer software responding. Note that
these lines are part of the BODY even though their format
resembles that of headers. Despite the apparently-fixed
order specified by the syntax above, they can appear in any
order, but there must be exactly one of each.
After those preliminaries, and an empty line to unambigu-
ously define their end, the remaining lines are the relayer
names (which MAY be accompanied by the corresponding domain
17 Jan 1994 - 67 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.5
names, if known) of systems which the responding system
passes the target newsgroup to. Only the names of news
relayers are to be included.
NOTE: It is desirable for a reply to identify its
source by both domain name and relayer name
because news propagation is governed by the latter
but location in a broader context is best deter-
mined by the former. The date and whogets message
ID should, in principle, be present in the MAIL
headers, but are included in the body for robust-
ness in the presence of uncooperative mail sys-
tems. The reason for the path list is discussed
below. Adding version information eliminates the
need for a separate message to gather it.
NOTE: The limitation of pass lines to contain only
names of news relayers is meant to exclude names
used within a single host (as identifiers for mail
gateways, portions of ihave/sendme implementa-
tions, etc.), which do not actually refer to other
hosts.
A relayer which is unaware of the existence of the target
newsgroup MUST not reply to a whogets message at all,
although this MUST not influence decisions on whether to
pass the article on to other relayers.
NOTE: While this may result in discontinuous maps
in cases where some hosts have not honored
requests for creation of a newsgroup, it will also
prevent a flood of useless responses in the event
that a whogets message intended to map a small
region "leaks" out to a larger one. The possibil-
ity of discontinuous recognition of a newsgroup
does make it important that the whogets message
itself continue to propagate (if other criteria
permit). This is also the reason for the inclu-
sion of the whogets message's path list, or at
least the leading portion of it, in the reply: to
permit reconstruction of at least small gaps in
maps.
Different networks set different rules for the legitimacy of
these messages, given that they may reveal details of orga-
nization-internal topology that are sometimes considered
proprietary.
NOTE: On Usenet, in particular, willingness to
respond to these messages is held to be a condi-
tion of network membership: the topology of Usenet
is public information. Organizations wishing to
belong to such networks while keeping their inter-
nal topology confidential might wish to organize
17 Jan 1994 - 68 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.5
their internal news software so that all articles
reaching outsiders appear to be from a single
"gatekeeper" system, with the details of internal
topology hidden behind that system.
UNRESOLVED ISSUE: It might be useful to have a way
to set some sort of hop limit for these.
7.6. checkgroups
The checkgroups control message contains a supposedly
authoritative list of the valid newsgroups within some sub-
set of the newsgroup name space:
checkgroups-arguments =
checkgroups-body = [ invalidation ] valid-groups
/ invalidation
invalidation = "!" plain-component *( "," plain-component ) eol
valid-groups = 1*( description-line eol )
There are no arguments. The body lines (except possibly for
an initial invalidation) each contain a description line for
a newsgroup, as defined under the newgroup message (section
7.3).
NOTE: Some other, ill-defined, forms of the check-
groups body were formerly used. See appendix A.
The checkgroups message applies to all hierarchies contain-
ing any of the newsgroups listed in the body. The check-
groups message asserts that the newsgroups it lists are the
only newsgroups in those hierarchies. If there is an inval-
idation, it asserts that the hierarchies it names no longer
contain any newsgroups.
Processing a checkgroups message MAY cause a local list of
newsgroup descriptions to be updated. It SHOULD also cause
the local lists of newsgroups (and their moderation sta-
tuses) in the mentioned hierarchies to be checked against
the message. The results of the check MAY be used for auto-
matic corrective action, or MAY be reported to the news
administrator in some way.
NOTE: Automatically updating descriptions of
existing newsgroups is relatively safe. In the
case of newsgroup additions or deletions, simply
notifying the administrator is generally the wis-
est action, unless perhaps the message can be
determined to have originated within a cooperating
subnet whose members are considered trustworthy.
NOTE: There is a problem with the checkgroups con-
cept: not all newsgroups in a hierarchy
17 Jan 1994 - 69 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 7.6
necessarily propagate to the same set of machines.
(Notably, there is a set of newsgroups known as
the "inet" newsgroups, which have relatively lim-
ited distribution but coexist in several hierar-
chies with more widely-distributed newsgroups.)
The advice of checkgroups should always be taken
with a grain of salt, and should never be followed
blindly.
8. Transmission Formats
While this Draft does not specify transmission methods
except to place a few constraints on them, there are some
data formats used only for transmission that are unique to
news.
8.1. Batches
For efficient bulk transmission and processing of news arti-
cles, it is often desirable to transmit a number of them as
a single block of data, a "batch". The format of a batch
is:
batch = 1*( batch-header article )
batch-header = "#! rnews " article-size eol
article-size = 1*digit
A batch is a sequence of articles, each prefixed by a header
line that includes its size. The article size is a decimal
count of the octets in the article, counting each EOL as one
octet regardless of how it is actually represented.
NOTE: A relayer might wish to accept either a sin-
gle article or a batch as input. Since "#" cannot
appear in a header name, examination of the first
octet of the input will reveal its nature.
NOTE: In the header line, there is exactly one
blank before "rnews", there is exactly one blank
after "rnews", and the EOL immediately follows the
article size. Beware that some software inserts
non-standard trash after the size.
NOTE: Despite the similarity of this format to the
executable-script format used by some operating
systems, it is EXTREMELY unwise to just feed
incoming batches to a command interpreter in the
anticipation that it will run a command named
"rnews" to process the batch. Unless arrangements
are made to very tightly restrict the range of
commands that can be executed by this means, the
17 Jan 1994 - 70 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 8.1
security implications are disastrous.
8.2. Encoded Batches
When transmitting news, especially over communications links
that are slow or are billed by the bit, it is often desir-
able to batch news and apply data compression to the
batches. Transmission links sending compressed batches
SHOULD use out-of-band means of communication to specify the
compression algorithm being used. If there is no way to
send out-of-band information along with a batch, the follow-
ing encapsulation for a compressed batch MAY be used:
ec-batch = "#! " compression-keyword eol compressed-batch
compression-keyword = "cunbatch"
A line containing a keyword indicating the type of compres-
sion is followed by the compressed batch. The only truly
widespread compression keyword at present is "cunbatch",
indicating compression using the widely-distributed "com-
press" program. Other compression keywords MAY be used by
mutual agreement between the hosts involved.
NOTE: An encapsulated compressed batch is NOT, in
general, a text file, despite having an initial
text line. This combination of text and non-text
data is often awkward to handle; for example,
standard decompression programs cannot be used
without first stripping off the initial line, and
that in turn is painful to do because many text-
handling tools that are superficially suited to
the job do not cope well with non-text data.
Hence the recommendation that out-of-band communi-
cation be used instead when possible.
NOTE: For UUCP transmission, where a batch is typ-
ically transmitted by invoking the remote command
"rnews" with the batch as its input stream, a
plausible out-of-band method for indicating a com-
pression type would be to give a compression key-
word in an option to "rnews", perhaps in the form:
rnews -d decompressor
where "decompressor" is the name of a decompres-
sion program (e.g. "uncompress" for a batch com-
pressed with "compress" or "gunzip" for a batch
compressed with "gzip"). How this decompression
program is located and invoked by the receiving
relayer is implementation-specific.
NOTE: See the notes in section 8.1 on the inadvis-
ability of feeding batches directly to command
17 Jan 1994 - 71 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 8.2
interpreters.
NOTE: There is exactly one blank between "#!" and
the compression keyword, and the EOL immediately
follows the keyword.
8.3. News Within Mail
It is often desirable to transmit news as mail, either for
the convenience of a human recipient or because that is the
only type of transmission available on a restrictive commu-
nication path.
Given the similarity between the news format and the MAIL
format, it is superficially attractive to just send the news
article as a mail message. This is typically a mistake:
mail-handling software often feels free to manipulate vari-
ous headers in undesirable ways (in some cases, such as
Sender, such manipulation is actually mandatory), and mail
transmission problems etc. MUST be reported to the adminis-
trators responsible for the mail transmission rather than to
the article's author. In general, news sent as mail should
be encapsulated to separate the mail headers and the news
headers.
When the intended recipient is a human, any convenient form
of encapsulation may be used. Recommended practice is to
use MIME encapsulation with a content type of "mes-
sage/news", given that news articles have additional seman-
tics beyond what "message/rfc822" implies.
NOTE: "message/news" was registered as a standard
subtype by IANA 22 June 1993.
When mail is being used as a transmission path between two
relayers, however, a standard method is desirable. Cur-
rently the standard method is to send the mail to an address
whose local part is "rnews", with whatever mail headers are
necessary for successful transmission. The news article
(including its headers) is sent as the body of the mail mes-
sage, with an "N" prepended to each line.
NOTE: The "N" reduces the probability of an inno-
cent line in a news article being taken as a magic
command to mail software, and makes it easy for
receiving software to strip off any lines added by
mail software (e.g. the trailing empty line added
by some UUCP mail software).
This method has its weaknesses. In particular, it assumes
that the mail transmission channel can transmit nearly-
arbitrary body text undamaged. When mail is being used as a
transmission path of last resort, however, the mail system
17 Jan 1994 - 72 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 8.3
often has inconvenient preconceived notions about the format
of message bodies. Various ad-hoc encoding schemes have
been used to avoid such problems. The recommended method is
to send a news article or batch as the body of a MIME mail
message, using content type "application/news-transmission"
and MIME's "base64" encoding (which is specifically designed
to survive all known major mail systems).
NOTE: In the process, MIME conventions could be
used to fragment and reassemble an article which
is too large to be sent as a single mail message
over a transmission path that restricts message
length. In addition, the "conversions" parameter
to the content type could be used to indicate what
(if any) compression method has been used. And
the Content-MD5 header [rrr 1544] can be used as a
"checksum" to provide high confidence of detecting
accidental damage to the contents.
NOTE: It might look tempting to use a content type
such as "message/X-netnews", but MIME bans non-
trivial encodings of the entire body of messages
with content type "message". The intent is to
avoid obscuring nested structure underneath encod-
ings. For inter-relayer news transmission, there
is no nested structure of interest, and it is
important that the entire article (including its
headers, not just its body) be protected against
the vagaries of intervening mail software. This
situation appears to fit the MIME description of
circumstances in which "application" is the proper
content type.
NOTE: "application/news-transmission", with a
"conversions" parameter, was registered as a stan-
dard subtype by IANA 22 June 1993.
8.4. Partial Batches
UNRESOLVED ISSUE: The existing batch conventions
assemble (potentially) many articles into one
batch. Handling very large articles would be sub-
stantially less troublesome if there was also a
fragmentation convention for splitting a large
article into several batches. Is this worth
defining at this time?
9. Propagation and Processing
Most aspects of news propagation and processing are imple-
mentation-specific. The basic propagation algorithms, and
certain details of how they are implemented, nevertheless
17 Jan 1994 - 73 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 9
need to be standard.
There are two important principles that news implementors
(and administrators) need to keep in mind. The first is the
well-known Internet Robustness Principle:
Be liberal in what you accept, and conservative in what you send.
However, in the case of news there is an even more important
principle, derived from a much older code of practice, the
Hippocratic Oath (we will thus call this the Hippocratic
Principle):
First, do no harm.
It is VITAL to realize that decisions which might be merely
suboptimal in a smaller context can become devastating mis-
takes when amplified by the actions of thousands of hosts
within a few hours.
9.1. Relayer General Issues
Relayers MUST not alter the content of articles unnecessar-
ily. Well-intentioned attempts to "improve" headers, in
particular, typically do more harm than good. It is neces-
sary for a relayer to prepend its own name to the Path con-
tent (see section 5.6) and permissible for it to rewrite or
delete the Xref header (see section 6.12). Relayers MAY
delete the thoroughly-obsolete headers described in appendix
A.3, although this behavior no longer seems useful enough to
encourage. Other alterations SHOULD be avoided at all
costs, as per the Hippocratic Principle.
NOTE: As discussed in section 2.3, tidying up the
headers of a user-prepared article is the job of
the posting agent, not the relayer. The relayer's
purpose is to move already-compliant articles
around efficiently without damaging them. Note
that in existing implementations, specific pro-
grams may contain both posting-agent functions and
relayer functions. The distinction is that post-
ing-agent functions are invoked only on articles
posted by local posters, never on articles
received from other relayers.
NOTE: A particular corollary of this rule is that
relayers should not add headers unless truly nec-
essary. In particular, this is not SMTP; do not
add Received headers.
Relayers MUST not pass non-conforming articles on to other
relayers, except perhaps in a cooperating subnet that has
agreed to permit certain kinds of non-conforming behavior.
17 Jan 1994 - 74 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 9.1
This is a direct consequence of the Internet Robustness
Principle.
The two preceding paragraphs may appear to be in conflict.
What is to be done when a non-conforming article is
received? The Robustness Principle argues that it should be
accepted but must not be passed on to other relayers while
still non-conforming, and the Hippocratic Principle strongly
discourages attempts at repair. The conclusion that this
appears to lead to is correct: a non-conforming article MAY
be accepted for local filing and processing, or it MAY be
discarded entirely, but it MUST not be passed on to other
relayers.
A relayer MUST not respond to the arrival of an article by
sending mail to any destination, other than a local adminis-
trator, except by explicit prearrangement with the recipi-
ent. Neither posting an article (other than certain types
of control message, see section 7.5) nor being the moderator
of a moderated newsgroup constitutes such prearrangement.
UNDER NO CIRCUMSTANCES WHATSOEVER may a relayer attempt to
send mail to either an article's originator or a moderator.
NOTE: Reporting apparent errors in message compo-
sition is the job of a posting agent, not a
relayer. The same is true of mailing moderated-
newsgroup postings to moderators. In networks of
thousands of cooperating relayers, it is simply
unacceptable for there to be any circumstance
whatsoever that causes any significant fraction of
them to simultaneously send mail to the same des-
tination. (Some control messages are exceptions,
although perhaps ill-advised ones.) What might,
in a smaller network, be a useful notification or
forwarding becomes a deluge of near-identical mes-
sages that can bring mail software to its knees
and severely inconvenience recipients. Modera-
tors, in particular, historically have suffered
grievously from this.
Notification of problems in incoming articles MAY go to
local administrators, or at most (by prearrangement!) to
the administrators of the neighboring relayer(s) that passed
on the problematic articles.
NOTE: It would be desirable to notify the author
that his posting is not propagating as he expects.
However, there is no known method for doing this
that will scale up gracefully. (In particular,
"notify only if within N relayers of the origina-
tor" falls down in the presence of commercial news
services like UUNET: there may be hundreds or
thousands of relayers within a couple of hops of
the originator.) The best that can be done right
17 Jan 1994 - 75 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 9.1
now is to notify neighbors, in hopes that the word
will eventually propagate up the line, or organize
regional monitoring at major hubs.
If it is necessary to alter an article, e.g. translate it to
another character set or alter its EOL representation,
strenuous efforts should be made to ensure that such trans-
formations are reversible, and that relayers or other soft-
ware that might wish to reverse them know exactly how to do
so.
NOTE: For example, a cooperating subnet that
exchanges articles using a non-ASCII character set
like EBCDIC should define a standard, reversible
ASCII-EBCDIC mapping and take pains to see that it
is used at all points where the subnet meets the
outside. If the only reason for using EBCDIC is
that the readers typically employ EBCDIC devices,
it would be more robust to employ ASCII as the
interchange format and do the transformation in
the reading and posting agents.
9.2. Article Acceptance And Propagation
When a relayer first receives an article, it must decide
whether to accept it. (This applies regardless of whether
the article arrived by itself or as part of a batch, and in
principle regardless of whether it originated as a local
posting or as traffic from another relayer.) In a cooperat-
ing subnet with well-controlled propagation paths, some of
the tests specified here MAY be delegated to centrally-
located relayers; that is, relayers that can receive news
ONLY via one of the central relayers might simplify accep-
tance testing based on the assumption that incoming traffic
has already passed the full set of tests at a central
relayer.
The wording that follows is based on a model in which arti-
cles arrive on a relayer's host before acceptance tests are
done. However, depending on the degree of integration of
the transport mechanisms and the relayer, some or all of
these tests MAY be done before the article is actually
transmitted, so that articles which definitely will not be
accepted need not be transmitted at all.
The wording that follows also specifies a particular order
for the acceptance tests. While this order is the obvious
one, the tests MAY be done in any order.
First, the relayer MUST verify that the article is a legal
news article, with all mandatory headers present with legal
contents.
17 Jan 1994 - 76 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 9.2
NOTE: This check in principle is done by the first
relayer to see an article, so an article received
from another relayer should always be legal, but
there is enough old software still operational
that this cannot be taken for granted; see the
discussion of the Internet Robustness Principle in
section 9.1.
Second, the relayer MUST determine whether it has already
seen this article (identified by its message ID). This is
normally done by retaining a history of all article message
IDs seen in the last N days, where the value of N is decided
by the relayer's administrator but SHOULD be at least 7.
Since N cannot practically be infinite, articles whose Date
content indicates that they are older than N days are
declared "stale" and are deemed to have been seen already.
NOTE: This check is important because news propa-
gation topology is typically redundant, often
highly so, and it is not at all uncommon for a
relayer to receive the same article from several
neighbors. The history of already-seen message
IDs can get quite large, hence the desire to limit
its length... but it is important that it be long
enough that slowly-propagating articles are not
classed as stale. News propagation within the
Internet is normally very rapid, but when UUCP
links are involved, end-to-end delays of several
days are not rare, so a week is not a particularly
generous minimum.
NOTE: Despite generally more rapid propagation in
recent times, it is still not unheard-of for some
propagation paths to be very slow. This can
introduce the possibility of old articles arriving
again after they are gone from the history. Hence
the "stale" rule.
Third, the relayer MUST determine whether any of the arti-
cle's newsgroups are "subscribed to" by the host, i.e. fit a
description of what hierarchies or newsgroups the site wants
to receive.
NOTE: This check is significant because informa-
tion on what newsgroups a relayer wishes to
receive is often stored at its neighbors, who may
not have up-to-date information or may simplify
the rules for implementation reasons. As a hedge
against the possibility of missed or delayed new-
group control messages, relayers may wish to
observe a notion of a newsgroup subscription that
is independent of the list of newsgroups actually
known to the relayer. This would permit reception
and relaying of articles in newsgroups that the
17 Jan 1994 - 77 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 9.2
relayer is not (yet) aware of, subject to more
general criteria indicating that they are likely
to be of interest.
Once an article has been accepted, it may be passed on to
other relayers. The fundamental news propagation rule is a
flooding algorithm: on receiving and accepting an article,
send it to all neighboring relayers not already in its path
list that are sent its newsgroup(s) and distribution(s).
NOTE: The path list's role in loop prevention may
appear relatively unimportant, given that looping
articles would typically be rejected as duplicates
anyway. However, the path list's role in prevent-
ing superfluous transmissions is not trivial. In
particular, the path list is the only thing that
prevents relayer X, on receiving an article from
relayer Y, from sending it back to Y again.
(Indeed, the usual symptom of confusion about
relayer names is that incoming news loops back in
this manner.) The looping articles would be
rejected as duplicates, but doubling the communi-
cations load on every news transmission path is
not to be taken lightly!
In general, relayers SHOULD not make propagation decisions
by "anticipation": relayer X, noting that the article's path
list already contains relayer Y, decides not to send it to
relayer Z because X anticipates that Z will get the article
by a better path. If that is generally true, then why is
there a news feed from X to Z at all? In fact, the "better
path" may be running slowly or may be down. News propaga-
tion is very robust precisely because some redundant trans-
mission is done "just in case". If it is imperative to
limit unnecessary traffic on a path, use of NNTP [rrr] or
ihave/sendme (see section 7.2) to pass articles only when
necessary is better than arbitrary decisions not to pass
articles at all.
Anticipation is occasionally justified in special cases.
Such cases should involve both (1) a cooperating subnet
whose propagation paths are well-understood and well-
monitored, with failures and slowdowns noticed and dealt
with promptly, and (2) a persistent pattern of heavy unnec-
essary traffic on a path that is either slow or costly. In
addition, there should be some reason why neither NNTP nor
ihave/sendme is suitable as a solution to the problem.
9.3. Administrator Contact
It is desirable to have a standardized contact address for a
relayer's administrators, in the spirit of the "postmaster"
address for mail administrators. Mail addressed to
17 Jan 1994 - 78 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 9.3
"newsmaster" on a relayer's host MUST go to the administra-
tor(s) of that relayer. Mail addressed to "usenet" on the
relayer's host SHOULD be handled likewise. Mail addressed
to either address on other hosts using the same news
database SHOULD be handled likewise.
NOTE: These addresses are case-sensitive, although
it would be desirable for sequences equivalent to
them using case-insensitive comparison to be han-
dled likewise. While "newsmaster" seems the pre-
ferred network-independent address, by analogy to
"postmaster", there is an existing practice of
using "usenet" for this purpose, and so "usenet"
should be supported if at all possible (especially
on hosts belonging to Usenet!). The address
`news" is also sometimes used for purposes like
this, but less consistently.
10. Gatewaying
Gatewaying of traffic between news networks using this Draft
and those using other exchange mechanisms can be useful, but
must be done cautiously. Gateway administrators are taking
on significant responsibilities, and must recognize that the
consequences of error can be quite serious.
10.1. General Gatewaying Issues
This section will primarily address the problems of gateway-
ing traffic INTO news networks. Little can be said about
the other direction without some specific knowledge of the
network(s) involved. However, the two issues are not
entirely independent: if a non-news network is gatewayed
into a news network at more than one point, traffic injected
into the non-news network by one gateway may appear at
another as a candidate for injection back into the news net-
work.
This raises a more general principle, the single most impor-
tant issue for gatewaying:
Above all, prevent loops.
The normal loop prevention of news transmission is vitally
dependent on the Message-ID header. Any gateway which finds
it necessary to remove this header, alter it, or supersede
it (by moving it into the body), MUST take equally effective
precautions against looping.
NOTE: There are few things more effective at turn-
ing news readers into a lynch mob than a malfunc-
tioning gateway, or pair of gateways, that takes
17 Jan 1994 - 79 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 10.1
in news articles, mangles them just enough to pre-
vent news relayers from recognizing them as dupli-
cates, and regurgitates them back into the news
stream. This happens rather too often.
Gateway implementors should realize that gateways have all
the responsibilities of relayers, plus the added complica-
tions introduced by transformations between different infor-
mation formats. Much of section 9's discussion of relayer
issues is relevant to gateways as well. In particular,
gateways SHOULD keep a history of recently-seen articles, as
described in section 9.2, and not assume that articles will
never reappear. This is particularly important for networks
that have their own concept analogous to message IDs: a
gateway should keep a history of traffic seen from BOTH
directions.
If at all possible, articles entering the non-news network
SHOULD be marked in some way so that they will NOT be re-
gatewayed back into news. Multiple gateways obviously must
agree on the marking method used; if it is done by having
them know each others' names, name changes MUST be coordi-
nated with great care. If marking cannot be done, all
transformations MUST be reversible so that a re-gatewayed
article is identical to the original (except perhaps for a
longer Path header).
Gateways MUST not pass control messages (articles containing
Control, Also-Control, or Supersedes headers) without remov-
ing the headers that make them control messages, unless
there are compelling reasons to believe that they are rele-
vant to both sides and that conventions are compatible. If
it is truly desirable to pass them unaltered, suitable pre-
cautions MUST be taken to ensure that there is NO POSSIBIL-
ITY of a looping control message.
NOTE: The damage done by looping articles is mul-
tiplied a thousandfold if one of the affected
articles is something like a sendsys message (see
section 7.3) that requests multiple automatic
replies. Most gateways simply should not pass
control messages at all. If some unusual reason
dictates doing so, gateway implementors and admin-
istrators are urged to consider bulletproof rate-
limiting measures for the more destructive ones
like sendsys, e.g. passing only one per hour no
matter how many are offered.
Gateways, like relayers, SHOULD make determined efforts to
avoid mangling articles unnecessarily. In the case of gate-
ways, some transformations may be inevitable, but keeping
them to a minimum and ensuring that they are reversible is
still highly desirable.
17 Jan 1994 - 80 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 10.1
Gateways MUST avoid destroying information. In particular,
the restrictions of section 4.2.2 are best taken with a
grain of salt in the context of gateways. Information that
does not translate directly into news headers SHOULD be
retained, perhaps in "X-" headers, both because it may be of
interest to sophisticated readers and because it may be cru-
cial to tracing propagation problems.
Gateway implementors should take particular note of the dis-
cussion of mailed replies, or more precisely the ban on
same, in section 9.1. Gateway problems MUST be reported to
the local administration, not to the innocent originator of
traffic. "Gateway problems" here includes all forms of
propagation anomaly on the non-news side of the gateway,
e.g. unreachable addresses on a mailing list. Note that
this requires consideration of possible misbehavior of
"downstream" hosts, not just the gateway host.
10.2. Header Synthesis
News articles prepared by gateways MUST be legal news arti-
cles. In particular, they MUST include all of the mandatory
headers (see section 5) and MUST fully conform to the
restrictions on said headers. This often requires that a
gateway function not only as a relayer, but also partly as a
posting agent, aiding in the synthesis of a conforming arti-
cle from non-conforming input.
NOTE: The full-conformance requirement needs par-
ticularly careful attention when gatewaying mail-
ing lists to news, because a number of constructs
that are legal in MAIL headers are NOT permissible
in news headers. (Note also that not all mail
traffic fully conforms to even the MAIL specifica-
tion.) The rest of this section will be phrased
in terms of mail-to-news gatewaying, but most of
it is more generally applicable.
The mandatory headers generally present few problems.
If no date information is available, the gateway should sup-
ply a Date header with the gateway's current date. If only
partial information is available (e.g. date but not time),
this should be fleshed out to a full Date header by adding
default values, not by mixing in parts of the gateway's cur-
rent date. (Defaults should be chosen so that fleshed-out
dates will not be in the future!) It may be necessary to
map timezone information to the restricted forms permitted
in the news Date header. See section 5.1.
NOTE: The prohibition of mixing dates is on the
theory that it is better to admit ignorance than
to lie.
17 Jan 1994 - 81 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 10.2
If the author's address as supplied in the original message
is not suitable for inclusion in a From header, the gateway
MUST transform it so it is, e.g. by use of the "% hack" and
the domain address of the gateway. The desire to preserve
information is NOT an excuse for violating the rules. If
the transformation is drastic enough that there is reason to
suspect loss of information, it may be desirable to include
the original form in an X- header, but the From header's
contents MUST be as specified in section 5.2.
If the message contains a Message-ID header, the contents
should be dealt with as discussed in section 10.3. If there
is no message ID present, it will be necessary to synthesize
one, following the news rules (see section 5.3).
Every effort should be made to produce a meaningful Subject
header; see section 5.4. Many news readers select articles
to read based on Subject headers, and inserting a place-
holder like "<no subject available>" is considered highly
objectionable. Even synthesizing a Subject header by pick-
ing out the first half-dozen nouns and adjectives in the
article body is better than using a placeholder, since it
offers SOME indication of what the article might contain.
The contents of the Newsgroups header (section 5.5) are usu-
ally predetermined by gateway configuration, but a gateway
to a network that has its own concept of newsgroups or dis-
cussions might have to make transformations. Such transfor-
mations should be reversible; otherwise confusion is likely
on both sides.
It will rarely be possible for gateways to provide a Path
header that is both an accurate history of the relayers the
article has passed through AS NEWS and a usable reply
address. The history function MUST be given priority; see
the discussion in section 5.6. It will usually be necessary
for a gateway to supply an empty path list, abandoning the
reply function.
It is desirable for gatewayed articles to convey as much
useful information as possible, e.g. by use of optional news
headers (see section 6) when the relevant information is
available. Synthesis of optional headers can generally fol-
low similar rules.
Software synthesizing References headers should note the
discussion in section 6.5 concerning the incompatibility
between MAIL and news. Also of interest is the possibility
of incorporating information from In-Reply-To headers and
from attribution lines in the body; an incomplete or some-
what conjectural References header is much better than none
at all, and reading agents already have to cope with incom-
17 Jan 1994 - 82 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 10.2
plete or slightly erroneous References lists.
10.3. Message ID Mapping
This section, like the previous one, is phrased in terms of
mail being gatewayed into news, but most of the discussion
should be more generally applicable.
A particularly sticky problem of gatewaying mail into news
is supplying legal news message IDs. Note, in particular,
that not all MAIL message IDs are legal in news; the news
syntax (specified in section 5.3, with related material in
5.2) is more restrictive. Generating a fully-conforming
news article from a mail message may require transforming
the message ID somewhat.
Generation and transformation of message IDs assumes partic-
ular importance if a given mailing list (or whatever) is
being handled by more than one gateway. It is highly desir-
able that the same article contents not appear twice in the
same newsgroup, which requires that they receive the same
message ID from all gateways. Gateways SHOULD use the fol-
lowing algorithm (possibly modified by the later discussion
of gatewaying into more than one newsgroup) unless local
considerations dictate another:
1. Separate message ID from surroundings, if necessary.
A plausible method for this is to start at the first
"<", end at the next ">", and reject the message if
no ">" is found or a second "<" is seen before the
">". Also reject the message if the message ID con-
tains no "@" or more than one "@", or if it contains
no ".". Also reject the message if the message ID
contains non-ASCII characters, ASCII control charac-
ters, or white space.
NOTE: Any legitimate domain will include at
least one ".". RFC 822 section 6.2.2 forbids
white space in this context when passing mail
on to non-MAIL software.
2. Delete the leading "<" and trailing ">". Separate
message ID into local part and domain at the "@".
3. In both components, transliterate leading dots
(".", ASCII 46), trailing dots, and dots after the
first in sequences of two or more consecutive
dots, into underscores (ASCII 95).
4. In both components, transliterate disallowed char-
acters other than dots (see the definition of
<unquoted-char> in section 5.2) to underscores
(ASCII 95).
17 Jan 1994 - 83 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 10.3
5. Form the message ID as
"<" local-part "@" domain ">"
NOTE: This algorithm is approximately that of Rich
Salz's successful gatewaying package.
Despite the desire to keep message IDs consistent across
multiple gateways, there is also a more subtle issue that
can require a different approach. If the same articles are
being gatewayed into more than one newsgroup, and it is not
possible to arrange that all gateways gateway them to the
same cross-posted set of newsgroups, then the message IDs in
the different newsgroups MUST be DIFFERENT.
NOTE: Otherwise, arrival of an article in one
newsgroup will prevent it from appearing in
another, and which newsgroup a particular article
appears in will be an accident of which direction
it arrives from first. It is very difficult to
maintain a coherent discussion when each partici-
pant sees a randomly-selected 50% of the traffic.
The fundamental problem here is that the basic
assumption behind message IDs is being violated:
the gateways are assigning the same message ID to
articles that differ in an important respect
(Newsgroups header).
In such cases, it is suggested that the newsgroup name, or
an agreed-on abbreviation thereof, be prepended to the local
part of the message ID (with a separating ".") by the gate-
way. This will ensure that multiple gateways generate the
same message ID, while also ensuring that different news-
groups can be read independently.
NOTE: It is preferable to have the gateway(s)
cross-post the article, avoiding the issue alto-
gether, but this may not be feasible, especially
if one newsgroup is widespread and the other is
purely local.
10.4. Mail to and from News
Gatewaying mail to news, and vice-versa, is the most obvious
form of news gatewaying. It is common to set up gateways
between news and mail rather too casually.
It is hard to go very wrong in gatewaying news into a mail-
ing list, except for the non-trivial matter of making sure
that error reports go to the local administration rather
than to the authors of news articles. (This requires atten-
tion to the "envelope address" as well as to the message
17 Jan 1994 - 84 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 10.4
headers.) Doing the reverse connection correctly is much
harder than it looks.
NOTE: In particular, just feeding the mail message
to "inews -h" or the equivalent is NOT, repeat
NOT, adequate to gateway mail to news. Signifi-
cant gatewaying software is necessary to do it
right. Not all headers of mail messages conform
to even the MAIL specifications, never mind the
stricter rules for news.
It is useful to distinguish between two different forms of
mail-to-news gatewaying: gatewaying a mailing list into a
newsgroup, and operating a "post-by-mail" service in which
individual articles can be posted to a newsgroup by mailing
them to a specific address. In the first case, the message
is already being "broadcast", and the situation can be
viewed as gatewaying one form of news into another. The
second case is closer to that of a moderator posting submis-
sions to a moderated newsgroup.
In either case, the discussions in the preceding two sec-
tions are relevant, as is the Hippocratic Principle of sec-
tion 9. However, some additional considerations are spe-
cific to mail-to-news gatewaying.
As mentioned in section 6, point-to-point headers like To
and Cc SHOULD not appear as such in news, although it is
suggested that they be transformed to "X-" headers, e.g. X-
To and X-Cc, to preserve their information content for pos-
sible use by readers or troubleshooters. The Received
header is entirely specific to MAIL and SHOULD be deleted
completely during gatewaying, except perhaps for the
Received header supplied by the gateway host itself.
The Sender header is a tricky case, one where mailing-list
and post-by-mail practice should differ. For gatewaying
mailing lists, the mailing-list host should be considered a
relayer, and the From and Sender headers supplied in its
transmissions left strictly untouched. For post-by-mail, as
for a moderator posting a mailed submission, the Sender
header should reflect the poster rather than the author. If
a post-by-mail gateway receives a message with its own
Sender header, it might wish to preserve the content in an
X-Sender header.
It will generally be necessary to transform between mail's
In-Reply-To/References convention and news's References/See-
Also convention, to preserve correct semantics of cross ref-
erences. This also requires attention when going the other
way, from news to mail. See the discussion of the differ-
ence in section 6.5.
17 Jan 1994 - 85 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 10.5
10.5. Gateway Administration
Any news system will benefit from an attentive administra-
tor, preferably assisted by automated monitoring for anoma-
lies. This is particularly true of gateways. Gateway soft-
ware SHOULD be instrumented so that unusual occurrences,
such as sudden massive surges in traffic, are reported
promptly. It is desirable, in fact, to go further: gateway
software SHOULD endeavour to limit damage in the event that
the administrator does not respond promptly.
NOTE: For example, software might limit the gate-
waying rate by queueing incoming traffic and emp-
tying the queue at a finite maximum rate (well
below the maximum that the host is capable of!)
which is set by the administrator and is not
raised automatically.
Traffic gatewayed into a news network SHOULD include a suit-
able header, perhaps X-Gateway-Administrator, giving an
electronic address that can be used to report problems.
This SHOULD be an address that goes direct to a human, not
to a "routine administrative issues" mailbox that is exam-
ined only occasionally, since the point is to be able to
reach the administrator quickly in an emergency. Gateway
administrators SHOULD arrange substitutes to cover gateway
operation (with suitable redirection of mail) when they are
on vacation etc.
11. Security And Related Issues
Although the interchange format itself raises no significant
security issues, the wider context does.
11.1. Leakage
The most obvious form of security problem with news is
"leakage" of articles which are intended to have only
restricted circulation. The flooding algorithm is EXTREMELY
good at finding any path by which articles can leave a sub-
net with supposedly-restrictive boundaries. Substantial
administrative effort is required to ensure that local news-
groups remain local, unless connections to the outside world
are tightly restricted.
A related problem is that the sendme control message can be
used to ask for any article by its message ID. The useful-
ness of this has declined as message-ID generation algo-
rithms have become less predictable, but it remains a poten-
tial problem for "secure" newsgroups. Hosts with such news-
groups may wish to disable the sendme control message
entirely.
17 Jan 1994 - 86 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 11.1
The sendsys, version, and whogets control messages also
allow "outsiders" to request information from "inside",
which may reveal details of internal topology (etc.) that
are considered confidential. (Note that at least limited
openness about such matters may be a condition of membership
in such networks, e.g. Usenet.)
Organizations wishing to control these forms of leakage are
strongly advised to designate a small number of "official
gateway" hosts to handle all news exchange with the outside
world, so that a bounded amount of administrative effort is
needed to control propagation and eliminate problems.
Attempts to keep news out entirely, by refusing to support
an official gateway, typically result in large numbers of
unofficial partial gateways appearing over time. Such a
configuration is much more difficult to troubleshoot.
A somewhat-related problem is the possibility of proprietary
material being disclosed unintentionally by a poster who
does not realize how far his words will propagate, either
from sheer misunderstanding or because of errors made (by
human or software) in followup preparation. There is little
that can be done about this except education.
11.2. Attacks
Although the limitations of the medium restrict what can be
done to attack a host via news, some possibilities exist,
most of them problems news shares with mail.
If reading agents are careless about transmitting non-
printable characters to output devices, malicious posters
may post articles containing control sequences ("letter-
bombs") meant to have various destructive effects on output
devices. Possible effects depend on the device, but they
can include hardware damage (e.g. by repeated writing of
values into configuration memories that can tolerate only a
limited number of write cycles) and security violation (e.g.
by reprogramming function keys potentially used by privi-
leged readers).
A more sophisticated variation on the letterbomb is inclu-
sion of "Trojan horses" in programs. Obviously, readers
must be cautious about using software found in news, but
more subtly, reading agents must also exercise care. MIME
messages can include material that is executable in some
sense, such as PostScript documents (which are programs!),
and letterbombs may be introduced into such material.
Given the presence of finite resources and other software
limitations, some degree of system disruption can be
achieved by posting otherwise-innocent material in great
volume, either in single huge articles (see section 4.6) or
17 Jan 1994 - 87 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 11.2
in a stream of modest-sized articles. (Some would say that
the steady growth of Usenet volume constitutes a subtle and
unintentional attack of the latter type; certainly it can
have disruptive effects if administrators are inattentive.)
Systems need some ability to cope with surges, because sin-
gle huge articles occur occasionally as the result of soft-
ware error, innocent misunderstanding, or deliberate malice,
and downtime at upstream hosts can cause droughts, followed
by floods, of legitimate articles. (There is also a certain
amount of normal variation; for example, Usenet traffic is
noticeably lighter on weekends and during Christmas holi-
days, and rises noticeably at the start of the school term
of North American universities.) However, a site that nor-
mally receives little traffic may be quite vulnerable to
"swamping" attack if its software is insufficiently careful.
In general, careless implementation may open doors that are
not intrinsic to news. In particular, implementation of
control messages (see sections 6.6 and 7) and unbatchers
(see section 8.1 and 8.2) via a command interpreter requires
substantial precautions to ensure that only the intended
capabilities are available. Care must also be taken that
article-supplied text is not fed to programs that have
escapes to command interpreters.
Finally, there is considerable potential for malice in the
sendsys, version, and whogets control messages. They are
not harmful to the hosts receiving them as news, but they
can be used to enlist those hosts (by the thousands) as
unwitting allies in a mail-swamping attack on a victim who
may not even receive news. The precautions discussed in
section 7.5 can reduce the potential for such attacks con-
siderably, but the hazard cannot be eliminated as long as
these control messages exist.
11.3. Anarchy
The highly distributed nature of news propagation, and the
lack of adequate authentication protocols (especially for
use over the less-interactive transport mechanisms such as
UUCP), make article forgery relatively straightforward. It
may be possible to at least track a forgery to its source,
once it is recognized as such, but clever forgers can make
even that relatively difficult. The assumption that forg-
eries will be recognized as such is also not to be taken for
granted; readers are notoriously prone to blindly assuming
authenticity. If a forged article's initial path list
includes the relayer name of the supposed poster's host, the
article will never be sent to that host, and the alleged
author may learn about the forgery secondhand or not at all.
A particularly noxious form of forgery is the forged "can-
cel" control message. Notably, it is relatively
17 Jan 1994 - 88 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. 11.3
straightforward to write software that will automatically
send out a (forged) cancel message for any article meeting
some criterion, e.g. written by a specific author. The
authentication problems discussed in section 7.1 make it
difficult to solve this without crippling cancel's important
functionality.
A related problem is the possibility of disagreements over
newsgroup creation, on networks where such things are not
decided by central authorities. There have been cases of
"rmgroup wars", where one poster persistently sends out new-
group messages to create a newsgroup and another, equally
persistently, sends out rmgroup messages asking that it be
removed. This is not particularly damaging, if relayers are
configured to be cautious, but can cause serious confusion
among innocent third parties who just want to know whether
they can use the newsgroup for communication or not.
11.4. Liability
News shares the legal uncertainty surrounding other forms of
electronic communication: what rules apply to this new
medium of information exchange? News is a particularly
problematic case because it is a broadcast medium rather
than a point-to-point one like mail, and analogies to older
forms of communication are particularly weak.
Are news-carrying hosts common carriers, like the phone com-
panies, providing communications paths without having either
authority over or responsibility for content? Or are they
publishers, responsible for the content regardless of
whether they are aware of it or not? Or something in
between? Such questions are particularly significant when
the content is technically criminal, e.g. some types of sex-
ually-oriented material in some jurisdictions, in which case
ignorance of its presence may not be an adequate defence.
Even in milder situations such as libel or copyright viola-
tion, the responsibilities of the poster, his host, and
other hosts carrying the traffic are unclear. Note, in par-
ticular, the problems arising when the article is a forgery,
or when the alleged author claims it is a forgery but cannot
prove this.
A. Archeological Notes
A.1. A-News Article Format
The obsolete "A News" article format consisted of exactly
five lines of header information, followed by the body. For
example:
17 Jan 1994 - 89 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. A.1
Aeagle.642
news.misc
cbosgd!mhuxj!mhuxt!eagle!jerry
Fri Nov 19 16:14:55 1982
Usenet Etiquette - Please Read
body
body
body
The first line consisted of an "A" followed by an article ID
(analogous to a message ID and used for similar purposes).
The second line was the list of newsgroups. The third line
was the path. The fourth was the date, in the format above
(all fields fixed width), resembling an Internet date but
not quite the same. The fifth was the subject.
This format is documented for archeological purposes only.
Do not generate articles in this format.
A.2. Early B-News Article Format
The obsolete pseudo-Internet article format, used briefly
during the transition between the A News format and the mod-
ern format, followed the general outline of a MAIL message
but with some non-standard headers. For example:
From: cbosgd!mhuxj!mhuxt!eagle!jerry (Jerry Schwarz)
Newsgroups: news.misc
Title: Usenet Etiquette -- Please Read
Article-I.D.: eagle.642
Posted: Fri Nov 19 16:14:55 1982
Received: Fri Nov 19 16:59:30 1982
Expires: Mon Jan 1 00:00:00 1990
body
body
body
The From header contained the information now found in the
Path header, plus possibly the full name now typically found
in the From header. The Title header contained what is now
the Subject content. The Posted header contained what is
now the Date content. The Article-I.D. header contained an
article ID, analogous to a message ID and used for similar
purposes. The Newsgroups and Expires headers were approxi-
mately as now. The Received header contained the date when
the latest relayer to process the article first saw it. All
dates were in the above format, with all fields fixed width,
resembling an Internet date but not quite the same.
This format is documented for archeological purposes only.
17 Jan 1994 - 90 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. A.2
Do not generate articles in this format.
A.3. Obsolete Headers
Early versions of news software following the modern format
sometimes generated headers like the following:
Relay-Version: version B 2.10 2/13/83; site cbosgd.UUCP
Posting-Version: version B 2.10 2/13/83; site eagle.UUCP
Date-Received: Friday, 19-Nov-82 16:59:30 EST
Relay-Version contained version information about the
relayer that last processed the article. Posting-Version
contained version information about the posting agent that
posted the article. Date-Received contained the date when
the last relayer to process the article first saw it (in a
slightly nonstandard format).
These headers are documented for archeological purposes
only. Do not generate articles using them.
A.4. Obsolete Control Messages
There once was a senduuname control message, resembling
sendsys but requesting transmission of the list of hosts
that the receiving host had UUCP connections to. This
rapidly ceased to be of much use, and many organizations
consider information about their internal connectivity to be
confidential.
Historically, a checkgroups body consisting of one or two
lines, the first of the form "-n newsgroup", caused check-
groups to apply to only that single newsgroup. This form is
documented for archeological purposes only; do not use it.
Historically, an article posted to a newsgroup whose name
had exactly three components of which the third was "ctl"
signified that article was to be taken as a control message.
The Subject header specified the actions, in the same way
the Control header does now. This form is documented for
archeological purposes only; do not use it; do not implement
it.
B. A Quick Tour Of MIME
(The editor wishes to thank Luc Rooijakkers; most of this
appendix is a lightly-edited version of a summary he kindly
supplied.)
MIME (Multipurpose Internet Mail Extensions) is an upward-
compatible set of extensions to RFC 822, currently
17 Jan 1994 - 91 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. B
documented in RFCs 1341 and 1342. This appendix summarizes
these documents. See the MIME RFCs for more information;
they are very readable.
UNRESOLVED ISSUE: These RFC numbers (here and
elsewhere in this Draft) need updating when the
new MIME RFCs come out.
MIME defines the following new headers:
MIME-Version
Content-Type
Content-Transfer-Encoding
Content-ID
The MIME-Version header is mandatory for all messages con-
forming to the MIME specification and carries the version
number of the MIME specification. Example:
MIME-Version: 1.0
The Content-Type header indicates the content type of the
message. Content types are split into a top-level type and
a subtype, separated by a slash. Auxiliary information can
also be supplied, using an attribute-value notation. Exam-
ple:
Content-Type: text/plain; charset=us-ascii
(In the absence of a Content-Type header this is in fact the
default content type.)
Important type/subtype combinations are
text/plain Plain text, possibly in a non-
ASCII character set.
text/enriched A very simple wordprocessor-like
language supporting character
attributes (e.g., underlining),
justification control, and multi-
ple character sets. (This pro-
posal has gone through several
iterations and has recently split
off from the main MIME RFCs into a
separate document.)
message/rfc822 A mail message conforming to a
slightly-relaxed version of RFC
822.
17 Jan 1994 - 92 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. B
message/partial Part of a message (supporting the
transparent splitting and joining
of messages when they are too
large to be handled by some trans-
port agent).
message/external-body A message whose body is external.
Possible access methods include
via mail, FTP, local file, etc.
multipart/mixed A message whose body consists of
multiple parts, possibly of dif-
ferent types, intended to be
viewed in serial order. Each part
looks like an RFC 822 message,
consisting of headers and a body.
Most of the RFC 822 headers have
no defined semantics for body
parts.
multipart/parallel Likewise, except that the parts
are intended to be viewed in par-
allel (on user agents that support
it).
multipart/alternative Likewise, except that the parts
are intended to be semantically
equivalent such that the part that
best matches the capabilities of
the environment should be dis-
played. For example, a message
may include plain-text, enriched-
text, and postscript versions of
some document.
multipart/digest A variant of multipart/mixed espe-
cially intended for message
digests (the default type of the
parts is message/rfc822 instead of
text/plain, saving on the number
of headers for the parts).
application/postscript A PostScript document.
(PostScript is a trademark of
Adobe.)
Other top-level types exist for still images, audio, and
video samples.
Some of the above types require the ability to transport
binary data. Since the existing message systems usually do
not support this, MIME provides a Content-Transfer-Encoding
header to indicate the kind of encoding used. The possible
encodings are:
17 Jan 1994 - 93 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. B
7bit No encoding; the data consists of short
(less than 1000 characters) lines of
7-bit ASCII data, delimited by EOL
sequences. This is the default encod-
ing.
8bit Like 7bit, except that bytes with the
high-order bit set may be present.
Many transmission paths are incapable
of carrying messages which use this
encoding.
binary No encoding; any sequence of bytes may
be present. Many transmission paths
are incapable of carrying messages
which use this encoding.
base64 The data is encoded by representing
every group of 3 bytes as 4 characters
from the alphabet "A-Za-z0-9+/", which
was chosen for its high robustness
through mail gateways (the alphabet
used by uuencode does not survive
ASCII-EBCDIC-ASCII translations). In
the final group of 4 characters, "=" is
used for those characters not repre-
senting data bytes. Line length is
limited and EOLs in the encoded form
are ignored.
quoted-printable Any byte can be represented by a three
character "=XX" sequence where the X's
are upper case hexadecimal digits.
Bytes representing printable 7-bit US-
ASCII characters except "=" may be rep-
resented literally. Tabs and blanks
may be represented literally if not at
the end of a line. Line length is lim-
ited, and an EOL preceded by "=" was
inserted for this purpose and is not
present in the original.
The base64 and quoted-printable encodings are applied to
data in Internet canonical form, which means that any EOL
encoded as anything but EOL must be an Internet canonical
EOL: CR followed by LF.
The Content-Description header allows further description of
a body part, analogous to the use of Subject for messages.
Finally, the Content-ID header can be used to assign an
identification to body parts, analogous to the assignment of
identifications to messages by Message-ID.
17 Jan 1994 - 94 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. B
Note that most of these headers are structured header
fields, as defined in RFC 822. Consequently, comments are
allowed in their values. The following is a legal MIME
header:
Content-Type: (a comment) text (yeah) /
plain (and now some params:) ; charset= (guess what)
iso-8859-1 (we don't have iso-10646 yet, pity)
NOTE: Although the MIME specification was devel-
oped for mail, there is nothing precluding its use
for news as well. While it might simplify imple-
mentation to restrict the MIME headers somewhat,
in the same way that other news headers (e.g.
From) are restricted subsets of the RFC-822 origi-
nals, this would add yet another divergence
between two formats that ought to be as compatible
as possible. In the case of the MIME headers,
there is no body of existing code posing compati-
bility concerns. A full-featured MIME reading
agent needs a full RFC-822 parser anyway, to prop-
erly handle body parts of types like mes-
sage/rfc822, so there is little gain from
restricting MIME headers. Adopting the MIME spec-
ification unchanged seems best. However, article-
level MIME headers must still comply with the
overall news header syntax given in section 4, so
that news software which is NOT interested in MIME
need not contain a full RFC-822 parser.
The second part of MIME, RFC 1342 (Representation of Non-
ASCII Text in Internet Message Headers), addresses the prob-
lem of non-ASCII characters in headers. An example of a
header using the RFC 1342 mechanism is
From: =?ISO-8859-1?Q?Andr=E9_?= Pirard <PIRARD@vm1.ulg.ac.be>
Such encodings are allowed in selected headers, subject to
the restrictions listed in RFC 1342.
The MIME effort has also produced an RFC defining a Content-
MD5 header [rrr 1544], containing an MD5-based "checksum" of
the contents of an article or body part, giving high confi-
dence of detecting accidental modifications to the contents.
The "metamail" software package [rrr] helps provide MIME
support with minimal changes to mailers, and may also be
relevant to news reading agents.
The PEM (Privacy Enhanced Mail) effort is pursuing analogous
facilities to offer stronger guarantees against malicious
modifications, unauthorized eavesdropping, and forgery.
This work too may be applicable to news, once it is
17 Jan 1994 - 95 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. B
reconciled with MIME (by efforts now underway).
C. Summary of Changes Since RFC 1036
This Draft is much longer than RFC 1036, so there is obvi-
ously much change in content. Much of this is just
increased precision and rigor. Noteworthy changes and addi-
tions include:
+ section 4.3's restrictions on article bodies
+ all references to MIME facilities
+ size limits on articles
+ precise specification of Date-content syntax
+ message IDs must never be re-used, ever
+ "!" is the only Path delimiter
+ multiple moderators in the Approved header
+ rules on References trimming, and the _-_ mechanism
+ generalization of the Xref rules
+ multiple message IDs in Cancel and Supersedes
+ Also-Control
+ See-Also
+ Article-Names
+ Article-Replacing
+ more precise rules for cancellation
+ cancellation authorization based on From, not Sender
+ "unmoderated" and descriptors in newgroup messages
+ restrictive rules on handling of sendsys and version
messages
+ the whogets control message
+ precise specification of checkgroups messages
+ compression type preferably specified out-of-band
17 Jan 1994 - 96 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. C
+ rules for encapsulating news in MIME mail
+ tighter specification of relayer functioning (section
9.1)
+ the "newsmaster" contact address
+ rules for gatewaying (section 10)
+ discussion of security issues (section 11)
D. Summary of Completely New Features
Most of this Draft merely documents existing practice, but
there are a few attempts to extend it. These are:
TBW
E. Summary of Differences From RFC 822+1123
The following are noteworthy differences between this
Draft's articles and MAIL messages:
+ generally less-permissive header syntax
+ notably, limited From syntax
+ MAIL header comments allowed in only a few contexts
+ slightly more restricted message-ID syntax
+ several more mandatory headers
+ duplicate headers forbidden
+ References/See-Also versus In-Reply-To/References
(section 6.5)
+ case sensitivity in some contexts
+ point-to-point headers, e.g. To and Cc, forbidden
(section 6)
+ several new headers
References
[Sanderson] "Smileys", David Sanderson, O'Reilly & Associ-
ates Ltd., 1993.
17 Jan 1994 - 97 - expires 15 March 1994
INTERNET DRAFT to be NEWS sec. -
TBW
Security Considerations
Section 11 discusses security considerations in detail.
Author's Address
Henry Spencer
henry@zoo.toronto.edu
SP Systems
Box 280 Stn. A
Toronto, Ont. M5W1B2 Canada
17 Jan 1994 - 98 - expires 15 March 1994
These are the contents of the former NiCE NeXT User Group NeXTSTEP/OpenStep software archive, currently hosted by Netfuture.ch.