article

Document Sample
article Powered By Docstoc
					 Chapter 10
Usenet News




  Part 2: Applications   1
                      NNTP (1)

• Network News Transfer Protocol
   – RFC 977, 1036
• an overview of a typical news setup

       news                                   news
       server                                 server
                     NNTP
                    news feed                 NNTP
                                             news feed

          news
         articles         news               news client
         on disk          server

                    NNTP                     NNTP


        host                                    host



  news client                news client            news client



                      Part 2: Applications                        2
                           NNTP (2)
• NNTP protocol
% telnet netnews 119
Trying 140.113.17.249...
Connected to netnews.
Escape character is '^]'.
200 netnews.csie.nctu.edu.tw InterNetNews NNRP server INN 2.2.2 13-
     Dec-1999 ready (posting ok).
help
100 Legal commands
 authinfo user Name|pass Password|generic <prog> <args>
 article [MessageID|Number]
 body [MessageID|Number]
 date
 group newsgroup
 head [MessageID|Number]
 help
 ihave
 last
 list […..]
 listgroup newsgroup
 mode reader
 newgroups yymmdd hhmmss ["GMT"] [<distributions>]
 newnews newsgroups yymmdd hhmmss ["GMT"] [<distributions>]
 next
 post
 slave
 stat [MessageID|Number]
 Xgtitle [group_pattern]
 xhdr header [range|MessageID]
 xover [range]
                          Part 2: Applications                        3
                           NNTP (3)
  xpat header range|MessageID pat [morepat...]
  xpath MessageID
Report problems to <usenet@netnews.csie.nctu.edu.tw>
.

% telnet netnews2 119
Trying 140.113.209.3...
Connected to netnews-gw.
Escape character is '^]'.
200 netnews2.csie.nctu.edu.tw InterNetNews NNRP server INN 2.1 24-Jul-
    1998 ready (posting ok).
post
340 Ok
From: hcshih@csie.nctu.edu.tw
Newsgroups: csie.test
Subject: testing...

testing post
.
240 Article posted

list
215 Newsgroups in form "group high low flags".
alt.1d 0000044961 0000044894 y
alt.2600 0000392576 0000389705 y
alt.2600.hope.announce 0000005279 0000005264 y
…...
news.announce.newgroups 0000001349 0000001340 m
news.groups 0000140334 0000139922 y
csie.course.cla100 0000000098 0000000099 y
.
205 .

                          Part 2: Applications                       4
                               NNTP (4)
group csie.test
211 13 18204 18216 csie.test

head 18216
221 18216 <8fqp8p$e63$2@netnews2.csie.nctu.edu.tw> head
Path: netnews2.csie.nctu.edu.tw!magpie.csie.nctu.edu.tw
From: hcshih@csie.nctu.edu.tw
Newsgroups: csie.test
Subject: testing...
Date: 16 May 2000 06:19:05 GMT
Organization: Computer Science & Information Engineering NCTU
Lines: 1
Message-ID: <8fqp8p$e63$2@netnews2.csie.nctu.edu.tw>
NNTP-Posting-Host: hcshih@magpie.csie.nctu.edu.tw
X-Trace: netnews2.csie.nctu.edu.tw 958457945 14531 140.113.209.21 (16
     May 2000 6:19:05 GMT)
X-Complaints-To: ta@csie.nctu.edu.tw
NNTP-Posting-Date: 16 May 2000 06:19:05 GMT
Xref: netnews2.csie.nctu.edu.tw csie.test:18216
.
body 18216
222 18216 <8fqp8p$e63$2@netnews2.csie.nctu.edu.tw> body
testing post
.
article 18216
…




                           Part 2: Applications                         5
                         NNTP (5)
• A simple news client
    –   LIST
    –   NEWGROUPS 000510 000000 <de>
    –   GROUP [newsgroup]
    –   ARTICLE [article number]
    –   HEAD [article numbers]
• A more sophisticated news client
    – GROUP [newsgroup]
         • .newsrc
    – XOVER [range of article numbers]
group csie.help
211 429 12020 12464 csie.help
xover 12020-12021
224 data follows
12020 有關硬碟的問題... retsamsu.bbs@Meng.twbbs.org (期末考加油!!) 17 Jan
    2000 12:21:53 GMT <3YT2dH$59E@Meng.twbbs.org> 919 20 Xref:
    netnews2.csie.nctu.edu.tw csie.help:12020
12021 bmp如何做透明圖? gychen@csie.nctu.edu.tw (孫小六) 18 Jan 2000 04:16:35
    GMT <860pf3$8cf$1@netnews2.csie.nctu.edu.tw> 41 10 Xref:
    netnews2.csie.nctu.edu.tw csie.help:12021
12022 Re: bmp如何做透明圖? mdhsieh@csie.nctu.edu.tw (無意自命清高,但看不
    慣世俗庸人之嘴臉) 18 Jan 2000 08:52:35 GMT
    <8619kj$qb2$1@netnews2.csie.nctu..edu.tw>
    <860pf3$8cf$1@netnews2.csie.nctu.edu.tw> 1438 14 Xref:
    netnews2.csie.nctu.edu.tw csie.help:12022
    – ARTICLE [article number]



                         Part 2: Applications                     6
                    INN (1)
• What is INN ?
  – A transport system - no readers included
  – an implementation of the NNTP RFC 977
  – can be used for UUCP sites, but targeted at
    IP-connected hosts
  – runs on almost all modern UNIX systems
• Hardware
  – RAM
     • the golden rule: more is better
     • get as much RAM as you can so swapping is
       limited. A running innd/nnrpd of 60MB is not
       unreasonable for a busy system
  – Swap space
     • the golden rule: more is better
     • on many versions of UNIX, when innd has to
       fork there must be at least as much free swap as
       innd’s current size
  – Disk space/speed
     • the golden rule: more/faster is vital
     • a full news feed in mid 1996 is around 3 GB per
       day
                  Part 2: Applications                   7
                     INN (2)
     • inn’s article storage scheme is hard on your disk
       arm actuators
         – one file per article
         – timehash (new version)
         – cnfs (new version)
     • available disk space and number of spindles is
       most important
     • low seek times are very important
     • CPU speed is probably least important for now,
       but with some new capabilities like SPAM
       filtering, it is becoming more and more important
• Getting INN
  – http://www.isc.org/products/INN/
  – Last official release of INN
     • 2.2.2
     • inn-1.7.2.insync-1.1d
• Configuring INN
  – run-time config files
     • under subdirectory “etc”
     • inn.conf, expire.ctl, incoming.conf, innfeed.conf
       newsfeeds, nnrp.access, control.ctl, storage.conf

                   Part 2: Applications                    8
                               INN (3)
    – configuring: inn.conf
organization:    Computer Science & Information
   Engineering NCTU
server:          netnews.csie.nctu.edu.tw
pathhost:        netnews.csie.nctu.edu.tw
moderatormailer: ta@csie.nctu.edu.tw
storageapi:      true
          • pathhost is what gets put in the Path header
    – configuring: expire.ctl
          • defines the expiry policy for the groups you carry
    ## TRASHAREA: class 1
    1:7:10:15
    – configuring: storage.conf
          • defines CNFS
    method cnfs {
        newsgroups: *.binrary.*,free.*
        class: 1
        size: 0,0
        expires: 0
        options: TRASH
    }

    – configuring: cycbuff.conf
          • defines CNFS files
    cycbuff:BUF06H:/Achernar/CNFS2/TRASH:640000
    cycbuff:BUF09H:/beach/CNFS/CSIE:960000
    cycbuff:GENERAL:/beach/CNFS/GENERAL:1920000
    metacycbuff:TRASH:BUF06H
    metacycbuff:CSIEAREA:BUF09H




                             Part 2: Applications                9
                         INN (4)
     – configuring: site/incoming.conf (hosts.nntp)
        • specifies which hosts can connect to you to feed
          you articles
        • you have to specify each host (no wildcards)
        • it’s sometimes wise to put IP addresses here
          rather than names for when DNS is broken (or
          just not running) when innd is started
        • any hosts not listed in this file will be given to
          nnrpd to process if they connect
streaming:                     true           # streaming allowed
                                                    by default
max-connections:               8              # per feed
peer ME {
  hostname:                    "localhost, 127.0.0.1"
}
peer netnews.csie {
  hostname:                   "netnews-
   gw.csie.nctu.edu.tw,       140.113.209.249"
  max-connections:            20      # override global
                                            value.
}
##   Group entries look like:
##
##           group <name> {
##                # body
##           }


                       Part 2: Applications                     10
                   INN (5)
– configuring: newsfeeds
   • specifies what you do with articles you receive
     (i.e. who you feed then to)
   • does not specify which newsgroups you’ll take
     articles for (the active file does that)
   • the ME entry’s newsgroup list gives the default
     subscription list for all neighbors, but this
     defaulting feature is usually not used
   • site[/exclude-list]\
         :newsgroup-list[/distrib,distrib...]\
         :Tm:innfeed!
   • site : the name innd will use to refer to this peer.
     is usually the same as what that the peer puts into
     the Path header of articles
   • if site or any of the exclude names are in the
     Path header, of a just-arrived article, then the
     article will not be offered to this peer
   • to subscribe to all of comp but none of the
     sources except for comp.sources.unix:
       – comp.*, !comp.sources.*, comp.sources,unix
   • Distribution header




                 Part 2: Applications                   11
                                 INN (6)
ME :*,!control*,!junk,!alt.binaries.*::
overview!:*:Tc,Ao,WhR,S30000:/home/inn/news/bin/overchan
netnews2.csie/netnews2.csie.nctu.edu.tw\
     :*,!control*,!junk,!alt.binaries.*,!news.lists.filters\
     :Tm:innfeed!

      – configuring: innfeed.conf
group csie-news-sites {
    peer netnews2.csie {
          max-connections: 10
          ip-name:         netnews2.csie.nctu.edu.tw
    }
}
      – New article processing sequence
            • look for group(s) in active file. If none there then
              drop
            • validate against distributions in ME entry of
              newsfeeds file
            • go through newsfeeds file and try every entry
                  – check entry’s name against Path header
                    components
                  – check entry’s pattern against the Newsgroup
                    header
Path: netnews.csie.nctu.edu.tw!ctu-peer!news.nctu.edu.tw!
      news.nctu.edu.tw!news.csie.ncu.edu.tw!Forest
From: catbear.bbs@bbs.mis.cycu.edu.tw (考...完啦!!!)
Newsgroups: tw.bbs.forsale.computer


                               Part 2: Applications               12
                               INN (7)
     – configuring: nnrp.access
           • tell nnrpd, the reader daemon, who can do what
           • format:
                 – hosts: perms: username:password:patterns
           • if username and password specified then client
             must send them to Post or Read
*:R:::csie.help
*.nctu.edu.tw:RP:::csie.* !csie.cc.* !csie.bbs.adm.* nctu.* nthu.* tw.*
*.csie.nctu.edu.tw:RP:::* !junk !csie.cc.* !csie.bbs.adm.* !twbbs.*
     – configuring: control.ctl
           • specifies how control messages (other than
             cancel) are handled
           • format:
                 – <message>:<from>:<newsgroups>:<action>
           • sample version is usually sufficient
newgroup:skhuang@csie.nctu.edu.tw:csie.*|twbbs.*|cna.*:doit=mail
newgroup:*:tw.bbs.*:log=tw.bbs.ctl
newgroup:ltc@news.cc.nctu.edu.tw:tw.bbs.*:doit=mail




                             Part 2: Applications                         13
                       INN (8)
• Installing INN
   – compile
      • configure
      • make install
• Database Initialization
   – the active file defines what groups you’ll
     accept
   – the format of an active file entry is:
      • groupname high# low# flags
   – example:
      • alt.algebra.help 0000013406 0000013290 y




                   Part 2: Applications            14
                     INN (9)
   – the most up to date active file can be
     found in
      • ftp://ftp.isc.org/pub/usenet/CONFIG/
   – you will also need an up to date
     newsgroups file
• About article transfer
   – normal way of feeding
   – possibly multiple remotes send articles via
     standard protocol
   – local INN determines article numbers
• About article replication
   – slave vs. master
   – used to make one system a mirror image of
     another
   – master feeds articles in such a way that the
     article numbers remain consistent across the
     two machines
   – news readers can therefore reuse
     a .newsrc file

                   Part 2: Applications            15
                         INN (10)
• About ctlinnd
   – the program that controls the innd server
   – check the state of the server
       • ctlinnd mode
   – unthrottle the server after it throttled itself
     due to low disk space (for example)
       • ctlinnd go ‘’
   – control article flow or file maintenance
   – ctlinnd -h
• Cross posting: hard vs. symbolic links
   – in a cross posted article the extra article
     references are links to the original article
   – for newsgroups located on the same
     filesystem, the links are hard
   – on different filesystems, the links are
     symbolic
   – this means that a system that doesn’t support
     symbolic links can only have one filesystem
     for article storage

                     Part 2: Applications              16
                    INN (11)
• Monitoring INN
  – use innwatch, innreport,
    ctlinnd status
  – look at your system log files
     • news.notice, news.err, news.crit, news
  – read the mail from news.daily
  – tail -f is your friend
  – to verify that news is coming in, tail the
    news log file




                   Part 2: Applications          17
                  INN (12)
• Troubleshooting
  – run inncheck (especially after initial
    install)
  – reread the install document
  – read the FAQ
  – telnet to port 119 and issue NNTP
    commands by hand
  – make sure connection is to correct program
    (innd vs. nnrpd). Check the banner message
• Group selection
  – alt.* is more than half of a “full feed”
  – alt.binaries.* is more than half of
    that.
  – Take only what people at your site read
  – alt.binaries groups are extremely popular
  – cut off groups at your feed, not in active
• Moderation
  – moderated groups are marked m in active
    file

                 Part 2: Applications            18
                     INN (13)
  – server sends posting only to first moderated
    group it finds
  – looks for address in moderators, then
    inn.conf
     csie.computer-center:whyang@csie.nctu.edu.tw
     csie.forgery:anon@csie.nctu.edu.tw
     …...
  – Moderator receives as mail, messages into
    news article, posts
  – header in articles
     • Approved: xxxxx@csie.nctu.edu.tw
• Control messages
  – newsgroups
     •   control.cancel
     •   control.checkgroups
     •   control.newgroup
     •   control.rmgroup
  – header
     • Control: cancel <3OH5ih$aqP@bbs.mis.cycu.edu.tw>




                    Part 2: Applications              19
                         INN (14)
    – example:
Path: netnews2.csie.nctu.edu.tw!netnews.csie.nctu.edu.tw
     !csiecancel
From: printscr.bbs@192.192.100.4 (PRINTSCR)
Newsgroups: tw.bbs.forsale.computer
Subject: cmsg cancel <3aQQdN$fzF@192.192.100.4>
Date: 16 May 2000 04:41:15 GMT
Organization: csiecancel
Message-ID: <csiecancel.3aQQdN$fzF@192.192.100.4>
Approved: news@netnews.csie.nctu.edu.tw
Control: cancel <3aQQdN$fzF@192.192.100.4>
Lines: 3
Xref: netnews2.csie.nctu.edu.tw control.cancel:9276463

• Related newsgroups
    – news.*
         • news.software.nntp
         • news.software.reader
         • news.admin.*




                        Part 2: Applications               20

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:4/26/2013
language:English
pages:20
tao peng tao peng fuzhou http://
About 1234567