Spyware anti virus software

Document Sample
Spyware anti virus software Powered By Docstoc
					                                   The Spyware Warrior Guide to

                              Anti-Spyware Testing
                                              by Eric L. Howes

                                               Oct. 2-4, 2004 /
                                               Oct. 8-9, 2004 /
                                               Oct. 13-15, 2004


On this page...                                         Anti-Spyware Test by Eric L. Howes

       Overview                                                 Guide & Overview
       The Tests: Summary & Description                         Critical Detections
       The Tests: Design & Methodology                          Test Results # 1 (Oct. 2-4)
       Test PC                                                  Test Results # 2 (Oct. 2-4)
       Disclaimers                                              Test Results # 3 (Oct. 8-9)
       Lessons & Conclusions                                    Test Results # 4 (Oct. 8-9)
       Short Table of Applications & Tests                      Test Results # 5 (Oct. 13-15)
       News & Other Information                                 Test Results # 6 (Oct. 13-15)
       Background & Bio
       Questions & Contact                             Other pages of interest...

                                                                 Anti-Spyware Programs: Feature Comparison
                                                                 Rogue/Suspect Anti-Spyware Products & Web
                                                                  Sites
                                                                 Spyware Warrior Blog & Forums
                                                                 Getting Help w/ Spyware
                                                                 Spyware Warrior Home
                                                                 Ben Edelman - Spyware Research



Overview

As the the threat of "spyware" and "adware" has escalated over the past few years, the number of "anti-
spyware" scanners available on the Net has grown equally fast. At present there are over 100 anti-
spyware scanners available for download -- some for free, some for pay. Spyware and adware are
themselves complex enough to prove bewildering to most average users, however. So confusing in fact
is the threat of spyware and adware that users often have trouble distinguishing effective anti-spyware
scanners from less effective ones. Although a number of "tests" of anti-spyware scanners have been
reported on the Net, many if not most of those tests are of limited value because the design,
methodology, and execution of the tests is not fully and publicly documented, leaving even
experienced users and experts to wonder just how meaningful those tests really are. Still worse, some
of those "tests" are touted by webmasters who are affiliates for the companies whose products were
"tested."

The tests documented on these pages are intended to partially remedy these several problems with our
knowledge of anti-spyware scanners and how well they perform. At present, there are three groups of
tests documented here.

Users looking for a short list of recommendations for anti-spyware products can find such a list HERE.
For a more comprehensive list of anti-spyware products, see HERE. For a comparative breakdown of
features available in the more reputable anti-spyware programs, see HERE. And if your PC is already
overrun with spyware or adware, see my tips for what to do HERE.

The Tests: Summary & Description

Three rounds of tests have been conducted. The results for each round are reported on two "test
results" pages. A table summarizing the applications tested can be found below. Tables summarizing
the "critical detections" identified for each round of tests are found here.

Group 1 (Oct. 3-4)

In the first group of tests, twenty anti-spyware scanners were pitted against a collection of 15 adware
and spyware programs that were installed with the latest version of Grokster available from CNET's
Download.com. The spyware and adware installed with Grokster were documented and then broken
down into 134 "critical" detections, which included a mix of files, processes, and Registry entries (see
this table for details). Each anti-spyware scanner was then allowed to scan and remove every instance
of spyware or adware that it could find. The results of each anti-spyware scanner's performance in
finding and removing the 134 critical detections are reported on separate "results" pages:

      Results Page # 1
      Results Page # 2

For an interesting and illuminating analysis of the Grokster installation, see Ben Edelman's excellent
write-up HERE.

Group 2 (Oct. 8-9)

In the second group of tests, the anti-spyware scanners were matched against a mish-mash of 25
different adware and spyware programs picked up via "drive-by-download" at the Innovators of
Wrestling web site. Once again the installed adware and spyware was broken down into "critical
detections," this time numbering 153 (again, see this table for a summary). The anti-spyware scanners
were then allowed to find and remove spyware and adware. As before, the performance of the 20 anti-
spyware programs is reported on two "results" pages:

      Results Page # 3
      Results Page # 4

Two substitutions were made in the anti-spyware scanners used for the second group or round of tests.
First, SpyBouncer was substituted for SpywareNuker 2004. SpywareNuker 2004 requires users to
activate the product online. As SpywareNuker 2004 was uninstalled after the first round of tests, it had
to be reactivated when it was reinstalled. TrekBlue's server refused to activate, indicating that the
registration number had already been used to activate a copy of SpywareNuker 2004. At that point,
SpyBouncer was substituted for SpywareNuker 2004 for the second round of tests.
Second, as BPS Spyware & Adware Remover crashed at the beginning of removals during the test,
Tenebril SpyCatcher was tested on Oct. 15 (a week later than the other applications for this round) and
substituted for BPS Spyware & Adware Remover.

Group 3 (Oct. 13-15)

In the third group of tests, the anti-spyware scanners were pitted against yet another hodge podge
collection of adware and spyware programs. These 23 different programs were picked up by surfing 3
web sites in succession (007 Arcade Games Games, LyricsDomain, and Innovators of Wrestling ). As
before, the installed spyware and adware was broken down into "critical detections," 138 total for this
third round (see this summary table for a breakdown). The anti-spyware scanners were then unleashed
on the PC to find and remove whatever spyware and adware they could. Their performance is reported
on two "results" pages:

       Results Page # 5
       Results Page # 6

One substitution was made in the anti-spyware scanners used for the third group of tests. As
ZeroSpyware 2004 froze at the outset of removals during the test, Tenebril SpyCatcher was tested
instead and substituted for ZeroSpyware 2004.

Notes

Before moving to the test results pages, please read the information below about the tests themselves,
esp. the Disclaimers section.

PC Pitstop publishes a "Top 25 Spyware and Adware" list, which is updated regularly. The three tests
documented here include all of the top 10 spyware/adware applications on the PC Pitstop list (as of
Oct. 18, 2004), and a good number of the remaining 15 in the top 25.

Users looking for a short list of recommendations for anti-spyware products can find such a list HERE.
For a more comprehensive list of anti-spyware products, see HERE. For a comparative breakdown of
features available in the more reputable anti-spyware programs, see HERE. And if your PC is already
overrun with spyware or adware, see my tips for what to do HERE.

The Tests: Design & Methodology

The same testing process was used for both rounds of tests.

Installation

Before testing, all "anti-malware" protections were disabled, including all resident "anti-malware"
scanners, spyware "immunizations," custom browser security settings, and other system configurations
designed to block the installation or execution of "malware." The spyware and adware was then
installed from the internet.

       For the first round of tests (reported on "results" pages 1 and 2) Grokster version 2.6 was
        installed from Download.com. In addition to installing the main P2P file sharing application,
       the stub downloader/installer (grokstersetup.exe) itself downloaded and executed a number of
       other installers for other applications.

      For the second round of tests (reported on "results" pages 3 and 4) Internet Explorer was
       pointed to iowrestling.com, where a flurry of ActiveX Warning boxes was encountered for
       automated installations of spyware and adware. No less than 7 different boxes were clicked
       through, initiating installation processes for around 25 different adware and spyware
       applications.

      For the third round of tests (reported on "results" pages 5 and 6) Internet Explorer was taken to
       three web sites in succession, all of which popped up ActiveX Warning boxes for automated
       installations of spyware and adware. Although only 5 boxes were clicked through, 23 different
       adware and spyware programs were installed on the test PC as a result.

After all significant hard drive and network activity had ceased, the PC was rebooted to allow the
various installers to finish setup activity. Once that activity had completed and the installed software
components were in a relatively "stable" state, the personal firewall installed on the computer was
configured to block all network traffic to prevent further installations or changes. An InCtrl5
installation log was generated as well as a preliminary HijackThis! log.

"Critical" Detections

From those logs as well as from information gleaned by manual inspection of the hard drive and
Registry, a list of "critical" detections was generated, with each detection being assigned a unique ID
(see this page for details). Included in these "critical" detections were:

      executable files (.EXE / .COM)
      dynamic link libraries (.DLL)
      BHO-related Registry entries
      toolbar-related Registry entries
      browser setting-related Registry entries
      browser extension-related Registry entries
      auto-start Registry entries

These "critical" detections comprise only a subset of the complete collection of files and Registry
entries added to the test PC by the installed spyware and adware. As such, the test results reported here
do not provide a complete picture of the performance of the anti-spyware applications tested.

Nonetheless, these detections are "critical" because they constitute the most important files and
Registry entries installed by the spyware and adware applications that accompanied Grokster. These
detections represent the changes that would most visible and/or important to users. Any good anti-
spyware application would necessarily have to succeed at detecting and removing a significant number
of these files and Registry entries in order to be considered useful or effective, even if it left a
significant number of less important files and Registry keys -- that is to say, inert "junk" -- behind.

Moreover, these "critical" detections do provide a useful measure of the performance of these anti-
spyware applications because they test how well the programs:
      find and remove files on the hard drive
      kill running processes and remove the associated files
      correctly uninstall BHOs, browser toolbars, and other browser extensions
      find and remove Registry entries critical to the functioning of the spyware and adware
       applications

One significant aspect of these applications that was tested only in the third round of tests, however,
was how well the applications remove Winsock LSP hijacks (if removed incorrectly, the network
connection of the PC may be broken).

It should also be noted that not all applications installed by the Grokster setup program are represented
in the detections for the first group of tests. Not included are:

      Grokster
      P2P Networking
      Flashtalk

By contrast, all programs installed by iowrestling.com are represented in the detections for the second
group of tests. The same holds true for the programs installed during the third group of tests.

Along with the list of "critical" detections, a full Registry backup and copy of all newly installed or
changed files was archived. This Registry backup, combined with the archived files, was used to
restore the test PC to a "newly installed" state before each anti-spyware scanner test.

Scanning & Removal

After the test PC had been restored to a "newly installed" state, each anti-spyware application was
allowed to scan and remove every instance of spyware and adware that it could find. Where possible,
each scanner was configured to scan only the C-drive and the L-drive (containing the Temporary
Internet Files directory and main TEMP directory) on the test PC. Each scanner was also configured to
perform a "full" or "deep" scan of the Registry. If the anti-spyware application requested a system
reboot to complete the detection and removal process, a reboot was performed. In all cases the latest
definitions databases available for the applications were used. Scan logs were archived when possible,
though this was not always feasible.

To check the performance of each anti-spyware scanner, a custom-built batch file was executed. This
batch file generated a list of the "critical" files and Registry entries that were not removed by the anti-
spyware scanner. In some cases anti-spyware scanners may have detected and attempted to remove
certain files and Registry entries only to fail. As the batch file checked for "critical" detections actually
left in place at the conclusion of a scan, the test results reported here reflect only actual removals, not
mere detections or attempted removals. Finally, false positives were noted and reported when they
were generated.

Readers should be aware that in some cases anti-spyware applications may not have removed the files
and Registry entries for particular adware or spyware programs because of deliberate policy decisions
by the vendors not to target those programs for removal.
Test PC

The PC used for these tests was a 1.8 Ghz Pentium 4 w/ 512 mb RAM. Installed on the computer were
Windows 2000 w/ SP4, Internet Explorer w/ SP1, and Office 2000 w/ SP3. The network connection
was provided by InsightBB's cable broadband service. The network connection was monitored by
Agnitum Outpost Firewall Pro 2.1.

Disclaimers

Although the test results reported on these pages are detailed, readers should be aware of several
significant limitations of the tests performed:

      The test results reported here constitute but a few tests with three collections of spyware and
       adware programs. The anti-spyware scanners tested here may perform differently with other
       collections of spyware and adware.

      The tests results report only actual removals of a select number of "critical" files and Registry
       keys, and thus do not give a complete account of the removals performed by any of the anti-
       spyware scanners tested.

      These tests do not pit the anti-spyware scanners against what is undoubtedly the toughest
       spyware application of them all, CoolWebSearch.

Given these limitations, readers should not regard the test results reported here as any kind of
"definitive" guide to anti-spyware scanners, nor should readers use these test results as the sole basis
for purchasing decisions. The information presented on these pages is designed to supplement other
information about anti-spyware applications found on the Net, not completely replace it.

Moreover, nothing in these test results should be taken as an endorsement of or recommendation
against the use of any particular anti-spyware scanner by the author of these web pages. These tests are
primarily intended to help users gain better insight into the issues surrounding anti-spyware scanners
and the kind of performance that might be expected from them.

Finally, it should be noted that I have no financial relationship with any of the companies or
individuals whose products were tested. I am not an employee, affiliate, representative, or other agent
of any of these companies or individuals.

Lessons & Conclusions

If any lessons or conclusions can be drawn from these tests at all, they are quite general:

      Spyware and adware can prove quite difficult to remove, even for dedicated anti-spyware
       scanners.

       In the second and third group of tests, for example, one of the installed programs prevented the
       anti-spyware scanners from running on reboot, a common method used by anti-spyware
       scanners to remove stubborn spyware and adware that is currently in memory on a PC. As a
       result, some spyware and adware was not removed by the anti-spyware scanners during reboot
       that otherwise might have.

      No single anti-spyware scanner removes everything. (1) Even the best-performing anti-spyware
       scanner in these tests missed fully one quarter of the "critical" files and Registry entries.

      It is better to use two or more anti-spyware scanners in combination, as one will often detect
       and remove things that others do not.

      Where possible, users should become familiar with the use of HijackThis! in order to remove
       stubborn spyware and adware that standard anti-spyware scanners fail to remove. Less
       experienced users should know how to get help from the expert volunteers who provide free
       HijackThis! log advice and analysis at major anti-spyware forums.

      Prevention is always preferable to scanning and removal, and users should securely configure
       their PCs and install anti-malware protection to prevent the installation of spyware and adware
       in the first place.

      Moreover, users should learn to practice safe computing habits, which include avoiding web
       sites and programs of unknown or dubious provenance and carefully reading End User License
       Agreements and Privacy Policies.

Users looking for a short list of recommendations for anti-spyware products can find such a list HERE.
For a more comprehensive list of anti-spyware products, see HERE. For a comparative breakdown of
features available in the more reputable anti-spyware programs, see HERE. And if your PC is already
overrun with spyware or adware, see my tips for what to do HERE.

                                                   Return to top...

Short Table of Applications & Tests

As explained above, three rounds of tests have been conducted. The results for those three rounds of
tests are reported on six different "test results" pages (two pages for each round). Each "test results"
reports the performance of ten anti-spyware applications.

Given that readers might find the various "test results" pages a bit confusing, included below is a table
summarizing the applications tested and where the results for each can be found.

                                                                      3: Oct 13-
Application / Round (Date)                  1: Oct 2-4   2: Oct 8-9
                                                                          15
            ...found on Test Results page   #1     #2     #3     #4   #5     #6
Aluria Spyware Eliminator 3.0.32            X             X           X
BPS Spyware & Adware Remover
                                                   X                         X
8.2.0.10
GIANT AntiSpyware 1.0                       X             X           X
Intermute SpySubtract Pro 2.51              X             X           X
Lavasoft Ad-aware SE Personal 1.05              X             X           X
McAfee AntiSpyware 1.00.1126                    X             X           X
NoAdware 2.01                                          X             X          X
OmniQuad AntiSpy 4.2                                   X             X          X
PC Tools Spyware Doctor 2.1.0.254               X             X           X
Pest Patrol 4.4.3.24                            X             X           X
Spybot Search & Destroy 1.3.1 tx                X             X           X
SpyHunter 1.5.83                                       X             X          X
SpyKiller 2005 1.00                                    X             X          X
Spyware C.O.P. 10.0                                    X             X          X
SpywareNuker 2004 2.13                                 X
SpywareStormer 1.4.7                                   X             X          X
Tenebril SpyCatcher 3.00.46                                          X          X
Webroot Spy Sweeper 3.2                         X             X           X
Xblock.com X-Cleaner Deluxe 4.0.0.249           X             X           X
XoftSpy 3.45                                           X             X          X
ZeroSpyware 2004 1.00                                  X             X
               ...found on Test Results page    #1     #2     #3     #4   #5    #6
Application / Round (Date)                       Oct 2-4      Oct 8-9     Oct 13-15



                                                       Return to top...
                                               News & Other Information

  These tests have received coverage and notice across the Internet. Here are a few samples of news
                                     stories and other reactions:

                           
                          Slashdot - Failing Grades for More Anti-Spyware Tools
                           
                         eWeek - Study: Tools Let Spyware Slip Through Cracks
        Ben Edelman - Grokster and Claria Take Licenses to New Lows, and Congress Lets Them Do
                                                     It

                                                     Background & Bio

   I am a graduate student in the Graduate School of Library and Information Science (GSLIS) at the
University of Illinois at Urbana-Champaign. For twelve years I taught business and technical writing at
  the University of Illinois. This year (2004-2005) I am teaching a course in GSLIS. For the past few
 years I have also been teaching composition courses at Parkland Community College in Champaign.

 Over the past four years I have maintained a personal web site at the University of Illinois to supply
   internet users with resources to protect their privacy and security on the internet. Among those
  resources are several utilities and "block lists" that allow users of Microsoft's Internet Explorer web
 browser to protect themselves against the flood of unwanted software and content pushed on them by
     aggressive advertising and marketing entities. In April of 2004 I attended the FTC's Spyware
                                                 Workshop.

 In recognition of my work to help internet users protect their privacy and security, Microsoft awarded
         me its MVP (Most Valued Professional) Award (http://mvp.support.microsoft.com/).

Full disclosure: since late November 2004 I have performed part-time consulting work as an independent contractor for
Sunbelt Software, makers of CounterSpy. Because of that relationship and the conflict of interest that it represents, I must
recuse myself from public comment on CounterSpy. That means that I cannot and will not publicly evaluate, test, or even
recommend Sunbelt's anti-spyware product. The anti-spyware products that I do recommend, all of which are competitors
to CounterSpy, are listed here.

                                                 Questions & Contact

If you have questions or comments about any of the information presented on these pages, please don't
                                          hesitate to ask.

                                                           Best regards,

                                                          Eric L. Howes

http://spywarewarrior.com/asw-test-guide.htm

				
DOCUMENT INFO
Description: Spyware is a type of malware that can be installed on computers, and which collects small pieces of information about users without their knowledge. The presence of spyware is typically hidden from the user, and can be difficult to detect. Typically, spyware is secretly installed on the user's personal computer. Sometimes, however, spywares such as keyloggers are installed by the owner of a shared, corporate, or public computer on purpose in order to secretly monitor other users.