Towards a Benchmark for Evaluating Design Pattern Miner Tools

Document Sample
Towards a Benchmark for Evaluating Design Pattern Miner Tools Powered By Docstoc
					Towards a Benchmark for
Evaluating Design Pattern Miner
Tools


 Date:102/1/17
 Publisher:IEEE Software Maintenance and Reengineering, 2008.
 CSMR 2008. 12th European Conference on
 Author:Lajos Jen˝o F¨ul¨op¤, Rudolf Ferenc and Tibor Gyim´othy
 組員:余世淇,游家瑋
Introduction

• Recovering design pattern usage in source code is a very
  difficult task. Several tools are described in the literature for
  this purpose, but there is little work invested in evaluating
  them. The main reason for this is the lack of an approved
  benchmark for these tools.
Architecture

• We use the well-known issue and bug tracking system called
  Trac [34] (version 0.9.6) as the basis of the benchmark. Trac is
  written in Python and it is an easily extendible and
  customizable plug-in oriented system.
Architecture
Architecture
Architecture
A. Initialization
B. Pattern Matching processing on GPU
C. Data Output
• True Positives (TP): true instances found by the tool (correctly).
• False Positives (FP): false instances found by the tool
  (incorrectly).
• False Negatives (FN): true instances not found by the tool
  (incorrectly).
• The precision value is defined as TP/TP+FP , which means the
  ratio of correctly identified instances with respect to all found
  instances. The recall value is defined as TP/TP+FN , which
  means the ratio of correctly identified instances with respect
  to all existing real instances.
Adding a New Tool
• Upload. Uploading the results of a tool requires the name of
  the tool, name of the mined software, the programming
  language of the software, source location, and information
  about the found design pattern instances in comma separated
  value (CSV) file format
Benchmark Contents
Conclusion and FutureWork
• This paper presents work in progress, the benchmark is
  operational, but will certainly need further development
  based on user feedback. A possible extension of the
  benchmark would be, for instance, a flexible solution for
  selecting siblings among the pattern instances. Currently,
  users can add comments to an instance to argue about the
  judgements. In the future, we want to extend this with
  categorizing the votes of a user to control the overrating of his
  own tool.

• http://www.inf.u-szeged.hu/designpatterns/

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:7
posted:11/4/2013
language:English
pages:14