Abstract—Automatic question answering (QA) is an
interesting and challenging problem. Generally such problems
are handled under two categories: open domain problems and
close domain problems. Here the challenge is to understand the
natural language question so that the solution could be matched
to the respective answer in the database. In this paper we used a
template matching technique to perform this matching. The
first part of the paper discusses about an automatic question
answering system that we have developed using template
matching techniques. The approach adopted is an automated
FAQ (Frequently Asked Question) answering system that
provides pre-stored answers to user questions asked in ordinary
English and SMS language. The system also has techniques to
overcome spelling and grammar mistakes introduced in
questions by its users and therefore user-friendly compared to
restricted syntax based approaches. The second part of the
paper studies three techniques for performance evaluation in
the above system which are based on template matching
approach: 1) Random classification of templates, 2) Similarity
based classification of templates, 3) Weighting template words.
Index Terms—Evaluation technique, question answering,
NLP, template matching, FAQ, answering system.
T. Gunawardena is with the Department of Mathematics and Computer
Science, University of Basilicata, Italy (e-mail: etilani@gmail.com).
N. Pathirana is with the University Politehnica of Bucharest, Romania
(e-mail: nishara.pdn@gmail.com).
M. Lokuhetti is with the IFS Software Company, Sri Lanka (e-mail:
medhavimpl@gmail.com).
R. Ragel and S. Deegalla are with the Faculty of Engineering, University
of Peradeniya, Sri Lanka (e-mail: roshanr@pdn.ac.lk,
dsdeegalla@pdn.ac.lk).
Cite: Tilani Gunawardena, Nishara Pathirana, Medhavi Lokuhetti, Roshan Ragel, and Sampath Deegalla, "Performance Evaluation Techniques for an Automatic Question Answering System," International Journal of Machine Learning and Computing vol. 5, no. 4, pp. 294-300, 2015.