Defining and Measuring Quality in Online Discussions – A Critical Review

Authors – Alexandru Spatariu, Kendall Hartley, and Lisa D. Bendixen

Year – 2004

Published in – The Journal of Interactive Online Learning, Volume 2, Number 4, Spring 2004, ISSN: 1541-4914, www.ncolr.org

Link – http://www.ncolr.org/jiol/issues/PDF/2.4.2.pdf

Importance to my Research – High

MY REVIEW
This paper studies different methods to evaluate quality in online discussion forums within an educational setting. The literature is categorised into four different classes as follows:

1. Levels of Disaggrement
2. Argument Structure Analysis
3. Interaction Based Coding
4. Content Analysis

All these techniques tag different sentences  or code different sentences in different quality attributes e.g. low, medium or high. For example, Levels of Disaggrement method tags sentences with varying level of disaggrement i.e. how much (or how less) is one forum post in aggrement with the previous forum post or vice-versa. Here level of disaggrement is the prime quality classifier, so posts with low disaggrement are low in quality whereas those with high disaggrement are high in quality.

The next technique Argument Structure Analysis involves quality evaluation by studing how arguments are made within each post, if arguments are supported by theory or facts they are weighted more than other. This technique can be used along with Levels of Disaggrement technique, to see if posts that are qualified as being high in quality, do actually have any supporting arguments as classified by Argument Structure Analysis.

Content based analysis takes all together different approach when analyzing quality of posts, it analyzes content or meta-content, e.g. in case of evaluating a student forum the contents would include – student participation rates, electronic interaction patterns, social cues, etc as identified by Hara, Bonk, and Angel (2002).

This paper provided a good insight into the existing quality evaluating parameters, however I rated this paper as being of High Importance to my research becuase these techniques are not automated, a person (editor, tutor or lecturer) has to manually code each post with different levels of quality. I would like this to be automated I am more interested in looking at NLP to do this for me.

Cite this article as
Critical Review on “Defining and Measuring Quality in Online Discussions” by V. Potdar, 29th Feb, 2008. Available Online – http://drvidy.wordpress.com/2008/02/29/defining-and-measuring-quality-in-online-discussions-a-critical-review/

Abstract
In support of research examining relationships between learner characteristics and the quality of online discussions, this paper surveys different methods for evaluating discussions. The paper will present coding methods used in our own research as well as methods used by others interested in quality online discussions. Key topics include what constitutes quality in online discussions and how that quality can be measured?

Important Terms

1. Asynchronous Communication
2. Toulmin Model
3. Idea Units

Reference Sheet
Some Useful Statistics – According to the U.S. Department of Education (2003), 89% of public, four-year institutions offered distance education courses during the 2000–2001 academic year. Of those offering distance education courses, 90% offered Internet courses.

Useful References