Home
Objectives
Management Committee
Working groups
Events
News
Documents
Recent updates
What is COST?
Links

"Towards a Framework for the Evaluation of Web-sites intended to Support On-line Communities"

Costin Pribeanu
November 7th 2002, Montegrotto, Italy
Since this paper is provided in annex VI, the main points from the presentation (and additional comments that did not appear in the paper) will be outlined here.

Many web-sites aimed at supporting local on-line communities are developed informally. This leads to usability problems and failure to meet users’ expectation since usability is at the moment not a priority in their design. In the future Costin anticipated that there would be more demand for better usability (taking a wide definition of usability to include not just HCI but the context and issues of social acceptability).

Various researchers have pointed out that creating on-line communities is difficult (even recommendations about how to do this can be found):

  • When setting up web-sites for local communities it is difficult to get funds
  • There are no recipes for success.
  • While such web-sites can set up on official sites this has disadvantages Ñ e.g. it may misrepresent the community by presenting just a good face to the outside world, it may be oriented to visitors rather than local people and its contents may become oriented to official concerns like tourism and e-Government Ñ which may not be the only things of potential interest.

Examples of principles and recommendations to support web-site sociability have included:

  • the importance of building trusting relationships,
  • developing social translucence,
  • how to stimulate cooperation,
  • how to govern such communities,
  • how to attract people to these communities (various examples were provided Ñ see the paper).

Costin then turned to different approaches to evaluating usability, which are discussed in more depth in the paper. These can be:

  1. Formal — an analysis technique.
  2. Empirical — experiments with human subjects.
  3. Involve a usability inspection — judgement of experts that are evaluating the interface.
  4. Automated — involve software tools for usability testing.

In particular two standards, ISO 9126 and 9241-11, relating to software and work processes were noted, with examples of what elements usability (and quality) should include.

Heuristics are general principles expressed in a straightforward way an example was decorating the web-site to celebrate special occasions, and remove them when the event is over). Meanwhile guidelines provide more detailed prescriptions, following a guideline template. Some examples were given and some drawbacks were discussed (e.g. 1000s of these guidelines exist, often developed in companies, but it is difficult to know how to choose from them and they are mainly technologically oriented for designers). An alternative that has become popular in the last few years is usability patterns, but the problem with these is that they are difficult to write up.

Based on this discussion, the presentation arrived at proposals for a framework for evaluating usability involving a layered structure of heuristics and guidelines moving from general heuristic to more detailed guidelines. These should provide a consistent framework for making evaluations.

Examples of general heuristic include:

  • Create an attractive social space.
  • Find partners able to foster the social interaction.
  • Record and present the collective actions carried on.
  • Ask people to identify themselves.
  • Provide with clear membership conditions.
  • Explain clearly the policies and rules of conduct.
  • Provide with shared assets and stimulate individual contributions
  • Provide with means for monitoring and sanctioning.
  • Find appropriate social moderators.

Examples of lower level heuristics, general guidelines and lower level guidelines were also provided.