D. R. Newman
Information Management Dept.
Queen's University Belfast
The research aims to study the impact of new electronic Community Information Networks (CIN) upon participating community and voluntary groups and their clients.
This project attempts to measure the advantages and disadvantages of electronic community information networks to the intended beneficiaries, by using as a case study the newly formed Community Information Network Northern Ireland (CINNI), and comparing this with other CINs.
This will involve an evaluation of:
Community and voluntary groups have specific information needs that relate to their client needs and the tasks their staff carry out to meet these needs.(1, 2) Much of their work relies on rapid information acquisition and dissemination to the clients. Often they are better at this than the government agencies.(3) The promise of electronic CINs is to allow faster and wider communication of information among them, to the benefit of their clients: from raising $0.5M to finding out how to desk-top publish a newsletter (2 examples from a Pittsburgh CIN(4)).
Different CINs concentrate on data retrieval, information networking or knowledge synthesis. Data retrieval from on-line databases of government and voluntary sector information (like Volnet (5, 6)) is seen as the main benefit in many library-based CINs. But information retrieval rarely brings benefits to organisations without information or business plans (see, e.g., the research on the Aston Science Park Information Service). There has to be some way to help guide those seeking information to get a clear idea of what they do not know, in a dialogue to determine which parts of this are mission-critical.
For this we need information networking. It works not only at the data retrieval level of the typical on-line database, but also the knowledge level, when knowledgeable or experienced people on these networks help find solutions to information needs. This is human networking where the people need not attend the same meeting at the same time, but take part in dialogues in electronic discussion groups. Many CINs are set up to facilitate such discussions, through electronic mail and computer conferencing, often with the intention of involving many members of the community in the discussions.(7) Doug Schuler's review(8) of Community Networks(9, 10) and Freenets(11) stresses their strengths as a participatory, interactive, medium. There is a divide between those who think communication or information is more important in CINs. Often the strategic decision on which to support in a CIN is made upon technical grounds, rather than research into the needs of the community groups. These approaches to information networking need to be compared, including the number and quality of networking contacts, and their perceived benefits
Beyond information networking, there is knowledge synthesis and dissemination: the building up of shared community knowledge, based upon information and skills available in the community, and then making it available to others. Research into the social uses of groupware has found techniques to support through computers the exploration of differences in the community, and the bringing together of divergent views into a shared understanding.(12) These shared understandings can then be developed into reports and published electronically, on the World-Wide Web. Such report generation and dissemination is a common activity in campaigning and voluntary groups, such as the CDF or NICVA. The dynamic and interactive nature of Web publishing opens new opportunities to such groups. It makes their reports more widely accessible, more easily updateable, and makes reader feedback easier to collect. This can potentially transform the marketing of their organisation and its message.(13)
This leads on to the final general point about CINs: the available technology is changing. No longer do users have to use simple text screens designed for terminals. Instead there are easier to use graphical user interfaces, such a Netscape for browsing the World-Wide Web (WWW) or PowWow for off-line reading of computer conference messages, as used in Metronets.(14) Distributed information standards allow users to access information anywhere on the Internet, instead of limiting them to what has been put on a particular host. As a result of these changes, it is possible to offer a wider range of services, without making the systems too complicated to be used by volunteers or people in the community. Since these changes go beyond simple usability (by allowing new uses), they need to be evaluated not just as an HCI issue, but also by measuring benefits and costs to the participating humans, organisations and their customers or clients. In particular, they make it possible to use Community Networks as a means of marketing a community's needs and views to people outside, such as donors, specialist advisers and other communities with similar interests.
There are important research issues in working out exactly how to apply these technologies to bring maximum benefit to the organisation, its staff and its clients. For some tasks current human networking or conventional paper-based information retrieval is better. We need to study the workings of a CIN, and the non-electronic alternatives, to identify the appropriate ways for maximising the benefits of the electronic services.
On 17 November 1994, representatives of 80 community and voluntary groups from Belfast met to see a presentation by organisers of a Pittsburgh CIN and Bytes for Belfast about the possibility of a community information network for Northern Ireland. Of the questionnaires returned, 96% were in favour, saying they intended to use such a service in the future, and how they would use could use it. For example, the East Belfast Community Development Agency is setting up a teleworking centre for people with learning disabilities. They wished to quickly discover 8 income-generating opportunities, one of which should be ready to go to a feasibility study in three months. Internet resources should help quickly discover ideas from around the world, not just those known to organisations in Belfast. NIACRO has groups of prisoner families twinned with similar groups in Germany. Email and computer conferencing should at least allow visits to be better planned, and at best provide a new means of continuous contact.
A 3-month trial was set up with the dozen most interested organisations. They have written proposals of what they wish to do with CINNI, and have agreed to take part in monitoring and evaluating this trial in collaboration with the organisers and Queen's University Information Management Dept. The results of this trial will be used to inform and support funding proposals for expansion of CINNI, and to help redesign CINNI to better meet the participants' needs. Since then a steering group has been set up, the lines have been put in, and the first users trained. It is about to go live in a few weeks time.
The trial will make use of Internet technology and services. The groups will dial into a local Internet POP, and make use of any Internet services or tools they wish. In addition, CINNI will have its own newsgroups for computer conferencing between local organisations, to discuss common issues of concern, and to draw on the experiences of people in other organisations. CINNI will also have space for WWW pages for groups who wish to provide information to others (e.g. potential donors). Some organisations like NICVA have databases that can be made available via CINNI, although it is unlikely that they will be set up within the first 3 months trial. The users will make use of GUI tools like Netscape to access CINNI.
This is an ideal opportunity to research the benefits of a Community Information Network, since CINNI is just starting. Unlike other researchers, we will not have to rely on people's memories of what they did before the CIN started.
The results will be used by several groups
These will be transferable to any CIN context, and some of the findings will also be relevant to any organisation planning to make use of the Internet.
There are two research issues being studied in this project:
The first will be studied using mainly quantitative techniques, the second by qualitative methods.
In this phase, the key objectives of each organisation in the trial will be determined.At the client level, we need to know how much they benefit, directly or indirectly from a CIN. For example, if information is acquired on a business opportunity that can be taken up by unemployed or disabled in a community centre. These are measures of changes in an organisation's effectiveness brought about through electronic networking.
At the organisation level, we need to look at how electronic community information networks affect the performance and working style of staff in their work tasks. For example, by measuring how much time is saved by using electronic information sources, or electronic communication with experts. These are measures of changes in efficiency.
In each case, the comparison is with what they would have done without the CIN.
The evaluation criteria will be based on the key objectives of those participating in the trial. Each organisation will be asked to identify a few key targets they wish to achieve within the trial period. These must include at least one to benefit the clients, and at least one to make some work in the organisation more efficient. Some organisations have already done this. In addition, the organisers of the CIN will be asked to provide key questions they need answering for potential sponsors.
For example, Nicola Skinner of East Belfast Development Agency identified the objective of: Finding eight money-making ideas for a teleworking centre for people with learning disabilities in 3 months, at least one of which must be strong enough to go to feasibility assessment.
Then key users in each organisation will be interviewed, to find out how they do or would achieve these key objectives without a CIN, what documentation they have to record these processes, how their clients would benefit, and how satisfied they are with these working methods.
From these objectives, we will derive a number of indicators. This focused approach should make the evaluation manageable, rather than trying to measure everything and then interpret the results.
In this phase, research instruments will be developed to measure the indicators identified in phase one, administered and analysed. This will provide the baseline measurements of current (pre-CIN) activities.
The choice of research techniques will depend upon the evaluation criteria identified in Phase 1. They are likely to include some of the following:
Where organisations have identified many targets in Phase 1, you can use repertory grid techniques with staff members to classify their objectives and their constructs.
Satisfaction questionnaires to find out client opinions of the effectiveness of achievements resulting from greater or lesser use of a CIN. In this phase we will need to identify how satisfied clients are with the range and performance of the services offered by the organisation before CINNI.
The first three techniques will enable us to clarify and expand upon our Phase 1 understanding of the organisations, their clients, and their objectives.
Satisfaction and perception questionnaires and records will be analysed to show the baseline level of performance relative to the identified targets. This needs nothing more sophisticated than calculating descriptive statistics of the data (particularly histograms) and plotting them as graphs. In some circumstances it would be possible to explore the data using machine learning techniques (such as machine induction) to identify patterns from which we can generate hypotheses to test in subsequent phases.
In this phase we aim to collect the data that shows the impact of the CIN on the organisations and the clients.
In addition, we may develop further tests to check on hypotheses suggested from Phase 1. Indeed, it is possible to use the electronic communications technology itself to investigate these hypotheses. For example:
Such techniques can be used throughout Phases 2 and 3 to investigate the research question of the moment, as is done in participatory action research.(16)
In this phase the data will be analysed and the report written. By this time, there will be comparable data from the early days of the use of CINNI (when most tasks are carried out by non-electronic means), and after it has been used for some time: in exactly those areas where the organisations have set out to evaluate the benefits of using an electronic CIN.
Depending on the nature of the objectives identified in Phase 1, some of the following analytical techniques will be used:
The report will bring together the quantitative and qualitative findings to answer both the main research issues identified.
The chart shows how the work will be done over the year. The phases are identified by numbers and arrows.