CTCNet Evaluation Project Update by Clifton Chow, Janet Cornebise, Jan Ellis, June Mark, Bart Wise At the time of the last issue of the CTCNet Review, the CTCNet research and evaluation team had completed the first phase of our research. Thanks to the support of affiliates, the initial phase of our work was very successful. During the first phase, we worked intensively with five sites to find out more about the impact of technology access on individuals. Our research consisted primarily of interviews with participants at the centers. We spoke with a very diverse group of individuals who participated in a variety of program offerings at the five centers. From these interviews, we have begun to develop a better understanding of how technology access is changing people's lives. (More detailed information on the report can be found in the Spring/Summer issue of the CTCNet Review or from the report, Community Technology Centers: Impact on Individuals and their Communities.) The information in this report already been used by several affiliates in grant proposals. Quantitative Participant Impact Study In the second phase of research, we are concentrating on developing a better understanding of impacts across CTCNet as a whole. Work on a national survey to collect data on the impact of technology center access on participants began in mid 1997. In June a copy of the initial draft of the survey was circulated at the All-Affiliates conference. Feedback from the conference led to revisions of the survey, which was then pilot-tested at two Boston area CTCNet computing centers. Results from the pilot-test, together with review and reconsideration of the goals of the evaluation and the findings from the qualitative study were incorporated into subsequent revisions. Feedback from CTCNet staff and the research advisors was also solicited. The survey seeks to collect information from participants (a) about the individuals' reasons for coming to a community technology center, (b) about the learning goals which they may have brought to the center and the extent to which participants perceive that these goals are being met, (c) about their satisfaction level with services and outcomes, and (d) about their demographic background. All 266 affiliates in the network were invited to participate in the study, and representatives at 70 sites responded. Of the 70 affiliates who expressed interest in the study 23 were not in a position to participate because computer centers at those locations were not yet in operation. Therefore, the directors at the 47 remaining sites were asked to estimate the number of surveys they could administer in a three week period. Based on these estimates three mailings totaling 3,180 surveys were sent out. Affiliates in the study were asked to seek responses from as many adult and teen participants as possible. Presently, these surveys are being conducted in both paper and electronic format. One of the affiliates, an assistive center, has arranged for the survey to be administered to the visually handicapped exclusively through a computer application format. Data entry is underway, and analysis is expected to begin in February. Each participating center will receive a summary of the data collected from their site, along with a copy of the final report from the national study. Longitudinal Impact Study The final design plan for the longitudinal study of impact on individual users was undertaken in the fall of 1997. Interview data collected in 1996 for the qualitative study were reviewed and prospective subjects selected. At the present time, the team has constructed parameters and methods to ensure a good level of representation by a diverse set of participants in the study. We have evaluated candidate participants from previous pilot studies at the centers and from the qualitative study interview group for suitability for inclusion in the longitudinal study. We will choose a set of participants whose stories are representative of a variety of compelling experiences and goals, e.g. a senior who uses e-mail to affordably maintain community with family and friends, a middle-aged man out of work and trying to gain new job skills, and a teenager earning money by creating World Wide Web versions of jobseekers' resumes. In each case that we choose, we will also collect information about the entire scope of the participant's interaction with the center, about the impact of that relationship on other parts of the participant's community, and about the center's community itself, to try to build a full picture of what the center means in that person's life. Directors of centers from which candidates will be drawn have been contacted in order to solicit their help and cooperation. During the coming month the team will finalize choices of participants and concurrently will be creating a protocol to guide the semi-structured interview. Interviews will begin in late February or early March. The longitudinal study will include approximately 12 subjects, with a projection that it will be possible to follow half of those original 12 through the full three years of the study. Subjects will be drawn from participants at the Somerville Community Computing Center in Somerville, MA and the United South End Settlement in Boston, MA, and perhaps from other sites, such as the Brooklyn Public Library Literacy Program in Brooklyn, NY, Virtually Wired in Boston, MA, the Hard Disk Cafe in Gloucester, MA, and Malden Access Television in Malden, MA. Community Resource Mapping Community resource mapping is a process by which certain assets within a community are inventoried. Such assets could be, for example, sources of funding, equipment, or other materials; individuals with useful knowledge and skills; and organizations with actual or potential interest in supporting efforts to strengthen conditions and relationships within a neighborhood or community. During the summer, a pilot mapping study was conducted by Elizabeth Goodsell, the CTCNet evaluation team's summer fellow, with the intent of developing materials to encourage interested affiliates to conduct their own community resource mapping project. As a result of the pilot study, a guide, entitled Forging Community Links: A Resource Mapping Guide for CTCNet Affiliates, was produced to help affiliates carry out a similar resource mapping process on their own. Drawing upon strategies described in Building Communities from the Inside Out: A Path Toward Finding and Mobilizing a Community's Assets, by John Kretzmann and John McKnight, a member of the research team identified community connections that had been developed by two Boston area affiliates: the Dorchester Family YMCA and the Asian-American Civic Association. These connections were identified by interviewing staff and volunteers, board members, visitors, and students at both sites, local businesses and passersby in the community, as well as funders and other partner organizations. For both affiliates, a map was constructed to show actual and potential linkages to other community-based or nonprofit organizations, government agencies, businesses, and individuals. Some benefits of community resource mapping mentioned in the guide are that it makes relationships between a center and a community visible for staff to use in the following ways: planning programs, seeking volunteers and other forms of support, and reflecting on the role of the center as a community-building agent. The guide details practical procedures and tools useful in carrying out such a project. It, along with other CTCNet publications, can be accessed on the Internet at http://www.ctcnet.org/publics.html. Research staff members presented findings from the report to center directors and other participants at the 1997 CTCNet All-Affiliates meeting in Pittsburgh, PA. We continue to seek additional ways for our findings to be useful to the community technology community, including staff at affiliate centers, volunteers, and users, as well as the broader research and funding community which supports this work. Projected Research Activities for Year 4 One component of the evaluation team's work for Year 4 will continue to revolve around the quantitative research effort. Each affiliate that participated in the national survey will receive a summary of the data submitted by individuals who access the technology offered at their site. These summaries will be helpful for affiliates to use in seeking funding and other forms of support, evaluating their impact locally, and developing press releases and other promotional materials. A second component of the work will be the continuation of the longitudinal study, as described elsewhere in this report. In addition to the dual research efforts, a tool kit will be created in Year 4 to help affiliates carry out their own needs assessments, impact evaluations, and mapping of community resources. This tool kit will contain a variety of instruments, such as interview protocols, observation protocols, staff questionnaires, participant questionnaires, and sign-in/tracking procedures. Several tools will be based on instruments used in this current project, while others may be ones that have been tested and recommended by staff at CTCNet or at CTCNet affiliates or they may be adaptations of instruments used by other community-based organizations and informal education programs. Considering the technology focus of CTCNet centers, computer-based tools will be included in the tool kit, in the form of spreadsheet or database templates for PCs and Macintoshes. In addition to these electronic tools, certain software applications such as survey-processing programs will also be described. The tool kit will be designed to help affiliates make informed choices about what tools to use and how to use them. A final component of next year's research and evaluation research work will be to disseminate the results of the national survey in a variety of ways: through reports that will be posted on the CTCNet web site, presentations to be offered at regional and annual affiliate meetings, and publications and presentations offered to the broader community of those interested in informal learning, technology and education, and programs that bring opportunities to disadvantaged populations. In addition to most of it being available online, Community Technology Centers: Impact on Individuals and Their Communities is also available from CTCNet in print form, with appendices included. The price is $10. Checks should be made out to EDC/CTCNet and sent to EDC/CTCNet, 55 Chapel Street, Newton MA 02158. Community Technology Center Review, January 1998 Table of Contents