Library Measurement Criteria
Isabel Danforth (danforth@tiac.net)
Fri, 12 Sep 1997 08:44:35 -0400
Date: Fri, 12 Sep 1997 08:44:35 -0400
Message-Id: <3.0.2.32.19970912085236.008095f0@sunspot.tiac.net>
From: Isabel Danforth <danforth@tiac.net>
To: Multiple recipients of list <conntech>
Subject: Library Measurement Criteria
This was posted to web4library as a follow up to people asking about
counting hits on a web page. I really like the included quote about
measurements. What do you think? Or do all public libraries really live
on circulation counts?
>Date: Thu, 11 Sep 1997 09:19:16 -0700
>Reply-To: hyman@sjrlc.org
>Originator: web4lib@library.berkeley.edu
>Sender: web4lib@library.berkeley.edu
>From: hyman@sjrlc.org
>To: Multiple recipients of list <web4lib@library.berkeley.edu>
>Subject: Re: counting internet usage
>X-Comment: Web4Lib Information - http://sunsite.berkeley.edu/Web4Lib/
>
>I am currently working with some social scientists who have expressed
>disdain for the assessment skills of librarians. I'm definitely sighing as
>I read and listen to the discussion of documenting Internet usage.
>
>Not being a purist, I have no objection to the generation of totally
>meaningless large numbers (i.e. web page hits) if it requires no cost or
>effort. You never know when a really large number may come in handy with
>someone who is not particularly knowledgeable about it's level of meaning.
>And it's great to know that I can increase my "number" exponentially by
>adding ten more "new" icons to my home page.
>
>It's a big problem, however, if we ourselves start to believe that these
>numbers are more than another big number to augment circulation
>statistics, with nothing close to the validity and comparability of even
>circ.
>
>When we finish counting web page hits, we still don't have the manangement
>information we need to plan, develop, assess, and document the service
>that we provide. It's getting harder and harder to capture the imagination
>of elected officials with abstract numbers, anyway. As a New Jersey
>politician said some years back --- I don't want to hear about
>circulation, I want to hear about people. Books don't vote.
>
>I think that Jane Goodwin, Coordinator of Planning and Evaluation at the
>Fairfax County (Va) Public Library said it really well in the
>aforementioned Public Libraries article (May/June 97):
>
>Goodwin expressed her desire to push measurement into the realm of impact
>assessment. She identified the following impacts, which could measure the
>difference such resources make for users:
>
>Who are the users of resources (demographic characteristics)
>Reasons for use of electronic resources
>The kinds of questions being answered with these resources
>Length of time of use by individual
>Length of wait (as an expression of demand and measure of frustration)
>Value of the resource as perceived by the user
>Level of enhancement to services resulting from use of the electronic
>resource
>
>This is her sole quote in the article which swings back to "the compelling
>need to count hits". I realize that Ms. Goodwin's impacts are at the other
>end of the risk-reward continuum from counting web page hits. They are
>certainly more expensive and time consuming to collect and process but
>the result is that we would actually know something worth knowing.
>
>We're in the information business. Do we really want to hang are hats on
>numbers so easily manipulated and discredited?
>
>Karen Hyman
>Executive Director
>South Jersey Regional Library Cooperative
>10 Foster Avenue, Suite F-3
>Gibbsboro, NJ 08026
>609 346-1222
>
>
>On Wed, 10 Sep 1997, Ernest Perez wrote:
>
>>
>> I agree with Dr. David R. Newman of Queens University about the
>> unreliability and questionable usefulness of counting hits as valid
>> statistical analysis. These comments were in regard to the original
>> message from Gerry Rowland of Iowa State Library, in this message
>> trail....
>>
>> Newman wrote:
>> > This only tells you the time pattern of WWW usage in the library, and
the
>> > popularity of different WWW information sources among your library
users.
>> > It tells you nothing about how useful, relevant or interesting the
>> > information is. Nor does it tell you why the site is popular, and
whether
>> > it is a result of technical factors (indexing, search results, time to
>> > download, time to connect) or factors to do with the material and the
>> > reader's interests.
>> >
>>
>> About all you can actually expect from analyzing hits on a Web server is
>> a continuing comparison with itself, assuming that's of any use. And
>> "about:global" results from the client/browser really tell you only what
>> an individual user happened to look at at a particular moment in time.
>> [For more on about:global, see "Hidden features of Netscape
>> Navigator ouuups Mozilla !"
>> at <http://wwwcn.cern.ch/~rigaut/about.html>
>>
>> Gerry Rowland, State Library of Iowa originally wrote:
>> > Internet use statistics are a high priority for libraries at the local,
>> > state and national levels.
>> > Counting hits by local users against remote Internet servers has been
a goal
>> > of the FSCS, the national public library statistics project, for
several years.
>>
>> I really don't understand why it's such a high priority. Are we trying
>> to measure how hard computers and telecommunications networks are
>> working, for some reason? Admittedly, Rowland's recent _Public
>> Libraries_ article uses the example of high hits on the Netscape server
>> contrasted to much lower hits on library servers, and suggests that we
>> might be "losing the war."
>> Seems to me that this is comparing apples and oranges. Are libraries
>> going to suffer because the telco's 411 information number gets a lot
>> more calls than library telephone reference numbers? Or that TV Guide
>> and Dilbert get looked at a bunch more than the 800's section of our
>> collections?
>>
>> Rowland writes:
>> > At today's meeting of the FSCS group, we learned that the command
>> > "about:global" in the Location: box of the Netscape browser returns a
list
>> > of files downloaded and a count of total files. I assume that Internet
>> > Explorer has a similar feature.
>>
>> The deceptive ease of collecting log files is perhaps exactly the
>> problem. It all really depends on if you're really recording anything
>> worthwhile in the first place. about:global shows you Netscape's
>> history.db file, which is simply a list of all the files you accessed in
>> browsing Web pages.
>> For instance, let's say you looked at a "single" page made up of an
>> HTML file, a photograph, two pretty line graphics, three "NEW" icons, a
>> CGI call, and a couple of other icons or graphic devices. Okay, the hit
>> count is going to show that you accessed that server something like 12
>> or 15 times. Is that information supposed to tell us something of ANY
>> significance?
>> And that webserver's log file would tally up 12 or 15 accesses.
>Well,
>> I s'pose you could maybe conclude that the particular site used a lot of
>> really complicated pages? No, the actual case might be that the
>> particular webserver uses really blah, plain, ho-hum pages that get used
>> a lot because of what's on them.
>>
>> Rowland writes:
>> > It would appear that a count of hits could be tallied over a period of
days
>> > or weeks, then multiplied to give an annual figure to provide a count
of hits.
>> >
>>
>> Yeah. I drive an average of 14,000 miles each year. So what?
>>
>> Rowland writes:
>> > Is this the way to generate Internet use statistics? I think it just
may be.
>>
>> Not!
>>
>> Newman goes on in his commentary:
>> > Think a bit about what sort of statistics would help you run a better
>> > library. Most of the questions relate to the needs and perceived
benefits
>> > of library patrons, and how well the WWW service meets those needs. So
>> > you actually need to ask them - be it in interviews, face-to-face or
>> > on-line questionnaires, or in single questions automatically presented
to
>> > a small random sample of accesses (the technique Jacob Palme used to
>> > research e-mail use and its benefits and costs to staff of the Swedish
>> > Defence Organization).
>>
>> On target. What are we trying to measure here, anyway? If it's patron
>> use or satisfaction, then, like Oregon State Librarian Jim Scheppke
>> says, "Why don't we ask them?" Interviews or questionnaires or focus
>> groups, and good sampling techniques will give a much more accurate
>> picture of how well or how poorly a library is performing in regard to
>> its network information services.
>> Counting network hits is a throwback to 1) the library profession's
>> mania for counting useless numbers for no particular purpose (because we
>> always have), and 2) the old days when connect time _maybe_ meant
>> something. And that's a qualified maybe. Connect time might also reflect
>> lots of coffeebreaks or interesting/attractive people at the next
>> workstation.
>>
>> I agree with the intent...that it's useful to measure value and quality
>> and success of ourinformation services. But I don't think that measuring
>> the present form of computer hit odometer is the way to do it. Let's do
>> think of some valid measures....
>>
>> Cheers,
>> -ernest
>>
>> Ernest Perez, Ph.D.//Oregon State Library//perez@opac.state.or.us
>> -----------------------------------------------------------------
>> Library: Like a software house, except the software's free.
>> It's not vaporware. And if it breaks, they help you fix it.
>> Quickly. Without a toll call.
>>
>
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Isabel L. Danforth Reference Librarian, Wethersfield Public Library
danforth@tiac.net Co-Director of Librarians' Online Support Team
http://www.gnacademy.org:8001/~lost/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~