Walt at Random

Thanks to all those who commented on my possible public library value ratio project, either on the original post, on the Wednesday, May 9 update when I extended the feedback period or on the third post on Friday, May 11–and to those who sent email rather than commenting on one of the posts.

The comments provided valuable feedback and helped me sharpen the model and concept, as well as pointing me to a range of studies done by state libraries and other agencies. Those studies and some additional thinking also helped me revise the formulas I was planning to use–although in one case that “thinking” resulted in a formula that’s just plain wrong (and has now been corrected). Those who love formulas and details will want to go read all the way through the May 9 post to see what was involved.

The error was too much cleverness on my part–and it would have been glaringly obvious as soon as I started sorting the spreadsheet for some overall comments in Chapter One. Namely, one particular library system showed more than $2 billion in PC-related benefits rather than about $26.1 million. Go read the bottom of the May 9 post: It has to do with large multibranch systems.

After reviewing the comments, looking at the other studies, and thinking about my own time and energy, I’m going ahead with the project–with the proviso that if I find that it’s not going well enough or rapidly enough, it may be abandoned midstream. My goal is to have the preliminary edition available at least a week before ALA Annual, although that’s not a hard-and-fast goal.

Preliminary edition? Yes. This one’s a proof of concept, with the hope of getting feedback on its usefulness. If it’s useful (and reaches an audience), I’ll plan to do a more polished study using the 2010 IMLS database–when that emerges.

Thanks again to all who commented, including those I may have given a bad time. Oh, and if you have a great idea for a title, let me know…right away. Right now, I’m probably going with the best of several suggestions from Library Society of the World members (my informal thinktank!)…

I’ve received great feedback–both in comments and in email–based on the second post on my possible public library value ratios book project.

Michael Golrick pointed me to a Wisconsin page with a much larger list of state, local and regional library ROI studies than I’d encountered previously. I’ve now read (or at least skimmed) all of those and checked a couple of other sources.

As a result, I’ve added a new section to the May 9th post explaining the actual model I’ll use for the Value Ratio–if I do this project at all. I think it’s a defensible model that’s also wildly incomplete (it’s a baseline, representing a countable portion of a library’s value, but nowhere near the totality), and could possibly be a valuable complement to some excellent state and local studies. But I’m still not sure.

I invite you to take another look–the new information is at the bottom of the post, under the line–and offer additional comments from now through Sunday, May 13, 2012. That’s when I’ll make a decision on whether or not to proceed. (You may also want to read the original post.)

Two days ago (Monday, May 7, 2012), I asked for instant feedback on a possible quick-book project providing a detailed set of charts and percentiles related to a “value ratio” for public libraries–that is, the ratio between library expense per capita and library value to its users and community (per capita).

A number of states have done studies of this ratio (usually defined as ROI, return on investment), some of them using fairly sophisticated models and surveys. Consistently, however, such studies yield a single number.

What I have in mind is both cruder (far less sophisticated and relying entirely on public information) and more detailed–namely, yielding a variety of percentile ranges that break down the overall picture into more detailed forms. To quote a little of the other post:

The four axes or, if you will, chapters, following an overall look:

• Clumps of libraries by LSA size (using the 10 HAPLR divisions)
• Clumps of libraries by expense/budget ranges
• Clumps of libraries by per capita expense ranges
• State-by-state analyses (one clump for states with few libraries, probably three by broad size categories for states with many libraries)

For each clump, as for the overall figures, I’d provide correlations as appropriate, plus mean, median, and percentile levels in two ways–the 90th, 80th, 70th, etc. percentiles, but also the percentage of libraries exceeding certain value ratio set points.

I wanted feedback on whether this might be useful, useless or even a bad idea. I wanted that feedback by the end of today (May 9, 2012), since my plan was to have the book available by ALA Annual.

Early feedback

I’ve received half a dozen responses. While most of them are mildly positive, by far the most detailed response–and the only one clearly coming from a state library employee–was less so. I’ll quote that feedback,, from Steve Mattthews, here:

For what it’s worth, it seems it would only by useful IF the Value Ratio to which you refer is an empirical statistically useful number. Simply calling some number “value” doesn’t make it valid. Are you using contingent valuation (CV) either willingness-to-pay (WTP) or willingness-to-accept (WTA)? Or are you using some arbitrarily assigned “value”?

It also seems like a no-brainer premise to say that “libraries that are better funded provide better service.” One of the pitfalls of using data collected for one purpose to apply to a different purpose is that it requires numerous questionable assumptions to make the numbers meaningful. How would this data be more valid or more valuable than the results from several states that conducted their own ROI study within the past few years? The 21st Century Library is More:

I replied, probably defensively, after looking at all of the ROI studies available from that link and from links within those links. My plan is to do something entirely different, possibly more valuable only because it’s something more than One Big Number for an entire state–but certainly less valid based on that commentary.

Extending the feedback request

If Steve Matthews’ response is typical of what well-informed librarians would say, or typical of how state libraries would feel about what I have planned, then I should abandon the idea: Way too much work for what may be perceived as valueless or even harmful.

I’m not quite ready to do that. Instead, I’ll ask for more feedback, through the weekend (that is, through Sunday, May 13, 2012). Apologies in advance if I respond in what seems a defensive manner; maybe I’ll try to avoid responding at all. Except, of course, to come to a decision on Monday.

So, either by mail to waltcrawford at gmail.com or in a comment here (noting that comments with multiple links sometimes get trapped as spam, and I get WAY too much spam to check each one):

Good idea? Lousy idea? Pointless idea?

[If you think my investigation into public library closings was actually a bad thing, you will most assuredly think this project is a bad idea as well–but you can think the first was good and that the second is pointless.]

Additional information (added 5/11/12)

I’ve now read or at least skimmed all of the studies linked to from the page Michael Golrick identifies (in one of the comments below), re-read some other sources, and modified (or clarified) the numbers I’ll use if (and it’s still if) I do this instabook–which, it should be clear, is intended to be complementary to the state ROI studies, the HAPLR ratings, the LJ Star ratings, and all those other things, not a competitor to any of them.

Here’s what I would be using to establish the Value Ratio and the two subratios (Explicit Value and Implicit Value):

Explicit Value

• $8.30 times the sum of Circulation, ILLTo, and ILLFrom. (This is one of the two Big Numbers–although ILLTo and ILLFrom are a small part of it. After looking at a wild diversity of arguably sound amounts for average circulation worth, I used the average price of mass market paperbacks in 2008.)
• $15 per reference question. (Assumes that, these days, there are relatively fewer but relatively more important/difficult reference questions.)
• $10 per program attendance. (Given the relatively small number of reported program attendances, changing this number wouldn’t have much effect on anything.)
• The larger of: $8 times the number of counted PC uses or $2.66 times (the number of Internet-connected computers available for users times the number of open hours the average number of open hours per outlet). This is the other Big Number. However calculated, it’s also a partial standin for uncounted items–database use, wifi use, etc. [If you’re wondering: That $2.66 figure is based directly on one of the carefully-done ROI studies, and assumes $4/hour value and that 2/3 of computers, on average, are in use. The $8 figure assumes an average of 2 hours per use, also at $4/hour.)

Implicit Value

• $60 per open hour (partly a standin for in-house use and the many uses of library as place–arguably, this figure should be much higher).
• $5 per visit (also partly a standin for the above).

Right now, it looks as though the median is a little more less than 5 and the mean is very close to the median (which is a good thing)–and EV is anywhere from two to three times as much as IV overall (about 2 for median, more than 3 for mean).

This information may or may not help late comments.

And no, I still haven’t made up my mind. The three or four additional hours spent yesterday in checking sources I hadn’t previously encountered, and the half-hour redoing overall numbers, don’t matter much in the overall scheme of things–they’re sunk costs. If this wouldn’t be valuable to libraries and the library community, I won’t do it.

Strikeouts and changes, Monday, May 14, 2012

As I was going over the various comments, asking for help with a good title for this study (from the Library Society of the World, and I’ve received some excellent ideas), and checking one item for interest (namely: how often is “actual PC use” a larger figure than “potential PC use”), I realized that there was one fundamental error in my ratios–it became fairly obvious when one multibranch library had more than $2 billion in PC use.

Namely, the original formula only works for central libraries with no branches. For multibranch libraries, it overstates the availability of PCs by a factor equal to the number of branches. That is: If a three-outlet library (central and two branches) has 200 PCs in central library (open 70 hours per week) and 100 each in the two branches (each open 50 hours per week), the original formula yields $2.66 * 400 * 170, or $180,880–and assumes that there are 400 PCs available 170 hours per week. (I would have discovered this error as soon as I started playing with the spreadsheet for the first chapter: It would be really obvious that something was wrong.)

The new formula is the best I can do, and it somewhat understates availability for libraries in which Central has many more PCs and is open many more hours than branches. To wit, it divides hours per week by the number of outlets–so, in the three-outlet case, the formula is $2.666 * 400 * 56.7, or $60,293.

If that seems like a big difference, consider a library with 89 outlets, open a total of 242,424 hours and with a total of 3,609 PCs: The original formula overstates the availability of PCs by close to 88 times. (Yes, there is such a library. As it happens, once the correct formula is applied, the two possible PC use benefit numbers–that is, $8 times reported PC uses and $2.66 times PCs times average hours–are very similar for that library.)

This change reduces the median ratio to 5.00 for 9,102 libraries or 4.88 for the 8,535 libraries included in most of the analysis (excluding 415 “libraries” with less than one quarter of a librarian, 152 libraries with less than $5 per capita funding, and 27 libraries with more than $300 per capita funding). Notably, the correlation between per capita expenditures and per capita benefits is now 0.63, which is a very strong correlation–stronger than it was with the erroneous calculation. It also means that circulation is The Big Number, representing 71% of direct benefits.

Does all this sound as though I’m almost certainly going ahead? That’s true, based on the whole set of comments received here and directly via email. And my thanks to all who commented. Look for an announcement, significantly before ALA Annual if all goes well–and probably in the next Cites & Insights.

I need instant feedback on this one–by Wednesday, May 9, if at all possible. A quick comment here or email to waltcrawford at gmail.com will do.

The idea

When I was finishing the Public Library Closures study, I remembered back to many years ago when I was doing state library conference keynotes with some regularity. For several of them, I did snapshots of the public libraries in the state (at the time, using state databases), looking at per-capita expenditures and circulation. I found that, with very few exceptions, “libraries that do a lot do a lot”–well-funded libraries had higher circulation per capita than poorly-funded libraries, to the point that they were at least as good bargains.

I wondered whether a slightly more sophisticated calculation would be useful to libraries and library groups in telling the positive story I believe public libraries should be telling: That is, it’s not about forestalling closure, it’s about providing the resources so libraries can enrich and enhance communities. And, with very few exceptions, public libraries are demonstrably good stewards of additional resources.

The status

I’ve taken the 2009 IMLS figures (the most recent available) and done two versions of a master spreadsheet–both using eight reported factors to determine direct value (countable events that are clearly valuable to community members) and indirect value (countables that provide a less clearly direct value to the community), calculating total operating expenses per capita, and preparing a Value Ratio: value per capita divided by expense per capita (with two sub-ratios for direct and indirect value).

One spreadsheet includes 8,936 libraries–excluding those that didn’t directly report expenses, hours, or circulation. It does include imputed figures for items other than expenses, hours and circulation. (There are some 300 libraries–mostly very small, quite a few not in the 51 states + DC, that just don’t report enough information for inclusion.)

The other begins with the 8,936, but moves 524 of them to a separate Outlier page based on one or more of these conditions:

• Less than one-quarter FTE librarian or total staff, or imputed staff levels rather than reported staff levels (350 libraries)
• Less than $5 per capita expenses (147 libraries)
• $300 or more per capita expenses (27 libraries))

That leaves 8,412 libraries. (Most of the 524 are small: While that’s 6% of the libraries, it includes only 1.8% of the population served–the remaining 8,412 include 293.7 million people in service areas, as compared to 5.3 million served by outliers.)

Also, these tables were cleaned up to zero out all numbers not directly reported by libraries–all imputed numbers became zeros.

There’s a strong correlation between expenses per capita and value per capita (>.6) for the 8,412 libraries. Libraries that are funded better generally provide more services, and–up to a point–it doesn’t seem to involve diminishing returns.

The plan

Here’s where I need immediate feedback: Is it a waste of time to break this down into a detailed set of charts and percentiles, using four different axes to look at subsets, yielding a reasonably compact book that I’d probably sell for $60/copy (and probably issue in a new version, with refinements, a few weeks after 2010 IMLS databases emerge)?

That is: Would a fair number of state libraries and possibly libraries find this analysis worthwhile (and possibly library schools), or would it be ignored or, worse, resented (as some folks seem to resent my finding that public libraries aren’t actually shutting down all over the place)?

The four axes or, if you will, chapters, following an overall look:

• Clumps of libraries by LSA size (using the 10 HAPLR divisions)
• Clumps of libraries by expense/budget ranges
• Clumps of libraries by per capita expense ranges
• State-by-state analyses (one clump for states with few libraries, probably three by broad size categories for states with many libraries)

For each clump, as for the overall figures, I’d provide correlations as appropriate, plus mean, median, and percentile levels in two ways–the 90th, 80th, 70th, etc. percentiles, but also the percentage of libraries exceeding certain value ratio set points.

(The overall value ratio for all 8,412 libraries is 4.59–that is, $4.59 in value for each $1 in expenses, including zeroing out all imputed numbers. Including the imputed numbers and the outlying libraries changes this to 4.62, a fairly trivial change. As it happens, the 4.59 is almost equally split between direct and indirect value.)

If you’re thinking either HAPLR or the LJ Star Libraries, there are similarities and differences.

Similarities: As with both of those, it’s based on nothing more than the IMLS database and a set of calculations.

Differences: It’s looking only at patterns, not “stars” or “top X.” In fact, I’ve removed library names from the primary spreadsheet, so I’m not even tempted to consider the library names themselves. (Yes, I can get back to the library name, but it’s a two-step process.) And it’s looking at more factors than LJ, although fewer than HAPLR.

Mostly, though, it’s not about naming names. It’s about showing value in general and providing appropriate benchmarks/comparisons.

Prepare or abandon?

I have the spreadsheet. Do I just say “that’s interesting” and drop it, or does it make sense to prepare the results in a manner that libraries/library groups might find useful?

Your feedback, please–and soon! (If I do this, the 2009 version will probably be ready by ALA Annual, which I won’t be attending.)