Walt at Random

I’ve done all the scanning I’m planning to do, looking for liblogs (that is, blogs by library people, as opposed to official library blogs). I’ve found 1,277 liblogs, and checked and excluded another 1,308 things that aren’t liblogs.

It’s Your Turn

I know I’m not going to get everything. That became fairly clear when, on checking 83 possibilities from eight blogrolls added while checking mostly-defunct liblogs, I came up with 31 legitimate liblogs…

I also know I’m not going to go through any more iterations.

So: If you know of liblogs I haven’t included, you’re invited to let me know between now and September 15, 2010.

What’s a liblog?

For the purposes of the current project, the most inclusive I’ve done, here are the criteria:

• The blog must be available on the web without using passwords or special permissions.
• Most posts in the blog must be in English.
• There must be at least one post prior to June 1, 2010 (that is, no later than May 31, 2010).
• The blog should be by a self-identified “library person” or group of people, not explicitly identified as not at all relating to libraries, or somehow related to libraries…but:
• The blog should not be an “official blog” from a library or a library-related group. (If you’re in doubt, include it.)

Please, Before Submitting Any Candidates…

• Check this page — Liblogs 2010 (with exclusions) — DRAFT. Use your browser’s Find function to check the name. (The list is in alphabetic order, but it’s idiot alpha order, with a few “A ” entries and a lot of “The ” entries. And, of course, cute punctuation can change sorting.)
• [Added 8/31/10:] If you see a blog twice, under a current name and an old name, that’s OK: In order to maintain my sanity while checking candidates, I include older names of renamed blogs in the exclusions list, under the “Renamed” category. There are 115 older names in the list as of now.

If You Have Candidates…

• Add a comment, with the blog name and URL–but give the URL as text, not as a link (omit the http://), and don’t combine the blog name with a link. (Why not? Because, particularly if you have more than one, it will cause Spam Karma 2 to flag it as spam–and with more than 100 spamments today, I’m not sure I’ll be able to sort through all the spam looking for legit posts.)
• Or send me email, waltcrawford at gmail dot com, using the same rules.

Sending candidates doesn’t guarantee they’ll be included–and since this project won’t involve blog profiles, it may not matter as much. Still, it will almost certainly be the most comprehensive look at English-language liblogs ever done, so it wouldn’t hurt…

Thanks. Now back to the Open Access project (and dealing with a keyed car, and, and, and…)

Postscript, September 16, 2010:

I’ve turned off comments, since it’s now past September 15, 2010.

Between comments here and direct email, and excluding blogs that obviously didn’t fit my criteria, I received 33 candidates. Of those, 25 met the criteria and have been added to the study, bringing the new total to 1,296 liblogs (that includes one more blog, noted in a response to another post, that had one single post some years back…). Six of the candidates are too new, having begun in June, July or August 2010. One is an official blog. And one just doesn’t seem to be there.

Thanks!

What’s that you say? How can this be Update 2 when you haven’t seen “Liblogs 2010” as the title of any previous post? All I can do is quote ol’ Ralph Waldo:

“A foolish consistency is the hobgoblin of little minds…”

Yes, I know, I’d thought it was the more general (and more nonsensical) “Consistency is the hobgoblin of small minds”; this version has the virtues of (a) apparently being the correct quotation and (b) making a lot more sense.

Anyway…this is an update to “The new project: update 1,” which was an update to “We grow too soon old...”

I guess calling it “Liblogs 2010” means I’m fairly certain I’m not going to abandon the project. Which, at this point, is true–barring some wonderful new paying opportunity that requires too much time. (I hope to start on one paying project very soon and to convert something else into a paying project–but both of those would leave enough time to continue this project after some delay.)

The candidates

I’ve completed the scan of 868 “candidates” from LISWiki, the ODP list of librarian blogs, the LISZen source list, Meredith Farkas’ “Favorite Blogs” list, Davey P’s “HotStuff” list and the Salem Press list–which began as 2,911 possibilities that boiled down to 868 after eliminating duplicates, items among the 606 liblogs (and 71 exclusions) that were either in previous studies or already identified from my Bloglines subscriptions, and items obviously not in English. The breakdown of those directories before deduping:

• Salem Press: 318
• LISWiki: 743
• ODP: 138
• LISZen: 795
• Hot Stuff: 748
• Farkas: 179.

The results

I found 415 more liblogs for the “broad look”–that is, English-language (or predominantly English-language) blogs by “library people” or somehow related to libraries, that aren’t official blogs (or at least don’t function as official blogs) and have at least one post visible on the web, no matter how old. The running total is now 1,021 liblogs for the broad look.

I also added 426 excluded candidates–names/addresses that aren’t visible (either deleted or password-protected), blogs not in English, official blogs, blogs with no apparent relation to libraries or library people, and blogs that have been incorporated into newer, renamed blogs.

Hmm. 426+415=841. The other 27 cases were either my mistakes (failing to delete blogs already there) or naming differences (the same blog appearing two or three times under different names).

Slightly more detailed results

Here’s how the 497 excluded candidates break down by reason, as I recorded them:

• Ten empty, blank, or dummy pages–some of them potential blogs but with no content at all. One had a single picture, nothing else.
• One malware site, popping up all sorts of windows asserting that my PC has viruses: Thanks a lot, Information Knot.
• 26 sites that aren’t blogs–including aggregators, newsletters and a variety of other things.
• 58 blogs that aren’t in English (as judged based on the first page of posts, except for all the posts in Farsi where the page title was enough… in one or two cases, where there are a handful of English posts among mostly-non-English posts, it was a judgment call)
• 62 blogs for which I could find no plausible library or librarian connection, either in the author info or categories or posts. I tended to err on the side of inclusion.
• 164 not viewable–mostly sites that have simply disappeared, but with a large handful of password-protected sites. I’m guessing that nearly all of these were liblogs at some point.
• 99 official blogs, including both library blogs and association/company blogs that appear to function as official publications. I also tended to err on the side of inclusion here, that is, if a blog had a library abbreviation (or ALA division or whatever) in its URL but was clearly the work of individuals who disclaimed organizational responsibility, I left it in the broad look.
• 77 renamed–blogs that have been incorporated into, or in a few cases, followed by, newer liblogs with different names.

I think that adds up to around 241 “former liblogs,” but that number might be high, since some not-viewable blogs may also be excludable for other reasons.

What about the broader look? Here’s an early summary:

• 597 liblogs have two or more posts during the March 1, 2010-May 31, 2010 test period. Another 63 have exactly one post during the quarter; at present, I don’t plan to include those in the deeper look.
• Of that 597, 417 have a Google Page Rank of 4 or higher–and, currently, those are the 417 I plan to use for the deeper look. Which is to say: I’ve recorded count, length, and comments for March-May 2010 (and going back to 2007) to the extent feasible for each of those blogs. I could add another 71 blogs with GPR 3, if I’m willing to do the extra metrics for those (in 19 cases, most of the metrics are there from earlier studies). It’s very unlikely that I’d add the 21 blogs with GPR 2, the six with GPR 1 (something I’ve almost never seen before, actually), or the 75 with GPR 0 (which can happen because a blog changes platforms or because it’s a corporate-platform blog and gets no link love on its own).
• So it appears that at least 660 liblogs are at least marginally active in 2010–and that a deep look could involve anywhere from 417 to 492 liblogs. There were 449 liblogs with countable posts in the 2009 study (including some two dozen with just one post), as a point of comparison.
• I have recorded blogging software when that was visible, but I’m going to recheck two dozen of the blogs where I recorded “other” as the software, before I started viewing source in cases where the software wasn’t obvious….and maybe 26 “unknown” that didn’t seem to be using any canned package.
• I recorded the country in which the blog was being written, when that was clear, and show 25 different countries.

What’s next?

Right away, nothing at all–I’m going to do some other work.

Then, well, it depends on other projects and energy.

• I’m likely to do the cut-and-paste trick with the some or all of the directories noted in LibWorld – library blogging worldwide. Although it’s fair to assume most of those blogs are either already in the spreadsheet or are non-English, there might be some exceptions.
• If I have loads of energy, I might cut-and-paste the first, say, 50 library-related blogrolls from blogs already in the deep study (or otherwise current), and see whether there’s enough yield to be worthwhile.
• I know that it’s not possible to say “here’s the universe”–but I suspect it will be fair to say that the final broad look will represent a very large majority of the English-language non-official liblog universe, at least of those blogs that have left any trace at all…
• And then, probably late this fall, possibly in early 2011, I’ll start working with the spreadsheet to prepare a new report, one that will probably come out as a (not quite so thick) book, with some details emerging here or in Cites & Insights. There’s quite a bit to be said about the broad range–after all, all I’m missing is length of posts and comment count for 2010, and pre-2010 metrics–and even more about the deep range and comparisons between the two.

Casual observations

I seem to have encountered a lot of blogs this year that I never encountered or at least didn’t include in earlier studies. I’m guessing that’s partly because blogs tend to gain GPR over time, partly because the Salem list actually includes quite a few blogs that aren’t in other directories, partly because the other directories have improved.

Quick observations:

• “PLN” seems to be a term that’s automatically understood to mean Personal Learning Network by many (most?) school library bloggers–and not, I think, by most others. I can assure you that the PALINET Leadership Network would not have had “PLN Highlights” as its alert blog name if that TLA (three-letter acronym in this case) was universally understood–or even prevalent outside of school librarianship.
• Either Will Manley or one of the commenters on his blog made a comment about this being the golden age of book reviews. Quite apart from Amazon and LibraryThing, I think that’s true, based on the number of high-quality, prolific book-review liblogs I’ve encountered this time around…particularly for YA and children’s literature, but also for books in general.
• Oh, in case you didn’t catch that: I did not require “active since December 2009” this time around, and I’ve recorded the starting month and lifespan for a good many blogs that weren’t around very long. Yes, Will Unwound is part of the deep study despite its January 2010 start date…and a total of ten liblogs that began in 2010 are part of the broad study. Of course, they had to have begun by May 2010, since May 31 is the cutoff point for all observations.
• What I’ve derided as The World’s Worst Blogging Platform, the LJ/SLJ construct, now turns out to be built on what I regard as the world’s best blogging software–but also the software that can be used to screw up a blog’s presentation perhaps more thoroughly than any other. That’s right: the LJ/SLJ blogs now use WordPress. So does this one, and I have no intention of changing. (I’ve encountered exactly one liblogger who explicitly moved away from WordPress–to Blogger. There may be others.)

Whew. That’s a lot more than I intended to say. So far, my promotional posts for But Still They Blog have been a total washout, with zero additional copies sold. So, you know, I’m aware that doing this 2010 study is unlikely to be remunerative…but it is fascinating.

I thought I’d update this post, now that I’ve spent a few hours days on the scan of 868 possible candidate blogs.

Then

I started out with 606 liblogs (English, visible–that is, reachable on the web, not official, somehow related to libraries or librarians) and 71 “rejects” (blogs that had been on my radar at some point but were either non-English-language, had wholly disappeared from the web, or were actually official blogs)–and 868 candidates, combined and filtered from 2,911 listings in six directories.

I guesstimated that I’d find 200 to 400 more liblogs from among those 868, but had no idea what the number would actually be. I also had no idea whether the process would be so grueling that I’d give up partway through–after all, there’s no economic incentive to complete this, just curiosity.

Now

I’m now almost through the “I”s in that crude-alphabetic list of 868 (“crude-alphabetic”: there were several “A something” blogs and there will be a LOT of “The something” blogs).

• There are 551 candidates left to check, so I’ve apparently done 317. In other words, I’m a little more than a third of the way done.
• It’s clearly feasible to do this. It’s not fast–I haven’t been doing any other writing this week–but it’s not so grueling as to be hopeless. I’ll certainly finish this scan, although not necessarily this month.
• It’s essentially impossible to estimate the time required, particularly since I’m backfilling data for newly-discovered blogs that go back more than one year. I might be able to check 30 blogs in one hour (if half of them are non-English, official, or disappeared and most of the rest were only there briefly); I might require more than half an hour just to handle one blog (say a 5-year-old Kidlit or YA lit blog with an enthusiastic audience). I’m guessing it averages about 20/hour overall, but that’s a very crude guess.
• At this point, there are 749 blogs on the broad-survey list and 229 excluded candidates. That’s an increase of 143 blogs and 158 exclusions. If the same ratio runs through the rest of the candidates, I’ll wind up adding just about 400 total (plus another almost-300 exclusions). That would mean roughly a thousand liblogs in the broad survey.

Future

Since I’ll start working on C&I again soon, and I hope to begin another (more lucrative) project in a week or so, this might go on the back burner–but I’m also interested in seeing how it goes (e.g., what percentage of those 1,000-or-so liblogs will turn out to be currently active?).

Assuming I come back to this, it now seems likely that I’ll make up a new list from the various national liblog directories in LibWorld (assuming some of them are still around and updated) and check that list. It’s less certain that I’ll try blogrolls, but who knows? It’s clearly not possible to be sure I’ve seen the whole universe; it’s not clear whether assembling blogrolls from 100 or 200 or 500 liblogs will yield any significant number of blogs not otherwise discovered.

Meanwhile, I suspect that I will include portions of But Still They Blog in Cites & Insights–but almost certainly not the whole non-profile manuscript, at least not in one big issue. So far, my new attempts at publicizing the book have yielded exactly zero sales, but that could change…

…and too late smart.

The setup

Four times, I’ve done analyses of liblogs (blogs by library people, as opposed to library blogs)–twice within Cites & Insights, twice as books.

Still available, still great bargains: The Liblog Landscape 2007-2008 and But Still They Blog: The Liblog Landscape 2007-2009. Note that Lulu’s still offering free shipping for any order over $19.95, making these even better bargains.

If you’re wondering: The two C&I analyses were Investigating the Biblioblogosphere in September 2005 and Looking at Liblogs: The Great Middle in August 2006.

In each case, and particularly for the two books (which attempted to cover a very large portion of the English-language “liblog landscape”), one of the biggest time-sinks in the project was the process of finding new liblogs–ones I hadn’t already included in a previous study.

There are several sources for such blogs, and the sources tend to repeat one another (as you’d expect)–and once you’re dealing with more than a hundred blogs or so, there’s no way I could remember which blogs I’d already looked at. I used a variety of techniques to make the situation somewhat manageable–after all, we’re talking about several thousand listings in the primary sources–but it still took scores or hundreds of hours, particularly when I started looking at blogrolls.

Last year, I concluded that, if I ever did do another similar study, I’d probably give up on blogrolls altogether: Too much work for too few new discoveries.

The occasion

Based on sales to date of the two books, it would be insane to do another study.

On the other hand… well, there were still things I wanted to know about the progress of (English-language) liblogs.

So I decided to start another, somewhat different, project and carry it out if it seemed feasible and didn’t get in the way of more directly-useful projects (such as a non-self-published book I hope to be doing later this summer and early fall).

The new project differs from the last two in two key respects:

• If there’s a book, it’s going to be much shorter–and the obvious way to do that is by leaving out individual blog profiles. Clearly, at least 90% of blog owners aren’t going to pay for a book that includes a profile of their blog, and those profiles are a lot of work (and take up a lot of space).
• The new project has two levels of inclusion, the first of which makes the project particularly interesting to me.

The two levels

• The broad look: As comprehensive a survey as possible of English-language blogs by library people (excluding official blogs) that have any visibility at all on the web in mid-2010. I can’t claim it will be a comprehensive survey of liblogs, because (a) quite a few have disappeared entirely, (b) a few are password-protected and won’t be included, (c) there will certainly be dozens or hundreds of blogs that I won’t encounter. But it will be the broadest look I’ve taken–albeit with less information on each blog:
  Birthdate: When it began (year and month)
  Lifespan: How many months it operated (through May 2010)
  Currency: The most recent post (prior to June 1, 2010)
  Nationality: The country (when obvious)
  Program: The blogging software (when obvious)
  Frequency: The number of posts from March 1, 2010 through May 31, 2010.
• The deep look: A deeper look at a large subset of those blogs, defined as:
  Blogs that have a Google Page Rank of 4 or higher (fairly visible blogs)
  that have at least two posts between March 1, 2010 and May 31, 2010 (active blogs).
  For those blogs, I’m also tracking the same metrics as in last year’s study (when available): Frequency, comments, and total post length–for March-May 2010 and, for blogs new to this year’s study, going back to March-May periods in 2009, 2008 and 2007.

The first part is more ambitious, in that I’m including–potentially–a lot more blogs.

The second part is less ambitious, both because I’m not doing blog profiles (a decision I could only change with up-front sponsorship–it’s a lot of work) and because I’m limiting that level of statistical analysis to blogs that are currently active.

(One difference: I’m not requiring that blogs have started before January 1, 2010. They must have started before June 1, 2010.)

To do this project, I once again need to dive into the directories, at least as a first cut, recognizing that I’d probably pick up some additional blogs from blogrolls…but only if I could take the time.

The breakthrough (the forehead-slap moment)

Last year, I used some teeny-tiny printouts to try to cut down the amount of extraneous checking, but it was still an enormous pain. This year, I was determined to avoid superfluous printouts, even if they only used a page or two of paper.

I had one small bright idea–at each stage (where I’ve finished a pass against a source of blogs), peel off copies of the blog names and excluded blog names to a separate spreadsheet, sort them, and use that spreadsheet in a narrow little column alongside the browser window when I’m looking at a new source. That worked nicely to add new blogs from my own Bloglines list–the process took half a day or less and yielded 43 new blogs (and nine new exclusions).

Well, so, I could do that with the other primary sources (LISWiki, the ODP list of librarian blogs, the LISZen source list, Meredith Farkas’ “Favorite Blogs” list and Davey P’s “HotStuff” list, the Salem Press list)–but that would still be an ordeal.

Or… I could cut-and-paste each of the directories, with HTML included, into a Word document; use global edits to normalize them, sort the blogs…and trim that document by comparing it to my existing list of already-included blogs. Then cut-and-paste the document back to a webpage to make it easy to check new candidates.

Why didn’t I think of this last year or the year before? Maybe because I never thought of Word and HTML in the same space…maybe because I’m getting old.

The results (so far)

This morning–after the usual Friendfeed time and editing for another project–I did the cut-and-paste for these six sources (the Salem Press list required more work than the others, but still not much); within an hour, I had a sorted Word document with–gasp–2,911 candidates.

This afternoon, after lunch and some errands, I trimmed that sorted document by comparing it to the spreadsheet, including special passes for Idiot Sorting (I’m being lazy this time, so there’s lots of blogs in the “A ” and “The ” areas–and some directories normalize those articles away). The process took about two hours, maybe less…and I now have a webpage (private) with 868 liblog candidates.

Which is still a lot of checking to do, but little enough to be feasible. How many of those 868 will I add to the 606 (not including “excluded blogs”) in the current list? I have no idea; I’d guess somewhere between 200 and 400, but I could be wrong.

If this process does turn out to go reasonably smoothly, I might–after taking an appropriate break and working on other stuff such as C&I–even change my mind about blogrolls. After all, they mostly use a consistent format, and I could cumulate a whole bunch of them in a Word/HTML document and… well, we shall see.

No promises

Am I certain there will be a 2010 survey? Not really. I’d say the odds are pretty good, but if paying gigs come up or there are other things that interfere, it could take a long time–and, frankly, I haven’t invested so many hours in it that I couldn’t just abandon it. (Although my track record for abandoning projects doesn’t suggest that this is highly probable.)

And for those of you who say “You idiot, you could have done this much more easily this time and the last two times by…” Well, you may be right. I certainly could have saved a lot of boring and annoying work in 2008 and 2009 if I’d thought of this. There may be an even better way, but this is a good start.