Walt at Random

Off to a good start on the book itself. If all goes well, it should be out (free PDF, production-cost paperback) by the end of the month.

While the final dataset won’t be uploaded until then, it’s fair to say that the odds are VERY high that there will be no changes to the temporary version mentioned in a previous post (last paragraph). Very high as in essentially certain…

The second pass is complete. A couple of Indonesian universities fixed their malware problem, which was encouraging enough that I did a quick rescan of the 618 xx2/xm2 cases (journals that had malware or were unreachable/unworkable both last year and this)–and recovered 43 journals to some other status. (Not always to a/default, but usually.)

Now there are a few days of “massaging”–making sure data is consistent, adding the derived data used for the book, preparing the uploadable version of the spreadsheet. Then, no earlier than next Monday, comes the actual preparation of the book, followed by uploading the shared dataset.[Impatient? See the end of this message.]

Where things stand

The final dataset includes just over 20,000 journals (20,109), with another 1,099 exclusions in a separate tab (not used in calculations), That’s the first time over 20,000 (albeit just barely), and the number of “clean” journals (no special codes and articles as recently as 2023) is up slightly, to 18,779 from 18,460. Unfortunately, article count is essentially flat: 1,441,979, down from 1,442,328 in last year’s report. In addition to the 18,779 “a” there are 499 “bi” (articles within the last five years but not since 2022), 370 “xm” (malware–up from 313 last year) and, sigh, 461 “xx” (unreachable/unworkable), up from 139 last year.

That “xx” figure notably includes a brand-new “grayed” category: 54 journals with incompetent html to ask for cookie options, but in a manner that freezes the screen with an overall gray cast AFTER offering cookie options–so that the screen is entirely frozen. (I intuit this from a split-second offering before the graying–and from the one journal that fixed it between the first and second passes: the cookie options appear on top of the gray, such that you can make a choice and recover the active screen. Yes, this is true on Edge, Chrome and Firefox–and even after clearing all cookies in Edge.) Many Iranian journals–and others–have the sensible approach of putting cookie opti0ns at the foot of the screen, leaving the rest of the frame visible and active. It seems pretty clear that these too-clever “let’s force them to make cookie choices by freezing everything else” cases are mostly copied from one source, and involve lots of different institutions. Do these people ever try to use their journals?

As always, all of these numbers are subject to change during the cleanup/massaging process.

Exclusions are up considerably, to 1,099, with the jump mostly being to 505 “xm2” (repeated malware, up from 335). Other categories are up only slightly: 253 expired (no articles since 2020; up from 230, and 155 of the 253 show evidence of being continued by another journal); 254 removed from DOAJ (essentially flat, from 251 last year); 70 xx2 (repeatedly unavailable/unworkable), up from 51 last year; and 17 “xn” (not an OA journal), up from 12 last year–including seven journals requiring a login and at least one that has apparently dropped its OA status entirely.

So, on to the massaging and writing. Oh, if you’re impatient: The final dataset won’t be shared until it’s, you know, final: when the book is ready (but before the Diamond OA book is written). It’s possible that a tentative dataset with a name something like tentgoa10.xlsx might perhaps possibly be available on my website (waltcrawford.name) earlier, perhaps as early as May 12 or 13. Use at your own risk: if it’s there, it’s tentative.

So far, I’ve checked 1,775 (after dealing with ones that don’t require rechecking–specifically journals that were defective both last year and this, the “xm2” and “xx2” problems). There are 685 left to do, so, barring disasters, I’ll finish the second pass this week and start massaging the data.

The good news: quite a few dsn, certificate, and similar problems have been fixed. The bad news: I don’t believe ANY cases of malware have been fixed–and the “grayed” problems (44 rechecked, and 12 more to go) haven’t been either. [Revised 5/5: In fact, a few cases of malware have been fixed. Not nearly enough, but some publishers/universities have cleaned up their acts.]

I’m pretty sure I know what’s causing the grayed problem: sites being too clever about insisting on cookie approval/disapproval or the like, and doing things in the wrong order, such that the entire frame is protected from any input (and usually unreadable). Net effect: the journal’s unworkable. Never happened until this year, and I suspect some journals found a “neat solution” and passed the code along… Oh, and yes, I have tried all three of the browsers I have access to (Firefox, Chrome, and Edge), and cleared the cookies from Chrome and Edge. No help. This is just plain bad coding.

Is “The Big Eleven” Worthwhile?

For the past three years, part of Chapter 6 has been discussions of The Big Eleven–eleven publishers and publishing groups that absolutely dominate OA fees–and The Long Tail (everybody else). That runs to eleven pages (purely coincidental) in the most recent edition.

I’m thinking of dropping that discussion. There’s significant labor in finding out who belongs in the “big” group (especially given changes in ownership), and I’m not convinced that the results justify the effort. Any reader could, of course, download the spreadsheet and prepare their own analysis.

Unless there’s convincing evidence that this work is worthwhile (it’s obviously not driving book sales and I don’t recall any feedback), I’ll drop it and save the eleven pages and the day or so of labor. Any feedback on this needs to reach me within the next week (say by May 11): waltcrawford@gmail.com

A check against last year’s count and some clarification has resulted in recognizing that 1,055 journals don’t need to be rechecked, leaving 2,460 that will be rechecked starting in a few days.

The 1,055 include 251 that ceased no later than 2020; 183 removed by DOAJ from its database; nine that either aren’t journals at all or can’t be counted as such; and the two “repeat offenders” categories: 79 that were unreachable or unworkable  both last year and this, and 533 that showed signs of malware or were otherwise dangerous both last year and this.

Of the remaining 2,460, many are either journals that hadn’t published any year-2024 issues when first checked or (about 550) had published some issues but in a pattern that suggested more 2024 issues were on the way. Others are journals with what are probably transitory problems–for example, 26 marked as being in maintenance, around 50 with apparently-temporary error codes.

There are 442 cases of malicious behavior (expanded to include not only malware but certificate problems that can expose a visitor to  malware) and 830 journals that were unreachable or unworkable (including most of the transitory problems. And then…well, there are close to five dozen “really bad HTML” cases: mostly journals where the home page is obscured by dark gray and won’t accept any commands, at least when tested in Chrome (which I use for the scan because of the fast/easy autotranslate). I feel it would be perfectly appropriate to say these are incompetent and shouldn’t be counted, but I’ll give them another try (using Edge or Firefox).

Oh, there are also 119 404 errors at the journal level, 14 where the journal blocked me from reaching it, more than 250 timeouts, and other miscellaneous problems. They’ll all be retested.

So how will it turn out? We shall see…in two or three weeks.

All 21,208 journals have been checked–but, as with all numbers here, that’s just at the end of the first pass: the second pass will assuredly reduce the number of journals and change other numbers.

Total articles for 2024: 1,432.325; for 2023, 1,405,501. Of the 21,208 journals, 7,360 have fees and 13,846 don’t (diamond OA). 1,893 are newly-added and 19,315 are ongoing. 3,220 may need rechecking, including most of these special cases (except bi and xd):

Inactive (bi): 481
Expired (xd): 251
Malware (xm): 9947
Not OA (xn): 197–mostly journals dropped from DOAJ.
Unavailable/unworkable (xx): 942.

Next steps (after a few days off): Determine which journals actually need rechecking; do the rechecking (probably 2-3 weeks); massage the data; prepare the main manuscript; post/publish the main book and dataset; and prepare the Diamond OA by Country book.

If it seems as though total gold OA (excluding “hybrid”) is pretty much flat, I believe that’s right.

I may or may not post daily (Mastodon) and weekly (here) updates on Pass Two.

So far, 19,600 journals have been checked. Total articles for 2024: 1,326,141; for 2023, 1,296,442. Of the 19,600 journals, 6,590 have fees and 13,010 don’t (diamond OA). 1,777 are newly-added and 17,823 are ongoing. 3,101 may need rechecking, including most of these special cases (except bi and xd):

Inactive (bi): 450
Expired (xd): 227
Malware (xm): 921
Not OA (xn): 141–mostly journals dropped from DOAJ.
Unavailable/unworkable (xx): 902.

So far, 18,200 journals have been checked (3,008 left to go). Total articles for 2024: 1,292,119; for 2023, 1,260,786. Of the 18,200 journals, 6,256 have fees and 11,944 don’t (diamond OA). 1,657 are newly-added and 16,433 are ongoing. 2,880 will be rechecked, including most of these special cases (except bi and xd):

Inactive (bi): 412
Expired (xd): 219
Malware (xm): 837–note the too-rapid increase here
Not OA (xn): 122–mostly journals dropped from DOAJ.
Unavailable/unworkable (xx): 851.

Guesstimate: the first pass will be done right around the end of April–but the recheck is slower.

So far, 16,825 journals have been checked (4,383 left to go). Total articles for 2024: 1,256,246; for 2023, 1,222,809. Of the 16,825 journals, 5,832 have fees and 10,993 don’t (diamond OA). 1,582 are newly-added and 15,243 are ongoing. 2,448 will be rechecked (the extra 2 last week was a typo), including most of these special cases (except bi and xd):

Inactive (bi): 388
Expired (xd): 203
Malware (xm): 613
Not OA (xn): 105–mostly journals dropped from DOAJ.
Unavailable/unworkable (xx): 738.

That xx number includes a surprising number of “grayed out” journals–a phenomenon new this year, where the journal home page is visible but covered with a light or dark gray overlay, and wholly nonfunctional. This is just bad HTML; using a different browser appears to clear it up, but it shouldn’t be happening at all. (There’s also one publisher–a commercial one–where, in Chrome but not in Firefox, journals appear without an overlay but are wholly unresponsive. For shame. Again, the recheck pass should clear these up, but they shouldn’t need clearing up.)

Guesstimate: the first pass will be done right around the end of April, give or take a week–but the recheck is slower.

So far, 15,500 journals have been checked. Total articles for 2024: 1,221,491; for 2023, 1,184,642. Of the 15,500 journals, 5,806 have fees and 9,694 don’t (diamond OA). 1,536 are newly-added and 13,964 are ongoing. 2,2217 will be rechecked, including most of these special cases (except bi and xd):

Inactive (bi): 360
Expired (xd): 189
Malware (xm): 572 [last week was a typo]
Not OA (xn): 91–mostly journals dropped from DOAJ.
Unavailable/unworkable (xx): 652