Academic Karma: a case study in how not to use open data

View the article’s original source
Author: nsaunders

Update: in response to my feedback, auto-generated profiles without accounts are no longer displayed at Academic Karma. Well done and thanks to them for the rapid response.

A news story in Nature last year caused considerable mirth and consternation in my social networks by claiming that ResearchGate, a “Facebook for scientists”, is widely-used and visited by scientists. Since this is true of nobody that we know, we can only assume that there is a whole “other” sub-network of scientists defined by both usage of ResearchGate and willingness to take Nature surveys seriously.

You might be forgiven, however, for assuming that I have a profile at ResearchGate because here it is. Except: it is not. That page was generated automatically by ResearchGate, using what they could glean about me from bits of public data on the Web. Since they have only discovered about one-third of my professional publications, it’s a gross misrepresentation of my achievements and activity. I could claim the profile, log in and improve the data, but I don’t want to expose myself and everyone I know to marketing spam until the end of time.

One issue with providing open data about yourself online is that you can’t predict how it might be used. Which brings me to Academic Karma.

Academic Karma came to my attention on Twitter via Chris Gunter.

Tipped off to an AcademicKarma profile I did not set up for myself. Looks like years of reviewing/editing mean zilch. http://t.co/45DrnoDvlZ

— Chris Gunter (@girlscientist) February 11, 2015

To which they replied:

@girlscientist Everyone with an @ORCID has an Academic Karma profile, think of it as an Academic directory.

— Academic Karma (@AcademicKarma) February 12, 2015

Everyone with an ORCID? I have one of those. Sure enough, appending my ORCID ID to their URL reveals that I have a profile.

You’ll note that my profile states “no review information shared” and that the data are sourced from ORCID. These are recent changes, brought about by one of my less polite tweets.

and if I don’t want one? More ResearchGate-style bullshit. MT @AcademicKarma Everyone with an ORCID has an Academic Karma profile

— Neil Saunders (@neilfws) February 18, 2015

Karma, apparently, according to someone

Karma, apparently, according to someone

Previously, profiles looked like the one shown in the image, right. In my case, as I have not included any reviewing or editorial activity in my ORCID profile, this resulted in a large, prominent “NA” for so-called “karma earnt”. This gave the misleading impression that I am a bad “corporate citizen”.

To their credit, the people behind Academic Karma made changes to profile views very quickly, based on my feedback. That said, they seemed genuinely bemused by my criticism at times.

@neilfws @AcademicKarma Wow! Genuinely trying to improve peer review here Neil. Value your feedback though on what we could do differently.

— Lachlan Coin (@lachlancoin) February 18, 2015

@neilfws @AcademicKarma @ORCID_Org designed for data re-use, what are we misrepresenting?

— Lachlan Coin (@lachlancoin) February 18, 2015

So let me try to spell it out as best I can.

  1. I object to the automated generation of public profiles, without my knowledge or consent, which could be construed as having been generated by me
  2. I especially object when those profiles convey an impression of my character, such as “someone who publishes but does not review”, based on incomplete and misleading data

I’m sure that the Academic Karma team mean well and believe that what they’re doing can improve the research process. However, it seems to me that this is a classic case of enthusiasm for technological solutions without due consideration of the human and social aspects.

Filed under: networking Tagged: academic karma, orcid, researchgate, social networking