Notes from Science Online – Real time statistics and new impact metrics in science

Virginia Barbour – PLoS

How can we assess the impact of any piece of research?

  • Usage
  • Media Coverage
  • Blog Coverage
  • Discussion Thread Activity
  • Social Bookmarking Activity
  • Expert Ratings
  • Effect on Public Policy
  • Where the Research was done
  • Who is citing it
  • Who is reading it

In general: articles have broken free of journals and can be assessed on their own merits. Some examples of paper statistics in New England Journal of Medicine and one other.

PLOs Steps for new article metrics: phase 1: acquire raw data which is open and non-proprietary…and can be independently verified

Phase Two…linkking,,.showing a PLoS paper with links to blog entries and papers citing this article. Also attaching Article usage in terms of viewing statistics – in the future add in more data sources.

Richard Grant – F1000

Showing real-time solo09 twitter feed.

We all want to make an impact – how do we do this? Who cares about impact?

  • Science Publishers
  • People who want to know where to read research
  • Funding agencies

What can and can’t metrics do?

For this meeting, for example, twitter is an immediate real time metric. Later then blogging and re-blogging. But no quality metric. In science we use the impact factor…it’s slow and takes time before it reflects any real interest. Can be gamed and does not tell you how good research is.

Now plugging f1000 – http://f1000.com

Victor Henning – Mendeley

Usage metrics for individual researchers – analogy to LastFM. Why not build a similar system to Last FM for science.

1. Build a system to measure article pervasiveness…how many people have the article in their library

2. Track article reading time in PDF viewers as a measure of interest

3. Track user tags and ratings

BUT preserve privacy – scientists may not wish to have information about their paper reading and interest behaviour displayed.

Now the system is Mendeley…cross platform…aggregates reading material, tag it, read it….etc….Ultimate goal is to aggregate these statistics for analysis….data segmantation with increasing pervasiveness..what are profs reading vs undergraduates, usage by geography etc…

Reblog this post [with Zemanta]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: