Crowdsourcing the Analysis and Impact of Scholarly Tweets

“Twitter is one of the fastest tools to discover newly published scholarly papers”, Martin Fenner wrote in one of his earlier posts. Now Fenner and Euan Adie finished the first phase of an interesting new experiment, the CrowdoMeter project.

In the last 2 months, they used crowdsourcing to analyze the semantic content of almost 500 tweets linking to scholarly papers (953 classifications by 105 users for 467 tweets). Their preliminary results show: 

  • 3 predominant subject areas: Medicine and Health, Life Sciences, and Social Sciences and Economics.
  • Most tweets (88%) discussed the papers, 10% were in agreement, 3% disagreed.
  • Most papers are not tweeted by their authors or publishers

In his recent guest post on Impact of Social Sciences, Fenner makes the argument that “social media, and Twitter in particular, provide almost instant, relevant recommendations as opposed to traditional citations.

A few years from now the ‘personalized journal’ will have replaced the traditional journal as the primary means to discover new scholarly papers with impact to our work.

What is still missing are better tools that integrate social media with scholarly content, in particular personalized recommendations based on the content you are interested in (your Mendeley or CiteULike library are a good approximation) and the people you follow on Twitter and other social media.

Fenner’s view is also based on Gunther Eysenbach’s study from 2011 that showed “highly tweeted papers were more likely to become highly cited (but the numbers were to small for any firm conclusions; 12 out of 286 papers were highly tweeted)”.

Fenner and Adie are using to track the scholarly impact of your research. Other tools – some of which we wrote about – include  ReaderMeterTotal ImpactPLoS Article-Level Metrics, and ScienceCard.

Have you heard of the Crowdsortium?

It’s a group of currently more than 75 crowdsourcing industry practitioners that share “best practices, education, data collection and public dialog.” Interesting for us, as problems become more complex, the Consortium could be helpful in defining the right crowdsourcing model for future crowdsourcing needs at UCSF and CTSI.

The Crowdsortium aims to provide each of these constituents with the knowledge, data and best practices to get the most out of participating in crowdsourcing.

The Crowdsortium recognizes that the crowdsourcing ecosystem is comprised of five participants: funders, practitioners, customers, the crowd and researchers.

As the problems crowdsourcing address become more complex, so do the problems that the crowdsourcing ecosystem face.

For example:                                                                         THE CROWDSOURCING ECOSYSTEM

• What crowdsourcing model should we use?
• How do we handle intellectual property rights?
• How do we logistically manage international participants?
• How do we fairly reward participants for their work?
• What are the benefits of monetary versus non-monetary rewards?
• How can game mechanics influence crowd dynamics?
• Should the crowd be anonymous or individually identified with their real names?

Anyone may participate in the public forums, feeds and email lists related to this website, members, however, enjoy additional benefits. Membership is currently available for funders, practitioners, customers and researchers at no cost. More at

Using Research Networking Effectively in Academia: UCSF-CTSI Team Presents On National AMIA Panel

Three of us from the Virtual Home team at CTSI went to this year’s AMIA (American Medical Informatics Assoc) meeting in DC and presented on a panel with Griffin Weber of Harvard University. The panel was called “Four Steps to Using Research Networking Effectively at Your Institution”

Griffin spoke on cutting edge features of research networking tools, such as linked open data and social network analysis.

Eric Meeks of UCSF spoke on standard APIs, such as OpenSocial, to leverage a community of developers, I spoke about incentivize usage and understand your audience, and to round it out, Brian Turner spoke about using data, tools and strangers to improve user interfaces.

The panel presentation was a 90 minute break out session and we were happy to have a good turnout and an engaged audience. I think that the work that UCSF has put into the ‘social engineering’ of the tool has really paid off. Our usage and engagement numbers are on the rise and comparatively speaking, Griffin mentioned that our traffic is about 5-times that of what Harvard Profiles is currently getting.

In addition, Eric also had a poster session at the meeting!

The UCSF presentations will be up on Slideshare, available on the CTSI channel and via our individual UCSF profiles:

“Using Prizes to Spur Open Innovation”: The National Institutes of Health (NIH) Explore Potential Approaches

This week’s NIH conference “Crowdsourcing: The Art and Science of Open Innovation” could be a hint that the research agency is seriously considering new ways to take advantage of the “processing power of lots of willing brains”.

ScienceInsider published a summary report that states:

NIH Director Francis Collins would soon sign papers that would ensure NIH is compliant with the America COMPETES Act, which gives federal agencies the authority to offer cash incentives for researchers to tackle high-risk, high-reward research questions that have eluded more traditional funding platforms, such as grants and sponsored research.

The America COMPETES Act was first passed in 2007 and was reauthorized in December. Under its authority, federal agencies outline a problem they’d like solved on, then open the competition to individuals or teams, evaluate the results, and award a money prize to whoever turns in the best solution.

Crowdsourcing for idea generation

Can big institutions get better ideas by including more people? High-profile crowdsourced ideation projects in our communities include:

  • Harvard Catalyst ran a contest last year, inviting Harvard and external community members to come up with their own answers to the question “what do we not know about Type 1 Diabetes?” They got over 190 responses. A panel of experts culled the list down to a dozen top award-winning ideas, and seven new projects have been funded to investigate these questions. Top ideas came from faculty, students, staff, and a patient.
  • The White House developed the SAVE Award, a national contest for federal government employees to suggest ways to make government processes cheaper and easier. The 2009 contest drew 38,000 entries, while the 2010 contest drew 18,000 entries, plus 160,000 votes. Winning ideas include not throwing away bulk medication at the VA, online scheduling at Social Security offices, and ending the mailing of print copies of the Federal Register. Check out the huge range of submitted ideas.
  • UCSF is running Bright Ideas, a campus-wide suggestion box. The campus community can share and vote on ideas. Previously-implemented Bright Ideas included a system to share unused office furniture and supplies, and the installation of audio signals for the visually impaired at Parnassus crosswalks.

(Image by Faith Grober)

Compelling Video Describes New Visualization Tool “Many Eyes”

It can be challenging to create animated video that conveys a complex message. Here is a great example that shows it’s doable – mind you, without a single spoken word.

A 60 second social story about developing and refining ideas, gaining insight and sharing through community; all based on the premise that many sets of eyes are better than one!

Take a look and let me know what you think. – Btw, the visualization tool “Many Eyes“, developed by IBM, is worth a look as well.