Finding the needle in a haystack

Share this on social media:

Paul Kudlow says the publishing industry is facing a monumental challenge: how do we ensure research is read?

Up to half of the 8,000-plus articles published daily are only read by their authors and journal editors. An effective, scalable method to maximise the reach and discoverability of published content is needed.

Social media channels had promising potential in this domain; however, despite their proven efficacy for marketing products and services, these channels have not been shown to be effective in driving usage of academic content.

We wanted to explore the role cross-publisher recommendations play in encouraging knowledge discovery. Last year, we showed that cross-publisher recommendations increase article page-views of scholarly content by 49 per cent in a study published in Learned Publishing. We have now conducted a follow-up, published in Scientometrics, that shows cross-publisher article recommendations also attract actively engaged readers to scholarly content.

The role of recommendations

A study published by Simon Inger Consulting in 2016 showed that 'Related Articles functionality is the only area that has become more useful [for content discovery] since 2012'.

David McCandlish, assistant professor at Cold Spring Harbor Laboratory, highlighted the value of interdisciplinary, cross-publisher recommendations in a follow-up to a recent SSP webinar. Researchers like David need to keep up-to-date with advances in their own field while scanning for applicable advances in other disciplines.

Cross-publisher recommendation engines push research to readers who don’t know exactly what they are looking for, driving serendipity in the discovery process. Ideally, this serendipity presents research to readers that will apply and extend that work as they continue advancing their field.

While other platforms (e.g. ScienceDirect) only provide recommendations to content they host, TrendMD provides recommendations both within and across journals and publishers. With more than 22 million articles in the TrendMD network, the breadth and variety of content that can be recommended is endless. Relevant recommendations are provided within the researcher’s workflow by incorporating key-phrase matching with collaborative filtering and personalisation.

Incorporating these additional aspects in recommendations provides a level of serendipitous discovery that is lacking when recommendations are generated by key-words alone.

The study

Four hundred Journal of Medical Internet Research (JMIR) articles were randomised to either the TrendMD arm (n = 200) or the control arm (n = 200). Articles in the TrendMD arm were recommended in the TrendMD recommendation widget throughout the TrendMD network (sponsored page-views) and on JMIR (non-sponsored page-views). Articles in the control arm were not included in any TrendMD recommendations. After four weeks, we compared mean Mendeley saves and other engagement metrics for articles in the two groups.

We selected Mendeley to be our primary source of data because the data is open and easily accessible, Mendeley saves have been correlated to future citations, and little is known about the effect of distribution strategies on Mendeley saves. A Mendeley save is counted when an article has been saved to a Mendeley user library account.

The results

Overall, articles randomised to TrendMD had significantly higher Mendeley save rates, total page-views, and pages per session than articles in the control group as well as significantly lower bounce rates. This suggests that TrendMD recommendations were effective in driving engaged readers to promoted content.

TrendMD-promoted articles had a 77 per cent increase in article saves on Mendeley relative to control with an average of 2.7 more Mendeley saves than the control group. Additionally, page-views driven by TrendMD were significantly correlated with this increase (Spearman’s rho = 0.6). While both sponsored and non-sponsored page-views had a high correlation with Mendeley saves, sponsored page-views had a larger effect. Taken together this suggests that cross-publisher distribution channels lead to an increase in page-views which are effective in increasing Mendeley saves.

These results are significant because ours is the first rigorous study looking at the effect of cross-publisher distribution channels on Mendeley saves. This study also highlights the benefit of providing recommendations from a diversity of publishers within the researcher workflow, as shown by engagement metrics. Though further study and replication are needed, these data suggest that cross-publisher article recommendations via TrendMD may enhance citations of scholarly articles.

Paul Kudlow is co-founder and CEO of TrendMD

Scientometrics article:
Learned Publishing article:
JMIR social media study:
Circulation social study:
Nature use of social media study:
SSP Webinar follow-up:
Simon Inger Consulting study:
Use of published articles: