使用R进行pubmed爬虫

工具

RISmed(library)
中文社区居然没有他的介绍让我匪夷所思,于是乎我做一个简易的介绍吧.

安装

install.packages('RISmed')

使用

library(RISmed)

1. 首先我们要定义一个搜索的tag:

search_tags = 'sleep'

然后就可以十分暴力的调用函数了:

• EUtilsSummary

Usage
EUtilsSummary(query,type="esearch",db="pubmed",url=NULL,encoding="unknown",...)
Arguments
query	
string query as given in NCBI search box

type	
string indicating type of NCBI Eutility

db	
string indicating the NCBI database used in query

url	
string of url to bypass query construction

encoding	
encoding to be assumed for input strings, see readLines

...	
additional limits added to query

tips

Windows的同学注意encoding

调用EUtilsSummary

> search_query = EUtilsSummary(search_tags,retmax=100)
> summary(search_query)
Query:
"sleep"[MeSH Terms] OR "sleep"[All Fields] 

Result count:  205279

query为search build:

“sleep”[MeSH Terms] OR “sleep”[All Fields]

retmax为文献爬取最大数目

后处理

pubmed_data = data.frame('Title' = ArticleTitle(records),'Abstract'=AbstractText(records))
pubmed_data$Abstract <- as.character(pubmed_data$Abstract)
pubmed_data$Abstract <- gsub(",", " ", pubmed_data$Abstract, fixed = TRUE)

> str(pubmed_data)
'data.frame':	99 obs. of  2 variables:
 $ Title   : Factor w/ 97 levels "A common hub for sleep and motor control in the substantia nigra.",..: 41 56 49 45 4 35 94 2 83 51 ...
 $ Abstract: chr  "BACKGROUND: Sleep apnoea is prevalent in dialysis patients. Previous studies identified excessive inflammation "| __truncated__ "AIMS: A large proportion of patients at memory disorders clinics are classified as having subjective cognitive "| __truncated__ "BACKGROUND: Despite the abundance of data documenting the consequences of poor sleep quality on blood pressure "| __truncated__ "Sleep difficulties are commonly reported by patients with epilepsy and can have a detrimental impact on overall"| __truncated__ ...

well done