A Short Survey for Google Scholar

A Short Survey for Google Scholar

1. An Overview
Google Scholar is a freely accessible web search engine that indexes the full text of scholarly literature across an array of publishing formats and disciplines. It allows users to search for digital or physical copies of articles, whether online or in libraries. Google scholar provides user-friendly and one-click search tools as well as advanced search services which can automatically narrow search results to a specific journal or article. Released in 2004, Google Scholar occupies a large market share in today's the academic search field. It is the most mainstream tool for knowledge discovery and academic informational retrieval. However, there are also criticisms which states that Google Scholar's database is still limited, unevenly covered and partially out-of-date.

2. Reviews
we have five categories of criterion to evaluate an academic search engine, respectively based on its performance on knowledge discovery, socialization, user experience, technical quality of services and extensibility. We tried to rate a score in each aspect, and finally, give an overall judgment to on value of the services.

2.1 Knowledge Discovery (Score: 24/30)
- Keyword Mining (Score: 9/10)

Google Scholar's database cover is broader than some popular academic search engines such as Microsoft Academic Research, and it is not restricted in the field of computer science. For two toy keywords, "Map Reduce" and "Plasmid DNA", Google Scholar got 2,710,000 and 1,550,000 results, obviously beats Microsoft's 1,485 and 4,512. The main reason is that Google Scholar crawl its data from many sources of information, not only academic conference and journals, but also from technical reports, preprints, theses, books, and other documents, including selected Web pages that are deemed to be scholarly. However, the cover advantage of Google scholar is not unique. One of its competitors, Scirus academic search, has similar performance on the number of retrieved results. Google Scholar ranks results with a combined ranking algorithm in a "way researchers do, weighing the full text of each article, the author, the publication in which the article appears, and how often the piece has been cited in other scholarly literature". Research has shown that Google Scholar puts high weight especially on citation counts and words included in a document's title. As a consequence the first search results are often highly cited articles. Different from Scirus, Google Scholar does not tend to rank the newest documents onto the top. So it may be harder to find what's recently going on in the academic field by using Google Scholar.

- Topic/Category Mining (Score: 2/5)

Google Scholar does not maintain topics or categories' list of its database. Actually, it receives fierce criticisms on the intransparancy of its database organization. Although the keyword search can complement the absence of topics/categories in some degree, it indeed bring inconvenience to users who want to find the papers in a specific topic.

- Knowledge Network(Score: 6/8)

Google Scholar organizes its knowledge network in two ways. First, its maintain links between the inter-reference papers, so that the user can easily find the group of papers which cite a previous papers, which is useful when the user is interested in the impact of a specific work. Second, it provides a service to list the related works to a paper. The evaluation metric of "relevance" is not clear, but it is indeed helpful if the user wants to collect all literatures in a narrow field. However, it is hard for the user to see the overall structure of the knowledge network.

- Acessibility(Score: 7/7)

Google Scholar do an excellent job in helping the user accessing academic materials given the search results. The user can read the abstract or a fregment of the paper directly on the search result page. It also help users find the pdf/ps files of electronicly available papers all over the internet.

2.2 Socialization (Score: 5/20)
- Academic Social Network Mining (Score: 0/5)

Google Scholar is not aiming at helping users discover the social relationship between scholars. All connections are maintained between papers, not authors. The user cannot learn which scholar has a closer relationship to a specific professor only depending on search.

- Author Information (Score: 2/5)

Yes, It is possible to learn an author by searching his/her name. The user can access the paper he/she published and their citations. However, no more information is available. There is also no staticitical processing to the raw data. Also, the author names are easily confused.

- Venue Information (Score: 3/5)

Since Google Scholar covers most international journals/conferences, so it is possible to learn some knowledge about this venues by the search results. It is also possible to restrict the search of some keyword on specific publication source. The drawback of Google Scholar is similar to its author information servies, considering that there is no more statistical information is provided to help the user judge whether a venue is suitable for him/her to publish the academic result.

- User Participation (Score: 0/5)

No, there is no way to participate in enriching the Google Scholar database or its ranking results.

2.3 User Experience (Score: 21/25)
- Page Layout (Score: 7/10)

This classic google search result page layout is concise and efficient. The information and links in the page are almost all necesseary and useful. One thing not good enough is that it uses a single line to represents the authors, publication source, time/date and stored website, which is quite crowed. Usually it is hard to read the full name of the conference/journal from this line. The name of authors are all in the abbreviated form, which causes confusion easily.

- Personalization (Score: 9/10)

Google provides a group of interesting and useful personaliation services to help the user work more efficiently. For example, the user can set his/her own search preference, including language, library links, windows layout and the bibliography manager. It also provides the email alert servies which periodically email the user newly published papers that match his/her search criteria. This is great in keeping up with the most recent research trend. The only bad point is that the user is not able to set their preference on the paper ranking algorithm, e.g. the relative importance of citation number, relevance to query, and document freshness.

- Responding Speed (Score: 5/5)

As a search query is submitted, the search result page can be returned immediately (<0.5s). The overall network condition is also good.

2.4 Technical Quality of Service (Score: 10/15)

- Update Rate (Score: 9/10)

Google Scholar normally add new papers several times a week. This update rate is quite satisfying. However, updates to existing records take 3-6 months to a year or longer, because in order to update our records, they need to first recrawl them from the source website.

- APIs (Score: 1/5)

Until now, Google Scholar is not yet available to the Google AJAX API. But there are a lot of people calling for it.

2.3 Extensibility (Score: 6/10)
- Potential to Be Improved (Score: 4/5)

Google Scholar is still a traditional academic search engine. However, it is indeed the most popular and the most useful one. Based on the current statues of Google Scholar, there is still a large space to improve, such as adding social network information, providing mobile services, and connecting with other google services like gmail and google books. If its API is released, I believe there are a huge amount of people who are excited to build their programs on Google Scholar.

- Uniqueness of the Services (Score: 2/5)

Google Scholar is unique mainly because it has the Google brand mark. However, as for the services, Google Scholar did not distinguish itself clearly with its competitors, maybe because that it is already the leader of this market.

3. Summary

In overall, Google Scholar got 66 out of 100 in the evaluation, which is a remarkable achievement.

 

---

Author: Yuchen Zhang