Solr or Elasticsearch–That Is the Question

blog source link:  http://www.datanami.com/2015/01/22/solr-elasticsearch-question/

January 22, 2015

Solr or Elasticsearch–That Is the Question

Otis Gospodnetić

That is the common question I hear: Which one is better, Solr or Elasticsearch? Which one is faster?  Which one scales better?  Which one can do X, and Y, and Z?  Which one is easier to manage?  Which one should we use?  Which one do you recommend? These are all great questions, though not always with clear and definite, universally applicable answers.

So which one do we recommend you use? How do you choose in the end? Well, let me share how I see Apache Solr and Elasticsearch past, present, and future, and let’s do a bit of comparing and contrasting, and hopefully help you make the right choice for your particular needs.

Early Days: Youth Vs. Experience

Apache Solr is a mature project with a large and active development and user community behind it, as well as the Apache brand.  First released to open-source in 2006, Solr has long dominated the search engine space and was the go-to engine for anyone needing search functionality.  Its maturity translates to rich functionality beyond vanilla text indexing and searching; such as faceting, grouping (aka field collapsing), powerful filtering, pluggable document processing, pluggable search chain components, language detection, etc.

Solr dominated the search scene for several years. Then, around 2010, Elasticsearch appeared as another option on the market. Back then, it was nowhere near as stable as Solr, did not have Solr’s feature depth, did not have the mindshare, brand, and so on.

But it had a few other things going for it: Elasticsearch was young and built on more modern principles, aimed at more modern use cases, and was built to make handling of large indices and high query rates easier. Moreover, because it was so young and without a community to work with, it had the freedom to move forward in leaps and bounds, without requiring any sort of consensus or cooperation with others (users or developers), backwards compatibility, or anything else that more mature software typically has to handle.

As such it exposed certain highly sought-after functionality (e.g., Near Real-Time Search) before Solr did.  Technically speaking, the ability to have NRT Search really came from Lucene, the underlying search library to both Solr and Elasticsearch use. The irony is that because Elasticsearch exposed NRT Search first, people associated NRT Search with Elasticsearch, even though Solr and Lucene are both part of the same Apache project and, as such, one would expect Solr to have such highly demanded functionality first.

Elasticsearch, being more modern, appealed to several groups of people and organizations:

• Those who didn’t yet have a search engine and hadn’t invested a lot of time, money, and energy in its adoption, integration, etc.
• Those who had to deal with large volumes of data and needed to more easily shard and replicate data (search indices) and shrink or grow their search cluster.

Of course, let’s admit it, there will always be those who like jumping on new shiny objects, too.

Evening the Search Playing Field

Fast forward to 2015.  Elasticsearch is no longer new, but it’s still shiny. It closed the feature gap with Solr and, in some cases, surpassed it. It certainly has more buzz around it. At this point, both projects are very mature. Both have lots of features. Both are stable. I have to say though, that I do see more Elasticsearch clusters with issues, but I think that is primarily because of a few reasons:

• Elasticsearch, traditionally being easier to get started with, made it possible for anyone to start using it out of the box, without too much understanding of how things work. That’s great to get started, but dangerous when data/cluster grows.
• Elasticsearch, lending itself to easier scaling, attracts use cases demanding larger clusters with more data and more nodes.
• Elasticsearch is more dynamic–data can easily move around the cluster as its nodes come and go, and this can impact stability and performance of the cluster.
• While Solr has traditionally been more geared toward text search, Elasticsearch is aiming to handle analytical types of queries, too, and such queries come at a price.

Although this may sound scary, let me put it this way — Elasticsearch exposes a ton of control knobs one can play with to control the beast. Of course, the key bit is that one has to be aware of all possible knobs, know what they do, and make use of that. For example, despite what you just read about Elasticsearch, we rely on it in our organization for several different products, even though we know Solr just as well as we know Elasticsearch.

Solr: Not Totally Eclipsed

What about Solr?  Solr hasn’t exactly stood still. The appearance of Elasticsearch was actually great for Solr and its community of developers and users. Despite being almost 10 years old, Solr development is going faster than ever. It, too, has a friendly API now. It, too, has the ability to more easily grow and shrink clusters, create indices more dynamically, shard them on the fly, route documents and queries, etc., etc. Note: when people refer to SolrCloud they specifically mean this form of very distributed, Elasticsearch-like Solr deployment.

I recently attended a Lucene/Solr Revolution conference in Washington D.C. and was pleasantly surprised by what I saw: A strong community, healthy project, lots of big name companies not only using Solr, but investing in it through adoption, contribution through development/engineering time, etc.

If you follow just the news you’d be led to believe Solr is dead and everyone is just flocking to Elasticsearch. That is actually not the case. Elasticsearch being newer, is naturally more interesting to write about. Solr was news 5-plus years ago. And of course there were some people going from Solr to Elasticsearch when Elasticsearch appeared–in the beginning there were simply no Elasticsearch users.

So which is better?  Which one should you use?  Where do Solr and Elasticsearch differ?  What does the future hold?

Here are some other things you should keep in mind:

• Both are released under the Apache Software License
• Solr is truly open-source–community over code. Anyone can contribute to Solr and new Solr developers (aka committers) are elected based on merit. Elasticsearch is technically open-source, but less so in spirit. Anyone can see the source, anyone can change it and offer a contribution, but only employees of Elasticsearch can actually make changes to Elasticsearch.
• Solr contributors and committers come from a number of different organizations, while Elasticsearch committers are from a single company.
• A number of Hadoop distributors have chosen Solr over Elasticsearch as their horses in the search race (e.g. Cloudera, Hortonworks, MapR being among them) even though they’ve also partnered with Elasticsearch.
• Both Solr and Elasticsearch have lively user and developer communities and are rapidly being developed.
• If you need to add certain missing functionality to either Solr or Elasticsearch, you may have more luck with Solr. True, there are ancient Solr JIRA issues that are still open, but at least they are still open and not closed. In Solr world the community has a bit more say even though at the end of the day it’s one of the Solr developers who has to accept and handle the contribution.
• Both have good commercial support (consulting, production support, integration, etc.)
• Both have good operational tools around it, although Elasticsearch has, because of its easier-to-work-with API, attracted the DevOps crowd a lot more, thus enabling a livelier ecosystem of tools around it.
• Elasticsearch dominates the open-source log management use case–lots of organizations index their logs in Elasticsearch to make them searchable. While Solr can now be used for this, too, it just missed the mindshare boat on this one.
• Solr is still much more text-search-oriented. On the other hand, Elasticsearch is often for filtering and grouping – the analytical query workload – and not necessarily text search.  Elasticsearch developers are putting a lot of effort into making such queries more efficient (lowering of the memory footprint and CPU usage) at both Lucene and Elasticsearch level.  As such, at this point in time, Elasticsearch is a better choice for applications that need to do not just text search, but also complex search-time aggregations.
• Elasticsearch is a bit easier to get started – a single download and and single command to get everything started. Solr has traditionally required a bit more work and knowledge, but Solr has recently made great strides to eliminate this and now just has to work on changing its reputation.
• Performance-wise, they are roughly the same. I say “roughly”, because nobody has ever done comprehensive and non-biased benchmarks.  For 95% of use cases either choice will be just fine in terms of performance, and the remaining 5% need to test both solutions with their particular data and their particular access patterns.
• Operationally speaking, Elasticsearch is a bit simpler to work with – it has just a single process. Solr, in its Elasticsearch-like fully distributed deployment mode known as SolrCloud, depends on Apache ZooKeeper.  ZooKeeper is super mature, super widely used, etc. etc., but it’s still another moving part.  That said, if you are using Hadoop, HBase, Spark, Kafka, or a number of other newer distributed software, you are likely already running ZooKeeper somewhere in your organization.
• While Elasticsearch has built-in ZooKeeper-like component called Xen, ZooKeeper is better at preventing the dreaded split-brain problem sometimes seen in Elasticsearch clusters. To be fair, Elasticsearch developers are aware of this problem and working on improving this aspect of Elasticsearch.
• If you love monitoring and metrics, with Elasticsearch you’ll be in heaven. The thing has more metrics than people you can squeeze in Times Square on New Year’s Eve! Solr exposes the key metrics, but nowhere near as many as Elasticsearch.

Boil It Down For Me

In conclusion, here are the bits that I think make the most difference for anyone having to make a choice:

• If you’ve already invested a lot of time in Solr, stick with it, unless there are specific use cases that it just doesn’t handle well. If you think that is the case, speak to somebody close to both Solr and Elasticsearch projects to save you time, guessing, research, and avoid mistakes.
• If you are a strong believer in true open-source, Solr is closer to that than Elasticsearch, and having one company control Elasticsearch may be a turn-off.
• If you need a data store that can handle analytical queries in addition to text searching, Elasticsearch is a better choice for that today.

If you expected a single definitive winner, I’m sorry to disappoint. We don’t have one here.  However, I hope this quick comparison of the two leading open-source search engines provides enough information and guidance to help you make the right choice for your organization.

 

About the author: Otis Gospodnetić is a Lucene, Solr, and Elasticsearch expert, co-author of Lucene in Action (1st and 2nd editions), and the founder and CEO of Sematext. Sematext is a globally distributed organization that builds innovative cloud and on premise solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), search analytics (SSA), and search enhancement. The company also provides Search and Big Data consulting services and offers 24/7 production support for Solr and Elasticsearch to clients worldwide.

请关注我们的技术创业项目 Terark，领先的数据技术提供商