Dealing with multiple config sets in Solr(+cloud)
Solr in “cloud” mode (solrcloud)
One of the first steps for solrcloud cluster setup is collection configuration preparation. Usually it’s done like this:
If you start Solr with above attributes, it will load configuration files under
./solr/collection1/conf to Zookeeper with
We need to execute this command only once. After configuration was uploaded we can just add more Solr instances without specifying configuration options. It works because if we have just one config set (
myconf in our example) in Zookeeper it will be automatically linked with all new collections we create.
What happens if we upload another config set with the new name?
zkcli.sh -zkhost localhost:9983 -cmd upconfig -confdir /opt/solr/collection2/conf -confname myconf1
If we use Solr in “cloud” mode, then we would have to use
linkconfig command to link collection with its configuration, otherwise Solr won’t be happy:
zkcli.sh -zkhost localhost:9983 -cmd linkconfig -collection collection2 -confname myconf2
Note - you can use
linkconfig action to link config and collection even before collection creation. That could be quite handy.
Traditional Solr (non-cloud)
It’s clear that sharing configuration between collections is very useful feature, but what if you are not in the “cloud” mode? Well, Solr 4.8+ added Config Sets specifically for this reason!
If we want to share configuration between cores, we need to place it into
mkdir -p solr/configsets/generic/conf/ cp -r solr/collection1/conf/* solr/configsets/generic/conf/
Then create couple new cores that would use our config set (note configSet attribute in the url):
curl 'http://localhost:8983/solr/admin/cores?action=CREATE&name=products&configSet=generic' curl 'http://localhost:8983/solr/admin/cores?action=CREATE&name=tags&configSet=generic'
At this point both cores should be created and point to same config set! You should be able to remove one of the cores without affecting the configuration, since it is stored in different (shared) location.
You can also use
configSet inside of core.properties file under your core dir:
Some popular ones
- Story behind X-Forwarded-For and X-Real-IP headers (23 Apr 2014)
- Internal redirect to another domain with proxy_pass and Nginx (14 Oct 2013)
- Secure data bag items with chef solo (04 Aug 2013)
My books recommendations
Great book for operations people. Helped me to design and build solid deployment pipelines. Awesome advices on automated testing as well. The author advocates against feature branches, every commit goes to master! Scary? I know, but it actually makes sense once you get the idea. Read the book to find out more.
One of those rare books where every word counts!
Classics from John Allspaw who is SVP of Infrastructure and Operations at Etsy (and used to work for Flickr). The book covers very important topics like metrics collection, continuous deployment, monitoring, dealing with unexpected traffic spikes, dev and ops collaboration and much more. Def recommend if you are starting out in the operations field or been doing it for a while ( in latter case you probably read this book already :).
This book is must read for every software engineer, no matter which language you use! It will change your perspective on writing code. I was amazed by the quality of material - very detailed and up to the point.
"The only way to make the deadline -- the only way to go fast -- is to keep the code as clean as possible at all times."
blog comments powered by Disqus