Manual way to feed old/new log entires to logstash

From time to time your Logstash process may crash and you start seeing gaps on your Kibana graph (you graph logs and monitor those, right? ). That may also, coincidentally?, correlate with downtime you had 10 minutes ago.. So you need to replay missing events back into Logstash and figure out what happened.  …

Continue reading  »

Build Zappos like faceted navigation with ElasticSearch

I’ve been looking at different facets implementations while working on REST Search API for my ecommerce store. Some of the solutions I saw were simple to achieve (like plain faceting by terms with single selection), while others required some thinking and some extra work.

Today we will talk about Zappos facets,  …

Continue reading  »

Little trick to get AWS instance_id as variable in Nginx

If you are running in AWS environment it might be useful to get Amazon instance_id in Nginx for different purposes (logging, headers etc).

To do that we need to compile Nginx with Perl support (–with-http_perl_module flag during compilation). After that is done we can put this snippet inside of our httpd Nginx config section:

Important: While the Perl module is performing a long-running operation,  …

Continue reading  »

Dealing with multiple config sets in Solr(+cloud)

Solr in “cloud” mode (solrcloud)

One of the first steps for solrcloud cluster setup is collection configuration preparation. Usually it’s done like this:

-Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf

If you start Solr with above attributes, it will load configuration files under ./solr/collection1/conf to Zookeeper with myconf name.  …

Continue reading  »

Openssl heartbleed autofix for EC2 Amazon AMI – be aware!

You probably heard about recent security hole discovered in openssl library called Heartbleed. If not read:

Anyways here is the catch – turns out Amazon can roll critical updates to all images based of Amazon AMI!  …

Continue reading  »