Manual way to feed old/new log entires to logstash
From time to time your Logstash process may crash and you start seeing gaps on your Kibana graph (you graph logs and monitor those, right? ). That may also, coincidentally?, correlate with downtime you had 10 minutes ago.. So you need to replay missing events back into Logstash and figure out what happened. …
Docker VM shortcomings and how Hodor can help
Bright side
Finally Docker hype reached FastCompany and here I am adding to it : ) I wish I started with Docker intro post first, but I bet there are tons of stuff like this already on the nets. Instead I’ll assume most readers will already be familiar with Docker in some way. …
Little trick to get AWS instance_id as variable in Nginx
If you are running in AWS environment it might be useful to get Amazon instance_id in Nginx for different purposes (logging, headers etc).
To do that we need to compile Nginx with Perl support (–with-http_perl_module flag during compilation). After that is done we can put this snippet inside of our httpd Nginx config section:
Important: While the Perl module is performing a long-running operation, …
Use Nginx basic auth to protect HTTP services like Solr
Sometimes it might be useful to add simple permissions layer on top of unprotected HTTP services ( especially if you would like to open those to the public! ). Recently we had to secure our Solrcloud install to be able to provide read only access to our FTS index to a partner, …
Dealing with multiple config sets in Solr(+cloud)
Solr in “cloud” mode (solrcloud)
One of the first steps for solrcloud cluster setup is collection configuration preparation. Usually it’s done like this:
-Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf
If you start Solr with above attributes, it will load configuration files under ./solr/collection1/conf to Zookeeper with myconf name. …
Improve and simplify Solr logging with Nginx proxy
In efforts to provide better visibility to all parts of our application framework we decided to collect and send Solr query logs to our logging farm for further analysis and smoke tests. For logging aggregation we use Logstash with Kibana – …
Story behind X-Forwarded-For and X-Real-IP headers
This was suppose to be 10 minutes fix..
Or so I thought. If you read my blog you probably know that for the most part I’m doing operations stuff for FastCompany. That means sometimes I deal with little tricky problems like the one I’m going to talk about today. …
Playing around with Jetty JVM Garbage Collection
First of all, I would never thought I’m going to write about JAVA GC. I’m not a JAVA guy, everything described in this post is my personal experience from the operational stand point. I hope it will be useful for some of you as well. …
Openssl heartbleed autofix for EC2 Amazon AMI – be aware!
You probably heard about recent security hole discovered in openssl library called Heartbleed. If not read:
http://www.openssl.org/news/secadv_20140407.txt
Anyways here is the catch – turns out Amazon can roll critical updates to all images based of Amazon AMI! …
SolrCloud in EC2. Dealing with changing IPs
If you use Amazon EC2 service for a while, you probably aware of the fact that any time you stop/start (reboot) your instance the underlying IP might change. We need to account for that when designing new services in EC2.
Lets see what happens to SolrCloud after IP change:
Note those grayed out IPs. …