Introduction Managing Elasticsearch clusters in environments is a common challenge for organizations that run search and analytics work that is vital to the business. Whether you are moving to infrastructure, setting up disaster recovery or changing operating systems, keeping Elasticsearch data consistent and minimizing downtime are the top priorities. The primary objectives were to: Achieve […]
Elasticsearch Has 20,000 Shards: What Actually Happens When You Restart a Node? Introduction Restarting an Elasticsearch node sounds simple: That assumption changes when a cluster contains thousands of shards. In one production environment I worked with, the Elasticsearch cluster had approximately 20,000 shards. A node restart could trigger extensive shard recovery, allocation, disk I/O, network […]
Introduction If you have ever spent 20 minutes writing a Kibana query just to find one error, this post is for you. I built a small project on a single AWS server where I could just type “show me the last 502 errors” in plain English, and get a real answer pulled straight from Elasticsearch. […]
When we run Elasticsearch in production, one of the common issues is imbalance in “shards”. There may be one node in the cluster that is out of disk space, while a few nodes with no shards on them. For example, here is a node with all the shards: Node Shards Disk Used Disk % Free […]
Introduction Elasticsearch has become a go-to choice for building fast, intelligent search experiences. But what if your source of truth is a relational database like MySQL? In this blog, we’ll walk through how we migrated structured relational data into Elasticsearch using Python — with company_registry as a working example. Why Migrate from MySQL to Elasticsearch? […]
Introduction We recently needed visibility into what our Java services were actually doing in production — response times, slow queries, errors, that sort of thing. We landed on Elastic APM. Here’s exactly how we set it up, including some decisions we made around multi-environment support that saved us a lot of headache later. The setup […]
Introduction Elasticsearch is a powerful search engine that’s commonly used for log and data analytics. Setting a multi-node cluster enhances the availability, fault tolerance, and performance of Elasticsearch, making it a preferred choice for production environments. In this blog post, I’ll walk you through the steps to create a multi-node Elasticsearch cluster using Terraform ensuring […]
Introduction In today’s competitive market, delivering a seamless and responsive user experience is crucial for retaining customers and driving business growth. Elastic APM (Application Performance Monitoring) has emerged as a vital tool for achieving these objectives. By providing deep insights into application performance and pinpointing bottlenecks, Elastic APM empowers businesses to optimize their applications, enhance […]
Elasticsearch offers highly useful plugin mechanism as a standard way for extending its core functionality such as custom analyzer, native scripts and more. While some plugins may contain static content which is served through its HTTP server, some others offer a graphical front-end for selected parts of the Elasticsearch REST API such as monitoring, managing cluster and […]
