Torsten Bøgh Köster's Blog

I blog #search and #observabiity

Solr integration tests with Docker and Testcontainers

When developing Solr plugins for the open source community (or your in-house Solr team) you’re faced with testing the plugin’s compatiblity against a range of Solr versions. The Solr Test Framework provides a great tool set for writing JUnit and integration tests against a single Solr version. In this post I’ll guide you through adding Integration Tests to a Solr plugin project. We want to ensure the plugin’s functionality through a configurable range of Solr versions. I will add links to working examples in the Querqy Query Parser as I recently added those kind of tests there.

Synthetic response and uptime monitoring using the Prometheus Blackbox Exporter

In e-commerce, response time is king. Regarding onsite search engines, response time tends to increase linearly with the number of products sold in an online shop. In a recent project, we enhanced a Solr search engine to overcome this barrier. We used the Prometheus Blackbox Exporter for progress validation (and regression detection).

Terraform state drift detection in GitHub Actions

One of the annoying things to deal with when it comes to Terraform infrastructure is Terraform state drift, which describes the mismatch between the configured Terraform resources and the infrastructure reality.

Configuration management in the clouds

When spinning up compute instances in the cloud, a proper and fully automated configuration management is a key success factor. I usually use Ansible for the job as it’s open source and widely adopted. But when it comes to highly volatile cloud infrastructure with instances spinning up and terminating on demand, configuration management systems utilizing a control master (like Ansible, Puppet or SaltStack) can be cumbersome.

Logstash: Geoip resolution using a REST api

Logstash comes pre-packaged with a geoip filter plugin and a Maxmind Geolite2 database. But when running Logstash in a container environment, updating the bundled database can be a hazzle. If you need geoip lookups in other systems, a rest service is the perfect fit to deliver consistent geoip data.

Logstash: exporting custom metrics to Prometheus

Logstashs regular expression engine Grok is the swiss army knife when it comes to mine gold in your logfiles. Besides pure log enrichment, metric gathering of log data is a common requirement. Unfortunately Logstash is lacking a native solution to export custom metrics to Prometheus. In this post, I’ll walk you through a solution utilizing the graphite-exporter as an adapter.