Torsten Bøgh Köster

Senior Search & Operations Engineer

Career Summary

As a software engineer I have expert level knowledge in constructing, developing and running high traffic web applications. I embrace open source software and am passionate about agile software development with Scrum. I am metric driven, have a passion for log management and love to talk about my findings at conferences and meetups.

Work Experience

Senior Search & Operations Engineer

Freelancer
2019 - present

As a Senior Search & Operations Engineer I cater both customers facing challenges in their search or operations environment (or both). I build Kubernetes clusters with infrastructure as code and observability in mind both in the cloud and on-premise.

Key customer projects:

  • Bridging the gap between devs and AWS cloud infrastructure in a e-commerce search project. Introduced Prometheus based observability and led a team overhauling the overall system architecture.
  • Introduced sharding into a e-commerce Solr search platform. Refactored all internal components and made algorithms sharding ready. Also updated the installation from the Solr 7.x release train to Solr 8.11 and onwards to Solr 9.
  • Acting as a trusted advisor for a junior search team taking over a aging Solr search infrastructure. Guided them through the overhaul of the autocomplete to get to know their search infrastructure.
  • Introduction of Ansible and infrastructure as code in a large on-premise infrastructure. Automated and scalable setup of Kubernetes clusters using Ansible and kubeadm. Log management and monitoring for two data centers using Graylog and Prometheus.
  • Seeding the idea of infrastructure as code for both on-premise and cloud infrastructure. Setup of SaltStack for the on-premise environment and Terraform for AWS infrastructure. Automated setup of Kubernetes (EKS) clusters including VPCs and ingress handling.
  • Migration from on-premise infrastructure to Google GCP including site-2-site VPN, dialin VPN and scalable Kubernetes clusters.

Technologies used:

  • Solr
  • Ansible
  • Kubernetes
  • Terraform
  • Logstash, Filebeat & Graylog
  • Prometheus & Grafana
  • Amazon AWS
  • Digital Ocean
  • Google GCP
  • Traefik
  • Keycloak

Engineering Lead (CTO)

shopping24 internet group
2012 - 2019

As CTO I am accountable for developing and operating the shopping24 product search platform. Together with the engineering team I maintain the api and search backend based on Apache Solr, the product backend systems (Java) and the portal frontends (PHP). I code on a regular basis in both operations and search and assist the teams in system- and software architecture challenges.

Key projects:

  • Insourcing of server operations: Development of a centralized configuration management using Ansible. Setup of a hybrid cloud hosting using bare metal, VMWare and KVM hypervisors as well as Amazon AWS. Recently pushing the transition into containerization using Docker, Kubernetes and Ceph as distributed storage.
  • Design and implementation of a inner source e-commerce search solution for the Otto Group based on Apache Solr. Onboarding and consulting of several Otto Group companies like otto.de and Limango.
  • As a in-house consultant I joined the implementation of the e-commerce search at otto.de and Limango. Implemented technical and relevance search monitoring, extended the CI/CD pipeline to do blue/green deployments.
  • Building and eventually spun off a SaaS to manage Google and Bing ads at large scale. Wrote an entity extraction as a Apache Solr plugin in order to categorize ad terms. The software is used by multiple tenants inside the Otto Group.

Technologies used:

  • Apache Solr
  • Ranklib
  • Kubernetes
  • Spring Boot
  • Java 11
  • Docker
  • Kubernetes
  • Logstash
  • Filebeat
  • Graylog
  • Nginx
  • HAProxy
  • Traefik
  • Varnish
  • Prometheus
  • Grafana
  • Graphite
  • Elasticsearch
  • Ansible
  • KVM
  • Ceph

Team lead Java development

shopping24 internet group
2010 - 2012

As a team lead responsible for restructuring and refactoring the Java API and product management backend.

Key projects:

  • Recruitment and training of the Java development team. Insourcing of Java development and application support for the existing Java applications.
  • Review of the existing system and software architecture. Refactoring towards an elastic and service oriented architecture. Moved traffic and storage intensive data to Amazon AWS.
  • Introduced Scrum as development process. Established a modern Java development infrastructure for Continuous Integration und Continuous Delivery.
  • Conceptual design and implementation of a modular REST api for delivering affiliate product data based on Apache Solr.
  • Centralizing logging, server, application and business metrics for real time analysis.

Technologies used:

  • Apache Maven
  • Amazon EC2
  • Amazon S3
  • Amazon CloudFront
  • MongoDb
  • Apache Solr
  • Apache Hadoop
  • Apache Mahout
  • Zookeeper
  • Jenkins
  • Sonar
  • Artifactory
  • Github
  • Redmine
  • New Relic
  • Graylog2
  • Logstash
  • StatsD
  • Graphite
  • Grafana
  • Collectd

Software Architect Java

Deutscher Ring
2005 - 2010

Developer, architect and in-house consultant for Java based projects focusing on integrating legacy systems into modern web UIs. Technical lead for internet and intranet web applications. Maintainer of the Java development and continuous integration infrastructure.

Key projects:

  • Prepared and led advanced in-house Java, Spring Framework, Maven & Eclipse trainings
  • Technical team lead for web applications serving customers and insurance brokers.
  • Switched the complete Java codebase from building with Ant to Maven, maintained build server for continuous integration and Maven infrastructure components
  • Created an Eclipse RCP based distribution platform for remote offline viewing of data warehouse data cubes

Technologies used:

  • Java
  • Eclipse RCP
  • Spring
  • Hibernate
  • Spring Security
  • Central Authentication Service (CAS)
  • IBM DB2
  • IBM Content Manager
  • IBM MQSeries
  • Websphere Application Server
  • Tomcat
  • Lucene
  • Hudson
  • Sonar
  • Artifactory

Software engineer & server administrator

Freelancer
2002 - 2005

Acquired and implemented Java and PHP development jobs. Gained knowledge of Linux server administration in high traffic environments as systems administrator for the largest soccer community in Germany.

Technologies used:

  • PHP
  • MySQL
  • Wordpress
  • Lighttpd
  • Debian Linux

Software developer PHP & Perl

Have-IT
2000 - 2002

Developed custom tailored applications for german high traffic websites in PHP and Perl.

Technologies used:

  • PHP
  • Perl
  • LAMP stack

Skills & Tools

Search

  • Solr & Lucene
  • Java / Spring Boot
  • PostgreSQL
  • NLP / NER / Learn To Rank
  • Elasticsearch

Operations

  • Ansible, Terraform
  • Prometheus, Grafana
  • Logstash, ELK & Graylog
  • Kubernetes & Docker
  • HAProxy, Nginx, Traefik & TLS

Others

  • CI/CD
  • Maven
  • Git & GitHub
  • AWS, Digital Ocean, GCP
  • Apple DEP & MDM
  • Jenkins & Travis CI
  • Filebeat
  • Artifactory
  • Tomcat
  • Nginx
  • Varnish
  • MySQL
  • Redis
  • LDAP
  • Zookeeper
  • Salt Stack
  • Munki

Community

Language

  • German (native)
  • English (professional)
  • Danish (intermediate)

Education

  • MSc Informatics
    (Diplom-Informatik)
    University of Hamburg
    1999-2006
  • A-Levels
    Carl-von-Ossietzky Gymnasium Hamburg
    1998

Interests

  • Road bike
  • Landscape photography
  • Travel
  • Cooking

Interested in hiring me for your project?