Logstash Component
The logstash component is available on all platforms. It
can be used to configure usage of
Logstash. Logstash is a
light-weight, open-source, server-side data processing pipeline that allows the
ingestion of data from a wide variety of sources, transform it on the fly, and
send it to Elasticsearch to be used with Kibana as part of the ELK stack.
Logstash overview:
- The Log/event processing engine written in JRuby and runs as a Java
application on the JVM.
- The log lines flow through 3 different stages - Input, Filters and Outputs.
- There are many standard input, filter and output plugins available.
- Logstash needs a simple configuration file in JSON format that specifies
input, filters and outputs.
- Logstash Forwarder is binary application written in Go. It tails log files and
forwards the lines to downstream Logstash servers using the lumberjack
protocol
Main goals of Logstash Forwarder tool design are
- Minimized resource usage where possible (CPU, memory, network)
- Secure transmission of logs
- Easy deployment with minimal moving parts
The Logstash forwarded run in userspace on the VM and gets installed as part of
compute component installation.