Logstash | Logging and Analytics for the Elastic Stack

Overview of Logstash

Logstash is a flexible, open source data collection, enrichment, and transportation pipeline. With connectors to common infrastructure for easy integration, Logstash is designed to efficiently process a growing list of log, event, and unstructured data sources for distribution into a variety of outputs, including Elasticsearch.

Process Any Data, From Any Source

  • Centralize data processing of all types
  • Normalize varying schema and formats
  • Quickly extend to custom log formats
  • Easily add plugins for custom data sources

Features of Logstash

Centralize Data Processing of All Types: Logstash is a data pipeline that helps you process logs and other event data from a variety of systems. With 200 plugins and counting, it can be connected to a variety of sources and stream data at scale to a central analytics system.

Normalize Varying Schema: Business-critical data is often scattered among different systems, each in its own format. Logstash allows you to parse this data and converge on a common format before inserting it into your analytics datastore of choice. Here are some of transformations that can be dynamically applied to your data:

  • Create structure for unstructured data using grok
  • Gain geographical data from IP addresses
  • Exclude sensitive PII data completely
  • Create an easier processing environment, regardless of the source

Extend to Custom Log Formats: Most logs written by infrastructure and applications have custom formats. Logstash provides a fast and convenient way to custom logic for parsing these logs at scale..

Add Plugins for Custom Sources: Built with extensibility in mind, Logstash provides an API for rapid plugin development by the community. With recent improvements to the plugin ecosystem, contributors can publish new plugins at any time.

Fully secured ingest pipelines: Deliver your data at-least-one using a persistant queue in the event of failure. The ingest queue is also scalable, absorbing throughput spikes without an external queueing layer.

Expand your deployment

X-Pack provides advanced monitoring and alterting features that expand your Logstash well beyond a conventional logging system. These features will help you to discover data bottlenecks in your pipelines, and lets you fully study and understand the flow of data between your deployments.

Want to experiment? Download a copy from the Elastic website and experience it for yourself.

Knowledge Focus – Pretoria, Gauteng, South Africa

If you’d like to know more about Logstash and what we can do to help, please fill in the form below and we’ll call you back.

Contact Knowledge Focus

Let us tell you more about enterprise search