ElastiFlow 6.4: Elasticsearch Output - support for TSDS

February 6, 2024

Elasticsearch Output: support for TSDS - TSDS output for Elasticsearch is now a fully supported feature and out of Technology Preview. Enabling Time Series Data Streams (TSDS), introduced in Elasticsearch 8.7, can result in storage savings of 50-70% depending on the content of flow records. Enabling TSDS does increase the ingest-related CPU load for Elasticsearch, which can be largely mitigated by the ingest CPU optimizations introduced in Elasticsearch 8.8. How to enable TSDS:

  • In Kibana, delete the 3 existing ElastiFlow index templates, as new ones will automatically be created once TSDS is enabled.

  • Stop your flow collector instance.

  • Open flowcoll.conf and set EF_OUTPUT_ELASTICSEARCH_TSDS_ENABLE to true.

  • Restart your flow collector instance.

Note: Enabling TSDS will not affect any existing data already in Elasticsearch. All dashboards will visualize data both before and after TSDS is enabled.