BitSwan Features v2006

BitSwan Features v20.06, launched on 18th of June 2020

New features v20.06:

Influx connection throttling

  • Throttling mechanism was introduced to the InfluxDB Connection, enabling it to become more fluent and reliable.

MySQL features

  • It is now possible to use the autocommit feature in MySQL connection's aiomysql.create_pool.
  • For more information on this feature, visit the official docs.
  • A new MySQL BinaryLogSource was added to extend the general MySQL module usability.

Integrity module

  • Integrity module is a new type of Processor, used for enriching JSON data with hashed events.
  • The data is encoded by JSON Web Token standards.
  • Usage of the new Processor is demonstrated in the example section (bspump-integrity-enricher.py)

Partition specification in the Kafka Source

  • It is now possible to specify topic partitions used by the Kafka Source, using configuration.
  • Partitions allow Kafka users to parallelize a topic by splitting the data in a particular topic across multiple brokers - each partition can be placed on a separate machine to allow for multiple consumers to read from a topic in parallel.

HyperLogLog

  • Implemented a HyperLogLog aggregation/approximation algorithm into the BSpump.
  • HyperLogLog is an algorithm for a count-distinct problem.
  • It is used to approximate the number of distinct elements in a multiset, which reduces computational load on large datasets.
  • An example using this new feature can be found in the examples section (bspump-hyperloglog.py)

BSPump Declarative language

  • A huge new feature, that is primarily connected to the BSPump's Logman.io
  • For the first time, users are able to build BSPump components (such as Pipelines), using declarative .yml configurations.
  • This whole new language is well documented and ready for use, while being extended and improved at the same time.
  • Several examples of this exciting new feature can be found in the examples section (bspump-declarative-*.py)

Windows Remote Management

  • Addition of a whole new source for Windows Remote Management, a SOAP based protocol enabling interoperation of various vendor's operating systems.
  • Usage is demonstrated in the examples (bspump-winrm.py)

Other changes

  • Fix in the subprocess source
  • Removed Elasticsearch connection retry option
  • Introduce changestream mechanic into the mongolookup
  • Add a new generator: Aggregator

  • Code refactoring

You Might Be Interested in Reading These Articles

BitSwan Features v1910

We've just launched new features: Analyzer profiling, bspump.lookup.MatrixLookup, bspump.lookup.BitMapIndex, MongoDBSink etc.

Continue reading ...

new release features analyzer profiler mongodb

Published on October 31, 2019

BitSwan Features v1903

We've just launched new features: Jupyter server in BitSwan ecosystem, First implementation of pipeline builder, BlankApp for Telco, Open ID connectivity, MySQL Transaction / Binary log, Session Analyzer to hold session data for a specified time, Throttles are now listed when accessing pipelines via REST call, TimeDriftAnalyzer sends fixed zero metrics when there is no history, Kibana Object Library support in bskibana tool. Enjoy and provide us with your feed back.

Continue reading ...

new release features Jupyter pipeline builder BlankApp Open ID MySQL Session Analyzer TimeDriftAnalyzer Kibana

Published on March 11, 2019

BitSwan Features v1907

We've just launched new features: Google Drive Sink, Tableau Sink, KafkaSink, analyzers and lookup refactoring, LatchAnalyzer etc.

Continue reading ...

new release features file google drive matrix kafka latch

Published on July 31, 2019