BitSwan Features v20.06, launched on 18th of June 2020
New features v20.06:
Influx connection throttling
- Throttling mechanism was introduced to the InfluxDB Connection, enabling it to become more fluent and reliable.
- It is now possible to use the autocommit feature in MySQL connection's aiomysql.create_pool.
- For more information on this feature, visit the official docs.
- A new MySQL BinaryLogSource was added to extend the general MySQL module usability.
- Integrity module is a new type of Processor, used for enriching JSON data with hashed events.
- The data is encoded by JSON Web Token standards.
- Usage of the new Processor is demonstrated in the example section (bspump-integrity-enricher.py)
Partition specification in the Kafka Source
- It is now possible to specify topic partitions used by the Kafka Source, using configuration.
- Partitions allow Kafka users to parallelize a topic by splitting the data in a particular topic across multiple brokers - each partition can be placed on a separate machine to allow for multiple consumers to read from a topic in parallel.
- Implemented a HyperLogLog aggregation/approximation algorithm into the BSpump.
- HyperLogLog is an algorithm for a count-distinct problem.
- It is used to approximate the number of distinct elements in a multiset, which reduces computational load on large datasets.
- An example using this new feature can be found in the examples section (bspump-hyperloglog.py)
BSPump Declarative language
- A huge new feature, that is primarily connected to the BSPump's Logman.io
- For the first time, users are able to build BSPump components (such as Pipelines), using declarative .yml configurations.
- This whole new language is well documented and ready for use, while being extended and improved at the same time.
- Several examples of this exciting new feature can be found in the examples section (bspump-declarative-*.py)
Windows Remote Management
- Addition of a whole new source for Windows Remote Management, a SOAP based protocol enabling interoperation of various vendor's operating systems.
- Usage is demonstrated in the examples (bspump-winrm.py)
- Fix in the subprocess source
- Removed Elasticsearch connection retry option
- Introduce changestream mechanic into the mongolookup
Add a new generator: Aggregator