BitSwan Features v1907

BitSwan Features v1907, launched on 1st of August 2019

New features v1907:

Google Drive Sink

Google Drive connection and GoogleDriveBlockSink is used to upload individual events as files to Google Drive.

Tableau Sink

New processor named FileTableauSink was added. FileTableauSink is used to output events to Tableau files using specified data schema.

KafkaSink refactoring

KafkaSink receives the KafkaProducer from the KafkaConnection.

Analyzers refactoring

New parameters were added to the constructor: analyze_on_clock and analyze_period; analyze_on_clock enables analysis by timer, which period can be set by analyze_period or Config["analyze_period"].

The performance of analyzers was improved.

Both predicate and evaluate method contain context attribute.

LatchProcessor fully refactored to LatchAnalyzer

The LatchAnalyzer accumulates events in the Latch variable and provides analyzer interface (analyze, evaluate etc.).

Add new native BSPump endpoint: /lookup/{lookup_id}/meta

The /lookup/{lookup_id}/meta endpoint returns information about the lookup (ID, ETAG, URL of the master).

Matrix refactoring

In the constructor, column_names and column_formats parameters were replaced by dtype. NamedMatrixABC was renamed to NamedMatrix etc.

Introducing bspump.cache

Module bspump.cache was introduced with LRUCacheDict implementing the LRU cache strategy: ([]

Introducing bspump.random

New processors named RandomDrop and RandomEnricher were added. RandomDrop simulates random drop of events, while RandomEnricher adds to event only one field with the name from Config['field'] and is mostly meant for testing.

Refactoring positioning of arguments in lookups and connections

Every default lookup and connection's constructors have the following header signature: __init__(self, app, id=None, config=None)

MySQL Lookup improvements

Living cursor was removed from MySQL lookup. The cursor is created when necessary, thus the performance and timeout issues were solved.

The configuration option in MySQLookup labeled "from" can now include a table name or a query string including joins.

Filters refactoring of internal code

More unit tests

OOB example extended to include work with ProactorService, which enables running long running tasks on threads

About the Author

Premysl Cerny

Premysl is senior developer in LibertyAces

You Might Be Interested in Reading These Articles

BitSwan Analyzers Tutorial

This tutorial is meant to go through all questions the software engineers might encounter during the work with BitSwan Analyzers and Matrices. There is a description of the initialization, parameters to tune, typical usage patterns and recommendations for optimization.

Continue reading ...

analyzer matrix numpy

Published on August 20, 2019

BitSwan Features v2101

We've just launched new features: WinAPI module, FTP support, and more!

Continue reading ...

new release bspump-declarative winapi async timeseries elasticsearch ftp

Published on April 21, 2021

BitSwan Features v1903

We've just launched new features: Jupyter server in BitSwan ecosystem, First implementation of pipeline builder, BlankApp for Telco, Open ID connectivity, MySQL Transaction / Binary log, Session Analyzer to hold session data for a specified time, Throttles are now listed when accessing pipelines via REST call, TimeDriftAnalyzer sends fixed zero metrics when there is no history, Kibana Object Library support in bskibana tool. Enjoy and provide us with your feed back.

Continue reading ...

new release features Jupyter pipeline builder BlankApp Open ID MySQL Session Analyzer TimeDriftAnalyzer Kibana

Published on March 11, 2019