Alphacast Friday Update - January 14th, 2022

As we presented in our previous update, we kept working on our new pipelines engine. We have now reached features parity with our previous engine, so we are ready to start testing internally before we launch it to everyone. In this update, we are showing how some of the new features compare with our previous ones, and we hope you will find the new experience easier (and more fun) to use:

Pipeline Migrations

We have automated the migration of our previous pipelines definitions. This means that once we launch the new pipelines platform, you will be able to edit your previously created pipelines, and extend them with new features included in the new platform (such as merge dataset). As an example, check the "lineal combination" definition in our previous admin tool:

Compared to the "Calculate Variable" step, with the given calculation already migrated:

Datasets search

When creating a pipeline, you usually need to indicate where data will come from, and where it should go to. In the past, you would need to remember in which repository your dataset was stored, in order to select it:

We are now using our Search Engine, and as you start typing the name of your dataset, you will get options to choose from. Notice that you don't need to remember the exact dataset name either, but you can search with keywords

Simplified resampling process

We now allow selecting a given grouping or interpolation function and apply it to all columns when changing the data frequency. You can still then modify the function for a given column:

Simpler transformations

When we started creating the pipelines engine in early-2021, we had a small group of transformations that quickly escalated into multiple transformation types. This turned our admin tool a bit chaotic, with a large grid of options where you had to scroll vertically and horizontally to define your transformations.

Most pipelines would use only a reduced set of these transforms, so we simplified the transformation step, where you first indicate the transformation and then the columns it should be applied to.

We really hope you'll enjoy the new experience we will be releasing soon. Beyond the pipeline editor, we are also doing lots of improvements in the way pipelines are processed, and we will soon offer new ways to interact with them. Stay tuned!