The rationale

This page goes into some of the rationale for the technical and business decisions we've made.

What does the architecture look like?

MF creates Notebook pipelines for each customer using core mf libraries.

This allows bespoke integrations to be completed easily at the expense of quickly punching out identical pipelines for new customers.

This is acceptable because integrations quickly become bespoke.

We don't have versioning issues because each application runs on its own instance of the core libraries until the application is updated. End to end testing is easy to do because there is always lots of data to work with.

The Stack

At Managed Functions, we are incredibly happy with how our tech stack has come together. Even though our customers never see what happens under the hood, if you're like us, you want to know how it all works.

Here's what we do when we bring on a new customer (or a new process for an existing customer).

1. We draw out the integration pipeline.
2. The development pipeline is generated automatically from the diagram
3. Our integration developers code the bespoke parts of the pipeline in Jupyter notebooks (an open-source development environment that is particularly strong in working with data)
4. Once built, they click Deploy which creates the cloud infrastructure to run the pipeline (serverless functions, storage, events and APIs). This can be deployed on our cloud or yours.
5. For customers on the Interactive plan, a user interface is generated that allows them to interact with the data as it moves through the pipeline.
6. The pipelines are monitored by the Managed Functions support team. If unexpected data comes through the pipeline, our support team is notified. They open up the Jupyter notebook and run that part of the pipeline. They can see the source of the failure, fix it and redeploy the pipeline thus not only fixing the problem but ensuring similar data does not create errors in the future.