Wondering if having some middleware might help later for 4.0 architecture?

Something like Apache Linkis ?

My thinking is along the lines of controllability as they mention (audit records, resource control, etc) which is useful in the context of FAIR practices. The bonus is that the bottom layers can change to whichever supported engines Quick Deployment | Apache Linkis For instance, I have a team that uses Anaconda(python) primarily for science workflows, but in news, they use MySQL and Spark. Sometimes the analytics are created and viewed by Zeppelin (apache.org)

Forget Apache Linkis. I now see its minimum requirements are 16 cores, 32GB ram. Only for large enterprise data orchestration needs.