This video gives insight on how a platform can be built on databricks . This is essentially a library that automates the table creation , airflow code generation etc .
this next video is about sputnik which is opensource courtesy – airbnb – it also provides ways for job logic and run logic