The setup describe here is a canonical way to create a collaborative and sustainable environment to conduct data-science and analysis. It is completely based on open-source tools and can be scaled up to larger instances. Indeed many of the component and best practices are also employed in scaled up organisational deployments. Hence the work-flow and components can be reused elsewhere. The key components:

• Linux server(s) geared towards data-science
• Jupyter lab instances
• Docker containers environment
• Portainer docker management interface
• Nginx proxymanager
• GPU enable environments and containers
• Xwiki based knowledge management and collaboration
• Nextcloud based file sharing

These tools match very well with our envisaged data/work-flow. It represent a step in our bootstrapping approach. The current configuration is very suitable for small scale deployment for training and mission purposes (although it is still considered beta-status).

A small team could use this as the a basis for an analysis hub. Considering the current nascent status a data-scientist/data-engineer should form part of the team to ensure proper functioning and troubleshooting. A large part of the required work however can be executed remotely.

The system is based on open-source technology but will allow for incorporation of proprietary and legacy elements. It is however strongly advised to steer clear of further cementing dependency on commercial packages like iBase, Analyst Notebook and ESRI ArcgisPro. Good (even better) alternatives for these tools exsist!

Instead of forking out yearly license and training fees for the above mentioned commercial products it would be far wiser to invest in knowledge within our own organisation and staff that will ultimately use it (it can come as no surprise that the highly competent IDF AI-units employ in-house developed tools leveraging on open-source software).

https://www.jpost.com/arab-israeli-conflict/gaza-news/guardian-of-the-walls-the-first-ai-war-669371