Introduction
Recently my team and I were discussing how to help our clients unlock the value of their industrial and manufacturing data. These industries often wrestle with terabytes of time-series data from a myriad of sensors, machines, processes and external sources. Each data source could potentially include different features with various formats, have non-rationalized time intervals and be generated from various proprietary technologies. The challenge of making this data available on a platform where workers can exploit the data and discover its hidden value can be overwhelming. Here is where the vision of AI Democratization often hits its first significant roadblock.
Managers who are tempted to feel elated at the prospect of having access to all the data they could ever wish for, soon realize it is a data sword of Damocles* hanging by a thread, ready to snap and bury them if they can’t find a way to unlock its value.
Managers who are tempted to feel elated at the prospect of having access to all the data they could ever wish for, soon realize it is a data sword of Damocles* hanging by a thread, ready to snap and bury them if they can’t find a way to unlock its value.
Richard Westall’s Sword of Damocles, 1812
* The parable of the sword of Damocles teaches us that no matter how good someone’s life may appear, it’s difficult to be happy living under existential threat.
* The parable of the sword of Damocles teaches us that no matter how good someone’s life may appear, it’s difficult to be happy living under existential threat.
Recent Posts
Recently my team and I were discussing how to help our clients unlock the value of their industrial and manufacturing data. These industries often wrestle with terabytes of time-series data from a myriad of sensors, machines, processes and external sources. Each data source could potentially include different features with various formats, have non-rationalized time intervals and be generated from various proprietary technologies. The challenge of making this data available on a platform where workers can exploit the data and discover its hidden value can be overwhelming. Here is where the vision of AI Democratization often hits its first significant roadblock.
Managers who are tempted to feel elated at the prospect of having access to all the data they could ever wish for, soon realize it is a data sword of Damocles* hanging by a thread, ready to snap and bury them if they can’t find a way to unlock its value.
Managers who are tempted to feel elated at the prospect of having access to all the data they could ever wish for, soon realize it is a data sword of Damocles* hanging by a thread, ready to snap and bury them if they can’t find a way to unlock its value.
Richard Westall’s Sword of Damocles, 1812
* The parable of the sword of Damocles teaches us that no matter how good someone’s life may appear, it’s difficult to be happy living under existential threat.
* The parable of the sword of Damocles teaches us that no matter how good someone’s life may appear, it’s difficult to be happy living under existential threat.
The Modern Historian
The Data Historian manages data from cradle to grave (assuming your data ever dies!)
This brings me to one of my favorite tech journalists, Rob O’Regan. He tackles some of these issues in an article written for CIO in May 2021. Specifically Rob makes the case for modernization of data/operational historians. These platforms must modernize: shift from collecting, dumping and dashboarding data for reporting on what has happened and perhaps what is happening now, into cloud-ready, end-to-end platforms that can use AI/ML to predict what will happen next. These platforms must be available to many personas in the organization and provide data in a way that presents a low-impedance environment for people to get what they need, quickly.
Features of a modernized historian
- Cloud-base
- Scale up/down to any workload requirements
- Real-time data ingest
- Integrated and standardize data from any industrialized data source
- Feature stores
- Versioning
- Low Code/No Code programming
- Advanced Analytics, AI/ML
MLOps No Longer an Option
Having said all of the above, little is accomplished if these advanced AI/ML platforms are not managed with MLOps best practices. The entire end-to-end data pipeline, from raw data intake to actionable insights, must itself be industrialized. MLOps ensures your ML pipeline can create, train, evaluate, validate, deploy and monitor the quality of ML models and update them when performance starts to degrade.
Data scientists require a laboratory for experiments which can then be automatically deployed into your ML pipeline for full lifecycle management. Everybody knows that Henry Ford didn’t make automobiles – he made the assembly line, which made automobiles. Your ML pipeline is the product, which if done well, will generate endless business value.
MLOps is the industrialization of data science
Benefits of MLOps Best Practices
| Optimize Operations and Reduce Cost | Increase Quality and Model Effectiveness |
| Automation eliminates manual practices | Capture knowledge and expertise |
| Continuous Integration/Delivery/Training | Version everything for repeatability |
| Reduce time-to-value | Transparency for regulatory/ethical compliance |
| Eliminate technical debt | Continuously monitor model performance |
| Onboard new employees quickly | Manage failures in production |
Final Thoughts
Everyone knows that there is value in data and that leveraging that value means a more successful business. The problem is very few organizations understand the way to derive value from data. It all starts with the data, skilled people and an analytic platform. These elements together make for an AI Factory. It is best to find a partner who has both the skilled consultants to show the way and a fully functional platform to provide the data historian and MLOps people with the technology they need to succeed. MLOps is no longer an option on this journey. Of course I would say that. I work for just such a company.
Check us out at www.tcgdigital.com.
Check us out at www.tcgdigital.com.
About tcg mcube
tcg mcube is an advanced analytics and AI Platform, which allows users to create compelling business solutions for tackling complex industry problems. With its modular architecture, tcg mcube handles many data sources, and it provides an efficient “cut-and-fit” into legacy environments if needed. Its mantra: Velocity to Value.
- Ingest structured and unstructured data from diverse sources
- Store ingested data within big data stores and data lakes
- Provide a library of algorithms for ML and AI
- Create stunning visualizations using a powerful BI library
- Empower the modern Data Historian
- Enable MLOps to optimize data value