The Data Ingestion Layer of tcgmcube comes with pre-built standard connectors to various source systems & instruments.
- Ingestion of structured, semi-structured, and unstructured data
- Options for real-time, near real-time, and batch ingestion
- Support for dynamic data pipelines
- Options for data transformation at various stages (ETL as well as ELT)
Support for data collection and management at the edge—handling events through data caches and synchronization
- Overlay of a semantic layer
The data storage layer comes with robust data management features. It leverages ontology and knowledge modeling capabilities, making it “easy to get data out” and has the following layers:
- Base data layer for source data processing, providing features to validate and catalogue the raw data
- Analytic Persistence layer with processed datasets for optimizing analytical queries and AI – driven processes
- Semantic Persistence Layer with contextualized data taxonomy through knowledge graphs
The analysis and visualization features of the platform are powered by the semantic layer that makes it “easy to get data out” for analysis needs, providing options for specifying deep ontologies for domain contextualization. This block provides:
- Traditional AI at scale with a wide assortment of statistical, ML, DL, and optimization algorithms.
- Comprehensive Gen – AI algorithms covering traditional LLM and multimodal LLM RAG models for fast information retrieval and traceability.
- Insights dissemination options include dashboards with easy business user self-service, operational reports, and low-code “upgrade safe custom screen painting”. These leverage the semantic layer for data interpretation and reporting.
- Action dissemination options – provides inputs to automated operational processes such as alerts, recommendations, action triggers, etc.
The Platform Services & Governance layer helps implement data management & governance practices for data quality, security, & compliance. Features include:
- Role-based Access Control and fine-grained access policies (row, column, and object-level access control)
- Data Encryption at rest and in transit
- Audit Logs for all data access and processing activities.
- Layered Security: Security can be defined at various levels- cluster, index, document, and field
- Metadata Management powered by knowledge graphs