The digital transformation for a future-ready enterprise requires redefining data estate modernization. As businesses accelerate their base towards the digital future, infrastructure robustness and scalability becomes extremely crucial to handle different business demands and customer preferences. The data framework of today requires high data processing capabilities and an integrated data platform accessible anywhere with top-notch security.

Share Market Live

NSE

To discuss the challenges of having a future-ready data framework and solutions, CNBCTV18.com along with Microsoft organised an enlightening panel discussion on “Redefining data estate modernization for a Future Ready enterprise”, comprising an elite set of panellists – Suchit Mishra, CIO and CISO, Dhani Stocks; Balakrishnan A, Executive Director, Geojit Financial; Ravindra Yadav, Director, Data Science, Meesho; Prashant Singh, CIO, Max Healthcare; Gajendra Gupta, Lead - Communications and Practices, Zensar Technologies; Ravi Hooda, Chief Digital Officer, Lifestyle Business, Raymonds Ltd.; Swapna Bapan, Senior Director, Data Analytics & AI, Microsoft and Mukul Singhal, CIO, Birlasoft Technologies.

Bridging the Gap Between Data & Decisions, A Comprehensive Narrative

A big challenge in achieving a robust data framework is limited data maturity. Organizations face a humongous issue in processing real-time unstructured data streams coming in from different sources, as most of them don't make business sense and are hard to combine for cohesive decision-making. This is where data estate modernization can help organizations in converting this seemingly unrelated data into business value and insights, necessary to make decisions. Swapna explained how this framework enables the right access for data to applications, people and devices, and brings about the ability to track that data to know where it is, who and who shouldn’t have access to it and what insights can be drawn to meet different business objectives for an organization.

Balakrishnan alluded to this point on how with the advent of multiple digital sources, legacy systems fell critically short in ensuring security, storage and data analysis, and added to the high cost of data management. This required organizations, especially those in transaction-based business, to undertake digital transformation. Using these systems, though, requires re-skilling and upskilling to create a modern data organization.

Digital Transformation is Essentially Data Transformation

Prashant pointed out how the healthcare sector got transformed through digital transformation enabled through connected systems. The emphasis on patient engagement through a unified data lake connected to different sources helped leverage the power of the cloud. Mukul opined that data is the new ‘oil’ for businesses to make the right decisions. AI/ML are important to derive critical insights and real-time business decisions using data coming from multiple heterogeneous systems. Ravindra added how adopting data lakes imbibed decision democratization in Meesho, generated additional business opportunities, revenue streams and demanded prediction at scale.

Contextualization and Flexibility is the Key

It is very easy to generalize how data modernization can be a panacea for all our business challenges. Gajendra emphasized how this one-size-fits-all data modernization approach does not work. The approach should be to scale and customize according to the priorities of an organization, its strategy, end-customer needs and business objectives.

According to him, three critical aspects ensured a successful data migration from a traditional warehouse to a more structured data lake and they include: optimizing the cost of the target landscape to establish appropriate return on investment, adopting the right security framework which is applicable globally, and scalable governance aspects to take care of end-to-end data management requirements.

Seconding this, Suchit held that data democratization is very important to establish data-driven analytics and decisions. This essentially means storing only value-adding data for organization consumption and preservation, considering the different stakeholders that the data is going to serve. The total cost of ownership goes a long way in determining the right and optimal approach, with the right data governance mechanism to guide the framework.

Since there is always a fine line between the total cost which goes into owning data, and profitability and business cases that the approach entails, Ravi rightly expressed the need to tread this fine line to succeed in a highly dynamic data-driven world. This approach emphasizes interactions happening between different data sources, sourced into a cloud lake and customized based on requirements.

Integrating Data Silos for a Data Lake

Data silos are the biggest impediments to a seamless, data-driven business approach. All the panellists agreed that such silos must be integrated across different organizational functions. Swapna emphasized this requires the discovery of all data and their sources. This, in turn, empowers the organization to facilitate end-to-end secure data frameworks such as role-based access controls, auto-archival processes, understanding of sensitive data and their annotation. It is also important to understand the behaviour of data in internal assets and external sources coming from social media. This helps establish an audit process and establish data governance throughout the entire lineage using tools.

Challenges in Migration to Data Lakes & the Way Forward

Dealing with the combination of structured-unstructured data, ensuring data security at different stages like encryption during transit and at rest is important to have a secure and robust data lake. According to Ravi, some of the challenges that legacy organizations face in this respect include defining which data is important, de-duplication, and skilling departments to handle data integration from different sources.

Some of the ways to handle these challenges include data normalization through innovative approaches, observability engines to monitor patterns, and ensuring data security in disparate systems.

The data journey is not a single step process. The panel signed off that the end-to-end data transmission should be tied to data governance mechanisms to enable the right access to the right people. It was agreed that an end-to-end audit mechanism will go a long way in ensuring a secure data framework. In the end, partnerships with like-minded agencies will ensure a smooth data journey!

This is a partnered post.