The world is awash in data. From social media interactions and financial transactions to sensor readings in factories and logistics tracking, the volume and velocity of data continue to explode. This presents both challenges and opportunities for businesses. How can they harness this data stream to identify and act on actionable insights, optimise operations, deliver exceptional customer experiences with the best cost?
A recent roundtable discussion organised by Volt Active Data, in collaboration with YourStory, on the sidelines of Tech Leaders Conclave, brought together industry experts to explore the transformative potential of real-time data analysis.
The roundtable discussion, “Building scalable and responsive applications for a hyper-connected world,” featured a distinguished panel of industry veterans. Shivani Muthanna, Director of Strategic Partnerships and Content at YourStory Media, moderated the discussion.
The participants brought expertise from a wide range of sectors. From the world of payments, Abhishek Madan, Vice President of Product at Paytm, and Prerit Khandelwal, Associate Director of Analytics at Razorpay, shared their insights. Sherene Kuruvilla, Associate Director of Product Design at Whatfix, represented design thinking. Jayaprakash Kavala, Chief Product Officer at Clari5, offered his perspective on building responsive applications.
The discussion also delved into the application of AI and automation in various industries. Srivatsa Subanna, Head of Intelligent Automation & AI at Axis Bank, shared his experiences. Mrinal Rai, Cofounder and CPO of Intugine, and Rohit Mittal, Founder, Chairperson, and CEO of Bert Labs, brought their expertise in AI solutions to the table. Bhavana Mittal, Cofounder, Executive Director, and Chief Growth Officer at Bert Labs, also participated in the discussion.
Other industry leaders who contributed their knowledge included Karan Makhija, Cofounder and CEO of Intellicar (logistics), Vikas Anand, Director of Engineering at Yubi (formerly CredAvenue) (data security); Ram Chandra, CIO of Systemantics, and Mohan Gupta, Sr Director of Product at Sharechat (social media).
Finally, the discussion benefitted from the technical expertise of Fahad Khan, Sales Director – APAC for Volt Active Data, and Biplab Banerjee, Principal Solution Architect at Volt Active Data.
While challenges in areas like data readiness and privacy were acknowledged, the overarching sentiment was one of optimism. Real-time data AI and IoT are poised to revolutionise industries from finance to manufacturing, logistics, and social media.
A robust data infrastructure is the foundation for leveraging real-time data. Panelists emphasised the need for high-speed data processing solutions to handle the ever-growing information influx. This includes not only the data itself but also tools to understand how applications are being used and how users are behaving. These insights are crucial for optimising the user experience and ensuring applications function effectively.
Value of Processing data within 10 milliseconds: Applications and challenges
The discussion delved into the specific applications of processing transactional data in various sectors. In the financial services industry, for instance, real-time data empowers institutions to detect fraud, create a personalised customer experience, and ensure regulatory compliance. Traditional core banking systems are built to manage the highly valuable financial transactions. Post-processing of data helps identify fraud or new business opportunities but loses out on opportunities to prevent it or provide the right offer. Experts suggest a shift towards microservices-based architectures that facilitate processing of in-flight transactions to prevent fraud or give the right offer without impacting the actual transactions.
Social media platforms can leverage data processing in 10 milliseconds to personalise recommendations and enhance user experiences. However, keeping pace with evolving user behaviour and ensuring data accuracy for models that rely on real-time information can be challenging.
Managing and analysing massive datasets in real time presents a unique set of hurdles for industries like manufacturing and retail. Organisations need to adopt modern systems that process the data, extract the value, and enable faster business operations. Additionally, identifying and reacting to anomalies in real time is crucial for optimising operations and preventing costly disruptions.
The Indian context: Digital transformation in banking
The discussion also explored the specific challenges faced by the Indian banking sector. Regulatory restrictions and limited investment in digital banking technologies are hindering adoption. Building strong tech teams and fostering a culture of data experimentation are critical steps for Indian banks to keep pace with the global shift towards digitalisation.
However, there are also positive developments. Unlike their US counterparts, who still rely on legacy systems, Indian banks are exploring and embracing new-gen data processing solutions and building microservices-driven architectures to cater to consumers’ growing demand for digital convenience.
The power of real-time processing
The final leg of the discussion focused on the importance of low-latency data processing. Processing data as it is generated empowers businesses to prevent fraud, personalise customer experiences, and optimise operations in real time. However, integrating low-latency data processing into existing workflows, managing high-velocity data streams, and balancing cost with performance optimisation are vital challenges that must be addressed.
Solutions like in-memory processing tools offer powerful capabilities for capturing and processing data. Additionally, Super fast processing engines like Volt Active Data provides the combination of Streaming, Relational Caching, and Complex Processing with Strong ACID guarantees and deployable at Cloud/Edge, enabling anomaly detection to fraud prevention
