Amidst the AI revolution, e27 presents a new series showcasing how organisations embrace AI in their operations.
Nick Eayrs serves as the Vice President of Field Engineering for Asia Pacific and Japan at Databricks.
Joining the company in January 2019, he currently leads the technical team of Data Engineers and Solutions Architects across the region. In his role, Eayrs offers thought leadership and guidance for implementing data and AI strategies at the C-level with major global customers.
In this edition, Eayrs shares how Databricks has embraced Artificial Intelligence.
Edited excerpts:
How do you perceive the AI revolution and its potential impact on your industry and workforce?
ChatGPT set off an awareness revolution last November when people could see and interact with AI, when it was already very much in our everyday lives – think Siri and your program recommendations on Netflix, among many others.
This sudden growing interest in Artificial Intelligence and large language models (LLMs) is reflected in what we have seen. The Databricks State of Data and AI report showed that the number of companies using LLMs has surged by 1,310 per cent between the end of November 2022 and the beginning of May 2023.
I’m excited about the AI revolution and its immense potential to make businesses and the workforce worldwide more productive and efficient. There is potential to discover lifesaving drugs quicker than ever with the help of AI; grocery retailers can reduce fresh produce wastage by correctly predicting the amount of fresh produce to stock up for different periods.
These are among the many other impactful AI use cases that improve our daily lives.
In what ways has your company embraced AI technologies to improve operational efficiency or enhance business processes?
Our company was founded because big data and AI are difficult problems.
Databricks is leveraging LLMs to build chatbots for our engineering teams — by training them with relevant proprietary data and manuals to understand our subject matter. Our internal chatbot is a helpful tool for our engineers to look for solutions that may otherwise take them much longer to solve.
On the creative front, our marketing teams are leveraging LLMs to help draft tweets so that our staff can quickly review the tweets.
Databricks has developed LakehouseIQ, a first-of-its-kind knowledge engine that continuously learns about your business, data, and relevant concepts. Most users will see this surfaced as a new assistant that helps them derive insights from their Databricks Lakehouse platform using natural language queries. It also provides intelligent search capabilities and allows for effective management/troubleshooting of user workflows.
LakehouseIQ also exposes all of these capabilities through an API so that you can build and power your own enterprise AI applications.
Can you share specific examples of how AI has been integrated into your workforce to streamline operations or drive innovation?
Beyond the specific examples that Databricks is using AI internally to make our workforce more productive, we also enable over 10,000 organisations all around the world with their data and AI:
- Financial institutions like Siam Commercial Banks use AI to modernise their loan application process in the financial services sector, offering instant loan approvals based on predictive analytics (transforming the manual evaluation, which used to take weeks).
- Car-sharing platforms like GetGo use AI to help with demand forecasting, fraud detection, and geospatial analytics in the transportation sector, optimising user experience.
- In the energy and utilities space, waste management companies like Cleanaway use AI to plan their route to deliver efficient waste and recycling services daily to millions of households and facilities.
What challenges or concerns did you encounter when implementing AI within your organisation, and how did you address them?
AI’s potential remains boundless. It is beneficial for organisations of all sizes because of its potential to provide value for internal stakeholders like employees and external stakeholders like customers but the key to AI is quality data.
Also Read: The value for biz lies in how humans, AI will enhance each other’s strengths: Mixpanel CEO
Our research with MIT Technology Review reveals that 72 per cent of the interviewed CIOs say that data is the biggest challenge for AI, and 68 per cent say unifying their data platform for analytics and AI is crucial. This reflects our conversations with organisations — the biggest challenge that enterprises struggle with is siloed infrastructure and disparate data platforms and tools, which are incompatible and challenging to integrate.
This is why Databricks pioneered the data Lakehouse, an open and unified data management architecture that combines the best of data lakes and data warehouses — so companies can effectively do both AI and BI on a single platform, maximising the value of AI.
How do you ensure transparency and uphold ethical considerations in using AI technologies within your organisation to mitigate privacy concerns?
Data privacy is a key concern for all companies intending to build their LLMs. Our customers’ first concern is this: ‘how can we build our own LLM models in-house without handing over our sensitive and proprietary data to a third party?’
That is why AI must be democratised so that every organisation — large or small, profit or non-profit — can benefit from the AI revolution while controlling how their data is used and keeping ownership of the value created.
Each organisation sits on a treasure trove — its data. This data only really has value when its business context is understood. This is why many organisations train LLM models in-house rather than handing their data over to third parties. With open-sourced LLM models that are now able to be used
companies can leverage these tools commercially to build their own LLMs on top of their data.
Also Read: AI has its advantages, but it can never fully replace humans: Asnawi Jufrie of SleekFlow
To do this, companies must have all their data in a unified platform like the Lakehouse. It enables businesses to carefully control their company-wide data and Artificial Intelligence development carefully, allowing them to better manage risks on one unified platform.
How do you ensure that AI technologies complement your workforce’s existing skills and expertise rather than replacing or displacing human workers?
AI is meant to enhance productivity and not replace workers. AI is not about job displacement but job augmentation.
In many instances, we will still want a human in the loop to oversee and check the output from AI while having AI do the mundane stuff more efficiently. AI is meant to solve specific business challenges and enhance productivity so employees can focus on their jobs more interesting, creative and high-value aspects.
How do you envision the future collaboration between humans and AI? What role do you see AI playing in augmenting human capabilities?
We partnered with MIT Technology Review and released a report on CIO perspectives on generative AI, and this is one of the many insights from CIOs across the globe:
- “We internally view AI/ML as being a helper, truly helping our people, and then allowing them to spend more time on other value-added activities.” — Cynthia Stoddard, Senior Vice President and Chief Information Officer, Adobe.
In many ways, Artificial Intelligence is not meant to perform extremely complicated work that requires a lot of planning fully automatically. Conversely, I don’t think there’s anyone whose job is just the super simple stuff that a language model can do.
What advice would you give to founders looking to leverage AI in their workforce?
There is a saying, ‘Garbage in, garbage out’, underscoring a vital principle in AI — that the quality of the given data determines the quality of AI’s output. The resulting AI models will likely be defective if the data is flawed, biased, or incomplete.
More importantly, if data is stored in silos and in disparate systems that aren’t compatible, companies will not be able to unlock the potential of the data and Artificial Intelligence fully.
This is where Lakehouse architecture comes in, an open and unified platform for data, analytics and AI.
—
Fundraising or preparing your startup for fundraising? Build your investor network, search from 400+ SEA investors on e27, and get connected or get insights regarding fundraising. Try e27 Pro for free today.
The post AI is not about job displacement but job augmentation: Nick Eayrs of Databricks appeared first on e27.