By Dr. Vivek Gupta, Founder & CEO · February 2020
Google's latest Meena Chatbot is astoundingly better than current chatbots. It is also a large dataset by itself comprising 2.6Bn parameters, and requires a humongous 300+ GB of data to train. Before this, Google's BERT algorithm used 20TB of data from the web combined with a 750 GB pre-training dataset and resulted in a model with 11 Bn parameters.
“The number of parameters in a model has gone from a few million in 2012 to many billions now and are likely to rise further.”
It is a significant advancement. As it rises further, this level of parameterized chatbots, large training datasets, and the inexact nature of intelligence will pose a new set of “single source of truth” management challenges to CIOs. The asymmetric benefits from AI — which lie in the abilities of networks to combine human-level intelligence with massively large sets of data — are impossible to ignore, and non-adoption is no longer an answer. The advancements are opening doors to new frontiers in research, knowledge usage, finding answers to questions, discoveries, customer communication, and massively personalized services, to name a few. As many companies are learning and racing to catch up, they realize that delays are expensive, and at times create an existential threat.
Under these imperatives, there are multi-fold organizational transformation challenges to adopt and deploy these technologies. It is messy, complicated, and rapidly changing. For CIOs, the problem is not just in adoption but in continuous management of these new technologies, which are unprecedented in their characteristics and nature. As these massive networks move into production and handle customer interactions, CIOs are facing the new challenge of managing a “single source of truthful intelligence.”
“Intelligence is not factual like data. CIOs need a new set of perjury laws for models to manage the challenge of differing truths.”
CIOs have traditionally been concerned about data and left the work of intelligence management to HR, as intelligence resided in humans. As intelligence migrates from humans to machines, and the need for managing training datasets and pre-trained networks grows, CXOs need to decide how to handle this exploding area. The challenges are not just operational but fundamental and philosophical.
“Will a push towards a ‘single source of truthful intelligence’ create regimented organisations and ‘GroupThink’?”
To tackle these challenges, CIOs not only need to engage their teams but include broader teams like L&D and HR as well in designing new rules and guides. This team will need to frame policies on many areas for a successful transition into AI-driven organizations. Some immediate areas where CIOs will need answers are:
Managing truth from competing models — As text models absorb knowledge from internal and external sources, they are likely to differ in their interpretations as well as factual accuracies. A policy for testing the truthfulness of models and a consistent framework for model governance is required.
Confidentiality testing — It will be challenging to judge privacy and confidentiality implications when organizations increasingly have to decide between learning effectiveness vs. privacy. How will organizations know that models don't know more than they should? How do you manage confidentiality vs. intelligence in a new generation of models where we don't know what is hidden in those 11 Bn parameters?
This is the beginning of a new role for CIOs. It is exciting, changing rapidly, and posing new questions.
This 2020 essay named the idea Softsensor now delivers — and it led directly to building Concord, our platform for governing AI agents and managing a single source of truthful intelligence in production. Originally published on LinkedIn →