We have spent some time in the discussions learning about Big Data
platforms and tools, so I thought I would switch gears and start a discussion
about operationalizing AI/ML and Big Data solutions. As a former cloud
operations engineer and administrator, I can appreciate the level of effort
and challenges around operationalizing any concept.
One should have many questions to answer before moving forward with any
plan to produce a concept. Our agency likes to deploy a potential solution as
a Proof of Concept (POC) to understand a potential solution's moving parts.
Our agency likes to know what works well, what does not., and what is
involved in developing, deploying, and supporting the concept. We like to
know how receptive the targeted user community will be, how much the
solution will cost, and what our IT Service Desk needs to know to support
the solution. These are but a few questions that one probably needs to
answer before operationalizing AI/ML.
First, as we all know from our studies in the MSDA program here at UMGC,
data is the fuel that powers AI/ML models, much like gasoline is the fuel
that powers our automobiles. However, data quality is always a concern. If
one pumps low octane, ethanol-rich gasoline into the tank, one may not get
the optimal power one needs to get from origin to destination. The same
goes for poor quality in data: garbage in equals garbage out. As a budding
data analyst, one can appreciate how Mr. John Parkinson puts a twist on the
whole "garbage in/garbage out" idiom in his article on managing Big Data –
"you have to solve for garbage in/gold out and prevent gold in/garbage out"
(Parkinson, 2021, para. 4). Notice how Mr. Parkinson points out a desired
state or garbage in/gold out.
In data science, one seeks to explore, transform, prune, and clean from the
garbage pile so that one can find the golden egg. One wants to offer valuable
insights to the user community to increase profits, predict disease, reduce
customer churn, etc. To operationalize AI/ML with Big Data, one must
formulate a strategy for handling massive amounts of data and addressing
some earlier questions.
Second, one must often choose between business priorities. Does the
business need to maintain a real-time view of the data originating from
multiple, disparate sources with loose convergence? (Parkinson, 2021). Does
the business need to maintain a comprehensive, historical view of the data
available in modern data warehouse systems? (Parkinson, 2021). This
priority choice is a critical decision point since the decision often drives data
storage and acquisition architecture and the AI/ML operational plan. For
example, an AI/ML model that predicts a potential airplane crash in real-
time would likely need an entirely different AI/ML operational plan than an
AI/ML model that predicts customer churn for a popular big-box retailer. g g
A discussion of these two challenges may make one question why we need
an AI/ML operational plan. One must understand the AI/ML use cases one
is attempting to operationalize and AI/ML capabilities. Different AI/ML use
cases consist of different Big Data platforms, services, and toolchains to
support the use case. For example, a conversational (chat) bot that leverages
Natural Language Processing (NLP) to determine user intent based on
utterances would likely need a different operational plan than a supervised
AI/ML model that classifies customer sentiment from online customer
reviews.
The chatbot use case would need a website or channel to interact with it.
Most chatbots then need to communicate to an NLP service to identify user
intent. Once the NLP service determines user intent, the service must send
one or more responses back to the chatbot. The more questions the chatbot
needs to answer, the more data needs to be stored (and the data needs to be
of high quality so that the chatbot responds with the appropriate answer).
One must consider how to maintain different versions of the NLP model,
which determines user intent and when new versions of the NLP model are
made available to the chatbot, and hence the user community. Before
deploying it to production and documenting test results, one must consider
testing the NLP model for accuracy and completeness. One may need to
adapt the chatbot rapidly to address unanswered questions, incorrect
answers, or service performance and reliability that impact users. One may
need a specific Integrated Development Environment (IDE) to import C#
programming language libraries with various methods to support the
solution.
The customer sentiment analysis would need to feed the text in one or more
customer reviews to a supervised AI/ML model that then labels the text
based on the trained model. Occasionally, one may need to retrain the model
to account for new words or slang in language that helps label the sentiment
as positive or negative. One must consider building and maintaining an
AI/ML pipeline that ingests the reviews, stores the data, and labels the data.
One may decide to train and evaluate the AI/ML model in a cloud-based
notebook that utilizes Python or R.
These are two examples of AI/ML use cases that can help one understand
that not all AI/ML operational plans are created equal. There are many
factors and nuances to consider when developing an AI/ML operational
plan, so one must proceed with caution and an eye toward the operating
costs and value generated for the organization. With that said, one can turn
attention to maturing operational plans to support various AI/ML with Big
Data use cases.
One way to mature AI/ML operational plans is to adopt an MLOps approach
to support various AI/ML use cases. The chatbot use case may require an
NLP service managed by one cloud-services provider. In contrast, the
sentiment analysis may require a text analytics engine managed by another
cloud-services provider. Therefore, one should consider adopting an MLOps
approach not tied to any language, framework, platform, or infrastructure
(Machine Learning Operations, n.d.).
In conclusion, it is not enough to acquire data, build models, test model
accuracy, and explain model efficacy to stakeholders. One must also
consider deploying and supporting AI/ML models in production. I look
forward to discussing MLOps in more detail in future discussions but
wanted to hear what others think about MLOps and its importance in
supporting AI/ML and Big Data operations. Does anyone have experience
with specific vendors with a good MLOps story or solution? Has anyone in
our class had experience deploying actual AI/ML models in production? Can
anyone shed light on common pitfalls to avoid or lessons learned? I would
like to hear about any experience with MLOps.
References:
Machine Learning Operations (n.d.). MLOps.org. Retrieved from https://ml-
ops.org/
Parkinson, J. (2021, May 12). Managing Big Data: Six Operational
Challenges. CIO Insight. Retrieved from https://www.cioinsight.com/news-
trends/managing-big-data-six-operational-challenges/
less
