We have spent some time in the discussions learning about Big Data
platforms and tools, so I thought I would switch gears and start a
discussion about operationalizing AI/ML and Big Data solutions. As a
former cloud operations engineer and administrator, I can appreciate the
level of effort and challenges around operationalizing any concept.
One should have many questions to answer before moving forward with
any plan to produce a concept. Our agency likes to deploy a potential
solution as a Proof of Concept (POC) to understand a potential solution's
moving parts. Our agency likes to know what works well, what does not.,
and what is involved in developing, deploying, and supporting the
concept. We like to know how receptive the targeted user community will
be, how much the solution will cost, and what our IT Service Desk needs
to know to support the solution. These are but a few questions that one
probably needs to answer before operationalizing AI/ML.
First, as we all know from our studies in the MSDA program here at
UMGC, data is the fuel that powers AI/ML models, much like gasoline is
the fuel that powers our automobiles. However, data quality is always a
concern. If one pumps low octane, ethanol-rich gasoline into the tank, one
may not get the optimal power one needs to get from origin to destination.
The same goes for poor quality in data: garbage in equals garbage out. As
a budding data analyst, one can appreciate how Mr. John Parkinson puts a
twist on the whole "garbage in/garbage out" idiom in his article on
managing Big Data – "you have to solve for garbage in/gold out and
prevent gold in/garbage out" (Parkinson, 2021, para. 4). Notice how Mr.
Parkinson points out a desired state or garbage in/gold out.
In data science, one seeks to explore, transform, prune, and clean from the
garbage pile so that one can find the golden egg. One wants to offer
valuable insights to the user community to increase profits, predict
disease, reduce customer churn, etc. To operationalize AI/ML with Big
Data, one must formulate a strategy for handling massive amounts of data
and addressing some earlier questions.
Second, one must often choose between business priorities. Does the
business need to maintain a real-time view of the data originating from
multiple, disparate sources with loose convergence? (Parkinson, 2021).
Does the business need to maintain a comprehensive, historical view of
the data available in modern data warehouse systems? (Parkinson, 2021).
This priority choice is a critical decision point since the decision often
drives data storage and acquisition architecture and the AI/ML operational
plan. For example, an AI/ML model that predicts a potential airplane
crash in real-time would likely need an entirely different AI/ML
operational plan than an AI/ML model that predicts customer churn for a
popular big-box retailer. k k
A discussion of these two challenges may make one question why we
need an AI/ML operational plan. One must understand the AI/ML use
cases one is attempting to operationalize and AI/ML capabilities. Different
AI/ML use cases consist of different Big Data platforms, services, and
toolchains to support the use case. For example, a conversational (chat)
bot that leverages Natural Language Processing (NLP) to determine user
intent based on utterances would likely need a different operational plan
than a supervised AI/ML model that classifies customer sentiment from
online customer reviews.
The chatbot use case would need a website or channel to interact with it.
Most chatbots then need to communicate to an NLP service to identify
user intent. Once the NLP service determines user intent, the service must
send one or more responses back to the chatbot. The more questions the
chatbot needs to answer, the more data needs to be stored (and the data
needs to be of high quality so that the chatbot responds with the
appropriate answer). One must consider how to maintain different versions
of the NLP model, which determines user intent and when new versions
of the NLP model are made available to the chatbot, and hence the user
community. Before deploying it to production and documenting test
results, one must consider testing the NLP model for accuracy and
completeness. One may need to adapt the chatbot rapidly to address
unanswered questions, incorrect answers, or service performance and
reliability that impact users. One may need a specific Integrated
Development Environment (IDE) to import C# programming language
libraries with various methods to support the solution.
The customer sentiment analysis would need to feed the text in one or
more customer reviews to a supervised AI/ML model that then labels the
text based on the trained model. Occasionally, one may need to retrain the
model to account for new words or slang in language that helps label the
sentiment as positive or negative. One must consider building and
maintaining an AI/ML pipeline that ingests the reviews, stores the data,
and labels the data. One may decide to train and evaluate the AI/ML
model in a cloud-based notebook that utilizes Python or R.
These are two examples of AI/ML use cases that can help one understand
that not all AI/ML operational plans are created equal. There are many
factors and nuances to consider when developing an AI/ML operational
plan, so one must proceed with caution and an eye toward the operating
costs and value generated for the organization. With that said, one can
turn attention to maturing operational plans to support various AI/ML with
Big Data use cases.
One way to mature AI/ML operational plans is to adopt an MLOps
approach to support various AI/ML use cases. The chatbot use case may
require an NLP service managed by one cloud-services provider. In
contrast, the sentiment analysis may require a text analytics engine
managed by another cloud-services provider. Therefore, one should
consider adopting an MLOps approach not tied to any language,
framework, platform, or infrastructure (Machine Learning Operations,
n.d.).
In conclusion, it is not enough to acquire data, build models, test model
accuracy, and explain model efficacy to stakeholders. One must also
consider deploying and supporting AI/ML models in production. I look
forward to discussing MLOps in more detail in future discussions but
wanted to hear what others think about MLOps and its importance in
supporting AI/ML and Big Data operations. Does anyone have experience
with specific vendors with a good MLOps story or solution? Has anyone
in our class had experience deploying actual AI/ML models in production?
Can anyone shed light on common pitfalls to avoid or lessons learned? I
would like to hear about any experience with MLOps.
References:
Machine Learning Operations (n.d.). MLOps.org. Retrieved from
https://ml-ops.org/
Parkinson, J. (2021, May 12). Managing Big Data: Six Operational
Challenges. CIO Insight. Retrieved from https://www.cioinsight.com/news-
trends/managing-big-data-six-operational-challenges/
less
