We have spent some time in the discussions learning about Big Data platforms
and tools, so I thought I would switch gears and start a discussion about
operationalizing AI/ML and Big Data solutions. As a former cloud operations
engineer and administrator, I can appreciate the level of effort and challenges
around operationalizing any concept.
One should have many questions to answer before moving forward with any
plan to produce a concept. Our agency likes to deploy a potential solution as a
Proof of Concept (POC) to understand a potential solution's moving parts. Our
agency likes to know what works well, what does not., and what is involved in
developing, deploying, and supporting the concept. We like to know how
receptive the targeted user community will be, how much the solution will
cost, and what our IT Service Desk needs to know to support the solution.
These are but a few questions that one probably needs to answer before
operationalizing AI/ML.
First, as we all know from our studies in the MSDA program here at UMGC,
data is the fuel that powers AI/ML models, much like gasoline is the fuel that
powers our automobiles. However, data quality is always a concern. If one
pumps low octane, ethanol-rich gasoline into the tank, one may not get the
optimal power one needs to get from origin to destination. The same goes for
poor quality in data: garbage in equals garbage out. As a budding data analyst,
one can appreciate how Mr. John Parkinson puts a twist on the whole "garbage
in/garbage out" idiom in his article on managing Big Data – "you have to solve
for garbage in/gold out and prevent gold in/garbage out" (Parkinson, 2021,
para. 4). Notice how Mr. Parkinson points out a desired state or garbage
in/gold out.
In data science, one seeks to explore, transform, prune, and clean from the
garbage pile so that one can find the golden egg. One wants to offer valuable
insights to the user community to increase profits, predict disease, reduce
customer churn, etc. To operationalize AI/ML with Big Data, one must
formulate a strategy for handling massive amounts of data and addressing
some earlier questions.
Second, one must often choose between business priorities. Does the business
need to maintain a real-time view of the data originating from multiple,
disparate sources with loose convergence? (Parkinson, 2021). Does the
business need to maintain a comprehensive, historical view of the data
available in modern data warehouse systems? (Parkinson, 2021). This priority
choice is a critical decision point since the decision often drives data storage
and acquisition architecture and the AI/ML operational plan. For example, an
AI/ML model that predicts a potential airplane crash in real-time would likely
need an entirely different AI/ML operational plan than an AI/ML model that
predicts customer churn for a popular big-box retailer. f f
A discussion of these two challenges may make one question why we need an
AI/ML operational plan. One must understand the AI/ML use cases one is
attempting to operationalize and AI/ML capabilities. Different AI/ML use
cases consist of different Big Data platforms, services, and toolchains to
support the use case. For example, a conversational (chat) bot that leverages
Natural Language Processing (NLP) to determine user intent based on
utterances would likely need a different operational plan than a supervised
AI/ML model that classifies customer sentiment from online customer
reviews.
The chatbot use case would need a website or channel to interact with it. Most
chatbots then need to communicate to an NLP service to identify user intent.
Once the NLP service determines user intent, the service must send one or
more responses back to the chatbot. The more questions the chatbot needs to
answer, the more data needs to be stored (and the data needs to be of high
quality so that the chatbot responds with the appropriate answer). One must
consider how to maintain different versions of the NLP model, which
determines user intent and when new versions of the NLP model are made
available to the chatbot, and hence the user community. Before deploying it to
production and documenting test results, one must consider testing the NLP
model for accuracy and completeness. One may need to adapt the chatbot
rapidly to address unanswered questions, incorrect answers, or service
performance and reliability that impact users. One may need a specific
Integrated Development Environment (IDE) to import C# programming
language libraries with various methods to support the solution.
The customer sentiment analysis would need to feed the text in one or more
customer reviews to a supervised AI/ML model that then labels the text based
on the trained model. Occasionally, one may need to retrain the model to
account for new words or slang in language that helps label the sentiment as
positive or negative. One must consider building and maintaining an AI/ML
pipeline that ingests the reviews, stores the data, and labels the data. One may
decide to train and evaluate the AI/ML model in a cloud-based notebook that
utilizes Python or R.
These are two examples of AI/ML use cases that can help one understand that
not all AI/ML operational plans are created equal. There are many factors and
nuances to consider when developing an AI/ML operational plan, so one must
proceed with caution and an eye toward the operating costs and value
generated for the organization. With that said, one can turn attention to
maturing operational plans to support various AI/ML with Big Data use cases.
One way to mature AI/ML operational plans is to adopt an MLOps approach
to support various AI/ML use cases. The chatbot use case may require an NLP
service managed by one cloud-services provider. In contrast, the sentiment
analysis may require a text analytics engine managed by another cloud-
services provider. Therefore, one should consider adopting an MLOps
approach not tied to any language, framework, platform, or infrastructure
(Machine Learning Operations, n.d.).
In conclusion, it is not enough to acquire data, build models, test model
accuracy, and explain model efficacy to stakeholders. One must also consider
deploying and supporting AI/ML models in production. I look forward to
discussing MLOps in more detail in future discussions but wanted to hear
what others think about MLOps and its importance in supporting AI/ML and
Big Data operations. Does anyone have experience with specific vendors with
a good MLOps story or solution? Has anyone in our class had experience
deploying actual AI/ML models in production? Can anyone shed light on
common pitfalls to avoid or lessons learned? I would like to hear about any
experience with MLOps.
References:
Machine Learning Operations (n.d.). MLOps.org. Retrieved from https://ml-
ops.org/
Parkinson, J. (2021, May 12). Managing Big Data: Six Operational
Challenges. CIO Insight. Retrieved from https://www.cioinsight.com/news-
trends/managing-big-data-six-operational-challenges/
less
