While Big Data can come in many forms, in my opinion the most useful forms
of Big Data come from places with user-generated content, such as Twitter,
YouTube, Facebook, etc. The data collected from sites such as these are
largely unstructured and as such can not easily be quantified and stored into
neat boxes. Examples of this unstructured data are large excerpts of text, such
as a post or status update, or a video uploaded from a phone, such as one you
would find on TikTok or YouTube. With smaller or manually entered data, it's
easy to find connections and draw conclusions from data. But with Big Data,
the connections and conclusions are not so easy to find and take a more
complicated approach.
This is where AI comes into play. With the introduction of AI and other
learning algorithms, unstructured data can now be collected and interpreted
more efficiently. A natural language AI can be taught how to look for patterns
in a block of writing by using numerous examples from a big data set, and then
used to make further analysis on other blocks of writing. The large amounts of
unstructured data usually associated with "big data" can best be analyzed with
the help of an advanced a sufficiently taught AI.
Big Data already has a crucial role in society, and it's use and importance will
only increase, specifically in the areas of marketing and health. Big Data is
already being used in marketing; when you see advertisements on the internet
that seem to match exactly to something you were just looking up or just
talking about, that's an example of a company utilizing your search history or a
conversation it heard, pulling out valuable data, and targeting its ads to offer
you exactly what it thinks you want. As time goes on, and more data streams
become available for analysis, Big Data will be used more and more to offer
people what they want with little to no effort. In the realm of health, Big data
could be used in a similar way, by analyzing a persons patterns either through
online traffic, purchase patterns, or location data to determine potential health
issues that could be used to adjust a persons health insurance premiums.
As I don't work in a capacity to use data science, there's no real way for me to
leverage Big Data in my profession currently. However, being able to leverage
Big Data is as essential as any other kind of data. As the world becomes
increasingly connected, either through our phones, social media, of the Internet
of Things, Big Data is increasingly collectible, and not collecting this data is a
missed opportunity that can result in weaker business practices. While I can't
use Big Data in my day-to-day, my goal in learning about Big Data and
associated techniques is to take this knowledge and either apply it to my
current job to help improve things, or to take it to a new profession in the future.
Hadoop is a framework that utilizes the MapReduce algorithm created by
Google to build applications that are able to analyze and process large amounts
of unstructured or semi-structured data, which means it is perfect of Big Data.
This basically means that Hadoop is essential when building an application
that disseminates Big Data and finds value through connection and correlation.
Since data is useless if the data provides no value, Hadoop is a very important
tool in turning Big Data into a return on investment.
