Assignment 1: Research Sources - AU ENG102 - Use attachments for reference

Running Head: QUALITY BIG DATA PROPOSAL 1

Quality Big Data

James Bates

Prof. Matt Beardmore

Composition II | ENG 102

November 21, 2018

Thesis

Roots of transparency in government and business dealings can be traced back to democratic societies and the Freedom of Information Act. The private sector, public sector, and government are currently all trying to this new concept in order to establish a change in a new model. It has been believed that transparency is crucial in strengthening democracy and promoting efficiency in government and businesses (Wamba et al, 2015). The information should be shared qualitatively objectively aimed by building informative websites or mobile applications. However, the phrase information is weakness comes to play since big data not only brings more understanding and benefits but it is also alarming challenges. As a result, more research is being done on the usability of data so that the transparency will result in improving the public sector. However, less concern has been assigned to intricacy in data gaining, its preparation and quality assurance. Several challenges are on the run-in managing data quality and reviewing good management practices would maximize big data usability. Challenges associated with managing big data quality and governance have discussed the paper and how these challenges could be managed to deliver usable data.

Data Quality

Data quality management involves how data is profiled and tools that institute data policy compliance. Profiling is necessary for analyzing, validating and monitoring data. To examine quality several dimensions should be checked such as accuracy, completeness, consistency, currency, and uniqueness (Saha & Srivastava, 2014). The data values should be correct and specific elements assigned their values and are up to date. The uniqueness of data refers to the representation of each item only once within the data sets. A company or the government can institute measurement and validation techniques to check on data quality in the various dimensions. However, the culture of big data concentrates most on the aggregate’s outputs from the analysis of the large datasets. Due to the possibility of such data originating from outside rather than inside the organization, data governance and quality assurance becomes a problem. Applying proactive data quality techniques such as profiling standardization, record matching and cleansing are likely to ultimate in quality data (Saha & Srivastava, 2014).

Challenges associated with attaining quality data.

The need to integrate data from various sources calls for applying corrections to data sets. However, when dealing with large data sets corrections are not easily done. As stated, the information gathered is not only from within the company’s firewall but also from the outside. Among the challenges encountered in data quality governance include data interpretation, huge volume, asserting tools, consistency and data longevity.

a) Data interpretation: It is associated with the understanding of various data values to deduce meaning. However, some data are not directly linked to a company and possibly do not have the same meaning as of those sourced from within. This brings great data conflict and would bring inaccuracies that reduce the quality of data (Saha & Srivastava, 2014).

b) Data volume: where huge data are collected, it becomes difficult to sort them out or analyze them. In most situation, the systems available in a given company cannot accommodate the analysis of such data. As a result, some information may be ignored which compromises data quality. The other option would be overwhelming their ability in an attempt to conform to expectation. However, if the capacity of a tool such as SQL queries is exceeded, its approach to meeting quality will as well be compromised (Cai, & Zhu, 2015.

c) Asserting Tools: even if a company was so consistent in its information, controlling its quality becomes very challenging when different parties are involved in editing or contributing their views. An organization will efficiently monitor what comes from within their administrative boundaries because they have their assertive tools but would be impossible to monitor additional information. An example of such a situation is always ensuring that tweeter feeds are off spelling errors (Cai, & Zhu, 2015).

d) Maintaining consistency: maintaining consistency when dealing with various sources is difficult. Allowing other parties to edit information may result in some of the data being erased. Cleansing data in an attained source becomes unwise when those data would bring data inconsistencies with the original data. Such kind of inconsistency would forbid consistency and traceability that could finally result in questionable application results (Cai, & Zhu, 2015).

Motivators for quality data.

Introducing proactive data quality methods are and techniques are motivated by various improvement factors that shape how data quality techniques are applied. To solicit data expectations from end users would aid in data quality assurance. These guidelines should come from the users so they are directly engaged in DQ processes to enhance assurance. Despite the limitation on assertive controls, datasets should also be monitored in order to validate the acquiescence to data expectations (Wamba et al, 2015). Proactive monitoring allows for a room to alert stewards whenever the data is not meeting expectations and as well as trigger authoritarian action. In addition, to ensure quality data, reference data management should be up to date. Standards for frequently use reference domains can be established and focus on the growth of entity identification and linkage. Governance should also be established in that, data quality dimensions should be well stated, rules defined since they are essential in soliciting expectations.

Strategic plan for quality big data.

The stated motivating factors call for a potential strategic plan to meet quality big data. Strategies channeled towards big data quality include understanding what qualities data is, metadata consistency, technology alignment and balancing validation and balancing (Kwon et al, 2014).

a) Quality data implies data that meets the data dimension mention above. The various understanding of quality data on internal and external sources should be distinct and when applied to structured and unstructured data.

b) Metadata consistency is a collaborative approach that ensures consistency in referencing domains. These metadata platforms would provide an environment for apprehending business terms, and define data elements (Hazen et al, 2014).

c) Balancing validation and cleansing. Members of big data users should be engaged in the identification of collection rules governing data quality. By this, a decision on when to standardize and when not to, how to carry out cleansing and other forms of enchantments are made easier contrary to using data controls for validation and monitoring.

d) Technology alignment: in no way should the applied data quality and governance tool inhibits the performance of big data platform. Any tool used, therefore, should be in alignment with the routine applied in big data platforms. The data quality tools should be capable of adapting to the big data environment.

Characteristics of Information Management Tools

From the strategic plan discussed, it is possible to discuss in details how Information Management Tools for big data should be. One key characteristic is an integrated platform. An organization should look for vendors who meet an information management that is the end to end process. cobbling different vendors would illusion the necessity of the best breed while this might not necessarily work out and might diminish the perceived benefits (Hazen et al, 2014). The tools should allow a business user engagement so that users are given access to manipulate, cleanse and allow an interface between the data clusters. Another quality is push-down data quality. The tool should employ big data technology so that its performance is not constrained. As a result, an organization should look for vendors whose tools can be pushed down to profiling, standardization and validation data methods. The management tool should also reduce the need for movement of data. Reducing data movements has a direct impact on positive performance (Kwon et al, 2014). The tools should allow manipulation or validation of data without necessarily moving it to a different location. It is worthwhile to also look for an information management tool that facilitates the adoption of data governance techniques. This is likely to drive involvement of business users to data quality expectations. As a result, efficiency in standardization and validating big data will be enabled.

Conclusion

Though it is challenging in ensuring big data quality, it is as well possible. It not only requires effective tools but also a collaboration with other users. With quality data, maximum use of it can be attained like easiness in data sharing, useful in decision making, creating trust among others. The paper generally has shown the challenges that makes quality big data unreachable, laid strategies to improve the qualities and suggested characteristics of a viable information technology tool.

References

Wamba, S. F., Akter, S., Edwards, A., Chopin, G., & Gnanzou, D. (2015). How ‘big data can make big impact: Findings from a systematic review and a longitudinal case study. International Journal of Production Economics, 165, 234-246.

Saha, B., & Srivastava, D. (2014, March). Data quality: The other face of big data. In Data Engineering (ICDE), 2014 IEEE 30th International Conference on (pp. 1294-1297). IEEE.

Cai, L., & Zhu, Y. (2015). The challenges of data quality and data quality assessment in the big data era. Data Science Journal, 14.

Hazen, B. T., Boone, C. A., Ezell, J. D., & Jones-Farmer, L. A. (2014). Data quality for data science, predictive analytics, and big data in supply chain management: An introduction to the problem and suggestions for research and applications. International Journal of Production Economics, 154, 72-80.

Kwon, O., Lee, N., & Shin, B. (2014). Data quality management, data usage experience and acquisition intention of big data analytics. International Journal of Information Management, 34(3), 387-394.