Smart City Urban Mobility Analysis using Hadoop and Predictive AI

1

Learner declaration I certify that the work submitted for this assignment is my own and research sources are fully acknowledged.

DEGREE: BSc Computer Science and Digitisation

Module: Big Data Analytics using AI ___________________________________________________________ Assignment Title:

Assignment Type: Report Word Limit: 3000 words (+/- 300) Weighting: 100% Issue Date: 4/9/2025 Submission Date: 30/9/2025 Feedback Date: 21/10/2025

Plagiarism: When submitting work for assessment, students should be aware of the InterActive/Canvas guidance and regulations in concerning plagiarism. All submissions should be your own, original work. Please note that you must not submit the same assignment for two different modules within your course. You must submit an electronic copy of your work. Your submission will be electronically checked.

Harvard Referencing: The Harvard Referencing System must be used. The Wikipedia, UKEssays.com or similar websites must not be used or referenced in your work.

Student signature: Date:

SmartCity UrbanMobilityAnalysisusingHadoopand Predictive AI

Introduction

2 Phase 2: Implementation & Analysis (LO 3) - 50% of Total Grade

LO1. Demonstrate the understanding of basic concepts of Big Data, its importance and need in business context. LO2. Explain the various components of Hadoop and HFDS along with their role in the Big Data ecosystem. LO3. Summarize the learning on Big Data analytics using Yarn, HDFS and MapReduce. Assessment Criteria: Weighting 100%

3000 words Task Description: You are a Data Engineer tasked with designing and implementing a proof- of- concept Big Data analytics solution for a city's transport authority.

Scenario: The city council wants to analyse urban mobility patterns using data from road sensors, taxi trips, or public transit records. The objective is to identify congestion hotspots, understand their causes, and predict future traffic patterns to enable proactive traffic management and better infrastructure planning.

Phase 1: Conceptual Design & Architecture (LO 1, LO 2) - 20% of Total Grade

The goal of this assignment is to provide you with hands-on experience in designing and implementing a Big Data analytics solution that incorporates a predictive AI component. You will address a hypothetical smart city challenge by using the Hadoop ecosystem to process large-scale data and derive actionable insights for urban planning. This assignment requires you to design a solution using Hadoop, HDFS, YARN, and MapReduce to analyse transportation data. The final step involves using the processed data to train a simple predictive model, thereby connecting Big Data processing with AI applications. This will help you understand the end-to-end pipeline from raw data to business intelligence in a modern context. Learning Outcomes:

1. Business Context and Problem Statement (5%) • •

Describe the smart city scenario, focusing on the challenges of urban mobility. Define a clear problem statement. For example: "To analyse historical traffic data to predict the likelihood of traffic congestion at key intersections based on time of day and day of the week". Explain how solving this problem provides tangible value to the city (e.g., reduced commute times, lower pollution, improved public safety).

•

2. Hadoop Ecosystem and Architecture (15%) • • • •

Explain why a Big Data approach is necessary for this scenario. Identify the roles of HDFS, YARN, and MapReduce in your proposed solution. Justify your choice of these components for the defined problem. Create a clear architectural diagram illustrating how data flows from source to HDFS, is processed by MapReduce managed by YARN, and is then used for analysis.

3

Submission Guidelines:

1. Data Acquisition & Preparation (5%)

• Select a suitable public dataset representing urban mobility (e.g., taxi trip records, traffic sensor data). Platforms like Kaggle or city-specific open data portals are good sources. Describe the dataset's structure, size, and key attributes relevant to your problem statement.

•

2. Hadoop Environment and Data Ingestion (10%) • Set up a local single-node Hadoop cluster (e.g., using the official Apache Hadoop

binaries or a Docker image). Document the key steps of your setup process. Load your chosen dataset into HDFS. Provide the commands and screenshots showing the data successfully stored in HDFS.

• •

3. Data Processing with MapReduce (20%) • Write a MapReduce program in Java or Python to process the data. Your program must perform data cleaning and feature engineering to prepare it for the predictive model. Example tasks: calculate average trip duration per route, count vehicle flow per hour, or identify other relevant features from the raw data. Explain the logic of your Mapper and Reducer classes and include the well- commented source code in your report's appendix.

•

•

4. Predictive Analysis and Visualization (15%) • •

Export the processed data from HDFS. Use the processed data to train a simple predictive model. You can use a library like Scikit-learn in Python to build a classification or regression model that addresses your problem statement. Analyze and interpret the output of your MapReduce job and your predictive model. Create meaningful visualizations (e.g., graphs showing congestion by time of day, a confusion matrix for your model) to present your findings.

• •

Phase 3: Reflection and Documentation (LO 1, LO 2, LO 3) - 30% of Total Grade 1. Critical Reflection (10%)

•

•

Reflect on the key challenges you encountered during implementation (e.g., data cleaning, debugging MapReduce, model accuracy) and how you addressed them. Critically discuss the performance and scalability of your MapReduce solution. Could it be optimized (e.g., by using a Combiner)?

2. Final Report Documentation (20%) •

•

•

Compile a detailed, professional report of no more than 3000 words documenting the entire project. The report must be well-structured with clear headings, proper grammar, and academic language. Ensure all phases (Conceptual Design, Implementation, Reflection) are thoroughly covered, including diagrams, code snippets, commands, and visualizations to support your work. Include a bibliography using the Harvard referencing style. •

4

•

•

•

•

•

• •

•

Document Format: Submit your assignment as a single document following theBSBI assignment template provided in Canvas. Writing Quality: Ensure clear and concise writing with proper grammar and spelling. Use headings and subheadings to organize your work logically according to the tasks outlined above. Visuals: Include visuals like diagrams (process flow, conceptual model sketches), tables (data assumptions, results), and graphs (simulation output) where appropriate to enhance understanding. Task Coverage: Address each part thoroughly, demonstrating your understanding of Big Data concepts and their application to the business scenario. Implementation Details: Provide relevant examples and details of your model implementation, including code snippets, commands, and calculations. Referencing Style: Use Harvard referencing style for your bibliography. Discussion: Discuss your findings, insights, and the implications of your recommendations. Reflect on the challenges faced and how you overcame them. Submission: Submit your assignment electronically (Canvas) by the specified deadline.

5

GUIDANCE ON ASSESSMENT Allmaterials must be properly referenced under Harvard conventions. The length required is 3000 words with tasks equally weighted. The writing style should be formal academic /

report writing style with in-text referencing to support your comments and observations. Originality, quality of argument and good structure are required. The report should demonstrate

sound understanding and ability to apply knowledge and theory of Simulation Techniques. Additional marks being awarded for juxtaposition and insight of issues.

Grading Criteria

Generic Criteria Knowledge of contexts, concepts, technologies and processes

The extent to which:

relevant contextual or theoretical issues are identified, defined and described historical or contemporary practices are identified, defined and described appropriate technologies, methods and processes are identified defined and described

90 - 100 80 - 89 70 - 79 60 - 69 50 - 59 40 - 49 30 - 39 0-29

Le ve

l 6

Understanding through application of knowledge

The degree to which research methods are demonstrated: relevant knowledge and information is compared, contrasted, manipulated, translated and interpreted knowledge and information is selected, analysed, synthesized and evaluated in order to generate creative ideas, practices, solutions, arguments or hypotheses

Exceptional ability to produce a range of creative practices and to critically evaluate them in a wider context, generating sustainable arguments and highly effective and individual results

Exceptional application of a range of research methodologies to projects and problems and hypotheses, with evidence of highly focused independent thought and some new insights into the subject

Exceptional breadth and depth of knowledge of contextual and theoretical issues, some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Exceptional knowledge of a range of relevant specialist techniques and processes

Systematic and thorough application of a range of research methodologies to projects and problems and hypotheses, with evidence of highly focused independent thought and some new insights into the subject

Outstanding ability to produce a range of creative practices and to critically evaluate them in a wider context , generating sustainable arguments and highly effective and original results

Outstanding breadth and depth of knowledge of contextual and theoretical issues, some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Extensive knowledge of a range of relevant specialist techniques and processes

Rigorous application of a range of research methodologies to projects , problems and hypotheses with evidence of highly focused independent thought and critical analysis

Strong ability to produce a range of creative practices and to critically evaluate them in a wider context, generating sustainable arguments and highly effective results

A breadth and depth of knowledge of contextual and theoretical issues, some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Significant knowledge of a range of relevant specialist techniques and processes

Confident ability to apply a range of research methodologies to projects, problems and hypotheses with clear evidence of independent thought and critical analysis

Confident knowledge of a range of relevant specialist techniques and processes

Strong ability to produce a range of creative practices and to evaluate them in a wider context , generating effective results

Confident knowledge of a range of contextual and theoretical issues, some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Sound knowledge of a range of relevant specialist techniques and processes

Sound ability to apply a range of research methodologies to projects, problems and hypotheses and to demonstrate independent thought and critical analysis

Familiar with a range of contextual and theoretical issues, at least some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Sound ability to produce arange of creative practices and to evaluate them in a wider context, generating effective results

Competent ability to produce a range of creative practices and evaluate them in a wider context to generate effective results

Familiar with a range of contextual and theoretical issues and their relationship to a range of historical and contemporary practices

Adequate knowledge of a range of relevant specialist techniques and processes

Competent ability to apply a range of research methodologies to projects, problems and hypotheses with some element of independent thought and critical analysis

Limited knowledge of a range ofrelevant specialist techniques and processes

Ability to apply a limited range of research methodologies to projects, problems and hypotheses with little evidence of independent thought or critical analysis

Limited ability to produce a range of creative practices and to evaluate them in a wider context to generate effective results

Some knowledge of a range of contextual and theoretical issues and their relationship to arange of historical and contemporary practices

Limited knowledge of contextual and theoretical issues and their relationship to a range of historical and contemporary practices

No significant knowledge of a range of relevant specialist techniques or processes No significant ability to apply research methodologies to projects, problems and hypotheses, and no evidence of independent thought or critical analysis

No significant ability to produce a range of creative practices or to evaluate them in a wider context to generate effective results

6

Application of technical and professional skills

The degree to which:

appropriate materials and media are selected, tested and utilised to realise and present ideas and solutions

appropriate technologies, methods and processes are demonstrated

transferable, professional skills are effectively demonstrated

self management and independent learning are demonstrated Exceptional ability to

manage own learning in a sustained manner and to critically evaluate own progress, making use of a wide range of feedback sources

Exceptional,individual andfluentapplication ofa range of specialist practical and technical skills

Outstanding accomplishment of a range of advanced transferable and professional skills applied to complex situations and problems

Outstanding ability to manage own learning in a sustained manner and to critically evaluate own progress, making use of a wide range of feedback sources

Accomplished,original andfluentapplication ofa range of specialist practical and technical skills

Outstanding accomplishment of a range of advanced transferable and professional skills applied to complex situations and problems

Accomplished application of advanced transferable and professional skills to complex situations and problems

Very high ability to manage own learning in a sustained manner and critically evaluate own progress making effective use of feedback

Accomplished and original applicationof a range of specialist practical and technical skills

Strong ability to manage own learning in a sustained manner and to critically evaluate own progress making effective use of feedback

Confident application of advanced transferable and professional skills to challenging situations and problems

Confident and imaginative application of a range of specialist practical and technical skills

Sound application of advanced transferable and professional skills

Sound application ofa range of specialist practical and technical skills

Sound ability to manage own learning in a sustained manner and critically evaluate own progress making effective use of feedback

Competent application of advanced transferable professional skills

Competent ability to manage own learning in a sustained manner and make effective use of feedback

Competent application ofa range of specialist practical and technical skills

Limited application of advanced transferable and professional skills

Basic application of a range of specialist practical and technical skills

Basic ability to manage own learning in a sustained manner and make use of feedback

Rudimentary application of a range of specialist practical and technical skills

Ineffective application of advanced transferable and professional skills

Evidence of a basic ability to manage own learning