This project is meant to be a substantial independent research or engineering effort related to material we have studied in class. Your project may involve a comparison of systems we have read about, an application of database techniques to a system you are familiar with, or be a database-related project in your research area.
This document describes what is expected of a final project and proposes some possible project ideas.
What is Expected
Good class projects can vary dramatically in complexity, scope, and topic. The only requirement is that they be related to something we have studied in this class and that they contain some element of research -- e.g., that you do more than simply engineer a piece of software that someone else has described or architected.
What to Hand In
There are two written deliverables: a midway report including a project proposal, due on Nov 29, and a final report, due on the day of your final exam.
Project Proposal: The proposal should consist of 1-2 pages describing the problem you plan to solve, outlining how you plan to solve it, and describing what you will "deliver" for the final project.
Final Report: You should prepare a report on your project with maximum length of 15 pages (10 pt font or larger, one or two columns, 1 inch margins, single or double spaced -- more is not better.) Your report should introduce and motivate the problem your project addresses, describe related work in the area, discuss the elements of your solution, and present results that measure the behavior, performance, or functionality of your system (with comparisons to other related systems as appropriate.)
Because this report is the primary deliverable upon which you will be graded, do not treat it as an afterthought. Plan to leave at least a few days to do the writing, and make sure your proofread and edit carefully!
Project Ideas
The following is a list of possible project ideas from past years. You are not required to choose from this list -- in fact, we encourage you to try to solve a problem of your own choosing! Note that these are not meant to be complete project proposals, but just suggestions for areas to explore -- you will need to flesh them out into complete projects.
· Loading large datasets (> 10 GB) can be very slow, especially when there are many indexes on tables. One simple trick is to sort data before inserting it, or to drop indexes, insert data, and reload indexes. The goal of this project would be to investigate other techniques to improve load performance in an open source database such as Postgres or MySQL/Innodb. One concrete idea might be "fast bulk load" a la Hbase for MySQL. HBase uses a MapReduce job to build its indexes. One possible project would be to build a prototype that does the same for a single (or multiple) MySQL Boxes.
· Twitter provides a fire hose of data. Automatically filtering, aggregating, analyzing such data can allow to harness the full value of the data, extracting valuable information. The idea of this project is to investigate stream processing technology to operate on social streams, or extract structure from them and store them in a database system. You can use what you have used in DS 200, but, you have to design and compare different database options to select a good backend for your application.
· Compare and contrast the performance of different SQL/NoSQL solutions on different workloads (e.g., MySQL vs Postgres vs Hive vs Pig vs Voldemort vs Mongo vs ...)
· Build an automatic tool to extract data from websites such as the car sales website that you have been working with and populate the data into a database. You can use off-the-shelf software to extract the data from the sites. Design a set of queries someone may want to run on the data from these sites. Justify the basis for your choice of database.
Adapted from: http://db.csail.mit.edu/6.830/assignments/final_proj.html
Z1DONUK3
project.docx
0
