The course explores two complementary roles for humans as applied to interactive data analytics: one, where humans are the analysts performing or supervising the analysis; here, the emphasis is on building usable tools for these analysts, and second, where humans are the crowdsourced workers assisting with the computation and analysis; here, the emphasis is on having humans process as little data as possible while gaining maximum benefit.
Students will read a number of papers: both important landmark papers as well as cutting-edge papers, act as a discussant for a paper at least once, and complete a semester-long implementation project. Familiarity with basic databases, machine learning, and algorithms expected.
Crowd-Powered Analytics: An IBM study estimated that 80% of the data recorded every day is unstructured: i.e., it consists of images, videos and text. Fully automated processing of unstructured data is not yet a solved problem. Humans, on the other hand, are very good at understanding, interpreting, and processing unstructured data. How do we use humans to effectively process large volumes of unstructured data?
Interactive Analytics: A McKinsey Big Data Study estimated that 10s of Millions of new data analysts will be needed by 2015. With so many novice data analysts interacting with data, how do we enable them to quickly get valuable insights? Quickly could mean generating the same results faster, but approximately; it could mean showing them visualizations instead of raw data; it could mean helping the users to ``guess'' the query or insight in mind.
You must use the following link to submit class reviews: Link.
Remember to cover the 5 key questions: what is the problem, why is it important, what sets it apart from previous work, what are the key technical ideas, what are the key flaws and open issues, all within 500 words.
The class reviews must be submitted by midnight the day before class.
You must use the following link to submit your list of top-5 papers: Link.
The papers you provide can be from the list given below. You are also free to list a paper of your choice, but it must match the themes of the class. This list must be submitted by midnight September 1. .
|Date||Paper||Presenter||Notes||8/24/2015||Introduction to Course Content||Aditya|
|8/26/2015||CrowdScreen: Algorithms for Filtering Data Using Humans||Aditya|
|8/31/2015||VLDB Conference--No Lecture|
|9/2/2015||VLDB Conference--No Lecture||Send ranked list of 5 papers by midnight September 1|
|9/7/2015||Holiday- Labor Day|
|9/9/2015||So Who Won: Dynamic Max Discovery with the Crowd||Vipul||First time for Class Review- Send it by midnight the day before; Student Presentations Start|
|9/14/2015||Crowdsourced Enumeration Queries||Ruihan|
|9/16/2015||Human-Powered Sorts and Joins||Aditya|
|9/21/2015||Turkit: Human Computation Algorithms on Mechanical Turk||Alex Morales|
|9/23/2015||CrowdDB: Answering Queries Using Crowdsourcing||Aditya|
|9/28/2015||Deco: Declarative Crowdsourcing||Andrew||Submit project proposal by midnight the day before. Proposals won't be graded, but are compulsory|
|9/30/2015||Potter’s Wheel: An Interactive Data Cleaning System||Donald|
|10/5/2015||Wrangler: interactive visual specification of data transformation scripts||Tifany|
|10/7/2015||Profiler: Integrated Statistical Analysis and Visualization for Data Quality Assessment ; Mid-term review for projects||Yulun|
|10/12/2015||Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing; Map-Reduce basics||Suhansanu|
|10/14/2015||Dremel: Interactive Analysis Of Web-Scale Datasets; Column Stores basics||Alexander Zahdeh|
|10/19/2015||Spark SQL: Relational Data Processing in Spark; Shark basics||Doris|
|10/21/2015||BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data||Liqi|
|10/26/2015||Trust Me, I’m Partially Right: Incremental Visualization Lets Analysts Explore Large Datasets Faster||Shengliang|
|10/28/2015||Polaris: A System for Query, Analysis, and Visualization of Multidimensional Relational Databases||Aditya|
|11/2/2015||SeeDB: Automatically Recommending Query Visualizations||Aditya|
|11/4/2015||GraphLab: A New Framework For Parallel Machine Learning||Stephen|
|11/9/2015||MLbase: A Distributed Machine-learning System||Henry|
|11/11/2015||MAD Skills: New Analysis Practices for Big Data||Ritwika|
|11/16/2015||dbTouch: Analytics at your Fingertips||Yishuo|
|11/18/2015||Gestural Query Specification||Himel|
|12/30/2015||Making Database Systems Usable||Aditya|
|12/2/2015||Google fusion tables: web-centered data management and collaboration||Donald Cha||Final Project Report Guidelines|
|12/7/2015|| Project Presentations in class ||Submit project report the same night by 11.59 PM|
As part of this course, you need to complete a semester-long project. See the instructor for ideas. Alternatively, you are free to look for ideas in your domain of expertise: for instance, if you work in computational journalism, building a new way to browse and manage large collections of textual archives could be a perfectly reasonable project. Either way, you must speak to the instructor to verify that the project is indeed "challenging" enough.