Industrial Big Data analysis with RHadoop

Project reference: 1923

The project will consist of:

  • Getting to know Hadoop and RHadoop;
  • Defining big data source related to Industry 4.0;
  • Creating and storing big data files (BD);
  • Preparing BD for basic analysis;
  • Defining predictive model and writing RHadoop code for building this model;
  • Evaluation and application of the developed model on new data.

The student will create a big data file, store it in a DFS; perform basic analysis and build a predictive model for new data using RHadoop.

Project Mentor: Prof. Janez Povh, PhD

Project Co-mentor: MSc.Timotej Hrga

Site Co-ordinator: Doc. Dr. Leon Kos

Learning Outcomes:

Student Prerequisites (compulsory):

  • Basics from data management;
  • R language
  • Basics from regression and classification

Student Prerequisites (desirable):
Basics from Hadoop.

Training Materials:
The candidate should go through PRACE MOOC:

https://www.futurelearn.com/admin/courses/big-data-r-hadoop/4

Workplan:

  • W1: introductory week;
  • W2: creating and storing industrial big data file
  • W3: coding and evaluating scripts for analysis;
  • W4-W5: coding and evaluating scripts for analysis;
  • W6: preparing materials for MOOC entitled MANAGING BIG DATA WITH R AND HADOOP.
  • W7: final report
  • W8: wrap up

Final Product Description:

  • Retrieved industrial big data files and stored in Hadoop;
  • Created RHadoop scripts for analysis and new prediction models;
  • Created a report on this example to be used in do PRACE MOOC with title MANAGING BIG DATA WITH R AND HADOOP.

Adapting the Project: Increasing the Difficulty:
We can increase the size of data or add more demanding visualization task.

Adapting the Project: Decreasing the Difficulty:
We can decrease the size of data or simply the prediction model.

Resources:
RHadoop installation at the University of Ljubljana, Faculty of mechanical engineering.

Organisation:
University of Ljubljana
project_1620_logo-uni-lj

Please follow and like us:
Tagged with: , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.