Database Visualisation Project : HUES Platform

ProjectSummary_GSoC2016.txt

Google Summer of Code 2016 : Project Repository , Database Visualization for HUES Platform Energy Technology Data
------------------------------------------------------------------------------------------------------------------------ 
Open source energy simulation database for HUES Platform: using MEAN Stack
Khushboo Mandlecha
------------------------------------------------------------------------------------------------------------------------ 

Github Link : https://github.com/Khushboo1192/DatabaseVisualizationProject

Introduction:
Data on the HUES Platform is highly diverse and rapidly growing. Currently, this data is stored in separate platform modules using different data structures and file formats, making it difficult to explore holistically. This project involves the development of an open source database to enable the linked exploration of data on the platform in the form of visual representations for e.g. histogram graphs for energy demand, supply and technology data.
First part was the development of server based scripts to automatically extract relevant data from the modules of the existing HUES platform, and write this data to a common database. I have chosen MongoDB as the new database and everything can be retrieved in the form of JSON objects. Further, the users can also upload technology data files to the portal which will get stored in the new database MongoDB.
The development of a web-based portal for querying, visualizing, uploading and downloading data was the second part of the project development. As the database is migrated to MongoDB we can achieve fast data retrieval. Moreover, visual representations using graphical structures will help the researchers and practitioners for the development of models and analyses of distributed energy systems.
 
------------------------------------------------------------------------------------------------------------------------ 
Technology and Working : In this section I have listed down the technological aspects of the project.

MEAN STACK:

The database from the HUES platform can be exported to files and can be stored as JSON then we can use MongoDB as database and use node and angular.

The MEAN stack is a web development stack made up of MongoDB, Express, AngularJS, and Node.js. in JavaScript on both the client and the server3. The MEAN stack enables a perfect harmony of JavaScript Object Notation (JSON) development: MongoDB stores data in a JSON-like format, Express and Node.js facilitate easy JSON query creation, and AngularJS allows the client to seamlessly send and receive JSON documents.

Node.js has different charting libraries for interactive data visualization like D3.js. It is open-source and free for all uses. Their charts are responsive (using media queries) and DPI independent and can be on used any kind of device – mobile, tablet or desktop. It is based on SVG which makes it future compatible.
------------------------------------------------------------------------------------------------------------------------ 

Work Completed:

Data Extraction: Previously Database was stored in MySQL server, migrated to MongoDB server using command line scripts for MongoDB. Created a new Database in mongoDB and all the schemas/ collections  have been added.
		a. Users
		b. Technology Data
		c. Upload Information (Which user uploaded what file)

Upload and Download Files: Registered users can upload Energy Technology files to the database. Technology Template files can also be downloaded by the user. 

Querying the Database: Options in the form of drop down menu is available to the user to query from several options on the database. The selected option will display the results accordingly.

Registration and Login: A user has to register to the portal in order to access the features to the database. As of now any user can register. After registering, login info can be used to login into the system.

Static Visualisation: A histogram for energy type vs number of entries can be plotted on the system.   

------------------------------------------------------------------------------------------------------------------------ 
Work Remaining : 

Graphical Visualisation: More interactive graphical visualizations needs to be made which will give more information to the user. 

Security Aspects: User Registration needs to be two level where an administrator will approve a new user.

Advanced Querying: As of now querying is limited because of drop down menu. Later it can be customised by understanding the user needs extensively.