Make the Neo4j raw data accessible via REST interface

(This suggestion for project https://github.com/marcelmaatkamp/steemit-data-analysis-platform)

Introduction

Now that version v1.0 of https://github.com/marcelmaatkamp/steemit-data-analysis-platform is ready and having the raw data available in a Neo4j database does not make it accessible for an application and giving end-users a direct connection to Neo4j is also not an option.

This suggestion will be on how to serve the data to 'end-users' where an end-user can be a user behind a browser or an application requesting the raw data for its own analysis and display.

The REST server

The data should come from a webserver serving one or more urls. Behind those urls is a controller which will pickup the the attributes given, make a query for Neo4j and serve the resulting data in a common format like 'json'.

An example which can be used is https://spring.io/guides/gs/rest-service. Because Spring has already a build-in webserver in the form of Jetty it can thus serve the requests and with the right settings added in the application.properties and a configuration bean it can also connect to the Neo4j datastore.

The controller should define the end-points where an application can query the data.

Since v1.0 focused on authors, votes and posts, in order to get (a subsection of) the data to analyse or display in an client application the following URL's are proposed:

Single node queries

These will al return exactly one instance of an object in the database with all its fields in json format.

/api/v1/author/
/api/v1/voter/
/api/v1/post/

Set of nodes queries

These will return a set of results of a single node_type like posts or votes in a given timeframe

/api/v1/votes?start_time=<start_time,stop=<stop_time> | time=1d,1month..
/api/v1/posts?start_time=<start_time,stop=<stop_time> | time=1d,1month..

Search queries

/api/v1/match?limit=

This will return the query "MATCH (n) RETURN n LIMIT ;"

Client application

The server can also serve static HTML pages like '/index.html' On that page a client library like Keylines or d3.js can be included which fetches the data from the url endpoint '/api/v1/match?limit=1000' for example.

Expected result

The expected result will be an interactive /index.html which renders the data from Neo4j in the browser and a REST interface on /api to let client applications connect to.

Example output with d3.js would be like this:

Posted on Utopian.io - Rewarding Open Source Contributors