Guide to Data Ingestion
To aggregate your data from different sources
Microshare.io is here to help you aggregate and use your data from several sources, especially IoT platforms.
Below are several ways to move your data as microshares™ in the data lake.
Upload data manually
Manual upload is the most basic way of loading data from your own database, or from an open data project.
To do so, send your data as the body of a POST /share call.
It will then be available to use from the data lake with GET /share calls
Set up a websocket Robot
Some IoT platforms act as websocket servers and allow websocket clients to listen and pull data live.
Orange Live Objects, The Things Network, or a Sagemcom private gateway support it.
In Microshare.io, you can setup a websocket client Robot that takes care of writing your data as a microshare as soon as it is available.
There is no UI to configure your own yet, but here is the WS client configuration we’ve gone with:
Set up a scheduled Robot to pull data
Alternatively, some platforms offer RESTful APIs to request for the data they store, such as Orange Live Objects, Bouygues Telecom Objenious, Sierra AirVantage, or Cumulocity.
In that case you can setup a scheduled Robot to perform GET calls to your IoT platform periodically.
You are at liberty to setup your Robot script the way you want, to decide when and what to store as a microshare from that data.
Below is a sample Robot script that performs a data pull from Orange LiveObjects, storing all pulled IoT packets to the microsahre data lake:
Set up your platform to post the data
For Actility ThingsPark or a Kerlink private gateway. Some platforms can be configured to POST data. Configure them to do a POST /share/:recType call.
Once your data is loaded in the data lake, you’ll want to get it ready to be used in Dashboards and Applications. Build your multisteps worflow with a Data Workflow to parse, transform and format your data automatically.