Comprehensive guide to understand Robots
What’s a Robot?
A Robot is an actor that automates the routine, but sometimes complex tasks, associated with managing your data workflow. Robots can transform and improve data from your microshares™. Robots can interact with external services to feed external systems and to pull new data into microshare to further enrich your data. A single Robot usually automates a single such task. Keeping each Robot simple ensures that it will be: 1) easy to maintain, 2) fast and efficient with higher volumes, and 3) reusable in multiple projects.
Robots can be chained together to cooperate in fulfilment of more complicated needs. Such a chain is called a Data Workflow. Data Workflows are capable of preparing your data to be easily consumed in Dashboards and multiple Applications. Each Robottic ‘step’ of your Data Workflow should create new data in the lake. Embrace it, there is no such thing as too much data with Microshare.io. These intermediary records preserve the transaction history, allow for fast error recovery, and keeps the stream processing fast and efficient.
All Robots run in parallel (at the same time) and react to events as they occur. In many cases, a Robot will react to an event by reading a microshare record, acting on it in some way, and writing a new record back into microshare.
What can I do with them?
To keep your data workflow clear, we suggest each one of your Robots takes care of only one single action. Below are some typical use cases we identified.
1. Data ingestion
For Robots tasked to pull or receive data sent by an external source.
- CSV files dropped in a target folder
- active web socket connection to a remote server
- interacting with an external service via API
2. Data parsing
Often managed with a Data Workflow to massage and enrich a bunch of records.
3. Data transformation
Perfect for decoding a formatted IoT payload.
4. Data formatting
Prepare your data to be used in a Dashboard.
5. External services triggers
For a Robot used to trigger another service on the web.
How do I use them?
Accessing the UI
From the management console available here, open the Manage -> Robots panels. Create and edit your Robots here.
First of all, this part is compulsory:
require method loads the library of function your Robot will be able to use,
main function is what’s actually run, so it must encapsulate all of Robot’s actions.
Its two parameters are
textthat is the metadata about the event that triggered the Robot.
authis your auth token to allow this Robot to access the data lake on your behalf.
You will use the two other functions 99.9% of the time:
lib.parseMsg(text);parses the message that triggered your Robot from the data lake.
lib.writeShare(auth, newRecType, newRecord, ['tags']);writes back a new record with a new recType in the data lake.
For more code samples, refer to the Data Workflow page.
Triggered vs scheduled
Your newly created Robot will first do… nothing!
To activate it, be sure to check the Active checkbox.
Once that is done, your Robot will trigger each time a new record with its RECORD TYPE is added to the data lake. So set that RECORD TYPE in your Robot accordingly.
Optionally, your Robot can also run on a schedule, without waiting for a new record to be created.
Delay time will run your Robot once after the specified amount of time, the countdown starts when you activate the Robot.
Interval time triggers your Robot periodically every interval, forever.
Set the Delay and/or Interval times, activate the checkbox (ticked means on), sit back and relax.
If you edit an existing Robot, you will see that a testing panel is present. Follow the instruction to simulate a run of the Robot with sample data.
Important: the lib.write is only simulated in a test, so you are not at risk to pollute your data.
How do they work?
Behind the scene, each Robot is an Akka Agent loaded with its ECMAScript 6 compatible script.
The Java libraries accessed by the Robots point to the adequate Service to read and write in the data lake.