Wednesday, August 16, 2017

Cool open source tool for ER diagrams

Recently one of my colleagues introduced me to a cool Java tool called SchemaSpy.
SchemaSpy is a java tool that can be used to create beautiful database documentation and also ER diagrams.

Would highly recommend perusing the following links and utilizing this tool:

Monday, August 07, 2017

Ruminating on Telematics standards

While creating training material for our internal staff on telematics, we found the below site that gives a very good introduction to novices on the basics of telematics -

I had covered the business benefits of telematics in this blog post. One of the fundamental challenge for the wide spread adoption of telematics has been the lack of standards around aggregating data from multiple TCU (Telematics Control Unit) vendors.

But now, the Association of Equipment Managers (AEM) and Association of Equipment Management Professionals (AEMP) have created a ISO standard to help systems
from different manufacturers all speak the same language -

The AEMP 1.0 standard defined a XML data format that is available here.
An example of the AEMP REST API by John Deere is available here (in both XML and JSON formats) -

The newer version (v 2.0) of the AEMP telematics standard is now an ISO standard (ISO 15143-3).
An example of ISO 15143-3 telematics API (with XML data format) is here.

As part of this standard, equipment makers would report the following data points and 42 fault codes using a standard protocol that will allow mixed equipment fleets to be managed from a single application.

Data Elements
  1. Serial Number
  2. Asset ID
  3. Hours
  4. Location
  5. GPS Distance Traveled
  6. Machine Odometer
  7. Fault Codes
  8. Idle Time
  9. Fuel Consumption
  10. Fuel Level
  11. Engine Running Status (on/off)
  12. Switch Input Events
  13. PTO Hours
  14. Average Load Factor
  15. Max Speed
  16. Ambient Air Temp
  17. Load Counts
  18. Payload Totals
  19. Active Regen Hours

Fault Codes
  1. Engine coolant temperature
  2. Engine oil pressure
  3. Coolant level
  4. Engine oil temperature
  5. Hydraulic oil temperature
  6. Transmission oil temperature
  7. Engine overspeed
  8. Transmission oil pressure
  9. Water in fuel indicator
  10. Transmission oil filter, blocked
  11. Air cleaner
  12. Fuel supply pressure
  13. Air filter pressure drop
  14. Coolant system thermostat
  15. Crankcase pressure
  16. Cool water temp, outlet cooler
  17. Rail pressure system
  18. ECU temperature
  19. Axle oil temp, front
  20. Axle oil temp, rear
  21. Intake manifold temperature
  22. Secondary steering pressure
  23. After-treatment reagent internal filter heater
  24. Crankcase ventilation 
  25. Boost pressure
  26. Outgoing brake pressure status
  27. Brake pressure, output
  28. Brake pressure, output
  29. All wheel drive hydraulic filter, blocked
  30. All wheel drive hydraulic filter, blocked
  31. Brake pressure acc status
  32. Injection control pressure
  33. Brake circuit differential pressure
  34. Brake pressure acc charging
  35. Brake pressure actuator
  36. Reserve steering control pressure
  37. HVAC water temperature
  38. Rotor pump temperature
  39. Refrigerant temperature
  40. Alert and alarms help power down machines quickly
  41. Multiple pressure indicators
  42. Switching gear events

Monday, July 17, 2017

Performance benefits of HTTP/2

The HTTP/2 protocol brings in a lot of benefits in terms of performance for modern web applications. To understand what benefits HTTP/2 brings, it is important to understand the limitations of HTTP/1.1  protocol.

The HTTP protocol only allows one 'outstanding' request per TCP connection - i.e. if a page is downloading 100 assets, only one request can be made per TCP connection. Browsers typically open 4-8 TCP connections per page to load the page faster (parallel requests), but this results in network congestion.

The HTTP/2 protocol is fully multiplexed - i.e. it allows multiple parallel requests/responses on the same TCP connection. It also compresses HTTP headers (reduce payload size) and is a binary protocol (not textual). The protocol also allows servers to proactively push responses directly to the browsers cache, thus avoiding HTTP requests.

The following links give an excellent explanation of the HTTP/2 protocol and its benefits.

The following site shows the performance of HTTP/2 compared to HTTP/1.1 when loading tons of images from the server -

There is another site - which states that HTTPS is faster than HTTP, but that is because HTTPS was using HTTP/2 by default in the background.

Sunday, July 16, 2017

Ruminating on DMN - a new modeling notation for business rules

We all know how the BPMN standard helped in interoperability of business process definitions across lines of business and also across organizations. But there was one element missing in the BPMN standard - i.e. the ability to model business rules/decisions in a standard way so that they can be interoperable.

Most of the current business rule engines (aka BRMS) follow a proprietary standard for modeling rules and migration from one BRMS suite to another was usually painful. Hence, we are pretty excited about DMN (Decision Model and Notation), a standard from OMG that can be considered complimentary to BPMN.

Many Rule Modeling tools such as IBM Decision Composer, OpenRules and Drools already support the DMN standard. Part of the DMN standard is also a Friendly Enough Expression Language (FEEL). FEEL defines a syntax for embedding expressions in the rule.
An excellent tutorial about DMN using decision tables can be found here - A good video tutorial on IBM decision composer is here -

It is important to understand the difference between a decision modeling tool and a decision execution engine. A decision modeling tool would give a GUI to define the decision model using the DMN standard. This decision model can be exported as a *.dmn file (which is in XML format - example here). The decision execution engine actually is the runtime to execute the model. Most of the existing rule engines have extended their support to run rules defined in DMN model. For example, the new IBM Decision Composer is a web-based rule modeling tool, but the modeled rules can be exported to run in the existing ODM engines.

So in theory, the DMN model created by one tool can be used to execute the model in another tool . Some folks have tested this and noted down the challenges in this whitepaper - The Effectiveness of DMN Portability

Sunday, April 16, 2017

Ruminating on the single-threaded model of NodeJS and Node-RED

Many developers and architects have asked me questions on the single-threaded nature of NodeJS and whether we can use it effectively on multi-core machines. Since Node-RED is based on NodeJS, whatever is valid for NodeJS is also valid for Node-RED.

NodeJS has a single thread per process model. When you start NodeJS, it would start a single process with one thread in it. Due to it's non-blocking IO paradigm, it can easily handle multiple client requests concurrently, as there is no thread that is blocking for any IO operation.

Now the next question is around the optimal usage of multi-core machines. If you have 8 core or 16 core machines on your cloud and just run one NodeJS process on it, then you are obviously under-utilizing the resources you have paid for. In order to use all the cores effectively, we have the following options in NodeJS:

1) For compute-intensive tasks: NodeJS can fork out child processes for heavy-duty stuff - e.g. if you are processing images or video files, you can fork out child NodeJS processes from the parent process. Communication between parent and child processes can happen over IPC.

2) For scaling REST APIs: You can start multiple NodeJS processes on a single server - e.g. if you have 8 cores, start 8 NodeJS processes. Put a load balancer (e.g. nginx) in front of these processes. You would anyways have some kind of load-balancing setup in your production environment and the same can be leveraged.

3) Cluster support in newer versions on NodeJS: In the latest versions of NodeJS, you have support for Clusters. Cluster enables us to start multiple NodeJS processes that all share the same server port - e.g. 8 NodeJS processes all listening to port 80.  This is the best OOTB option available today in NodeJS.

Hence it is indeed possible to effectively utilize NodeJS on multi-core machines. A good discussion on the same is available on StackOverflow here.

Another interesting question that is often asked is whether Java developers should jump ship and start development in NodeJS because of its speed and simplicity. Also, NodeJS evangelists keep harping about the fact that the single-threaded nature of Node removes the complexity of multi-threading, etc.

Here are my thoughts on this:

1) Today Java has first class support for non-blocking IO, similar to NodeJ. Take a look at the superb open-source  Netty library that powers all the cloud services at Apple.

2) Most of the complexity of multithreading is abstracted away by popular frameworks such as Spring - e.g. Spring Cloud enables developers to write highly scalable and modular distributed applications that are cloud-native without dealing with the complexities of multi-threading.

3) The Servlet 3.0 specification introduced async requests and the Spring REST MVC framework also supports non-blocking REST services as described here.  

Thus today, all the goodies of NodeJS are available on the Java platform. Also plenty of  Java OSS to kick start your development with all the necessary plumbing code.

Ruminating on a nifty tool called 'ngrok'

Many a times developers want to test their APIs running on their development machine against a mobile app. To do so, developers typically do the following:

Option A) Create a hotspot Wifi on their development machine and connect the mobile phone to this Wifi. Since both the mobile phone and their development machine is on the same network, the mobile app can call the APIs running on the localhost of the developer machine.

Option B) We host the APIs on a public cloud (with a public IP), so that the mobile app can access it. For this, the developer needs to setup a automated CI/CD process that would deploy the REST APIs on the middleware.

There is a third option where-in, the developer can run the API server on his local machine and just create a proxy server on the internet that would securely tunnel the request to this local machine through the firewall. This can be done using ngrok -

ngrok can be setup on your local machine within minutes and can serve as a great debugging tool. It has a network sniffer built in that can be assessed from http://localhost:4040
We would highly recommend this tool for any mobile/API developer. 

Monday, February 13, 2017

IoT energy standards in Europe for Smart Home / Smart Grid

European governments are pretty energy conscious with a number of initiatives in the EMEA region around smart energy grids. Jotting down some of the energy standards that are relevant in Europe.

In order to successfully build a smart grid system, it is important to have standards through which home appliances can publish information on how much energy they are using in real-time, which in turn can be provided to consumers and energy management systems to manage energy demands more effectively.

Smart appliances are also technically very heterogeneous, and hence there is a need for standardized interfaces. The standardization can happen at two levels - the communication protocol level or the message-structure level.
  • SAREF (Smart Appliances REFerence) is a standard for exchange of energy related information between home appliances and the third-party energy management systems. SAREF creates a new reference language for energy-related data. 
  • EEBus - A non-profit organization for interoperability  in sectors of smart energy, smart home & building. EEBus has defined a message-structure standard called as SPINE (Smart Premises Interoperable Neutral-message Exchange). SPINE only defines message structure at the application level (OSI Layer 7) and is completely independent from the used transport protocol. SPINE can be considered a technical realization of the SAREF onthology. 
  • Energy@Home - Another non-profit organization that creating an ecosystem for smart-grid. 
  • SAREF4EE - The extension of SAREF for EEBus and Energy@Home initiatives. By using SAREF4EE, smart appliances from different manufacturers that support the EEBus or Energy@Home standards can easily communicate with each other using any energy management system at home or in the cloud.

A good presentation illustrating the current challenges in Smart Home is available here -