26. April 2017

How to publish topic by mosquitto_pub to Bluemix

In the previous article about LampESP, I’ve described how to subscribe ESP8266 to MQTT at Bluemix cloud. The remaining question is how to publish data.

Let’s define simple scenario for server monitoring: Server is sending every minute it’s average load to Bluemix for further processing. How to solve it?

You’ll need to create a new device type at Bluemix, let’s call it “server”. Then create new device using the same step like in the previous article.

Then you can run a simple shell script which will push all the data to Bluemix:

#!/bin/bash
# Replace following values by your own
ORG_ID="myid"
DEVICE_NAME="reactor"
DEVICE_TOKEN="psst, something secret"

LOAD=`cat /proc/loadavg`
mosquitto_pub -h "${ORG_ID}.messaging.internetofthings.ibmcloud.com" \
-i "d:${ORG_ID}:server:${DEVICE_NAME}" \
-u use-token-auth \
-P "${DEVICE_TOKEN}" -r \
-t "iot-2/evt/${DEVICE_NAME}/fmt/text" -m "${LOAD}"

If you need more detailed information about communication just add option “-d” to mosquitto command.

Then you can check also logs from the device at Bluemix. You should see something like this:

Closed connection from YOUR_IP. The connection has completed normally.
Token auth succeeded: ClientID='d:ORG_ID:server:DEVICE_NAME', ClientIP=YOUR_IP

Please keep in mind that Bluemix requires certain format of topic name.

In the next article, we will take a closer look how to transport event from a server to ESP8266 by Node-RED.

21. April 2017

How to connect ESP8266 to Bluemix

Connecting ESP8266 by MQTT to a server like Mosquitto is relatively easy. You need just one dependency (in platformio.ini file):

[env:d1_mini]
lib_deps =
  PubSubClient

Code for a connection is:

WiFiClient espClient;
PubSubClient mqttClient(espClient);

static void callback(char* topicChar, byte* payloadByte, unsigned int length) {
  String topic = topicChar;

  // Default size is defined in PubSubClient library and it's limited to 128
  // https://github.com/knolleary/pubsubclient
  char buf[MQTT_MAX_PACKET_SIZE];
  if (length >= MQTT_MAX_PACKET_SIZE) {
    length = MQTT_MAX_PACKET_SIZE - 1;
  }
  snprintf(buf, length + 1, "%s", payloadByte);

  String payload = String((char *)buf);
  Serial.print(payload);
}

void setup() {
  mqttClient.setServer("iot.georgik.rocks", 1883);
  mqttClient.setCallback(callback);
}

void reconnect() {
  if (mqttClient.connect("display")) {
    subscribeTopics()
  } else {
    Serial.println("Connection failed");
  }
}

Code for a subscription to topic is:

void subscribeTopics() {
  mqttClient.subscribe("some/topic/data");
}

Code for publishing to topic is:

mqttClient.publish("other/topic/data", "123");

This works fine with Mosquitto in trusted environment. You can find sample implementation in LampESP branch v0.3 in file LampMQTT.ino.

When you switch to cloud environment you have to take in consideration security model of the cloud. E.g. Microsoft Azure IoT Hub does not support direct communication between devices. Similar limitation is true also for IBM Bluemix. The architecture of messaging must be little bit different.

How to change the code to connect to IBM Bluemix?

First of all you need to create an account in IBM Bluemix. There is 30 day trial. Then you can use part of services for free, but you have to enter your credit card. The free 370 GB-hours is sufficient to run 512 MB virtual machine during whole month for free.

Then you have to create “Internet of Things Platform” service. This service is available only in regions US and United Kingdom. The region Germany does not support this service yet. Once the service is running, go to Dashboard for your service.

You’ll be redirected to URL like: https://ORG-ID.internetofthings.ibmcloud.com/dashboard/#/boards/

There you’ll see IBM Watson IoT Platform Dashboard. Go to Devices.

Click “+ Add Device” button. Then click “Create device type”.

Click “Create device type”.

Set name e.g. to ESP8266 and proceed with registration of device type.

Once the type is ready you can define device. Click “+ Add device”. Select your type ESP8266 from drop down. Now careful!

The Next button is in lower right corner. It seems that UX engineers were not validating the interface.

Fill in the name of the device and description.

You can skip Metadata. Leave Security set to Auto-generated authentication token.

Skip summary and click Add.

Now you’ll see page with Device. The most important part is section with Authentication token.

The cloud is ready. Now it is necessary to update the code. There is a small gotcha in the server name. If you look at Watson dashboard you might think that’s the hostname for MQTT: ORG-ID.internetofthings.ibmcloud.com. You have to inject subdomain “messaging”. The correct hostname is: ORG-ID.messaging.internetofthings.ibmcloud.com.

...
void setup() {
  mqttClient.setServer("ORG-ID.messaging.internetofthings.ibmcloud.com", 1883);
  mqttClient.setCallback(callback);
}
...

The next important step is to add token which will be used to establish a session between the device and cloud. This is a little bit tricky. You’ll need three parameters which will be composed in the following fashion and supplied to connect method of PubSubClient:

  • id = “d:ORG-ID:ESP8266:DEVICE-NAME”
  • username = “use-token-auth”
  • password = “TOKEN-FROM-DEVICE-PAGE”
void reconnect() {
  if (mqttClient.connect(id, username, password)) {
    subscribeTopics()

In the case of cloud version of MQTT you could not subscribe or publish just to any random topic. You have to follow the format. In the case of subscription to command you have to use following topic: iot-2/cmd/COMMAND/fmt/json. Replace COMMAND, by any of your commands that the device should receive.

void subscribeTopics() {
  mqttClient.subscribe("iot-2/cmd/COMMAND/fmt/json");
}

To publish the device status there is also special topic:

mqttClient.publish("iot-2/evt/status/fmt/json", "{\"d\":{\"value\":\"online\"}}");

Now you’re ready to connect to the Bluemix cloud. If you experience any problems, just go to device detail and find Connection log.

Now you can connect Node-RED or other tools and communicate with the device. You can find working implementation at LampESP v0.4 project.

Let’s a make summary. There are three gotchas:

  • MQTT must be connected to subdomain: ORG-ID.messaging.internetofthings.ibmcloud.com
  • Subscription and publishing must follow specific format of topic name
  • Auth information must be composed in precisely

2. April 2017

Custom font for OLED display connected to ESP8266 via SPI

Small OLED displays can easily extend the functionality of ESP8266.

I made an experiment with 128×64 OLED display from Com-Four.

The first challenge was how to connect the display to ESP8266. The recommended way for high performance is to use Serial Peripheral Interface Bus (SPI). The advantage of this approach is the speed, the disadvantage is that it will take more pins.

The display could be connected in following way (also described in the example of ESP8266_SD1306 library):

ESP8266 - SD1306
GND     - GND
3V      - VDD
D5      - SCK (also known as CLK)
D7      - SDA (also known as MOSI/DOUT)
D0      - RES
D2      - DC
D8      - CS

If you’re using PlatformIO, just add ESP8266_SD1306 library to dependencies in platfromio.ini:

lib_deps =
 ESP8266_SSD1306

Now you can run any example from Squix78 library. The library contains 3 sizes of Arial font: 10, 16 and 24px.

My goal was to display temperature from Observatory in Brno. Retrieving temperature and sending it to MQTT for ESP8266 was quite easy.

#!/usr/bin/env python3

import paho.mqtt.publish as publish

import urllib.request
f = urllib.request.urlopen('http://www.hvezdarna.cz/meteo/lastmeteodata')
content = f.read().decode('utf-8')

items = content.split(' ')

publish.single('/home/monitor/display/0', items[4], hostname='localhost')

I used default Arial 24 font. The problem was that the number was too small and barely readable from a distance. Luckily Daniel Eichhorn published great online tool which is able to generate font of any size for OLED display: http://oleddisplay.squix.ch.

My first attempt was to generate Roboto Light 54px font. It was working, just number 4 was not visible. I discovered a bug in the generator, that too big font will overflow default size of char in the jump table.

After several attempts I’ve found the right font for me DejaVu Sans 52px. This font was far more readable.

The last touch to make the font more readable was to tune down contrast little bit by the command:

display.setContrast(10);

I can definitely recommend this type of OLED display. It has good readability even during a sunny day. The code is available at GitHub in LampESP project.

1. March 2017

Microsoft Azure IoT Hub MQTT device to device communication is not supported

Microsoft Azure provides IoT Hub. It might sound great because it has support for MQTT. It is very important to mention that Azure IoT Hub has a different way how it operates than Mosquitto which means that architecture of your IoT solution must be different.

Let’s check differences.

Default port:

  • Mosquitto is 1883 which is not secure, it is possible to enable TLS
  • IoT Hub 8883 secured by TLS/SSL (only)

Publishing and subscription of messages:

That’s a huge difference. In the case of Mosquitto you can easily build network of devices which can communicate together and management logic could be injected by Node-RED. In the case of IoT Hub whole interaction is managed by the cloud.

If you still want to play with IoT Hub and MQTT, I recommend article written by Satish Pagare who explains how to use mosquitto_sub and mosquitto_pub to interact with IoT Hub.

20. February 2017

How to send command from Python via MQTT to RGB LED connected to ESP8266

I’ve described how to send server load as a number to MQTT in the previous article. The number could be then translated via Node-RED to command for LampESP with RGB LED. The result is simple. LED indicates server load by displaying different colors.

The other option is to deliver color command directly from the server using Python.

Just install paho-mqtt:

pip install paho-mqtt

Here is small snippet of Python code (publish_server_load.py):

#!/usr/bin/env python3

import paho.mqtt.publish as publish
import os

color = 'red'
load = os.getloadavg()[0]
if load < 0.7:
    color = 'black'
elif load < 1.5:
    color = 'blue'
elif load < 3:
    color = 'green'
elif load < 7:
    color = 'orange'

publish.single('/server/monitoring/command', color, hostname='iot.sinusgear.com')

ESP should listen to /server/monitoring. Code of LampESP 0.3 is available at GitHub.

Put this code into crontab

* * * * * /usr/local/bin/publish_server_load.py

If you’re using virtualenv the command should be:

* * * * * /opt/my-python-env3/bin/python /usr/local/bin/publish_server_load.py