11. February 2017

Display server load on Linux Mint desktop delivered via MQTT

In previous article I wrote how to subscribe to MQTT topics and display also timestamp with color. There was one MQTT topic which represented server load and it demonstrates that MQTT could be used for server monitoring.

Here is an example how to measure server load and publish it as MQTT topic. Open crontab:

crontab -e

Add following line:

* * * * * /usr/bin/mosquitto_pub -t /server/reactor/load -m `cat /proc/loadavg | sed -e 's/ .*//g'`

Cron will invoke the command every minute. Command will publish data to MQTT topic /server/reactor/load (note: reactor is the name of my server).

We can subscribe at a workstation to the topic and execute some commands. mosquitto_sub will keep running in the background and we can use xargs to invoke a new command once the new message arrives.

Following example shows how to display notification on Linux Mint – Cinnamon using

mosquitto_sub -t '/server/reactor/load' -h iot.sinusgear.com | xargs -d$'\n' -L1 sh -c 'notify-send "Server load: $0"'

The result at workstation:

The only downside is that notifications are spamming desktop every minute. It does not make sense to notify admin when server load is too low. We can add a little bit of logic and display notifications about load which exceeded a certain threshold.

mosquitto_sub -t '/server/reactor/load' -h iot.sinusgear.com | xargs -d$'\n' -L1 /bin/zsh -c 'if (( $(echo "$0 > 1.2" | bc -l ) )); then notify-send "Server load: $0"; fi'

Note: just be aware that message from MQTT is not sanitized. Run such command only in trusted environment or add type check of value received from MQTT.

Enjoy :-)

5. February 2017

Colorful log from MQTT with timestamp

The easiest way to subscribe to all messages on MQTT server like Mosquitto is following command:

mosquitto_sub -v -t '/#'

The only problem is that result is just plain text without timestamp which makes it harder to spot some messages.

The solution to the problem is quite simple. Just pipe output of subscription command to xargs which will inject date and to ccze which will add colors. Here is the command:

mosquitto_sub -v -t '/#' | xargs -d$'\n' -L1 sh -c 'date "+%d.%m.%Y %T $0"' | ccze -m ansi

Result:

1. December 2016

ESP8266 WiFiManager gotchas

Thanks to PlatformIO it is very easy to add further functionality from libraries to the code for ESP8266. When you need to install library, just start Library Manager from ide and type:

pio lib install WiFiManager

PlatformIO will resolve dependencies and download all necessary stuff. Even better option is to add dependency to platformio.ini file:

[env:d1_mini]
platform = espressif8266
board = d1_mini
framework = arduino
lib_deps =
  ArduinoJson
  esp8266_mdns
  PubSubClient
  WiFiManager

PlatformIO will manage the installation after you save the file. That is very neat.

Then new problems occurred when I started with integration of WiFiManager. Do not take me wrong. WiFiManager is very handy library, but you may experience some tricky issues.

The first problem that I hit was that WiFiManager interface was not visible at 192.168.4.1 port 80. ESP was apparently running in AP mode for configuration, but there was no web interface. This issue was caused by this line in my code:

ESP8266WebServer server(80);

I had web server in the main code before I merged the code from WiFiManager. Web server was serving simple REST API for controlling relay. Unfortunately this construct blocked the port so the AutoConnect web server. The server was not able to bind the port. Solution was simple. It was sufficient to instantiate web server after WiFi was working.

The second problem was that AutoConnect from WiFiManager was always turning in AP mode. I was hunting the problem for several hours without any result. One idea was that WiFiManager is not saving the password. That was not true. Other idea was that the memory is corrupted and reformat would help. Unfortunately it didn’t. I gave up.

That solved the problem. Seriously. I gave up and that solved the problem. I let the device running and then ding. It was online. For some strange reason ESP8266 was not able to establish connection with particular WiFi router at first shot. Then it turned on AP mode and after timeout of Config Portal it connected to the WiFi network. Remedy? Just shorten ConfigPortalTimeout:

wifiManager.setConfigPortalTimeout(60);

The third problem was that Config Portal was not displaying information stored as extra custom parameters. I stored there host name of MQTT broker and other options so they won’t be hardcoded. The problem was caused by WiFiManagerParameter constructed in global scope. Issue was similar the first problem with web server.

Solution was to move construction of WiFiManagerParameter to the method after loading configuration values from file config.json stored on file system.

File configFile = SPIFFS.open("/config.json", "r");
...
WiFiManagerParameter custom_mqtt_host("mqtt_host", "MQTT host", configMqttHost, 40);

After resolving these issues WiFiManager is working and it is possible to set values for configuration via Config Portal and there is no need to hardcode them anymore.

Original code is stored at GitHub – LampESP branch v0.1. The new version is at Github – LampESP branch master. I made also small refactoring and the functionality was divided into smaller files which resembles modules for better code maintenance.

27. November 2016

Controlling relay connected to ESP8266 via MQTT from Android

After successful implementation of MQTT into Lamp Relay with ESP8266 there was one remaining challenge. Communication of Android Lamp App with ESP8266 via MQTT. It was little bit harder than I expected, because examples were not very clear.

Sending commands to MQTT broker was relatively easy. There was just a catch. It is necessary to declare the MQTT service in AndroidManifest.xml:

<service android:name="org.eclipse.paho.android.service.MqttService">
</service>

Thanks to tutorial from HiveMQ I was able to send messages to MQTT broker.

The problem was the opposite direction. How to receive messages?

Well, there is support in the same MQTT library for topic subscriptions. The issue is that incoming message is being processed on background thread and it’s not possible to update UI. It took me some time to realize the source of problem. I was hunting ghosts. I thought that there is some problem in ImageButton or TobbleButton.

Correct solution was to move communication to Android service class. Then bind the service so UI can send commands to the service and the last part was to add broadcast receiver to pass message from service to UI.

Here is small schema which describes delivery of message from broker to app.

mqtt-broker-to-android

It was necessary to make also small update in ESP8266 in order to tell Mosquitto to persist the state of a lamp (the last bool parameter):

mqttClient.publish(topicName("relay"), "off", true);

The result is nice. I can turn on the lamp via web interface and all connected Android apps are updated and user can see the state of the lamp.

lamp-app-mqtt

The code for Android app is available at Github – LampApp. Previous code with REST API was moved to branch v0.1. The code for ESP8266 module is also at Github – LampESP. Feel free to customize the app. You can find instructions in README.md.

I see the next challenge for LampApp: display lamp control widget at lock screen. Maybe I’ll be more successful than the last time.

22. November 2016

IoT and waiting for the sun

It took me some time to implement all features to ESP8266 controlling relay of my lamp.

smart-home-lidl-salt-lamp

Thanks to examples from PlatformIO I was able to use all neat features in one code:

You can find complete code at github repo LampESP. Just replace HOST_NAME_HERE and ROOM_NAME_HERE with your MQTT server and your room name.

Once you deploy code to ESP8266 you can use Over-the-air update (OTA). Just change the value in platformio.ini and set upload_port to IP address of your device.

The implementation allows control via simple GET API by accessing the web of device or LampAndroidApp.

As I wrote few weeks ago, my goal was to add MQTT. I’ve installed Mosquitto and connected the device to the server.

It is possible to watch states of device is possible via MQTT client:

mosquitto_sub -v -h HOST_NAME_HERE -t '/#'

Here is example how to change states of relay:

mosquitto_pub -h HOST_NAME_HERE -t '/home/ROOM_NAME_HERE/command' -m 'on'
mosquitto_pub -h HOST_NAME_HERE -t '/home/ROOM_NAME_HERE/command' -m 'off'

I had an idea about next cool feature: turn on/off lamp based on sunset and sunrise. I found very simple and elegant solution that I would like to share with you.

Download Sunwait and compile it. The program just starts and waits until the time of sunrise or sunset based on geographic coordinates. You can also specify offset (how long before or after event) should program end.

Now you can put the sunwait into your local cron table and chain it with command that should be executed after sunrise/sunset occurs.

Here is example how to configure cron to turn on the lamp 15 minutes before sunset and turn it off 15 minutes after sunrise.

crontab -e
10 15 * * * /usr/bin/sunwait sun down -0:15:00 49.230738N, 16.576802E; /usr/bin/mosquitto_pub -h HOST_NAME_HERE -t /home/ROOM_NAME_HERE/command -m "on"
4 15 * * * /usr/bin/sunwait sun up +0:15:00 49.230738N, 16.576802E; /usr/bin/mosquitto_pub -h HOST_NAME_HERE -t /home/ROOM_NAME_HERE/command -m "off"

The lamp is now working. The only problem is the Android app. I thought that adding MQTT client to Android app will be as easy as adding it to ESP8266, but it’s far from truth. There are still some challenges ;-)