26. August 2020

How to run multiple Bash commands in parallel

If you’d like to run multiple commands in shell which is executed one by one, you can simply type:

command1; command2; command3.... commandn

What if you need to run all these jobs in parallel. When you have just single command you can add & and the command will be executed in the background. So the temptation for a solution might be:

command1 &; command2 &;... commandn &

Well, it does not work. The correct solution for oneliners is simple. Just replace ; by &

command1 & command2 & command3 & ... commandn

The same solutions works also for other shells like zsh.

4. June 2017

Workaround: Failed to activate the platformio-ide package

PlatformIO was working without any problem, but after several updates, there was following error message:

Failed to activate the platformio-ide package

The bug is also mentioned at GitHub. It seems that the bug has a relation to the default OS shell.

Try following workaround (it worked for my Linux/ZSH):

  • change default shell to Bash
    sudo chsh -s /bin/bash YOUR_LOGIN
    
  • start Platform IO IDE
  • revert the shell back to your favorite
    sudo chsh -s /usr/bin/zsh YOUR_LOGIN
    

This solution worked for me. Let me know whether you have the same experience.

26. April 2017

How to publish topic by mosquitto_pub to Bluemix

In the previous article about LampESP, I’ve described how to subscribe ESP8266 to MQTT at Bluemix cloud. The remaining question is how to publish data.

Let’s define simple scenario for server monitoring: Server is sending every minute it’s average load to Bluemix for further processing. How to solve it?

You’ll need to create a new device type at Bluemix, let’s call it “server”. Then create new device using the same step like in the previous article.

Then you can run a simple shell script which will push all the data to Bluemix:

#!/bin/bash
# Replace following values by your own
ORG_ID="myid"
DEVICE_NAME="reactor"
DEVICE_TOKEN="psst, something secret"

LOAD=`cat /proc/loadavg`
mosquitto_pub -h "${ORG_ID}.messaging.internetofthings.ibmcloud.com" \
-i "d:${ORG_ID}:server:${DEVICE_NAME}" \
-u use-token-auth \
-P "${DEVICE_TOKEN}" -r \
-t "iot-2/evt/${DEVICE_NAME}/fmt/text" -m "${LOAD}"

If you need more detailed information about communication just add option “-d” to mosquitto command.

Then you can check also logs from the device at Bluemix. You should see something like this:

Closed connection from YOUR_IP. The connection has completed normally.
Token auth succeeded: ClientID='d:ORG_ID:server:DEVICE_NAME', ClientIP=YOUR_IP

Please keep in mind that Bluemix requires certain format of topic name.

In the next article, we will take a closer look how to transport event from a server to ESP8266 by Node-RED.

12. August 2013

IntelliJ Idea support for Bash scripting

Unix users write scripts in their favorite editor. That’s ok, but when you have bigger project then power of IDE could help you. E.g. when editing project based on mix of Python, Bash and Java.

There is very useful plugin for IDEA known as BashSupport.

It will give you syntax highlight, code hinting and CTRL+Click is working the same way like in other programming languages. Very handy.

Here is example:

idea-bash-support

How to install this plugin?

File -> Settings. Search for word Plugins. Click Browse repository. Type BashSupport to search field. Right click on plugin name and choose install. restart Idea.

idea-plugin-installation

Enjoy :-)

Note: This plugin works also on Windows, you need to configure just path to bash (e.g. Bash from Cygwin).

28. December 2011

CGI server? Bash one-liner!

Author: Lordrat

By now you know, that when you need simple server to serve static pages just for you to play, you use Python’s SimpleHTTPServer.

But this time I wanted more: I wanted to to able to use serve images caught by my webcam. CGI server would be quick answer. While there is class in python doing that, I did not wanted to play with it. I needed something quick to do and maybe I was just little lazy to do so this day.

First thought: one window while cycle with cam-grabber, in the other window SimpleHTTPServer. Well, the solution lacked elegance, as it was not one-liner-styled enough. And then it occurred to me: why not use Netcat?

Building Blocks

 

At first, I needed to get data from camera. I had figured how to do that some time ago.

vgrabbj -i vga -d /dev/video0 -a -z 10 -q 100

Let’s just elaborate on that a little bit. vgrabbj takes `640×480` image from video4linux device `/dev/video0`, with enabled brightness adjustment, taking 10 images just to adjust, and using 100% quality of JPEG, and sending it to stdout.

Another building block is “server” to serve something that returned command. NetCat is our fiend. Little-bit of HTTP-fu is needed, but not much.

{ echo -ne 'HTTP/1.0 200 OK\r\nConnection: Close\r\n\r\n'; cat file; } | nc -l 1234

We will be serving JPEG, so we add appropriate HTTP header (unless you can see things in binary garbage).

Content-Type: image/jpeg

It may happen that file-size of image will cripple user’s comfort over slower network. What about speeding it up by sending response gzipped? We will need header

Content-Encoding: gzip

and we will have to use… you guessed it: gzip.

gzip -f

However, gzip will not reduce the size of JPEG much, as it already has entropy quite high. We can decrease quality of image i.g. by switch -q of vgrabbj, but in this case I would prefer ImageMagick for reasons that will become clear in a moment. Another possibility how to deal with this issue is scaling image down, which can be done here as well.

convert - -quality 75 -

It will also be nice to have a non-crippled copy of served file. Can do that: tee.

Serving repeatedly can be done by i.e. by while loop. Small functional enhancement: it will be nice to have so called “auto-refresh”. Adding one HTTP header will do that for us. Say every 5 seconds.

Refresh: 5

Some cooling enhancements? Server header:

Server: CoolServer!

Assembling

 

And now just put it all together.

while :; do { echo -ne 'HTTP/1.0 200 OK\r\nServer: Kwak!\r\nConnection: Close'\
'\r\nRefresh: 5\r\nContent-Type: image/jpeg\r\nContent-Encoding: gzip\r\n\r\n'\
; vgrabbj -i vga -d /dev/video0 -a -z 10 -q 100 | tee omg-`date +'%Y-%m-%d-%H'\
'%M%S'`.jpg | convert - -quality 75 - | gzip -f; } | nc -l 1234; done

Feel free to unwrap it ;-).
And a non-one-liner (little bit more readable) version:

#!/bin/bash
PORT=1234
QUALITY=75
REFRESH=5
HEADER='HTTP/1.0 200 OK\r\n'
HEADER=$HEADER'Server: CoolServer!\r\n'
HEADER=$HEADER"Refresh: $REFRESH\r\n"
HEADER=$HEADER'Connection: Close\r\n'
HEADER=$HEADER'Content-Type: image/jpeg\r\n'
HEADER=$HEADER'Content-Encoding: gzip\r\n'
HEADER=$HEADER'\r\n'
while :; do
    {
        echo -ne "$HEADER"
        vgrabbj -i vga -d /dev/video0 -a -z 10 -q 100 \
        | tee omg-`date +'%Y-%m-%d-%H%M%S'`.jpg \
        | convert - -quality "$QUALITY" - \
        | gzip -f
    } \
    | nc -l "$PORT"
done

Notes

 

  • Some versions of nc may need to be called more like `nc -l -p 1234`.
  • First photo will be served from time you started script, every other from time the last shoot was taken. (Exercise for patient reader: why it is so? ;-) ). This might not be issue, as it speeds up reading of photo on request.
  • Previous can be solved by creating script, that can be called by nc on connection. That way you can create even interactive “CGI” script, that will read (and parse) input from std in and write response to stdout. Not all versions of nc are able to do that.
  • Another exercise: Will you be able to secure your communication using stunnel.

Enjoy ;-)