Basics about dimming LEDs – EMC and flickering

This article is not directly related to the RevPi. I wrote it as a by-product of the DMX article. Since I have done some research on dimming LEDs, I would like to share my current – probably incomplete – state of knowledge on the subject.

What is the issue with dimming LEDs?

LEDs are dimmed by PWM. This is not very complex and saves energy. The basic requirement is that the frequency is sufficiently high so that you no longer perceive flickering. But, you might run into EMC problems. This is simply due to the fact that a PWM is a square-wave signal. Therefore the signal contains a large proportion of harmonics.  These in turn can feed back into the grid or be emitted by the LED luminaire.

But what is so special about LEDs that I am writing an entire article to the subject? LEDs switch much faster than incandescent lamps. Furthermore LEDs are often used as long LED strips. Both may lead to unwanted effects.

The right frequeuncy to use

On the internet you can find various articles on the right frequency for dimming LEDs. In my opinion, many articles are written by “would-be professionals”. The quality of the statements in such articles may therefore be questioned. However, I was able to read out a consistent tendency of many articles regarding the PWM frequency to be used.

Many LED dimmers allow the PWM frequency to be set. Most sources recommend min. 500Hz and an upper limit in the low kHz range. The lower limit can be explained simply by the human eye, which can detect frequencies up to about 70Hz by flickering. In motion, even higher frequencies are perceived. The 500Hz, with a certain safety margin, therefore represents the frequency at which the human eye no longer perceive flickering.

PWM and EMC

I am not an EMC professional and would therefore like to reserve judgement, but I have had my (bad) experiences with dimmers, especially with dimmer switches in connection with guitar amplifiers.

Basically you can assume that the higher the frequency, the greater the risk that the LED lamp will act as an antenna. This should apply especially to the installation of long LED strips. So it makes sense to keep the frequency as low as possible for EMC reasons.

However, problems, e.g. with interference in the music system, can also occur below 500Hz. Impractically, the frequently recommended PWM frequency range lies exactly in the audible spectrum. If such effects occur, it is probably due to a mains feedback. Therefore, the first thing I would do is replace the transformer with another model. If effects still occur after the exchange, I would try to shorten the supply lines or use shielded lines.

PWM-dimmed LEDs during video shooting

If you shoot a video with dimmed LED lighting, e.g. with a mobile phone camera, the LED lighting can flicker in the video. This can be explained by the fact that LEDs switch very quickly and are therefore not permanently on as it looks to the human eye, but switch on and off with PWM frequency. The human eye is simply too slow to perceive flickering above a certain frequency.

A video sensor, however, may be much faster than the human eye. Since the PWM frequency is significantly higher than the repetition rate of the video, the video recording is technically an undersampling of the LED frequency. This results in so-called alias effects. If you are not familiar with this topic and would like to know exactly what it is, I recommend the Wikipedia article on the Nyquist theorem.

One solution would be to synchronise the PWM frequency exactly to the repetition rate of the video, which is rather impractical, at least for home use. The much simpler way is to increase the frequeuncy so that the flickering of the LEDs is faster than the video sensor.

This in turn speaks in favour of raising the PWM frequency. If you want to make slow motion recordings, you even need a higher PWM frequency, because the frame rate of the video is higher. Some PWM dimmers offer frequeuncies of up to a few 10 kHz especially for this purpose.

Effects can also occur in photos. Exposure times of 1/1000 s are common in the photo range. This means that the photo may be triggered at the exact moment when the PWM is in the “off state”. When taking photos, this can be avoided by increasing the exposure time.

Summary

The PWM frequency used is a compromise between EMC and possible flickering effects. You can probably only find out exactly which frequency is the right one through practical trials. As long as no problems become apparent, a PWM frequency of 500Hz seems a good choice to me.

Home Automation – Using the DMX Gateway with LED dimmers

In my previous articles I laid a good foundation for working with the RevPi. Now I would like to focus more on a practical application. Since I want to continue working in C++, I will also use the library I already created in one of my previous article.

The Revolution Pi in home automation

I thought about using the Revolution Pi for home automation a long time ago. A few months ago, I did some renovations in the house and renewed the electrical installation. All the lines from this part of the house now run together centrally. This allows me to switch lights and, if necessary, sockets flexibly. I would now like to realise this with the RevPi. In addition, I have two LED lights that are too bright for me at night. I would like to dim them. Conveniently, the lights are constructed extremely simply from a few LED board elements and a transformer. The individual parts can be easily separated.

DMX dimmers

The most practical solution for dimming LEDs with the RevPi is probably the use of DMX. Kunbus offers a DMX gateway for the RevPi and you can find various LED dimmers on the market that can be controlled via DMX. Depending on the range of functions, the dimmers cost between 40 and 100€ for a 4-channel dimmer, e.g. for RGBW. I found a cheap solution (approx. 20€) in the form of a simple circuit board.  The board is small enough that you can integrate it into a lamp.

Dimming of LEDs

I would like to say a few words about dimming LED lamps in general, because you have to consider a few things. I have spent some time researching on the internet and thinking about this topic myself and I would like to share the state of my knowledge. In order not to overload this article with too much theory, I put the information in a separate article.

The DMX protocol

DMX is a very simple protocol. There is one master in each string and up to 512 addresses that can be accessed. Theoretically, each of the addresses can be a slave. However, a slave often occupies several addresses. Slaves are pure receivers and the received data is not acknowledged in any way. This makes the protocol very clear.

DMX is based on RS485. The master cyclically sends out one byte per address. Each slave can occupy one or more – typically consecutive – addresses. The address(es) are set on the slave unit. The data is simply sent in sequence after a preamble. The meaning of the data content is not defined by the DMX specification, nor does the DMX master know anything about it. Furthermore, the master has no knowledge about the addressing of the slaves.

For a single-colour dimmer, one byte is usually sufficient, but for moving lights, for example, which are often used in event technology, considerably more is required. In this case, axis positions (usually 2 bytes each), the colour wheel, the shutter, the gobo wheel, etc. must be addressed. For this purpose, each slave is manually assigned a start address. The slave then picks up the necessary data from the start address. For example, 4 addresses are needed for an RGBW dimmer like the one I use.

Test setup and configuration

Somewhere I still had a few short pieces of LED strips lying around. I also had a 12V power supply in my inventory. Add to that a DMX gateway and the first test setup was ready.

Setting up the Gatway

First I had to find out how the terminal on the DMX module is wired. The tutorial is currently not mentioning this. The DMX gateway’s operating instructions help.

Photo of the front of a DMX gateway with labelling of the three relevant contacts: GND, DMX+ and DMX-

After a little test to see if my soldering attempts on the LED strips were successful, I moved the installation to my wall.

Photo of the test setup with LED strips, power supply unit and DMX dimmer board

In order to operate the gateway with the RevPi, you have to configure it first. This is done on the one hand with the help of the three rotary switches on the front of the housing and on the other hand in PiCtory. The DMX gateway can be operated in two modes:

  • Fixed configuration via the rotary switches on the front of the housing.
  • Setting via software → The rotary switches must be set to 0,0,0.

Settings in PiCtory

For now, I decided to use the fixed configuration. The DMX gateway is to function as the master. According to the documentation, any address greater than 512 can be set. This is because a maximum of 512 participants can be addressed or 512 bytes can be transmitted with DMX. All addresses above this are therefore not valid DMX addresses.

Screenshot of PiCtory with DMX gateway and expanded selection menu for the outputs

In PiCtory, you must also set whether the fixed configuration or the configuration via software is to be used. There are four options for this under “Outputs”. Only the two “512” are relevant for the master. The upper “512” corresponds to the configuration via software, the lower “512” to the fixed configuration.

The difference can be seen in the Value Editor at the bottom right. In the fixed configuration, only the variables DMX_Out_1, DMX_Out_2, etc. are displayed here. In the software configuration, you will find two configuration bytes Config_1 and Config_2 at the beginning of the list. The exact function of the two bytes you will find in the documentation. These are then designated DMXOut_1, DMXOut_2, etc., i.e. without “_” between “DMX” and “Out”.

If you set the wrong mode in PiCtory, the gateway basically works, only the mapping of the output variables no longer fits. This happened to me in the beginning. In addition, the DMX slave I used started numbering the DMX addresses at 0, contrary to the DMX definition. This led to an additional shift in the mapping and made the confusion complete.

The first test

I set the DMX dimmer to address 0, which actually corresponds to DMX address 1. The dimmer has four channels. Accordingly, addresses 1,2,3,4 are now used by the dimmer.

With piTest -d I first check whether the gateway has been correctly recognised.

Found 2 devices:

Address: 0 module type: 105 (0x69) RevPi Connect V1.0
Module is present
     input offset: 1024 length: 6
    output offset: 1030 length: 5

 

Address: 31 module type: 100 (0x64) Gateway DMX V1.0
Module is present
     input offset: 0 length: 512
     output offset: 512 length: 512

That looks good!

With

piTest -w <variable name>, <value>

you can write to output variables. The following piTest calls make the installed LED strips turn on properly, as you can see in the picture above.

piTest -w DMXOut_1,255
...
piTest -w DMXOut_4,255

Controlling I/Os on the Revolution Pi with Java

After laying the foundations for the use of Java on the RevPi and JNI in the previous article, I am now porting piControlIf to Java. At the beginning I had to clarify the question whether I should write a wrapper for the Linux system functions or one for piControlIf. Writing the wrapper for piControlIf seemed more manageable to me.

In retrospect, it was quite a lot of hard work, since accessing the arrays of objects via JNI is quite cumbersome. Probably a different approach would have been better. But afterwards one is always wiser. Finally, I would like to show in this article how PiControl can be used in Java. Everyone can implement this in the way that suits them best.

Porting the API to Java

First of all, we need to create a new project, which I call PiControlIfJava. Create a class with the follwing methods:

private native void open();
private native void close();
public native int reset();
public native int read(long Offset, long Length, short pData);
public native int write(long Offset, long Length, short pData);
public native int getDeviceInfoList(SDeviceInfo pDev);
public native int getBitValue(SPIValue pSpiValue);
public native int setBitValue(SPIValue pSpiValue);
public native int getVariableInfo(SPIVariable pSpiVariable);
public native int findVariable(String name);

In addition, a lot of constants from piControl.h are needed, such as PICONTROL_LED_A1_GREEN. I simply copied all the constants into the Java file and did not check which of them are actually needed.

As far as possible, I took over the structure of piControlIf 1:1 from the C++ implementation. I named classes, methods etc. the same and merely adapted them to the Java style, e.g. methods always begin with a lower case letter. I put the classes in a package with the name com.kunbus.revpi.

I also removed the Hungarian notation, which already has no place in C code  and even more so in Java. Linux Tovarlds put it in his typical blunt way as follows:  “Encoding the type of a function into the name (so-called Hungarian notation) is brain damaged […]”. Even Microsoft, probably the main cause of the spread of this bad habit, now advises against it. If you need something to laugh about, you’ve come to the right place.

Converting the variable types

Since the variable types in C differ in some corners from those in Java, e.g. there are no unsigned types, I used the following conversions:

C

Java

Unsigned byte

int

Unsigned short

int

Unsigned int

long

Char *

String

After the porting work is done, you have to included the reference to the external library. I named the library that I am about to create with the JNI interface PiControlIfWrapper.

static {
    System.loadLibrary("PiControlIfWrapper");
}

Creating the JNI interface

First of all I have to export the interface to C-Files. This is done via

<path>\javac -h . PiControl.java SPIValue.java SPIVariable.java SDeviceInfo.java

The Java compiler now creates an .h-file with entries of the following type:

JNIEXPORT jint JNICALL Java_com_kunbus_revpi_PiControl_findVariable (JNIEnv *, jobject, jstring);

I create a project PiControlWrapper, as mentioned above. Now I insert the generated header file and create a cpp file with the same name in the project. I configure the project so that a dynamic library is generated by the compiler.

Don’t forget to include the libpiControlIf.so  and remember to specify it in Netbeans for all configurations (debug/release). Otherwise, you will only notice the error at the very end when you run the Java test program. Or, like me, you’ll spend an hour looking for the cause.

Accessing variables with JNI

It is noticeable that the method name is extremely bulky, as Java uses the so called fully qualified name, i.e. including the package. Two parameters are always given to each method. A JNIEnv* and a jobject. The jobject is simply the Java object from which the call was made, i.e. a kind of this reference. JNIEnv* allows access to all data that would also be possible from the Java object.

Now it’s getting a little tricky here. As great as Java is, as complicated is JNI. You have to get the impression that JNI is intentionally kept complicated to discourage users from working with it. Explaining all the code is beyond the scope of this blog. Therefore, I will only show a few extracts.

Primitives such as int are simply passed as jint and treated directly in C like a normal integer. With strings it’s getting a bit more complicated. If I want to access the string in the parameters in the findVariable method, it looks like this:

const char *varNameCStr = env->GetStringUTFChars(name, NULL);

 if (varNameCStr == NULL)
        return;

//do something with varNameCStr

 env->ReleaseStringUTFChars(name, varNameCStr);

If fields of a class have to be read out, the tip effort becomes even more:

class spiValueClass = env->GetObjectClass(spiValue); //get the class identifier

jfieldID addressFieldID = env->GetFieldID(spiValueClass, "address", "I"); // get the field identifier inside the class

if (addressFieldID == NULL) {
    return;
}

jint = env->GetIntField(spiValue, addressFieldID); //  geht the actual value of the field

“I” is the so-called field signature, in this case it is an integer value, for a double it would be a “D”, for example. Here you will find a complete table with the field signatures.

Accessing fields with JNI

I don’t want to go into the finer points of JNI. There is enough documentation on the net. However, I would like to point out a pitfall that I have picked up. If a field in an object is another object, e.g. a string, I need the fully-qualified-class for the field signature. Instead of a simple “I” for integer, it then becomes L<fully-qualified-class>;. It could hardly have been made more complicated.

If I want to access a string within a field, it looks like this:

jfieldID varNameFieldID = env->GetFieldID(spiVariableClass, "varName", "Ljava/lang/String;");

That’s enough about JNI and on to the test program. After you finished to implementation of the wrapper,you created the dynamic library. Finally you have to copy the .so file into the directory of the Java project PiControlIfJava so that it can be found by Java.

A test program for the javanese piControl

For this I used an old friend: RevPiBlink. I would now like to implement this in Java. After I created a new Project called RevPiBlinkJava, I just copied the code from RevPiBlink and adapted it to Java. Fortunately, Java is not far away from C++. So this looks quite familiar:

public class PiControlIfJavaTest {
    public static int getRevPiLEDAddress(PiControl piC) {
        SPIVariable spiVariableOut = new SPIVariable("RevPiLED", 0, 0, 0);
        if (piC.getVariableInfo(spiVariableOut) != 0) {
            return -1;
        } else {
            return (spiVariableOut.address);
        }
    }

    public static void main(String[] args) throws InterruptedException {
        PiControl piC = new PiControl();

        SPIValue spiValue = new SPIValue(0, 0, 0);
        spiValue.address = getRevPiLEDAddress(piC);
        spiValue.bit = 0;
        spiValue.value = 1;

        while (true) {
            spiValue.value = spiValue.value ^ 0x01;
            piC.setBitValue(spiValue);
            sleep(1000);
        }
    }
}

Now you only have to use the already created Java library PiControlJava. In Netbeans right click on the Libraries folder in your project and Add JAR/Folder and then add PiControlIfJava.jar.

Screenshot of Netbeans showing the context menu of the libraries folder

Now you can create and run the project.

The garbage collector and its influence on performance

If you need a certain temporal reliability, you make sure that the garbage collector does not start if possible. This leads the Java principle somewhat ad-absurdum, because the garbage collector is actually a sensible thing. It ensures that no memory leaks occur due to lost references. Unfortunately, the Java inventors have fully relied on the GC and Java does not support the possibility of explicitly destroying objects, as one is used to from C++ with delete. Objects are simply dereferenced and at some point the garbage collector comes along and releases the memory.

In my student days (a few years ago) we did measurements with the garbage collector. We had a program that created many objects and released them again. It was easy to see that after a while the garbage collector started up and from then on it was almost permanently working in the background. From this point on, the performance was significantly lower than at the start of the program.

A way around the garbage collector

Tests with my project at university showed that the program became significantly faster when objects that were no longer needed were not simply released into nirvana (dissolve reference). Instead, I did object recycling. A class accepted objects that were no longer needed and managed them in a list. Instead of simply creating new objects, it was first queried whether an orphaned object of the class already existed. During the runtime of the program, there was no longer any drop in performance. The overall performance also suffered almost not at all.

As mentioned, the tests were carried out several years ago and I cannot say how far the statements are still valid. If the garbage collector is a real problem for the project, you have to pay attention to the handling of objects. Specifically, avoid classes that implicitly create new objects, e.g. strings.

Accessing system functions in Java via JNI

In the last article I took the first steps with Java on the RevPi. Currently, however, we do not yet have access to the extension modules. For this we need access to system functions. Since Java is designed to be fundamentally platform-independent, Java (to my knowledge) does not provide any direct possibility to access Linux system functions. Java does, however, offer an interface for system-related programming, the so-called Java Native Interface or JNI.

Originally, I would have liked to use a ready-made library for accessing the Linux system functions. Interestingly, the web did not offer much in this regard. I would have guessed that there are countless use cases for this. Very likely I lack some knowledge in this context and was simply looking for the wrong thing.

A test program

Now comes the interesting part. But before I implement piControlIf right away, I’ll start with a test program. It’s been years since my last attempts with JNI.

package jnitest;

public class JniTest {
    public native void helloWorld();

    public static void main(String[] args) {
        JniTest jniTest = new JniTest();   
        jniTest.helloWorld();
    }
}

Before we can execute the program, we must first include the C code. To do this, the Java compiler creates a corresponding header file with

javac -h . <files>

If necessary, the complete path must be specified instead of just javac if the path is not stored. In my case it looks like this:

"c:\Program Files\Java\jdk-15.0.1\bin\javac" -h . JniTest.java

Issues with the tool

Now things are going to get messy again, because another gap appears, caused by the not yet finished transfer of Netbeans to Apache. I installed a new Netbeans. Netbeans comes with rudiemntary C-support by default. The old C/C++ plug-in is no longer available for the current Netbeans version. Actually, however, old plug-ins should still be able to be installed. For some reason, however, the old plugin would not install.

After a long search, it looks as the fault was not so much with Netbeans as with Java. So back to the old Java version. To do this, simply change the relevant lines in C:\Program Files\NetBeans-12.1\netbeans\etc\netbeans.conf. It is best you just copy the line and comment out the original line:

#netbeans_jdkhome="C:\Program Files\Java\jdk-15.0.1"
netbeans_jdkhome="C:\Program Files\Java\jdk-14.0.1"

And lo and behold, the plugin installs without complaining. Exit Netbeans and write again the current version into netbeans.conf.

netbeans_jdkhome="C:\Program Files\Java\jdk-15.0.1"
#netbeans_jdkhome="C:\Program Files\Java\jdk-14.0.1"

The matching dynamic library in C++

Create a new C++ project JniTestJava for a dynamic library. Create jnitest_JniTest.cpp and move the created header file from the Java project into the C++ project. Simply move the file to the appropriate project directory.

#include<jni.h>
#include<stdio.h>

#include jnitest_JniTest.h

JNIEXPORT void JNICALL Java_jnitest_JniTest_helloWorld(JNIEnv *, jobject) {
    printf("Hello JNI\n");
}

When you try to create an empty library, you will see the files jni.h and jni_md.h  are missing. With the following lines on the RevPi you will find the missing files.

/usr/find | grep jni.h
/usr/find | grep jni_md.h

Lo and behold, the files exist, only the symlink is missing. So.

sudo -s ln /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/include/jni.h jni.h
sudo -s ln /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/include/linux/jni_md.h jni_md.h

Now you can create the library. The file libJniTestJava.so will be generated. I don’t know if there is a nomenclature for wrapper libraries in the Java world. I thought it appropriate to add Java at the end to make it clear that the .so file is intended for Java (hence the name). Now copy the .so file into the appropriate directory.

When I looked at the RevPi, I was a little surprised why the program did not end up in the directory I was used to from C. As a standard, with Java, Netbeans simply puts the file in /home/pi/NetBeansProjects and not in the long path as with the C/C++ compiler. Makes things easier. However, the directory can be adjusted in the settings of the Java platform.

Provided you are in the directory where the .so file is located, copy the library to the java project folder.

sudo cp libJniTestJava.so /home/pi/NetBeansProjects/JniTest.

Back to the Java program

After I created the .so file, I still have to tell the Java program what the file is called. To do this, I add the following lines to the class JniTest:

static {
    System.loadLibrary("JniTestJava");
}

We also have to tell Java-VM where to find the .so file. We do this with the argument -Djava.library.path=<path>.  Since we put the file directly into the project directory, <path> is simply “.”. We add that under PropertiesRun VM Options.

I was a bit puzzled by the compiler warning:

warning: [options] bootstrap class path not set in conjunction with -source 8
1 warning

After a short investigation, however, the all-clear was given. The warning only means that although Java8 is compiled, newer libraries are used and the project may not run in a Java8 environment. A closer look reveals that, in contrast to C/C++, the compilation process takes place on the work computer. Only the compiled program is copied to the RevPi and executed there. Of course, Java13 is running on the workstation and not Java8.

The procedure is actually consistent, since the Java code is portable. Let’s see if this will causes trouble in the future with the libraries at some point. According to research, a –release 8 is sufficient to ensure that the compiled program really runs in a Java8 environment. For lack of a use case, I could not try it out yet. If necessary, however, you can always fall back on compiling the program on the RevPi.

How to use Java on the Revolution Pi – Netbeans setup

With Java, there are clearly two camps: the opponents and those who use it. During my studies I once heard the saying: “I don’t like Java. My computer crashes so slowly”. The fact is that Java is resource-hungry and slow, at least at start-up. This is due to the basic principle of Java. Java only generates an intermediate code when creating a program. The actual compilation takes place only during execution, the so-called Just-In-Time Compiling. In return, I get largely platform-independent code, and once the code is executed, it is optimised for the system. Current Java compilers even optimise continuously in the background.

Java in industrial use?

Deterministics are not a given in Java, which is mainly due to the garbage collector. Although there were efforts to do this in the past, they seem to have fallen asleep. If the requirements for real-time capability are not excessively high, however, this is not a problem. Many things can be solved much more elegantly than in C(++).

I am a big Java fan. Java is like C++, except that it makes my life more comfortable in many places. Dealing with strings or anonymous classes, to name two examples.

Setting up Netbeans for Java on the RevPi

My first attempts came to nothing because my installtion of Netbeans couldn’t find anything. After a bit of research on the internet, I came to the conclusion that it might have something to do with the dirty hack to get Netbeans 11 to work. Fortunately, there is now Netbeans 12 and it works without workarounds.

Now the RevPi must be set up as a platform in Netbeans. You can do this via the main menu Tools Java Platforms. You can now setup a new platform via the Add Platform button.

Screenshot of netbeans showing the Java Platform Manager and the Button for adding a platform

The RevPi is logically a remote platform, so we choose Remote Java Standard Edition.

Screenshot of netbeans showing the "Add Java Platform" dialog form for selecting the platform type

On the following page of the dialogue window, the name and data of the platform are specified. In our case, for example, “pi” and the IP address of the RevPi, as well as the user name and password. The Remote JRE Path you have to enter:

/usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre

Screenshot of netbeans showing the dialogue form "Add Java platform" with the input form for the platform setup, such as host name, etc.

When the setup of the new platform is complete, it looks like this in the Platform Manager:

A first Java project

After creating a new project, I wanted to reconfigure the project to run on the RevPi. Via Project PropertiesRun Runtime Platform, you should now be able to set the RevPi as the platform. However, the list with the platforms remained empty.

At this point I searched for the error for a long time. I looked at many tutorials on the internet, but none of them described that Netbeans only offers the platforms that support the set format in the Runtime Platform. With my Raspian version (Jessy), it only goes up to JDK8. Therefore, we first have to set SourcesSource/Binary Format → to JDK8. Now the runtime platform can also be set.

Now create a new configuration via RunConfigurationNew. Since I have several RevPis running, I called the configuration pi@<IP address> for better differentiation.

Then quickly write a line that outputs “Hello”.

package revpijavatest;

public class RevPiJavaTest {
    public static void main(String[] args) {
        System.out.println("Hello");
    }
}

Unusual from C programming is that the output of the program appears in the same window as the compiler messages. That’s why I didn’t see my “Hello” at the beginning and suspected an error.

Now you can actually get started with Java development on the RevPi. What is missing, however, is access to the RevPi specifics, i.e. the GPIOs.  To do this, we need to access system functions, which can be done with the so-called “Java Native Interface”, or JNI for short.

 

MQTT – Subscribing to Topics

In the last article I described how you can easily publish data via MQTT in C/C++ with the help of the library paho. Receiving data is just as easy. MQTT provides that you can subscribe to topics for this. Data published under these topics is then sent.

Waiting for callback

As in the last article, I will again use the library paho. The core of the subscription is a callback method that informs the application about the arrival of new data.

int MQTTClient_messageArrived(void *context, char *topicName, int topicLen, MQTTClient_message *message)

The parameters are largely self-explanatory. context refers to a user-defined date that is passed at the time of registration of the callback method. contex can also simply be NULL if it is not needed.

The callback method is registered with

int MQTTClient_setCallbacks (MQTTClient  handle, void *context, MQTTClient_connectionLost *cl, MQTTClient_messageArrived *ma, MQTTClient_deliveryComplete *dc)

With this method, callbacks can be set up for various purposes. In the specific case, I only want to receive published messages. With this, the call looks like this:

int MQTTClient_setCallbacks(client, NULL, NULL, messageArrived, NULL);

I implemented the callback method in such a way that it simply outputs the topic and the message. In a productive case, the data would perhaps be stored in a database or prepared in a GUI.

int MQTTClient_messageArrived(void *context, char *topicName, int topicLen, MQTTClient_message *message) {
    printf("Topic: %s \t Message: %.*s\n", topicName, message->payloadlen, (char*) message->payload);
    MQTTClient_freeMessage(&message);
    MQTTClient_free(topicName);
    return 1;
}

The return value 1 or true signals to paho that the message could be processed properly. Otherwise, paho tries to deliver the message again.I didn’t do any further experiments with this. However, I would use this function with great caution. I could not find a way to define something like max retries or see if a fixed number is set. There is a danger of getting the application into a kind of infinite loop and possibly provoking buffer overflows.

It is important to know that topicName and message are not automatically released again after the callback method returns. For this, MQTTClient_freeMessage and MQTTClient_free must be called. In the case of topicName, the memory can also be released with the usual C methods, which, according to the documentation, leads to problems under Windows when using different compilers. It is therefore recommended to always use MQTTClient_free.

Download the project here.

The IP address in the demo project must be adapted to the respective case. Make sure that the MQTT server is online. Then the programme can be started.

For testing, a message is generated with the command line tool mosquitto_pub:

mosquitto_pub -t test -m hello

Now you should see the following output in the terminal:

Terminal session that displays the output of revpimqttsub when the above message is sent.
Output of revpimqttsub

Synchronous or asynchronous, that is the question

As an alternative to the callback method described above, the following method can also be used to poll for new messages:

int MQTTClient_receive ( MQTTClient handle,char **topicName, int *topicLen, MQTTClient_message **message,unsigned long timeout )

As you can see from the two ways of receiving messages, paho can be operated synchronously and asynchronously. Depending on the method used, a few things have to be taken into account, which I have disregarded for my simple demos.

If callback methods are registered with MQTTClient_setCallbacks(), paho runs asynchronously. This means that paho also takes care of ensuring the quality of service. So this should be the preferred way. If the API is to be accessed by several threads, however, the MQTTAsync API must be used, as the normal API is not thread safe.

If no callback methods are stored, paho runs synchronously and the application must ensure via regular polling that the keep-alive mechanism is running and Quality of Service is ensured. This can be done by regularly calling MQTTClient_receive(). If the application only sends data but does not subscribe to topics, MQTTClient_yield() can be used instead. If QoS1 or QoS2 is to be used, MQTTClient_waitForCompletion() must also be called after sending a message.

 

MQTT – Publishing Messages

One of the main fields of application for the RevPi is in the (I)IOT area. MQTT is almost always mentioned in this context. MQTT is a protocol for exchanging data. Its popularity does not stem from the fact that it is very extensive, but because it is so incredibly simple.

A MQTT system basically consists of a MQTT broker (server) and clients. The server is basically not much more than a postman. Clients can send and receive messages. Each message is grouped into a so-called topic. A message of a client always consists of a message text and the topic under which the message should be published. Other clients can subscribe to Topics and receive all messages of the respective topics from the MQTT-Server without being asked. For example, a thermometer could regularly publish the current measured values under the topic temperature. Another client could display the current temperature and yet another could, for example, save the data.

The topics can be arranged hierarchically. This then works in the same way as a directory tree. If, for example, there are several thermometers, one in the house and one outside, the measured values could be published under temperature/outside, temperature/inside. If a client wants to receive all temperature data, it can subscribe to temperature/# and will receive all published messages in temperature and below.

MQTT offers even more functions. There has already been enough written about this on the internet. Therefore I will focus on the actual topic.

Two more notes, MQTT is case sensitive. The topics test and Test are different Topics. Also important to know that MQTT does not know any data types. All messages are strings. The application must take care of the conversion.

Preparation

First of all I need a MQTT server. One of the probably most common is Mosquitto. Practically it is included in the archives of many Linux distributions. So before I harass my work computer, my private computer has to be used. Of course I can run the MQTT-Server on the RevPi. Mosquitto also comes with Raspian/RaspberryPi OS. But that would only be half as interesting if I send the messages on the RevPi in a circle. A second RevPi serves the same purpose.

Install the Mosquitto-Broker and the Mosquitto-Clients:
sudo apt-get update
sudo apt-get install mosquitto
sudo apt-get install mosquitto clients

After the installation a small test on the server computer.
In one with

mosquitto_sub -t test

a client is started, start listening to the topic “test”.
In another terminal window

mosquitto_put -t test -m hello

a message is sent for the topic. If everything works now, the message should appear in the first terminal window.

Next I check if the connection between RevPi and my computer works correctly. For this the Mosquitto-Clients must be installed on the RevPi:

sudo apt-get update
sudo apt-get install mosquitto clients

Now a message is to be sent from the RevPi to the MQTT server on the PC. For this we need to know the destination IP address, in my case the 192.168.178.49.

mosquitto_pub -h 192.168.178.49 -t test -m helloagain

Publish Messages

Now I want to integrate MQTT into my application. There are several open source libraries for this. I chose paho, because I found simple examples for this right away. The necessary headers for C do not seem to come with the current distribution on the RevPi. Therefore I take the data from GIT. Before the make-run I have to install the developer files for SSL.

git clone https://github.com/eclipse/paho.mqtt.c.git
cd org.eclipse.paho.mqtt.c.git
sudo aptitude install libssl-dev
make
sudo make install

Now paho is ready for my purposes. Based on the example on the Eclipse page I wrote a small program:

#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <MQTTClient.h>

using namespace std;

const char *ADDRESS = "tcp://192.168.178.49";
const char *TOPIC = "test";

int main(int argc, char* argv[]) {
    const char *MESSAGE_STRING = "Hello, this is a message from the RevPi";

    MQTTClient mqttClient;
    MQTTClient_connectOptions connectionOptions =     MQTTClient_connectOptions_initializer;
    MQTTClient_message message = MQTTClient_message_initializer;
    MQTTClient_deliveryToken deliveryToken;

    MQTTClient_create(&mqttClient, ADDRESS, "RevPi", 
    MQTTCLIENT_PERSISTENCE_NONE, NULL);

    connectionOptions.keepAliveInterval = 20;
    connectionOptions.cleansession = 1;

    if (MQTTClient_connect(mqttClient, &connectionOptions) != MQTTCLIENT_SUCCESS) {
        printf("Failed to connect\n");
        return (-1);
    }

     // Copy the message string to avoid compiler warnings
    char *s = new char[strlen(MESSAGE_STRING)+1];
    strcpy(s,MESSAGE_STRING);

    message.payload = s;
    message.payloadlen = strlen(s);
    message.qos = 0;
    message.retained = 0;

    MQTTClient_publishMessage(mqttClient, TOPIC, &message, &deliveryToken);
    MQTTClient_disconnect(mqttClient, 5000);
    MQTTClient_destroy(&mqttClient);

    return (0);
}

The IP address in the constant ADDRESS must be adapted to the corresponding server.

The library paho-mqtt3c must be given to the linker. How this is done in Netbeans is shown in the previous articles.

Make sure that the MQTT server is online and a client is subscribed to the topic. The result should look like this:

Screenshot of a terminal session showing mosquitto_sub running, subscribed to to topic "test"
Terminal session with mosquitto_sub

The demo project

Versions and Packages

Versioning

As mentioned in one of the previous articles, libraries should be versioned. Here it becomes a bit confusing. The versioning of a library is shown on the one hand in the file name and on the other hand in the so-called so-name. The filename typically represents the real version of the library. The version is assigned to the file name libXYZ.so.<period1>.<minor>.<period2>.<release>

<period2> and <release> are optional. This nomenclature is established, but is not a technical necessity. As mentioned above, the file name is not everything. A library also has an internal name, the so-name. It typically only changes if the so-called Application Binary Interface (ABI), the binary interface through which an application communicates with the library, does not change backwards compatible. The so-name is set when linking via the parameter -Wl,-soname=<so-name>.

The ABI should not be confused with the API. A change of the API may result in an ABI change. But there are other reasons for ABI changes, especially when working with C++, see here. Another reason to change the version in so-name is if the behaviour of the library changes incompatibly.

Typically the so-name corresponds to the first one or two digits of the real version. If you put the library to /usr/lib or /usr/local/lib and execute ldconfig, ldconfig automatically creates a symlink with the so-name pointing to the real library. For example, if the library file name is libDemo.so.2.3 and the so-name is libDemo.so.2, then ldconfig will create a symlink named libDemo.so.2 pointing to libDemo.so.2.3.

For an existing library you can read the so-name with the following call:

objdump -p <library> | grep SONAME

A new version of libpiControlIf.so

As basis I used the dynamic library piControlIf from the previous article. Netbeans doesn’t seem to offer a special setting for the so-name, so I added the parameter -Wl,-soname=libpiControlIf.so.1 to the Project Properties in Build → Linker → Additional Options. In addition, I added the current version, in our case .1, to the file name Build → Linker → Output.Screenshot of Netbeans Project Properties dialog showing where to set the so-name

After rebuilding the library, the library must be moved to /usr/local/lib. Don’t forget to delete the old file first or make sure to overwrite it.

What is still missing is the link name of the library. This is again a symlink, but this time without any version. It typically points to the latest version of the library. The link must be created manually:

ln -s libpiControlIf.so.1 libpiControlIf.so

If you recreate RevPiBlinkDynamicLib from the previous article, the dynamic linker now requires libpiControlIf.so.1. A look with ldd shows

ldd revpiblinkdynamiclib

Screenshot of a terminal session showing the output of ldd revpiblinkdynamic

Creating a Debian package

libpiControlIf is now ready for use, at least on one RevPi. If I want to use the library on other RevPis, I have to copy the library + header to the appropriate place, run ldconfig and set the linker name. It is much easier to create a deb package from the library. Then we can install it on every RevPi using the package manager.

If you want to take the rocky road to creating a Debian package, you should read the various tutorials on the Internet, like this one. I tend to make things as complicated as possible, but here the working minimalism got me, because Netbeans can create a package for me without much effort.

In the Project Properties at Build → Packaging → Package Type set Debian Package.

Screenshot of Netbeans' Project Properties showing where to set the Package Type to Debian Package

Then two lines below on Packaging Files (Build → Packaging → Packaging Files). Here in the tab enter the data of the library, like version etc. In the line Package the name of the package is entered. Netbeans also uses this name for the file name of the package. By default Netbeans name the file according to the file name of the program, i.e. libpiControlIf.so. Netbeans automatically adds the file extension .deb to the file name. To name a package something .so.deb seems pointless to me, because the package contains also additional file like the header files. Therefore I named the package libpiControlIf (without .so).

Screenshot of Netbeans Packaging options showing where to setup the package name, etc.

It is important to enter the right one in Architecture, because the package manager must after all know whether the package is suitable for the platform.

Which architecture is currently running on a Debian system can be found out here:

dpkg --print-architecture

When executed on the RevPi, armhf is displayed:

Screenshot of a terminal session showing the outout of dpkg --print-architecture

So register armhf in Packaging Files → Info → Architecture

Besides the library the deb package should also contain the header files piControlIf.hpp and piControl.h. We can add these via Packaging Files → Files → Add Files from Directory. As target directory (File or Directory Path in Package) we specify /usr/lib or /usr/include (without local/), because this time everything is done by the package manager.

Screenshot of Netbeans Packaging options showing where to manage the containing files and symlinks

Last but not least we add the symlink for the linker name via Packaging Files → Files → Add Softlink.

Screenshot of a Netbeans dialog for adding a new symlink to a package

With a right click on the project More Build Commands → Build Package the package is created on the RevPi in PiControlIf/dist/Release/GNU-Linux/package.

The package can now be installed with apt. Important, do not forget ./ before the file name.

apt install ./libpiControlIf.so.deb

The files should then end up in /usr/lib and /usr/include as set. You may have to delete old versions from /usr/local/lib and /usr/local/include, because the files in the /usr/local directories are used first. This can then lead to undesired effects. Probably it is deliberately chosen to overide existing files in /usr/lib and /usr/include with a new version during development.

Please note: The deb package is not an official Kunbus package and is not maintained by the development. Maybe I can persuade my colleagues to offer it officially in the future.

 

A library for piControlIf

A static library for piControlIf

First a new project piControlIf is created for a static library. Then the following files are added

    • piControlIf.cpp
    • piControlIf.hpp
    • piControl.c

Since I didn’t change anything in the files there shouldn’t be much to debug. Since I want to create a library that can be used productively, I change the configuration to Release by right-clicking on Project, Set Configuration → Release.

After building the project, the files are all located in the project directory on the RevPi. I would like to make the library and the corresponding header files globally available. As described in the previous article, the directories in /usr/local are provided for this purpose. /usr/local/lib for the libraries and /usr/local/include for the header file. So the files are copied as follows:

sudo cp lilcontrolif.a /usr/local/lib/
sudo cp piControl.h /usr/local/include/
sudo cp piControlIf.hpp /usr/local/include/

A test program for the library

Now quickly create a test program. For this I simply used RevPiBlink from one of the previous articles and made RevPiBlinkStaticLib out of it. Remove the unnecessary files piControlIf.cpp, piControlIf.hpp, piControl.h, which are already available on the RevPi. In the include statement for piControlIf.hpp the quotes must now be replaced by <>, so that the compiler no longer searches for the files in the project directory:

#include <piControlIf.hpp>

Last but not least the linker has to be told that libpicontrolif.a should be included. As described in the last article this is done via Project Properties → Build → Linker → Libraries → Add Library. Alternatively, you can also use Project Properties → Build → Linker and enter -lpicontrolif in Additional Options.

The parameter -l<library> tells the linker to include the corresponding library. It is important to include the library without the preceding lib or the file extension .so. If the project is executed, A1 flashes every second.

A dynamic library for piControlIf

Why always compile the piControl library permanently into a program. It makes much more sense to make a dynamic library out of it. For this I use the existing library project “piControlIf”. To create a dynamic library instead of a static one, simply select Project Properties → Build → Configuration Type → Dynamic Library.Screenshot of Netbeans' Project Properties showing the configuration type setup for a dynamic library

When the library is created, you will find the file libpiControlIf.so in the output directory. What happens in the background? The project is compiled with the parameters -fpic or -fPIC. PIC stands for “Position independet Code”. This is a prerequisite for a dynamic library. -fPIC (capitalised) is only relevant for special architectures. On X86 systems it makes no difference according to man page. ARM systems do not seem to be affected either. With -shared the linker gets the instruction to create a .so file.

Wisely, Netbeans creates the file this time in upper and lower case while static libraries were all lower case. This avoids naming conflicts when there is an equivalent static and dynamic library on the system.

As with the static library, the files must now be copied to /usr/local/lib. Since the header files already exist, only the library file must be copied:

sudo cp libpiControlIf.so /usr/local/lib

A dynamic library must be found by the dynamic linker at runtime. In contrast to the static linker, the dynamic linker does not search through all files in the corresponding directories, but consults a table. This table is updated with ldconfig.

sudo ldconfig

With ldconfig -p you can check if the library was found correctly.

ldconfig -p | grep libpiControlIf

Now we need a test program. Again I copied RevPiBlinkStaticLib and called it RevPiBlinkDynamicLib. In Project Properties → Build → Linker → Libraries the static library picontrolif must be replaced by the dynamic piControlIf.

With ldd you can see which dependencies our program has

ldd RevPiBlinkDynamicLib

 

Using libraries on Linux

A few (more) words about libraries

So far I have always included piControlIf.cpp, piContorlIf.hpp and piControl.h in my projects. Since the files are always the same, it makes much more sense to make a library out of them. Linux (and other operating systems) distinguish between static and dynamic libraries. Static libraries are basically nothing more than a precompiled piece of program that is linked into the finished target program. The advantage is that the corresponding code does not have to be recompiled for each project. In addition, instead of many source code files, I may have only one library to include. Libraries in general have under Unix systems a “lib” at the beginning of the file name. Static libraries end with an “.a” and are usually written in lower case.

Dynamic libraries, unlike static ones, are not part of the finished target program, but a reference is set. The library is only loaded at runtime. Under Linux dynamic libraries are called “Shared Objects”, which reflects in the file extension “.so”. Shared Object is actually more appropriate, because these libraries are usually only available once in the system. Many programs share the library. This saves memory and allows an easy exchange of the library without touching the actual program.

For the sake of completeness, it must be mentioned that there is also a weakness of dynamic libraries. The target program usually does not bring the library itself, but depends on the library being brought by the distribution. Often the libraries depend on other libraries. If the version of a library changes so that it is not downward compatible, all dependent programs and libraries must be updated. Although libraries can be kept in several versions, dependencies of the libraries among each other quickly lead to a confusing mess of versions and dependencies. Despite this lack, dynamic libraries are part of the basic structure of the Linux world, not least because security updates can be inserted very easily.

First steps with libraries

First I would like to create a small test library. Hierfür erzeugen wir ein neues Projekt. Netbeans already brings along a template for static libraries in which everything is already preset: New ProjectC/C++C/C+ Static Library. Creative as I am, I call my project “StaticTestLibrary”.

Screenshot of Netbeans' New Project dialog showing how to create a new project for a static library

A file called statictestlibrary.cpp is added to the project. In this file I added a method with the meaningful name test(), which displays a text on the console.

#include<stdio.h>

void test() {
    printf("Hello, this is the static test library");
}

The compiler creates the file statictestlibraray.o. This can be found in the project directory on the RevPi under build/Debug/GNU-Linux. When creating an application the linker would simply link all .o files together to the finished program. But I can also link the file to any program. If I want to create a static library I have to create an archive from it:

ar crs libstatictestlibrary.a statictestlibrary.o

The parameters crs mean

    • c: force new version
    • r: create, if not yet existing
    • s: create index

The linker can use an archive like an .o file. An archive can of course contain several .o files. On several internet pages I read that it is necessary to execute ranlib. The man-page of ar says: “Running ar s on an archive is equivalent to running ranlib on it.”. This looks to me strongly as if a separate ranlib run is superfluous.

Let’s see if everything worked out well. For this purpose, I am creating a new application project StaticTestLibraryDemo. I put the created library on the development computer into the directory of the new project with

scp pi@192.168.178.32:~/netbeansprojects/StaticTestLibrary/build/Debug/GNU-Linux/statictestlibrary.a .

To add the library open the project properties and choose Add Existing Item and select the file libstatictestlibrary.a. Adding the file to the project causes Netbeans to copy the file to the RevPi before creating the project.

To include the library in the build process, choose Project Properties at BuildLinker and type in the library name in the line Additional Options.

Screenshot of the Netbeans Project Properties showing where to add the StaticTestLibrary to the project

I created a small test program that simply calls the method test() from the library.

void test();

int main(int argc, char** argv) {
    test();
    return 0;
}

The declaration void test(); is important. Otherwise the compiler will complain, because it does not know test(). Normally you create a header file with exactly this declaration for statictestlibrary and include it with #include in the target program. But for demonstration purposes I kept it simple.

In the end, I also did some unnecessary work, because Netbeans creates the library completely automatically. When creating StaticTestLibrary you can find the file libstatictestlibrary.a under StaticTestLibrary/dist/Debug/GNU-Linux. Netbeans is also smart enough to add a “lib” to the front of the project name. Nevertheless it is always good to know what is happening behind the scenes.

System-wide availability

To make libraries available system-wide Linux offers two possible storage locations: usr/lib and /usr/local/lib. The difference between the two directories is that /usr/lib should only contain files installed by the package manager. Files that are installed past the package manager should be stored in /usr/local.

Basically, /usr/local maps the structure as in /usr (only without local). The general rule is, files from the package manager are placed in the directories directly under /usr, everything that runs outside the package manager should be stored into /usr/local. This helps to keep an overview on your system. There is also no risk that the package manager will overwrite manually added files. The basic structure of the file system is described in the Filesystem Hierarchy Standard (FHS), so that the structure of Unix systems is largely identical and programs can be ported easily.

The file is copied to /usr/local/lib as described:

sudo cp libstatictestlibrary.a /usr/local/lib/

For testing, the compiler or linker must be told that it should include the library. This is done at Project Properties → Linker → Libraries → Add Library

Strangely you get to see a complete file dialog, but you just have to enter libStaticTestLibrary.a in the line File name: and click Select.

Screenshot of a Netbeans dialog box in which the library name is entered

Now the programme can be created and started as usual.

Here are the projects for StaticTestLibrary and the corresponding demo program.

External devices and the watchdog

The ominous EEPROM

In one of my last posts I explained how the watchdog of the RevPi Connect can be used. If you look into our tutorial, there is also an ominous EEPROM mentioned. How exactly is that connected? And what do the Illuminati have to do with it?

The relay can be used to reset external devices when the watchdog triggers. But not for every application it is desired that the relay switches with the watchdog. In addition, the relay can be used as a normal relay output.

Therefore this function was designed to be configurable. This is where the EEPROM comes into play, because it carries the configuration for the relay.

The tutorial provide two ready-to-use Python scripts to program the EEPROM. enable_relay_watchdog.py configures the EEPROM to trigger the relay with the watchdog. enable_relay_watchdog releases the connection between watchdog and EEPROM. The logic of the Python scripts can of course be recreated in C. So the behaviour of the relay can be set directly from the application, without having to use the Python script.

The FTDI library

I noticed that the necessary library libftdi1 is present, but not the header. Here you will find the FTDI driver inclusive header file: https://www.intra2net.com/en/developer/libftdi/download.php. Please note, Raspian Stretch is delivered with the old version 1.2 of the library. Meanwhile there is 1.4 available. In case the headers have changed, I played it safe and downloaded the old version 1.2: https://www.intra2net.com/en/developer/libftdi/download/libftdi1-1.2.tar.bz2.

After creating a new project in Netbeans, I added the new library to the project. That killed my nerves a bit, because the library is there, but I could only get the linker to find the file if I specified the path absolutely. For this purpose, I specified “/usr/lib/arm-linux-gnueabihf/libftdi1.so.2” under “Build->Linker->Additional Options”.

Screenshot of the Netbeans Project Properties showing where to type in the path of the FTDI library

Later I created a symlink libftdi1.so -> libftdi1.so.2. with

ln -s libftdi1.so.2 libftdi1.so

Then I could also use “Libraries”→”Add Library” and just enter ftdi1 here. Should look like this

Screenshot of the Netbeans Project Properties showing the list of the libraries used in the project

Netbeans simply makes the compiler option “-lftdi” from this.

Translation of the Python script

For the program I translated the Python script enable_relay_watchdog.py one to one in C. The string handling is a little more complex than in Python. I have made it easy for myself at this point. The code only works as long as devnum consists of one digit, i.e. is less than or equal to 9.

int devnum = c - '0'; //convert char to int

char str[6] = "";
sprintf(str, "d:1/%i", devnum);

If you expect devnum to be larger, a simple devnum = c – ‘0’ is no longer sufficient for conversion. The string must also be longer than 6 characters. If you plan to use the code in a real application, I advise you to implement the conversion mechanism safely!

Here you will find the Netbeans project.

When executing the program, make sure that it is called with root privileges, i.e.

sudo <path>/revpiwatchdogrelayenable

If you want to test if the delay relay really switches, let the watchdog respond. Now you should hear the quiet clattering of the relay.

Here is the same program that matches the phyton script “disable_relay_watchdog.py”.

Push-in terminals for the RevPi

Opinions differ on the question of whether to use screw terminals or spring terminals. The RevPi is delivered with screw terminals. This decision was made years ago and is not so easy to change without upsetting the regular customers.

I personally prefer push-in clamps. Little known is that there are also push-in terminals for the RevPi. These have the nice advantage that they are double-row, i.e. two terminals are available for each contact on the RevPi. This is helpful, for example, to loop the power supply through.

X2 and X4 push-in terminal blocks with double row terminals

The terminals can be obtained as spare parts for Pilz products. The following Pilz articles are the appropriate terminals for X2 and X4, as single or 10-piece packs:

The terminals can be obtained directly from Pilz or from various distributors. I am still trying to convince my colleagues that we offer the push-in terminals ourselves.

 

Watchdog of the RevPi Connect

Woof woof

Here is the article my boss has been waiting for and the actual trigger for my blog. One single article was not enough from my point of view, so I started to write articles about all my work with the RevPi.

The RevPi Connect has a built-in watchdog that works independently of the main controller. In addition there is the integrated relay output, which optionally switches when the watchdog is triggered.

If you have already looked at our tutorial for the Watchdog (https://revolutionpi.com/tutorials/ubersicht-revpi-connect/watchdog/) and just thought: “huh?“. At first I felt the same. After a little bit of trial and error and reading the schematic, you get behind it quite quickly.

How the Watchdog works

The watchdog is deactivated in the delivery state. This is done by connecting input WD to ground (0V). If this connection is released, the watchdog is armed. Therefore, do not touch the wire for the time being. Or who is like me and must have seen everything first, just try it out. If you still have a revision 1.0 of the RevPi Connect, like me, you have an eight-pin connector with more wire bridges. These are for the power supply of the RevPi, so keep your hands off. Starting with revision 1.1 these wire bridges are omitted.

So how does the Watchdog work? Here is a simple drawing. It does not exactly match the schematic. But it doesn’t matter from the user’s point of view. If you still want to know exactly, here you can find the schematic.

Schematic drawing, which explains how the Watchdog works: the output of the timer and the input WD are connected to an and gate of which output is connected to the reset

The RevPi Connect is equipped with a watchdog timer, which is configured to set its output after 60 seconds, if it ist not reset in time. In the drawing the timer is simply marked with T. The watchdog reset is done by toggeling the reset input of the timer (WD-Reset). However, the timer only triggers a reset of the RevPi when the input WD of the & becomes true. For this it is sufficient to disconnect the WD input from 0V and leave it open.

How do you get WD-Reset now? It is well hidden on bit 7 of RevPiLED. Why RevPiLED? Originally one byte was reserved in the process image for the LEDs on the core. In addition there was the third LED, the relay and the watchdog on the connect. The name has remained. Maybe I can motivate the colleagues to come up with a nicer name for it.

A demo program for the Watchdog

I have named my example program RevPiWatchdog. As basis I used the sample program RevPiTestOutput from a previous article.´. I replaced the for loop with an endless while loop and bit 0 with bit 7, because I want to reset the watchdog and not switch any LED. I also set the sleep time to 10s. This is quite enough, because the watchdog triggers after 60.

SPIValue spiValue = {0, 0, 0};
spiValue.i16uAddress = spiVariableOut.i16uAddress;
spiValue.i8uBit = 7;
spiValue.i8uValue = 1;

while(true){
    spiValue.i8uValue = spiValue.i8uValue^0x01;
    piC.SetBitValue(&spiValue);
    sleep(10);
}

For testing I started the program by hand and fiddled the cable on the terminal, which is a bit cumbersome while the program is running. One should not actually do that. If you have tried to remove the wire before, you can see now that it works. It works on until the program is finished. Then it takes another 60s and the RevPi resets itself.

systemd

To avoid having to start the program manually every time, it makes sense to start it automatically at startup. Usually it should be sufficient if the watchdog program is started “sometime” by systemd. Then it is sufficient to copy the systemd service from the last article.

For the sake of clarity I copied the executable file of the program back into ~/projects. Then I created the systemd service file in /etc/systemd/system. For this I simply copied the existing revpiblink.service

sudo cp revpiblink.service revpiwatchdog.service

With sudo nano revpiwatchdog.service I edited the file and replaced in the line ExecStart, revpiblink with revpiwarchdog, so

ExecStart=/home/pi/projects/revpiwatchdog

After saving the file you have to update systemd and activate revpiwatachdog.

Systemctl daemon-reload
Systemctl enable revpiwatchdog

Now RevPiWatchdog will be started automatically at every start of the RevPi and will also run with open cable bridge.

When things take a bit longer

My Connect+ needs about 35s until an application is started via systemd with multi-user target. So we have enough time until the 60s have expired after the start and the watchdog triggers. But if you increase the load on the RevPi during startup and you come close to or even exceed the 60s for complete startup, the moment comes when you have to deal with systemd. In our example the default for systemd is that our program needs it from the multi-user target. Exactly when to start is in systemd’s hands. If time is short, you have to start earlier.

Practical applications

As it is shown in the example, the program makes no sense in reality, of course. The watchdog should react when the system stops responding. In the case of the example, the watchdog would of course only react if the watchdog program was no longer running. If any other process has started to slip, the watchdog will not notice this. So you either have to build the watchdog directly into your target application or design the watchdog program to monitor the function of the other processes. The latter is of course much more extensive. But if you have several processes running in parallel, this is a reasonable solution. As long as the actual watchdog program is running, you also have the chance to restart processes that have hung up or to perform a soft reboot in the case of an error.

 

Starting processes automatically

RevPi makes itself independent

Revolution Pi is built for the tough industrial environment. Normally no software developer sits next to the plant to operate the RevPi. So at some point there comes the moment when the RevPi leaves the parental home and has to manage on its own. The first step to independence is that the RevPi, whatever he has to do, starts by itself. So we have to make sure that processes are started automatically at startup.

If you search the internet you will often find rc.local as a solution. Admittedly this was my first idea. But a closer look showed me that rc.local is completely outdated and should not be used anymore. The magic word is now systemd, in lower case. The developers attach great importance to this.

systemd

If you are now wondering “what the heck is systemd”, here are a few words about it. systemd is now the most common init process under Linux. init process is the first process that is started after the kernel is loaded and takes care of starting all other processes. Classically, almost all Linux systems have used the so-called SysVinit. This is basically nothing more than a collection of start scripts that are executed in a predefined order. SysVinit cannot exhaust modern computer architectures, which leads to unnecessarily slow starting systems. Therefore several alternatives to SysVinit have been created. systemd seems to have finally been accepted, although there is much criticism of it and possibly better systems exist.

If you look at various pages about systemd, it kills you, but the core of the matter is very simple. Granted, before I wrote this article, I knew nothing about systemd except that it exists and what makes it crude.

For demonstration I used RevPiBlink from the previous article again. For simplicity I copied the executable file into ~/projects. Now I want to start this program automatically with RevPi. Therefore I have to create a service for systemd. This is simply a text file with configuration settings. For this I go into the directory /etc/systemd/system and start nano with a corresponding file:

sudo nano revpiblink.service

The file consists of a few lines:

[Unit]
Description=Let LED A1 blink permanently

[Service]
Type=simple
ExecStart=/home/pi/projects/revpiblink

[Install]

WantedBy=multi-user.target

What do the individual lines mean? The lines in square brackets mark different sections. Under [Unit] is the description. Description is free text. That’s all we need for now.

Chapter [Service] defines what is to be done. Type=simple, just leave it like that. Whoever is interested will find countless articles on the net. ExecStart describes what should be done when starting the process, similarly there is also ExecStop. But we don’t need that right now.

[Install] tells systemd when the process should be started or who needs it. WantedBy thus defines a target state at which the corresponding process should run. In our case this is multi-user.target. This stands for multi-user operation on a console basis. Usually this is the “end station” for the RevPi. Above that there is graphical.target, if a graphical user interface is running. Enough talk about things I only know a little bit about myself.

After revpiblink.service is created, systemd has to know that the file exists and the service has to be activated. That works with

sudo systemctl daemon-reload

and

sudo systemctl enable revpiblink

If you restart the RevPi now, LED A1 should flash after powering up. A look into the process list with ps -All | grep revpiblink should now list the corresponding process.

Shutdown

Thereupon I realized a first practical application, namely the automatic shutdown. As a basis I use RevPiTestInput. I modified the new program RevPiShutdown in a way that periodically the input IN is scanned and as soon as it is true, the shutdown is triggered. For this, I simply replaced the line

    piC.GetBitValue(&spiValue);
    printf("%i, ",(int)spiValue.i8uValue);

… with these…

while (true) {
    piC.GetBitValue(&spiValue);

    if (spiValue.i8uValue == 1) {
        system("sudo shutdown -h now");
    }

    usleep(100000);
 }

The last line of the loop simply pauses. Since we want to query a key, the query interval must be much smaller than one second, otherwise keystrokes might be missed. Therefore I use usleep. The parameter specifies the waiting time in microseconds.

Again, I copy the program into ~/projects. Then I create a new service in /etc/systemd/system. For this I simply made a copy of revpiblink.service

sudo cp revpiblink.service revpishutdown.service

and modified the ExecStart entry

ExecStart=/home/pi/projects/revpishutdown

and then…

Systemctl daemon-reload
Systemctl enable revpishutdown

Tiny test (my wife insisted that “tiny” was the right word here)…

I set the input with my great cable solution

Connection to 192.168.178.32 closed by remote host.
Connection to 192.168.178.32 closed.

Background Processes

Parallel universes

When I start a program on the console, the console remains occupied by this program for the time being. If I want to do something else on the RevPi, I have to wait until the program is finished (or open a second console). If the program is designed as an endless process, I can wait until the never ending day. Often I also want to run several programs in parallel. Then the time has come when I have to deal with background processes.

A small program for demonstration

To demonstrate the functionality of background processes I wrote a little program RevPiBlink. As basis I use RevPiTestOutput from a previous post. I only changed the program to replace the for loop with an endless while loop.

while(true) {
    spiValue.i8uValue = spiValue.i8uValue^0x01;
    piC.SetBitValue(&spiValue);
    sleep(1);
}

Jobs

When I start the program in the command line, LED A1 flashes until I end the current appointment session or manually stall the program. The latter I can do with Ctrl-C. As long as the program is running, I cannot use the command line session.

Linux makes it really easy to use background processes. If I start a program and append & to the command, the program starts in the background and I can continue working on the console.

So I start my example program with

./revpiblink &

Now I get two numbers displayed, one of them in square brackets.  SSH console showing the output of the command "./revpiblink &"

The number in square brackets is the Job ID, the other is the Process ID. The process ID is valid globally for the entire system. The job ID is always only valid within the respective terminal session.

If I want to bring a process back into the foreground I can do this with fg %<Job ID>. So in my case

fg %1

SSH console showing the output of the command "fg %1"

Now the process can be terminated with e.g. Ctrl-C (we will not do that now). Much more interesting is that I can pause the process with Ctrl-Z. Now I have control over the command line again. I can now start the process in the background again with bg <Job ID>. If I cannot remember the job ID, the jobs command helps.

SSH console showing the output of the command "jobs": [1]+ Running ./revpiblink &

If I want to end a process in the background I can do this with kill %<job ID>.

SSH console showing the output of the command "kill %1": [1]+ Terminated ./revpiblink &

Declaration of independence

If I start a background process in a console, this process only lives as long as the terminal session exists. This can easily be shown with RevPiBlink. If you start the program with revpiblink & and end the SSH session, A1 stops flashing.

This can be prevented by uncoupling the process from the console session. This is done with disown <Job ID>. If I start ./revpiblink & again and run disown %1, the program is preserved even if the SSH session is terminated. I can also combine the commands to

./revpiblink & disown

If you now run jobs, the list remains empty. Sure, the current session has no more “own jobs”.

If we want to end the process now, we need the process ID, pid for short. With kill <pid> I can then send the process to the eternal hunting grounds. If I don’t know the process ID, I can simply find it out with the ps command. Without any further parameters, ps only lists me processes that were started from the current terminal session and that run under the current user. Processes that were started with sudo, for example, will not appear. With ps -A I get all processes displayed. In addition, ps allows countless parameters for filtering and sorting the processes and other options.  The easiest way to find my revpiblink is with

ps -A | grep revpiblink

SSH console showing the output of the command "ps -A | grep revpiblink": 2892 pts/0 00:00:00 revpiblink

The first number is the pid. I can now give that to kill.

By the way, kill without parameters tries to end processes “gently”. To do this, kill sends a signal to the process, telling it to terminate the process. So the process has the possibility to finish its work cleanly.  However, the process can also veto it. If a process should prove to be stubborn, it can be terminated hard with kill -9 <pid>.

Shutting down the RevPi

Caution, data loss!

Like any computer, the RevPi should not just be taken off the power, but shut down properly. Otherwise there is a risk of data loss. The article about shutting down is therefore actually a bit late. I only became aware of this topic after I accidentally disconnected the RevPi from the power supply a few times, because the power supply is connected to the switch socket of the computer and I turn it off in the evening. Fortunately nothing happened so far.

Manual shutdown

The most obvious is to simply connect to the RevPi via SSH and shut it down manually. To do this you need the command shutdown, where the command needs root privileges.

shutdown supports several parameters which I would like to mention briefly. With -h <time> you can define when the RevPi should shutdown. With now the RevPi shuts down immediately.

sudo shutdown -h now

The parameter -h instructs the system to stop or power off after shutdown. This is the standard behavior of shutdown. Therefore, you can conveniently omit the -h.

The following command will instruct shutdown to shut down in 60 minutes

sudo shutdown 60

SSH session showing the output for the command "shutdown 60"

You can also set a time, e.g. if you want the RevPi to shut down after work

sudo shutdown 18:45

SSH session showing the output for the command "shutdown 18:45"

If you do not specify a time span or a time, shutdown will shut down after two minutes.

If you change your mind, you can abort a once triggered shutdown with -c. Kindly shutdown reminds you of this when you execute it.

With sudo shutdown -r <time> you can also restart the RevPi. The time specifications work in the same way as for -h.

Shutdown with remote command

In the previous post I described how to execute commands remotely. This is of course useful for shutdown.  With the key set up, I can run RevPi with OpenSSH just like that:

ssh -i revpi_connect_33093_openssh pi@192.168.178.32 -t sudo shutdown -h now

Alternatively, I created a script shutdown.txt for Putty.

This can then be executed just like that:

putty -i revpi_connect_33093.ppk pi@192.168.178.32 -m shutdown.txt

Execute commands remotely

Just automate it

In one of my previous articles, I already hinted that I do not particularly like stupid work. So it often happens when working with the RevPi, that you have to execute a single command on the RevPi. It is really a waste of time to manually open a console for SSH every time and do whatever you want. It is much easier if you can send a command directly remote.

OpenSSH

The easiest way to do this is with OpenSSH. Here you can simply append a command with -t. For example, to start RevPiTestOutput from one of the previous articles, the following line is sufficient.

ssh -i revpi_connect_33093_openssh pi@192.168.178.32 -t netbeansprojects/RevPiTestOutput/dist/Debug/GNU-Linux/revpitestoutput

Putty

It’s a little more complicated with Putty. Putty does not seem to support to execute single commands as a parameter. Maybe it is due to the limitations of the Windows command line (speculation). Instead Putty supports executing whole scripts via file with -m. According to the official documentation, however, this should be treated with caution, as not all systems support multiline commands.

I created a text file named command.txt and inserted the same command as in the OpenSSH example

Screenshot of the Windows editor showing the contents of command.txt to run revpitestoutput

The script can be executed as follows:

putty -i revpi_connect_33093.ppk pi@192.168.178.32 -m command.txt

OpenSSH on Windows

How times change

Since Steve Balmer had to leave the executive chair at Microthingamabob, a lot has changed here. First of all, the attitude towards Linux has changed. So it happened that Windows 10 now comes with an OpenSSH client. By chance I stumbled upon it while researching the internet. Up to now I was dependent on Putty under Windows. Now I can also access the RevPi with Windows tools. Officially OpenSSH is still beta under Windows. But during my short experiments I could not find any problems.

Windows Home seems to be excluded at the moment. Apparently Microsoft does not trust the standard user to use such a complex tool like SSH.

The OpenSSH client is installed via Settings > Apps > Apps & features > Manage optional features > Add a Feature. Choose OpenSSH Client (Beta) and click Install.

If necessary after a restart, you can use SSH on the Windows command line. With ssh pi@192.168.178.32 I can now open the SSH connection to RevPi.

Screenshot of the Windows command line showing an established OpenSSH session

Even though OpenSSH for Windows is officially still in beta status, I didn’t find anything abnormal in my tests.

If you have used Putty from the command line before, you will notice some differences in the usage, as the command line parameters of Putty and OpenSSH are different. For example, OpenSSH does not allow you to specify a password on the command line. The execution of remote commands also works differently. I will deal with this in a follow-up article.

OpenSSH with keys

Just like in Putty, I can of course use keys for authentication in OpenSSH. However, the file format of the keys differs between Putty and OpenSSH. But puttygen can convert the keys. To do so, open the existing private key in puttygen with File > Load. The key can be converted via Conversions > Export OpenSSH key.

Screenshot of PuTTY Key Generator showing the menu for exporting the OpenSSH key

Now comes the obligatory question again, if you really want to save the key without a passphrase.

Although the keys for OpenSSH don’t have an extension, I named my key revpi_connect_33093_openssh to make it easy to distinguish. With -i you can now give ssh the key as command line parameter. Provided I am in the correct directory, it will look like this in my case:

ssh -i revpi_connect_33093_openssh pi@192.168.178.32

So SSH welcomes us without any password entry:

Screenshot of the Windows command line showing an established OpenSSH session with keys

Authentication with keys in Putty

Password is enough, isn’t it?

Lazy people are supposed to be the most productive. That’s probably not true for everyone. I’m sure a certain amount of due diligence is part of it. So if, like me, you’re annoyed by every job where you have to do exactly the same thing more than twice, you’ll look for a way to automate such work steps. The possibilities of the Unix world are exemplary here.

While working with the RevPi I was annoyed after a short time by having to enter the password in Putty again and again. At least for experimental use, you can make your work much easier here by storing a key instead of a password.

Generating The Keys

Putty comes with a small tool called puttygen, which is responsible for key generation. For my current work I left the default settings (RSA and 2048Bit). For security issues in the experimental stage these are more than enough. Just click on Generate and, as you can read, fiddle around a bit with the mouse. Moving the mouse creates entropy from the PC’s point of view and contributes to good keys.

Screenshot of PuTTY Key Generator (puttygen) while creating a new key pair

After the key was created, I connected to the RevPi and created a new file for the keys with nano ~/.ssh/authorized_keys2. I copied the public key from puttygen into it. This works by selecting the public key in puttygen (the contents of the box under “Public key for pasting into OpenSSH authorized_keys file:“) and right-clicking and copying it.

Screenshot of PuTTY Key Generator showing the new public key for clipboard copy

In Putty (with the nano open), simply right-click in the text area. Exit nano with Ctrl-X and answer the question for saving with “y”.

Screenshot of editor nano, in which the new public key is inserted

After the public key is stored on the RevPi, the private key must still find its way into Putty. The private key can be secured with a passphrase. From a security point of view this is definitely recommended. For experimental operation, a password is usually not necessary. After all, I don’t want to have to deal with that annoying password.

On the PC I created a directory .ssh analogous to the RevPi and saved the key there by Save private key. puttygen considers a private key without passphrase to be questionable and therefore asks again if you really want to save the key without passphrase, which I of course answered with Yes. I named the key revpi_connect_<serial number> to keep the overview when using several RevPis. However, both the directory and the file name can be chosen freely.

Setting Up Putty

First of all the current SSH session should be terminated and Putty should be started again. Select configuration (or set it up if not already done). Then enter “pi” under Data > Auto-login > Auto-login username.

Screenshot of PuTTY Configuration showing where to type in the Auto-login username

Click on Browse in SSH > Auth and select the private-key.

Screenshot of PuTTY Configuration showing where to set up the private key

Do not forget to save, so back to Session and Save. Now the session can be opened as usual and no username or password should be requested anymore.

Operating Putty via command line

Putty can also be controlled via the command line. So if clicking around all the time is too annoying or e.g. you want to include the dial-in in a script you can start a session with putty -load <session-name>. Alternatively Putty can be called directly with the parameters for configuration: putty -i <Path to private key>\<private key.ppk> pi@192.168.178.32. If the path or the file name contains spaces, put it in quotation marks (“”).

If the directory in which the key is located is synchronized with OneDrive – as in my case – make sure the path is correct. This would be something like C:\Users\<user>\OneDrive – <xyz>\<Path to private key>\private key.ppk. This is where Windows tricked me for a while, because the folders for documents etc. are usually located directly under “C:\Users\<user>\…”. The ways of Microsoft are quite unfathomable…

First steps in C/C++

Programming the RevPi in C/C++

In the previous article I set up the development environment and wrote the first small C program for the RevPi. But I could have executed the program on any computer.

In this article I show how to access the process image of the RevPi via C++. For this I use the files from the piTest demo program. The files piControlIf.hpp, piControlIf.cpp and piControl.h are required.

Probably, the most minimalist light show in the world

First I created a new program, of course again with RevPi as build host. I named it RevPiTestOutput and added the three files from the piTest demo (piControlIf.hpp, piControlIf.cpp, piControl.h).

Now comes the moment when you have to take a closer look at the philosophy of RevPi. The RevPi is designed as a modular system. Accordingly, it is not known from the beginning which inputs and outputs are actually present. Furthermore, the RevPi is intended for industrial applications. Therefore, all inputs and outputs are transferred collectively in a so-called process image.

RevPi Connect already have an input, a relay output and several LEDs on board, that can be freely switched by the user. So I do not have to deal with the configuration of the expansion modules yet.

In the first sample project RevPiTestOutput I use LED A1 to demonstrate how to access an output. To use the C++ version of the library, you must first include piControlIf.hpp in your program. Now, you need an instance of piControl, which I called piC. Then a structure of type SPIVariable has to be filled with the symbolic identifier of the output. The symbolic identifier for the LEDs on the RevPi is RevPiLED. This is then passed to piControl::GetVariableInfo. All in all, this looks like the following:

piControl piC;
SPIVariable spiVariableOut = {"RevPiLED", 0, 0, 0};

piC->GetVariableInfo(&spiVariableOut);

Now SPIVariable::i16uAddress contains the address of the LEDs in the process image. RevPiLED is one byte. The LEDs are represented by individual bits within the byte. The green LED A1 is on bit 0. You find the assignment of the LEDs in our tutorial Statusbytes RevPi Connect. To switch the LED, a structure SPIValue must be filled with the address and the data:

SPIValue spiValue = {0, 0, 0};
spiValue.i16uAddress = spiVariableOut.i16uAddress;
spiValue.i8uBit = 0;
spiValue.i8uValue = 1;

piC.SetBitValue(&spiValue);

In my sample program I switch the LED on and off every second. The whole thing 10 times with a for loop . The entire relevant code will look like this:

piControl piC;
SPIVariable spiVariableOut = {"RevPiLED", 0, 0, 0};
    
piC.GetVariableInfo(&spiVariableOut);

SPIValue spiValue = {0, 0, 0};
spiValue.i16uAddress = spiVariableOut.i16uAddress;
spiValue.i8uBit = 0;
spiValue.i8uValue = 1;
    
for (int i = 0; i < 10; i++) {
    spiValue.i8uValue = spiValue.i8uValue^0x01;
    piC.SetBitValue(&spiValue);
    sleep(1);
}

The first time I ran the program I got an error, because the compiler could not find “piControl.h” in “piControlIf.hpp”. If you replace the angle brackets in #include <piControl.h> with quotation marks, so #include “piControl.h” it works. I suppose the files were originally located in different directories at the developer colleagues. In piTest the C++ version is not used at all. Therefore the error was not noticed.

Reading an input

Reading an input works basically the same way as writing an output. First the position in the process image must be determined. With GetBitValue(&spiValue) the corresponding position in the process image can then be read.

In the sample program RevPiTestInput I want to read the input IN of the RevPi Connect and display it on the console. The input IN is confusingly located at RevPiStatus and there in bit 6 (see Statusbytes RevPi Connect). Altogether the program looks as follows:

piControl piC;

SPIVariable spiVariableOut = {"RevPiStatus", 0, 0, 0};

piC.GetVariableInfo(&spiVariableOut);

SPIValue spiValue = {0, 0, 0};
spiValue.i16uAddress = spiVariableOut.i16uAddress;
spiValue.i8uBit = 6;
spiValue.i8uValue = 0;

piC.GetBitValue(&spiValue);
printf("IN: %i\n",(int)spiValue.i8uValue);

If the input is at 0V, the correspondoing  bit is false, so the output is 0 (zero). If the input is at 24V, a 1 is displayed. If the input is open, i.e. neither 0V nor 24V, the bit is also false. For testing I simply plugged a cable into IN and connected it to the 0V or 24V terminal on the bottom of the Connect.

Picture of my RevPi setup with cable connected to input IN

If the input is at 0V or open, the output in SSH looks like this:SSH console showing the output of revpitestinput

A Symlink for the Path

As described in the last article, Netbeans places the programs in a somewhat bulky path. I made your work a bit easier by symlinking to the appropriate directory. You can do this with ln -s <path> <identifier>. For example I have set a symlink in the home directory, called netbeansprojects.

ln -s ~/.netbeans/remote/<IP address RevPi>/<PC name>/<Path on 
PC>/NetBeansProjects netbeansprojects

The symlink can be used like a normal directory. The home directory looks like this:SSH console with output of comand "ls -l" after creating new symlink netbeansprojects

 

Setting up the development environment

Let there be Netbeans…

To develop on the Revolution or the Raspberry Pi itself is extremely laborious. For the use of a modern development environment the Pi lacks a little bit of power. It makes more sense to develop on the PC and run the software on the RevPi, if possible without manual interaction.

After a short research my choice fell on Netbeans. Netbeans brings everything I need: an extensive editor, remote host support, etc. Eclipse probably brings similar features. But I am used to Netbeans.

Netbeans was given to the Apache Foundation by Oracle a few years ago. The transition of the code to Apache is being done piecemeal and does not seem to have been completed yet. Initially it made working with Netbeans very difficult. But true fans remain loyal to their software. Meanwhile Netbeans almost shines again in its old glory. Two times, however, manual action was required to get Netbeans running as desired.

Installation

First of all I downloaded the current Netbeans Version 11.3 and the current Java SE Development Kit 14. Now I installed the JDK.

Here comes the first tripping point. When you try to install Netbeans, an error message is displayed

Error Message Window: Critical Error. An unexpeced exception happend in thread main. Excption: java.lang.NoClassDefFoundError: java/util/jar/Pack200

After a short research on the net it is clear that the installer is not compatible with the current JDK version 14. Here is a post on Stack Overflow on this topic: How to fix Netbeans IDE installation error?. The solution: install an older version of the JDK.

Now I could have simply downloaded an older JDK version. For that I would have had to register with Oracle, which I did not feel like doing. I decided to take the harder path and used openJDK 13.

The easiest way seems to be to overwrite the existing JDK installation with the old version. This can be found under C:\Program Files\Jave\jdk-14.0.1. First I made a copy of jdk-14.0.1 in jdk-14.0.1_copy in the same directory. Now I unpacked openjdk-13.0.2_windows-x64_bin.zip. The ZIP file contains a directory “jdk-13.0.2”. I copied the content of this directory into C:\Program Files\Jave\jdk-14.0.1 and overwrote all existing files. Now the Netbeans installation worked. In contrast to a post on stack overflow you should not switch back to JDK 14 yet. The plugin manager of Netbeans, does not seem to be able to cope with that either.

To develop in Netbeans for C/C++, an appropriate plugin must be installed. Here comes the second point where the change from Oracle to Apache becomes noticeable. There is no official C/C++ plugin for Netbeans 11 yet, but the plugin for Netbeans 8 works fine. I have activated the NetBeans 8.2 Plugin Portal in Tools > Plugins > Settings.

Screenshot of the Netbeans plugin manager, with activated Netbeans 8.2 Plugin Portal

In the tab Available Plugins, the list of plugins must now be updated via Check for Newest. Now the plugin C/C++ appears in the list. I have installed it.

Screenshot showing Netbeans plugins with selected C/C++ plugin

You can probably switch back to the JDK 14 now. But I haven’t tried that.

Setting Up the Build Host

Via Menu > File > New Project > Select ‘C/C++ Application I have created a new project.

Screenshot of Netbeans dialog for new project with selected C/C++ Application

Via “Menu > File > New Project > Select ‘C/C++ Application” I have created a new project. I have assigned the creative project name “RevPiHello”. Now Netbeans may complain that no build environment is installed. But it doesn’t matter, we want to work with Revolution Pi.

To compile and execute programs on the RevPi, a build host must be set up. In the default setting of Netbeans you will find a tab “Service” on the left side. If you select the “C/C++ Build Host” with the right mouse button, you can add a new build host.

Screenshot of Netbeans context menu for C/C++ Build Host

At Host-Name the IP address of the RevPi must be entered and on the following page pi as Login name.

Screenshot of Netbeans setup for new remote host with entered IP address for the RevPi

Screenshot of Netbeans setup for new remote host with entered login "pi"

On the next page the password must be entered (sticker on the side of the device)  and with Finish the setup of the the build host is completed.

Now the building host must still be set in the current project. This is done by right-clicking on the project (RevPiHello) and selecting Properties > Build. By default two configurations Release and Debug are created. The settings for the Build Host should be made accordingly for both. Select pi@<IP address> on the right-hand side of the uppermost entry Build Host.

Screenshot of Project Properties - Build, showing the menu for "Configuration" and with new Build Host for the RevPi selected

Now switch to Run > Console Type and select Standard Output.

By the way, if you create a new C/C++ project, you can also choose RevPi directly as build host. I noticed that afterwards.

The First Project

I have already created a project. Now I have extended the empty example program in main.cpp by a few lines. Here you can download the whole project.

#include <cstdlib>
#include <cstdio>

using namespace std;

int main(int argc, char** argv) {
     printf("Hello Revolution Pi\n");
     return 0;
}

In Netbeans the hammer icon is used to build the project. If it’s successful, does it look like this in something…

Screenshot of Netbeans showing a successful build with the newly configured build host

If you connect via SSH, you will find the executable file somewhat hidden under ~/.netbeans/remote/<IP address RevPi>/<PC name>/<Path on PC>/NetBeansProjects/RevPiHello/dist/Debug/GNU-Linux. I have found no way to change this path except by mounting fixed shard folders. A tutorial can be found e.g. here. I haven’t tried it, so without guarantee.

If you run the program with ./revpihello you will get the following output. So everything seems to have worked.

SSH console showing the output of revpihello

Alternatively, you can run the program directly in Netbeans by clicking the green arrow icon to the right of the two hammer icons. Netbeans shows you a terminal below.

Screenshot of Netbeans showing the output of revpihello directly in the integrated console

I have now experimented a little bit and stumbled over a small unpleasantness, which made me doubt my abilities a little bit. If you run the program directly via Netbeans, via Run Project (green arrow), change something in the source code while the program is running, and perform Run Project again shortly after the program has finished, it may happen that the executable file is not changed. I suspect that a process has not yet released the file in time. Interestingly, the build process seems to finish properly. At least I do not get any error messages. A Run Project is useless, because Netbeans thinks that the state on the RevPi is up to date and does not trigger a new build process, but simply runs the program.

Ideally always wait a few seconds shortly after the program has finished. Worked reliably for me. Alternatively you can run “Build Project” manually.

Let’s get started!

Revolution Pi has conquered a firm niche in the industrial world. But we at Kunbus are far from satisfied with this. Therefore, we have asked ourselves the question, what can we do even better. What could be more obvious than to ask our own employees. Not necessarily those colleagues who are already involved in RevPi, but those who have had little or nothing to do with RevPi. That’s how the idea of a blog was born, in which I tell you about my experiences with RevPi.

About me: My name is Thomas and I am actually responsible for technology sales at Kunbus. So I usually deal with things that you can’t touch (which is not because it bites). So it’s a nice change to do something practical. This I can say after a short time: if you have a certain nerd potential like me, the RevPi really has an addictive factor. As a convinced Linux user I bring along some useful experiences in handling the RevPi. Actually I have also experimented with a Raspberry Pi. But that was ages ago.

After my colleagues sent me a nice RevPi Connect+, I armed myself with my tools and got to work. If you are wondering what that strange thing at the top right is, it is a stripping tool.

Crimping pliers, screwdriver, cable, wire stripper and wire end sleeves

After the first loose setup was on the desk, I took my first steps. So I installed Putty and looked up the IP address via the Fritzbox, in my case 192.168.178.32. In order to avoid entering the IP address by hand every time, I created a session right away.

Screenshot of Putty with saved session

Afterwards then the registration with the data on the housing side. Of course I won’t tell you. ?

I could have started now, actually. But I didn’t really like the loose setup. Anyway, there’s already enough stuff on the desk. So I quickly decided that I had to set it up quite neatly. After some digging, I had found a board, a multiple socket and even a piece of top hat rail. Four holes in the wall and a few screws later, the assembly was ready.

Unfortunately, the board is actually one or two centimetres too narrow for the socket. But it fits for now. The CAT6 network cable has proven to be extremely stubborn. After I got stuck on it a few times, I screwed an eyelet and fixed the network cable with a cable tie (It can be imagined behind the USB cable which is inserted from the left).

Board for RevPi assembly including multiple sockets and power supply

So now it can really start. Logged in again with Putty and with the help of piTest a LED is turned on and off.

SSH console on which LED A1 is switched on and off with piTest

Image of the RevPi Connect with green LED A1 on and off

Okay, that wasn’t a challenge yet. Let’s see what the colleagues have done nice with demos. Oh right… Python. I don’t know anything about that. After an admittedly short glance I decided, we probably won’t become friends. So the decision was made to program in C/C++. But more about that in my next article.

Please note, this blog is a field report. It is not meant to be an official tutorial, although I think you can find some useful things here. After all, not everyone has to reinvent the wheel.  If you have questions about the blog entries, please answer the blog entry and don’t bother my colleagues from the support department.