piControl process communication errors after opening /dev/ttyAMA0 with pySerial

FMeinicke · Post by **FMeinicke** » 13 Jul 2022, 09:41

Yesterday I discovered a strange problem with our RevPi system: When I open the /dev/ttyAMA0 device using pySerial the piControl process seems to get stuck in an unrecoverable state that can only be resolved by a reboot.

Here's my setup:

RevPi Connect+ (Raspi CM3+) 8GB running Raspbian Buster with the 5.10.103-rt62-v7 kernel
- Connected via LAN A to our company network
- nothing connected via the 2 UBS ports or the RS485 interface
RevPi DIO module

Steps to reproduce:

Open an interactive python3 console

Run the following code

Code: Select all

  import serial
  ser = serial.Serial("/dev/ttyAMA0", timeout=2, write_timeout=2)

This code runs without any errors, however, the power LED on the RevPi turns red and I can't communicate with the DIO module anymore. I tried resetting the piControl process using

Code: Select all

piTest -x

which at least has the effect that the power LED turns green again but then the DIO module's power LED turns red, i.e. I still can't communicate with the module. The only thing that helps is to restart the RevPi Connect.

This is the output from journalctl:

Code: Select all

$ journalctl -b -k | grep piControl
Jul 13 08:58:15 RevPi69786 kernel: piControl: loading out-of-tree module taints kernel.
Jul 13 08:58:15 RevPi69786 kernel: piControl: built: Tue May 24 11:46:00 UTC 2022
Jul 13 08:58:15 RevPi69786 kernel: piControl: RevPi Connect
Jul 13 08:58:15 RevPi69786 kernel: piControl: MAJOR-No.  : 243
Jul 13 08:58:15 RevPi69786 kernel: piControl: MAJOR-No.  : 243  MINOR-No.  : 0
Jul 13 08:58:15 RevPi69786 kernel: piControl: read file finished, f_pos=6044
Jul 13 08:58:15 RevPi69786 kernel: piControl: 2 devices found
Jul 13 08:58:15 RevPi69786 kernel: piControl: 95 entries in total
Jul 13 08:58:15 RevPi69786 kernel: piControl: cl-comp:  0 addr 70  bit ff  len   8
Jul 13 08:58:15 RevPi69786 kernel: piControl: cl-comp:  1 addr 71  bit ff  len   8
Jul 13 08:58:15 RevPi69786 kernel: piControl: cl-comp:  2 addr 119  bit ff  len   8
Jul 13 08:58:15 RevPi69786 kernel: piControl: filp_open -2095457024
Jul 13 08:58:15 RevPi69786 kernel: piControl: set priority of spi0 to 54
Jul 13 08:58:15 RevPi69786 kernel: piControl: piIO thread started
Jul 13 08:58:15 RevPi69786 kernel: piControl: RevPiDevice_init()
Jul 13 08:58:15 RevPi69786 kernel: piControl: Enter Init State
Jul 13 08:58:15 RevPi69786 kernel: piControl: Enter PresentSignalling1 State
Jul 13 08:58:15 RevPi69786 kernel: piControl: PADS 0 = 0x1b   slew=1  hyst=1  drive=3
Jul 13 08:58:15 RevPi69786 kernel: piControl: PADS 1 = 0x1b   slew=1  hyst=1  drive=3
Jul 13 08:58:15 RevPi69786 kernel: piControl: PADS 2 = 0x1b   slew=1  hyst=1  drive=3
Jul 13 08:58:15 RevPi69786 kernel: piControl: piControlInit done
Jul 13 08:58:15 RevPi69786 kernel: piControl: Enter InitialSlaveDetectionLeft State
Jul 13 08:58:15 RevPi69786 kernel: piControl: Enter ConfigLeftStart State
Jul 13 08:58:15 RevPi69786 kernel: piControl: Enter ConfigDialogueLeft State
Jul 13 08:58:15 RevPi69786 kernel: piControl: GetDeviceInfo: Id 96
Jul 13 08:58:15 RevPi69786 kernel: piControl: found 2. device on left side. Moduletype 96. Designated address 31
Jul 13 08:58:15 RevPi69786 kernel: piControl: input offset     11  len  70
Jul 13 08:58:15 RevPi69786 kernel: piControl: output offset    81  len  18
Jul 13 08:58:15 RevPi69786 kernel: piControl: Enter SlaveDetectionLeft State
Jul 13 08:58:15 RevPi69786 kernel: piControl: Enter EndOfConfig State
Jul 13 08:58:15 RevPi69786 kernel: piControl: Device  0: Addr  0 Type 105  Act 1  In   6 Out   5
Jul 13 08:58:15 RevPi69786 kernel: piControl:            input offset      0  len   6
Jul 13 08:58:15 RevPi69786 kernel: piControl:            output offset     6  len   5
Jul 13 08:58:15 RevPi69786 kernel: piControl:            serial number 1  version 1.0
Jul 13 08:58:15 RevPi69786 kernel: piControl: Device  1: Addr 31 Type  96  Act 1  In  70 Out  18
Jul 13 08:58:15 RevPi69786 kernel: piControl:            input offset     11  len  70
Jul 13 08:58:15 RevPi69786 kernel: piControl:            output offset    81  len  18
Jul 13 08:58:15 RevPi69786 kernel: piControl:            serial number 73213  version 1.5
Jul 13 08:58:15 RevPi69786 kernel: piControl:
Jul 13 08:58:15 RevPi69786 kernel: piControl: Adjust: base 113 in 113 out 119 conf 0
Jul 13 08:58:15 RevPi69786 kernel: piControl: Adjust: base 0 in 0 out 70 conf 88
Jul 13 08:58:15 RevPi69786 kernel: piControl: After Adjustment
Jul 13 08:58:15 RevPi69786 kernel: piControl: Device  0: Addr  0 Type 105  Act 1  In   6 Out   5
Jul 13 08:58:15 RevPi69786 kernel: piControl:            input offset    113  len   6
Jul 13 08:58:15 RevPi69786 kernel: piControl:            output offset   119  len   5
Jul 13 08:58:15 RevPi69786 kernel: piControl: Device  1: Addr 31 Type  96  Act 1  In  70 Out  18
Jul 13 08:58:15 RevPi69786 kernel: piControl:            input offset      0  len  70
Jul 13 08:58:15 RevPi69786 kernel: piControl:            output offset    70  len  18
Jul 13 08:58:15 RevPi69786 kernel: piControl:
Jul 13 08:58:15 RevPi69786 kernel: piControl: start data exchange
Jul 13 08:58:15 RevPi69786 kernel: piControl: piDIOComm_Init done 0
Jul 13 08:58:15 RevPi69786 kernel: piControl: set BridgeState to running
Jul 13 09:04:15 RevPi69786 kernel: piControl: too many communication errors -> set inputs to default 0 11 0 0   0 0 0 0
Jul 13 09:04:15 RevPi69786 kernel: piControl: too many communication errors -> set inputs to default 0 12 0 0   0 0 0 0
Jul 13 09:04:15 RevPi69786 kernel: piControl: too many communication errors -> set inputs to default 0 13 0 0   0 0 0 0
[more messages like these]

I discovered this problem because I was running a python script that I wrote for a RaspberryPi that is connected to a device via RS232. I extended that script to also do some stuff with the I/Os of the DIO module and ran it on the RevPi. As I said, I have no serial devices connected to the RevPi at the moment because I just wanted to test the I/O part of the script. Later, there will be serial devices connected to the RevPi.

Anyway, the script basically scans for available serial (COM) ports and tries to open them, send a message and depending on the received message decide whether there is a particular device connected or not. To list all available serial ports I use pySerial's

Code: Select all

serial.tools.list_ports.comports()

This gives me 3 devices: /dev/ttyUSB0, /dev/ttyUSB1, and /dev/ttyAMA0. I assumed that these are the 2 USB ports and the RS485 interface of the RevPi. Since there's nothing connected to either of them I thought that it shouldn't be a problem to open these ports and send a message. My script would detect that there's no response coming back which in turn means that there are no devices connected.
This works as expected for /dev/ttyUSB0 and /dev/ttyUSB1 but not for /dev/ttyAMA0.

Out of curiosity I looked at /dev:

Code: Select all

$ ls -al /dev/
total 4
drwxr-xr-x 17 root root        4020 Jul 13 08:58 .
drwxr-xr-x 18 root root        4096 Feb 14  2019 ..
...
lrwxrwxrwx  1 root root           7 Jul 13 08:58 serial0 -> ttyAMA0
lrwxrwxrwx  1 root root           5 Jul 13 08:58 serial1 -> ttyS0
...
crw-rw----  1 root dialout 204,  64 Jul 13 09:29 ttyAMA0
lrwxrwxrwx  1 root root           7 Jul 13 08:58 ttyConBridge -> ttyUSB1
crw-------  1 root root      5,   3 Jul 13 08:58 ttyprintk
lrwxrwxrwx  1 root root           7 Jul 13 08:58 ttyRS485 -> ttyUSB0
crw-rw----  1 root dialout   4,  64 Jul 13 08:58 ttyS0
crw-rw----  1 root dialout 188,   0 Jul 13 08:58 ttyUSB0
crw-rw----  1 root dialout 188,   1 Jul 13 08:58 ttyUSB1
...

This would suggest that /dev/ttyUSB0, /dev/ttyUSB1, and /dev/ttyAMA0 are not behaving like I originally thought.

So my question now is what are these serial devices actually representing (hardware-wise)? I.e. is /dev/ttyUSB0 referring to the first USB port as I initially thought or is it referring to the RS485 interface as the directory listing of /dev seems to suggest? Also, what is /dev/ttyAMA0 and why does piControl not like it when I open the device using pySerial?

Thanks in advance,
Florian

nicolaiB · Post by **nicolaiB** » 13 Jul 2022, 10:08

Hi Florian,

/dev/ttyAMA0 is the RS485 bus of the piBridge and should not be used by any application (as long you're using modules). The other /dev/ttyUSB devices are FTDI serial interfaces (one for the RS485 front interface and one for the con bridge).

Nicolai

FMeinicke · Post by **FMeinicke** » 13 Jul 2022, 10:37

Hi Nicolai,

thanks for the fast reply. This makes sense to me now. I'll avoid using /dev/ttyAMA0 then

Florian