problem rebooting fitlet XA10 after watchdog kicked in

Post Reply
lutzn
Posts: 3
Joined: Fri Jan 29, 2016 7:18 pm

problem rebooting fitlet XA10 after watchdog kicked in

Post by lutzn »

Hello,

we do have a problem rebooting fitlet XA10 after watchdog kicked in.

Problem:
System seems to reboot after watchdog kicked in but does not come up again.

Configuration:
XA10 Rev. 1.2
Samsung 8GB DDR3L
Samsung EVO 850 120GB
BIOS 08.08
Plain Debian 8 Jessie
Watchdog enabled with BIOS defaults

What we did:
We used the standard 3.16 Debian kernel and added patches to sp5100_tco and gpio_fch. We compiled those modules and they loaded fine. We installed the watchdog and did a basic configuration. We enabled the BIOS mechanism for watchdog.

We tested with "echo c > /proc/sysrq-trigger", the kernel panicked - fine. After the timeout the machine started to reboot.

What we observed:
The led switched to orange. Then it switched to green. But the display stayed blank. We turned the XA10 off and on again via push button, but the XA10 does not come up again. The device seems to be dead except that the led is green now. We have to pull the plug to bring the device up again.

We did reload BIOS defaults and did a reset. No change. It seems to be BIOS related.

It is exactly the same behaviour described in this thread:
No operation after Bios Update to SBCFLT_0.08.08 2015.11.30 (http://www.fit-pc.com/forum/viewtopic.php?f=143&t=5811)
and this thread:
fitlet iA10 bios update and RAM problems (https://www.fit-pc.com/forum/viewtopic.php?f=145&t=5812)

By policy we have to use Debian, we can not even use derivates like Ubuntu or Mint. But that should be no problem as Debian is the Original and fitlet is not tied to a specific OS.

We would be happy to get advice as we are currently evaluating XA10 for our customers. Before we start configuring any image for XA10 we need to be sure to have removed all road blocks. Otherwise we have to stop the evaluation because watchdog functionality is one of the "must have"; we must deliver rock-solid devices.

UPDATE START
We did several fresh installs and we tested and documented every step. We can say that it is not purely watchdog related. It's enough to load the gpio-fch.ko module. This module is responsible for the strange behaviour. You can reproduce the problem in the following way:
1. Patch the kernel modules.
2. Install the gpio-fch.ko only.
3. Do depmod.
4. Reboot.
5. echo c > /proc/sysrq-trigger.
As you have no watchdog, the machine will hang.
6. Turn off and on via button.
That's it.
UPDATE END

Thank you very much for any help!
-R

mbirger

Re: problem rebooting fitlet XA10 after watchdog kicked in

Post by mbirger »

Please try it with Linux Mint 17.3.
I will let our Linux expert to respond.

lutzn
Posts: 3
Joined: Fri Jan 29, 2016 7:18 pm

Re: problem rebooting fitlet XA10 after watchdog kicked in

Post by lutzn »

We tested for several hours on plain vanilla Debian. At the end we can say for sure that it is related to gpio-fch only. The relevant module started development in 2013 by Denis Turischev from CompuLab, so you have the developer in place. It is simple to test: Load the module => hang, do not load the module => no hang. Not only "echo c > /proc/sysrq-trigger" causes the machine not to reboot but also sometimes a simple "reboot" is enough.

We can say that this line (and maybe the following iowrites, too) causes this behaviour to start:
iowrite8(0x01, pm_ba + 0xEA);
So, poking around on these ports seems to be a bad idea.

It is noteworthy that loading and unloading gpio-fch and "echo c >..." is enough to reproduce. This is a clear sign that the module's initialization is doing something wrong. And bad things start with the line above.

Mint is based on Ubuntu which in turn is based on Debian. Both of them focus mainly on usability and not on kernel stuff.
... Btw, and what if it works with Mint? Do you want to cross Debian from the list of OS being supported? My suggestion is to use the original as a reference and not "a copy of a copy". It is simple for you to test and may help other customers that use Debian. Sorry, we won't install Mint as we have started development of a standard image including testing. It is really not economical for us.

Post Reply

Return to “fitlet BIOS”