Fitlet2 Watchdog

Moderator: Andrey.Mazlin

jamessinton
Posts: 3
Joined: Fri Apr 06, 2018 12:50 pm

Fitlet2 Watchdog

Post by jamessinton »

I'm doing some tests on a fitlet2 x5 E3930. Specifically I am trying to test the watchdog. Clearly a device out in the real world needs to recover from unexpected crashes. I can enable the watchdog from with in Linux using the guides on the support site. What I can't see is where I can enable the watchdog in the UEFI/BIOS.

Does anyone know how to enable this feature in the UEFI/BIOS?

I have managed to get the fitlet2 to hang during boot a few times while simulating a power cut. Enabling the watchdog at a hardware level is needed to get around issues like that.

tamir
Site Admin
Posts: 419
Joined: Mon Dec 25, 2017 4:21 pm

Re: Fitlet2 Watchdog

Post by tamir »

Watchdog is enabled by default on the fitlet2.
It is hidden in the BIOS menu, hence you won't be able to see it.

In order to configure on Linux, we recently updated the manual, and you could find it in the link below:
http://fit-pc.com/wiki/index.php/Watchd ... tion_guide

hope that helps

jamessinton
Posts: 3
Joined: Fri Apr 06, 2018 12:50 pm

Re: Fitlet2 Watchdog

Post by jamessinton »

Hi Tamir,

Thanks for the reply. I have finally found time to go through what you've told me. In short I can't get the Watchdog to work. I did a fresh install of Linux Mint in an attempt to get the watchdog module to load as per the instructions. The operating system is not finding the hardware watchdog. I have included system details below. As you can see we're loading the iTCO_wdt module, but we're not seeing any hardware being detected.

BIOS FLT2.NBR.0.38.04.00 as shipped from supplier.

uname -a
Linux imsadmin-N-A 4.13.0-38-generic #43~16.04.1-Ubuntu SMP Wed Mar 14 17:48:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
No LSB modules are available.
Distributor ID: LinuxMint
Description: Linux Mint 18.3 Sylvia
Release: 18.3
Codename: sylvia

dmesg | grep iTCO
[ 125.180288] iTCO_vendor_support: vendor-support=0
[ 125.182840] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11

systemctl status watchdog
● watchdog.service - watchdog daemon
Loaded: loaded (/lib/systemd/system/watchdog.service; disabled; vendor preset: enabled)
Active: active (running) since Wed 2018-04-18 15:50:31 BST; 11s ago
Process: 4292 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
Process: 4288 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
Main PID: 4294 (watchdog)
CGroup: /system.slice/watchdog.service
└─4294 /usr/sbin/watchdog

Apr 18 15:50:31 imsadmin-N-A systemd[1]: Started watchdog daemon.
Apr 18 15:50:31 imsadmin-N-A watchdog[4294]: interface: no interface to check
Apr 18 15:50:31 imsadmin-N-A watchdog[4294]: temperature: no sensors to check
Apr 18 15:50:31 imsadmin-N-A watchdog[4294]: test=none(0) repair=none(0) alive=none heartbeat=none to=root no_act=no force=no

tamir
Site Admin
Posts: 419
Joined: Mon Dec 25, 2017 4:21 pm

Re: Fitlet2 Watchdog

Post by tamir »

We'll try to reproduce this here and get back to you.

Thank you.

tamir
Site Admin
Posts: 419
Joined: Mon Dec 25, 2017 4:21 pm

Re: Fitlet2 Watchdog

Post by tamir »

Sorry for the confusion as the Watchdog component on the fitlet2 should be initialized in the next BIOS update which should be coming soon.

You may follow our Google+ software updates page for when it should be released:

https://plus.google.com/104822148679589455238

jamessinton
Posts: 3
Joined: Fri Apr 06, 2018 12:50 pm

Re: Fitlet2 Watchdog

Post by jamessinton »

Hi Tamir,

Thanks for the update. Do you have an indication of when the BIOS update is likely to be released. The watchdog functionality is essential for an IoT device.

tamir
Site Admin
Posts: 419
Joined: Mon Dec 25, 2017 4:21 pm

Re: Fitlet2 Watchdog

Post by tamir »

We hope to release it in the next 2 weeks.

Thank you.

jeffdivert
Posts: 4
Joined: Wed Jun 26, 2019 4:50 pm

Re: Fitlet2 Watchdog

Post by jeffdivert »

I am also having trouble enabling the watchdog on the Fitlet2. I have Fitlet2 E3930 running BIOS FLT2.0.46.01.00 which appears to be the latest on the wiki and came after a fix for enabling the watchdog was applied on 0.40.1...

I have enabled watchdog service in Linux following the wiki guide, but the watchdog does not appear to be fully recognized by the OS. Here's some session input and output showing all the settings required by the guide should be setup properly but I can confirm that after forcing a kernel panic, the system did not recover via watchdog restart. I waited 5 minutes before manually rebooting

iotfresh@iotfresh-fitlet2:~$ cat /etc/default/watchdog
# Start watchdog at boot time? 0 or 1
run_watchdog=1
# Start wd_keepalive after stopping watchdog? 0 or 1
run_wd_keepalive=1
# Load module before starting watchdog
watchdog_module="iTCO_wdt"
# Specify additional watchdog options here (see manpage).
iotfresh@iotfresh-fitlet2:~$ cat /etc/watchdog.conf
#ping = 172.31.14.1
#ping = 172.26.1.255
#interface = eth0
#file = /var/log/messages
#change = 1407

# Uncomment to enable test. Setting one of these values to '0' disables it.
# These values will hopefully never reboot your machine during normal use
# (if your machine is really hung, the loadavg will go much higher than 25)
#max-load-1 = 24
#max-load-5 = 18
#max-load-15 = 12

# Note that this is the number of pages!
# To get the real size, check how large the pagesize is on your machine.
#min-memory = 1
#allocatable-memory = 1

#repair-binary = /usr/sbin/repair
#repair-timeout = 60
#test-binary =
#test-timeout = 60

# The retry-timeout and repair limit are used to handle errors in a more robust
# manner. Errors must persist for longer than retry-timeout to action a repair
# or reboot, and if repair-maximum attempts are made without the test passing a
# reboot is initiated anyway.
#retry-timeout = 60
#repair-maximum = 1

watchdog-device = /dev/watchdog

# Defaults compiled into the binary
#temperature-sensor =
#max-temperature = 90

# Defaults compiled into the binary
#admin = root
#interval = 1
#logtick = 1
#log-dir = /var/log/watchdog

# This greatly decreases the chance that watchdog won't be scheduled before
# your machine is really loaded
realtime = yes
priority = 1

# Check if rsyslogd is still running by enabling the following line
#pidfile = /var/run/rsyslogd.pid


iotfresh@iotfresh-fitlet2:~$ sudo modprobe iTCO_wdt
[sudo] password for iotfresh:
iotfresh@iotfresh-fitlet2:~$ dmesg | grep -i iTCO
[ 31.043566] iTCO_vendor_support: vendor-support=0
[ 31.045938] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
iotfresh@iotfresh-fitlet2:~$ cat /lib/systemd/system/watchdog.service
[Unit]
Description=watchdog daemon
Conflicts=wd_keepalive.service
After=multi-user.target
OnFailure=wd_keepalive.service

[Service]
Type=forking
EnvironmentFile=/etc/default/watchdog
ExecStartPre=/bin/sh -c '[ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module'
ExecStart=/bin/sh -c '[ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options'
ExecStopPost=/bin/sh -c '[ $run_wd_keepalive != 1 ] || false'

[Install]
WantedBy=default.target

iotfresh@iotfresh-fitlet2:~$ sudo systemctl start watchdog
iotfresh@iotfresh-fitlet2:~$ sudo systemctl status watchdog
● watchdog.service - watchdog daemon
Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2019-10-30 14:54:20 EDT; 2min 26s ago
Process: 1604 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
Process: 1599 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
Main PID: 1606 (watchdog)
Tasks: 1 (limit: 4359)
CGroup: /system.slice/watchdog.service
└─1606 /usr/sbin/watchdog

Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: pidfile: no server process to check
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: interface: no interface to check
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: temperature: no sensors to check
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: no test binary files
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: no repair binary files
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: error retry time-out = 60 seconds
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: repair attempts = 1
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: alive=/dev/watchdog heartbeat=[none] to=root no_act=no force=no
Oct 30 14:54:20 iotfresh-fitlet2 watchdog[1606]: cannot open /dev/watchdog (errno = 2 = 'No such file or directory')
Oct 30 14:54:20 iotfresh-fitlet2 systemd[1]: Started watchdog daemon.

tamir
Site Admin
Posts: 419
Joined: Mon Dec 25, 2017 4:21 pm

Re: Fitlet2 Watchdog

Post by tamir »

Escalated to our R&D, will return asap...

jeffdivert
Posts: 4
Joined: Wed Jun 26, 2019 4:50 pm

Re: Fitlet2 Watchdog

Post by jeffdivert »

Thanks Tamir,

I should mention that I tested this both on Linux Mint 19 Tara (Cinnamon) as well as Ubuntu 18.04.1 LTS with the same results:

iotfresh@iotfresh-fitlet2:~$ dmesg | grep -i iTCO
[ 31.043566] iTCO_vendor_support: vendor-support=0
[ 31.045938] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11

and nothing further, which leads me to believe that the watchdog may not be enabled in the BIOS?

Jeff

Post Reply

Return to “General fitlet2 questions”