Original post is here: eklausmeier.goip.de
I purchased an Odroid-XU4 for ca. 80 EUR including power-supply and case from Pollin. The original manufacturer is hardkernel. I intended to use this small ARM computer as a router and firewall. In the past I had used routers from multiple vendors, e.g., Linksys/Cisco, TP-Link, AVM/FritzBox, Netgear, and so on. There is a rule of thumb with all these devices: Usually you have to reboot them once or twice a month, otherwise they misbehave somehow. At least three of these device went completely catatonic. Now I had enough of this, I also wanted a command line interface to the router, ideally a real Linux system with bash, cron, gcc, etc. Although I already own an Intel NUC and I am very happy with this computer, an Intel NUC is a little bit too expensive to be used as just a router.
[youtube] wtqfC9v0xB0 [/youtube]
I recommend to additionally purchase a RTC backup battery. The Odroid has a realtime clock, but loses all date and time information once powered off. This way the log of the computer is garbled.
[more_WP_Tag]
1. Installing Arch Linux on Odroid #
I followed the description in archlinux|ARM. Below statements are just copied verbatim from that link.
1dd if=/dev/zero of=/dev/sdX bs=1M count=8
2fdisk /dev/sdX
3mkfs.ext4 /dev/sdX1
4mkdir root
5mount /dev/sdX1 root
6wget https://os.archlinuxarm.org/os/ArchLinuxARM-odroid-xu3-latest.tar.gz
7bsdtar -xpf ArchLinuxARM-odroid-xu3-latest.tar.gz -C root
8cd root/boot
9sh sd_fusing.sh /dev/sdX
10cd ../..
11umount root
All commands above proceed quickly. Only downloading the almost 300MB big Arch Linux image file takes some time according your internet speed.
Once Arch Linux is installed the computer looks like this.
1$ df -h
2Filesystem Size Used Avail Use% Mounted on
3dev 931M 0 931M 0% /dev
4run 997M 1.4M 996M 1% /run
5/dev/mmcblk1p1 30G 2.2G 26G 8% /
6tmpfs 997M 0 997M 0% /dev/shm
7tmpfs 997M 0 997M 0% /sys/fs/cgroup
8tmpfs 997M 4.0K 997M 1% /tmp
9. . .
Here is information from cpuinfo:
1$ cat /proc/cpuinfo
2processor : 0
3model name : ARMv7 Processor rev 3 (v7l)
4BogoMIPS : 78.00
5Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
6CPU implementer : 0x41
7CPU architecture: 7
8CPU variant : 0x0
9CPU part : 0xc07
10CPU revision : 3
11
12processor : 1
13model name : ARMv7 Processor rev 3 (v7l)
14BogoMIPS : 78.00
15Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
16CPU implementer : 0x41
17CPU architecture: 7
18CPU variant : 0x0
19CPU part : 0xc07
20CPU revision : 3
21
22processor : 2
23model name : ARMv7 Processor rev 3 (v7l)
24BogoMIPS : 78.00
25Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
26CPU implementer : 0x41
27CPU architecture: 7
28CPU variant : 0x0
29CPU part : 0xc07
30CPU revision : 3
31
32processor : 3
33model name : ARMv7 Processor rev 3 (v7l)
34BogoMIPS : 78.00
35Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
36CPU implementer : 0x41
37CPU architecture: 7
38CPU variant : 0x0
39CPU part : 0xc07
40CPU revision : 3
41
42processor : 4
43model name : ARMv7 Processor rev 3 (v7l)
44BogoMIPS : 120.00
45Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
46CPU implementer : 0x41
47CPU architecture: 7
48CPU variant : 0x2
49CPU part : 0xc0f
50CPU revision : 3
51
52processor : 5
53model name : ARMv7 Processor rev 3 (v7l)
54BogoMIPS : 120.00
55Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
56CPU implementer : 0x41
57CPU architecture: 7
58CPU variant : 0x2
59CPU part : 0xc0f
60CPU revision : 3
61
62processor : 6
63model name : ARMv7 Processor rev 3 (v7l)
64BogoMIPS : 120.00
65Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
66CPU implementer : 0x41
67CPU architecture: 7
68CPU variant : 0x2
69CPU part : 0xc0f
70CPU revision : 3
71
72processor : 7
73model name : ARMv7 Processor rev 3 (v7l)
74BogoMIPS : 120.00
75Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae
76CPU implementer : 0x41
77CPU architecture: 7
78CPU variant : 0x2
79CPU part : 0xc0f
80CPU revision : 3
81
82Hardware : SAMSUNG EXYNOS (Flattened Device Tree)
83Revision : 0100
84Serial : 0000000000000000
One can clearly see that the Samsung Exynos processor 5422 has 8 cores, and uses this ARM big.LITTLE architecture, where there are two different core types:
- The slow one, 1.5 GHz quad-core Cortex-A7
- The fast one, 2.1 GHz quad-core Cortex-A15
See also A big.LITTLE scheduler update as referenced in ODROID-Magazine February 2017:
1$ cat /sys/devices/system/cpu/cpu0/cpufreq/affected_cpus
20 1 2 3
3$ cat /sys/devices/system/cpu/cpu7/cpufreq/affected_cpus
44 5 6 7
Output from lstopo.
For further information see Odroid-XU4 user manual. For a thorough performance test see ODROID-XU4: Much Better Performance Than The Raspberry Pi Plus USB3 & Gigabit Ethernet @ $60 in Phoronix.
2. Network Cards #
The Odroid has two network cards, named eth0 and ethusb0. ethusb0 is directly connected to the cable modem, and gets its IP address per DHCP from the cable modem. eth0 has a static IP address. I used systemd-networkd for this.
1$ ls -l /etc/systemd/network
2total 12K
3-rw-r--r-- 1 root root 95 Apr 1 22:15 10-ethusb0.link
4-rw-r--r-- 1 root root 55 Feb 1 02:24 eth0.network
5-rw-r--r-- 1 root root 41 Apr 2 13:52 ethusb0.network
I wanted a fixed device name for the network card using USB, i.e., gigabit-USB-dongle, so I used the MAC address of this card to give it a fixed name, here ethusb0:
1$ cat /etc/systemd/network/10-ethusb0.link
2[Match]
3MACAddress=74:da:38:9f:c8:fa
4
5[Link]
6Description=USB to Ethernet Adapter
7Name=ethusb0
The built-in gigabit ethernet card gets a fixed IP address but does not use any gateway!
1$ cat /etc/systemd/network/eth0.network
2[Match]
3Name=eth0
4
5[Network]
6Address=192.168.178.1/24
I do not know how the card gets its name eth0. Probably I should add an entry in /etc/systemd/network using the MAC address.
The network card connected to the cable modem, gets its IP address via DHCP.
1$ cat /etc/systemd/network/ethusb0.network
2[Match]
3Name=ethusb0
4
5[Network]
6DHCP=yes
Once connected to the "real" internet, this will lead to the following routing table:
1$ route
2Kernel IP routing table
3Destination Gateway Genmask Flags Metric Ref Use Iface
4default ip-95-222-192-1 0.0.0.0 UG 1024 0 0 ethusb0
595.222.192.0 0.0.0.0 255.255.248.0 U 0 0 0 ethusb0
6ip-95-222-192-1 0.0.0.0 255.255.255.255 UH 1024 0 0 ethusb0
7192.168.178.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
8
9$ ifconfig
10eth0: flags=4163 mtu 1500
11 inet 192.168.178.1 netmask 255.255.255.0 broadcast 192.168.178.255
12 inet6 fe80::21e:6ff:fe31:a552 prefixlen 64 scopeid 0x20
13 ether 00:1e:06:31:a5:52 txqueuelen 1000 (Ethernet)
14 RX packets 5097790 bytes 515594426 (491.7 MiB)
15 RX errors 0 dropped 0 overruns 0 frame 0
16 TX packets 11049724 bytes 2851984084 (2.6 GiB)
17 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
18
19ethusb0: flags=4163 mtu 1500
20 inet 95.222.195.135 netmask 255.255.248.0 broadcast 95.222.199.255
21 inet6 fe80::76da:38ff:fe9f:c8fa prefixlen 64 scopeid 0x20
22 ether 74:da:38:9f:c8:fa txqueuelen 1000 (Ethernet)
23 RX packets 12352839 bytes 2937916965 (2.7 GiB)
24 RX errors 0 dropped 0 overruns 0 frame 0
25 TX packets 5128371 bytes 566073980 (539.8 MiB)
26 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Below two photographs show the connections.
The Odroid-XU4 has a red light, indicating that it has power. A blue light blinks, if the kernel is up and running.
3. Forwarding #
Enabling IP forwarding in the Linux kernel, see e.g., IP forwarding.
1$ cat /etc/sysctl.d/network.conf
2net.ipv4.ip_forward = 1
It is important to have the extension .conf for entries under /etc/sysctl.d, as systemd only applies setting for /etc/sysctl.d/*.conf files.
The essential part of this forwarding is that IP packets may hop from one ethernet card to another card. Setting ip_forward=0 would disallow this, i.e., not good for a router.
4. iptables #
For a tutorial see Iptables Tutorial by Oskar Andreasson.
Adding masquerading to both cards would make the router complete. But I additionally needed the following:
- Ports 80+443 should be directed to another server
- I need to
sshto two other machines in my network, here 24+118 - I want to
sshto the Odroid from outside but not via port 22. Within my network I want tosshandrsyncto the Odroid without fuzz, i.e., using port 22 - Block access to ports 53 (DNS), 67+68 (bootp), and 5355 (LLMNR = Link Local Multicast Name Resolution)
So I needed to NAT port 80 to my web-server, NAT = network translation. For the two other ssh-machines I arbitrary chose ports 8022 and 9022, which then direct to port 22 on the respective machines. Internet traffic coming from outside, i.e., coming from ethusb0, for port 22 on the Odroid is redirected to some unused port, here 15001, essentially quiescing port 22 for the outside world. "Secret" port 7022 is NAT'ed to port 22 internally. So sshd on Odroid has its port on 22, as usual, so all ssh/rsync traffic within my network does not need any ssh-config trickery or explicit port specification.
1$ iptables-save
2# Generated by iptables-save v1.6.0 on Thu Apr 6 22:55:46 2017
3*nat
4:PREROUTING ACCEPT [39276:4616181]
5:INPUT ACCEPT [14683:1093947]
6:OUTPUT ACCEPT [11747:870197]
7:POSTROUTING ACCEPT [0:0]
8-A PREROUTING -i ethusb0 -p tcp -m tcp --dport 8022 -j DNAT --to-destination 192.168.178.118:22
9-A PREROUTING -i ethusb0 -p tcp -m tcp --dport 9022 -j DNAT --to-destination 192.168.178.24:22
10-A PREROUTING -i ethusb0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.178.24
11-A PREROUTING -i ethusb0 -p tcp -m tcp --dport 443 -j DNAT --to-destination 192.168.178.24
12-A PREROUTING -i ethusb0 -p tcp -m tcp --dport 7022 -j DNAT --to-destination 192.168.178.1:22
13-A PREROUTING -i ethusb0 -p tcp -m tcp --dport 22 -j DNAT --to-destination 127.0.0.1:15001
14-A POSTROUTING -o ethusb0 -j MASQUERADE
15COMMIT
16# Completed on Thu Apr 6 22:55:46 2017
17# Generated by iptables-save v1.6.0 on Thu Apr 6 22:55:46 2017
18*filter
19:INPUT ACCEPT [97:7702]
20:FORWARD ACCEPT [95:13175]
21:OUTPUT ACCEPT [59:7201]
22-A INPUT -i ethusb0 -p tcp -m tcp --dport 53 -j DROP
23-A INPUT -i ethusb0 -p udp -m udp --dport 53:68 -j DROP
24-A INPUT -i ethusb0 -p tcp -m tcp --dport 5335 -j DROP
25-A INPUT -i ethusb0 -p udp -m udp --dport 5335 -j DROP
26COMMIT
Above output of iptables-save is later stored in /etc/iptables/iptables/iptables.rules.
Using MASQUERADE for ethusb0 is clear: Outgoing traffic to the "real" internet must be hidden, as the internal IP addresses make no sense in the "real" internet.
Remark: --to-destination 127.0.0.1:15001 is really -j DROP, but is not allowed in table nat.
iptables uses tables and chains. In above output *nat designates the table "nat". Similarly, *filter designates the table "filter". If you want to type in the single commands, the first rule would be, for example:
1iptables -t nat -A PREROUTING -i ethusb0 -p tcp -m tcp --dport 8022 -j DNAT --to-destination 192.168.178.118:22
In the same vein:
1iptables -t filter -A INPUT -i ethusb0 -p tcp -m tcp --dport 53 -j DROP
Leaving port 22 open to the outside can be a nuisance, see appendix below.
Finally enabling iptables in systemd.
$ systemctl enable iptables
$ systemctl start iptables
$ systemctl status iptables
iptables.service - Packet Filtering Framework
Loaded: loaded (/usr/lib/systemd/system/iptables.service; enabled; vendor preset: disabled)
Active: active (exited) since Mon 2017-04-03 22:01:12 CEST; 15s ago
Process: 600 ExecStart=/usr/bin/iptables-restore /etc/iptables/iptables.rules (code=exited, status=0/SUCCESS)
Main PID: 600 (code=exited, status=0/SUCCESS)
Apr 03 22:01:12 odroid systemd[1]: Starting Packet Filtering Framework...
Apr 03 22:01:12 odroid systemd[1]: Started Packet Filtering Framework.
If you feel shaky about fingering with iptables, then write a script, which deletes all firewall rules, like
1iptables -t nat -F
2iptables -F
which runs periodically via cron. This way you can get back in to your machine, once you messed up the firewall rules.
5. DNS #
I use dnsmasq as DNS server, see dnsmasq Arch ARM package. I also used dnsmasq first on my laptop to test the Odroid acting as DHCP client on ethusb0.
Important entries in /etc/dnsmasq.conf are:
address=/klm.no-ip.org/192.168.178.24
address=/klm.ddns.net/192.168.178.24
address=/edh.ddns.net/192.168.178.24
address=/klmport.no-ip.org/192.168.178.24
address=/borussia.no-ip.org/192.168.178.118
address=/www.eklausmeier.tk/192.168.178.24
address=/2o7.net/127.0.0.1
address=/adbrite.com/127.0.0.1
address=/adimg.uimserv.net/127.0.0.1
address=/adition.net/127.0.0.1
address=/adition.com/127.0.0.1
address=/ads.t-online.de/127.0.0.1
address=/adtech.de/127.0.0.1
address=/doubleclick.net/127.0.0.1
address=/ivwbox.de/127.0.0.1
address=/intellitxt.com/127.0.0.1
address=/kontera.com/127.0.0.1
address=/nuggad.net/127.0.0.1
address=/tfag.de/127.0.0.1
address=/tribalfusion.com/127.0.0.1
address=/quality-channel.de/127.0.0.1
address=/vibrantmedia.com/127.0.0.1
The first block gives special values to my No-IP addresses. The second block is a simple ad-blocker.
If you use dnsmasq, then you have to disable systemd-resolved:
1systemctl disable systemd-resolved
2systemctl stop systemd-resolved
6. Speed Test #
I made multiple speed tests. They showed the router is working as expected.
1$ speedtest
2Retrieving speedtest.net configuration...
3Testing from Unitymedia (95.222.195.135)...
4Retrieving speedtest.net server list...
5Selecting best server based on ping...
6Hosted by LWLcom GmbH (Frankfurt) [15.28 km]: 18.666 ms
7Testing download speed....................
8Download: 97.78 Mbit/s
9Testing upload speed......................
10Upload: 5.31 Mbit/s
In above example I used Arch package community/speedtest-cli, see speedtest-cli Arch package.
Here is another example.
1$ speedtest
2Retrieving speedtest.net configuration...
3Testing from Unitymedia (95.222.195.135)...
4Retrieving speedtest.net server list...
5Selecting best server based on ping...
6Hosted by 23media GmbH (Frankfurt) [15.25 km]: 12.446 ms
7Testing download speed....................
8Download: 105.46 Mbit/s
9Testing upload speed......................
10Upload: 5.11 Mbit/s
So speed is sufficient for the internet if you have for example 100 MBit/s. The network interface on the Odroid, though, is limited to max. 800 MBit/s. Below is a test with iperf3:
$ iperf3 -c R
Connecting to host R, port 5201
[ 5] local 192.168.178.5 port 38814 connected to 192.168.178.20 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 89.3 MBytes 746 Mbits/sec 0 115 KBytes
[ 5] 1.00-2.00 sec 88.1 MBytes 742 Mbits/sec 0 141 KBytes
[ 5] 2.00-3.00 sec 100 MBytes 839 Mbits/sec 0 410 KBytes
[ 5] 3.00-4.00 sec 102 MBytes 852 Mbits/sec 0 410 KBytes
[ 5] 4.00-5.00 sec 102 MBytes 853 Mbits/sec 0 431 KBytes
[ 5] 5.00-6.00 sec 102 MBytes 860 Mbits/sec 0 431 KBytes
[ 5] 6.00-7.00 sec 101 MBytes 851 Mbits/sec 0 452 KBytes
[ 5] 7.00-8.00 sec 102 MBytes 852 Mbits/sec 0 452 KBytes
[ 5] 8.00-9.00 sec 102 MBytes 855 Mbits/sec 0 452 KBytes
[ 5] 9.00-10.01 sec 103 MBytes 855 Mbits/sec 0 452 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 991 MBytes 830 Mbits/sec 0 sender
[ 5] 0.00-10.04 sec 990 MBytes 827 Mbits/sec receiver
iperf Done.
7. Problems #
a) As I forgot to purchase a backup battery, the Odroid loses its date and time, whenever the machine reboots. Once internet is available I use the following commands to get current date and time:
1ntpdate ptbtime2.ptb.de
2sleep 1
3ntpdate ptbtime2.ptb.de
4hwclock --systohc
I ask the NTP server twice, so the drift is not that huge.
Better than this is to automate it via systemd-timers.
Create a file /etc/systemd/system/date-fetch.timer as
1[Unit]
2Description="Runs periodic time synchronisation using a custom script"
3
4[Timer]
5OnBootSec=45sec
6OnUnitActiveSec=1w
7
8[Install]
9WantedBy=timers.target
The corresponding "service" is:
1$ cat /etc/systemd/system/multi-user.target.wants/date-fetch.service
2[Unit]
3Description="Runs periodic time synchronisation using a custom script"
4
5[Timer]
6OnBootSec=45sec
7OnUnitActiveSec=1w
8
9[Install]
10WantedBy=timers.target
11
12odroid /etc/systemd/system% cat date-fetch.service
13[Unit]
14Description="Fetch the current date only under certain circumstances"
15
16[Service]
17Type=simple
18Restart=on-failure
19RestartSec=5sec
20ExecStart=/usr/local/bin/date-fetch
21
22[Install]
23WantedBy=multi-user.target
The final bash-script /usr/local/bin/date-fetch is as follows:
1#!/bin/bash
2# Set date via NTP if internet is available
3# Gordian Edenhofer, 11-Apr-2017
4
5set -eu
6
7if [[ -n "$(ip addr show ethusb0 | sed -n 's/.*inet \([0-9\.]*\).*/\1/gp')" ]]; then
8 ntpdate ptbtime2.ptb.de
9 hwclock -w
10
11 # Cleanly exit since an IP address has been assigned to the network interface
12 # and the above commands completed without a failure (ensured by `set -e`)
13 exit 0
14else
15 # Return a failure exit code since an IP has not yet been assigned
16 exit 1
17fi
b) I refrained from using systemd-timesyncd as this may quickly flood your log if the Odroid is not connected to the internet. It is not uncommon that my ISP has an outage, albeit seldom.
c) Rebooting the Odroid via " I had difficulties with
shutdown -r now" does not always work. Many times I have to disconnect the power cord, then connect again.shutdown -r now, but they are now long gone.
d) When I am within my own network, I cannot http/https/ssh to internal servers using the external IP address.
e) It is not clear whether the Odroid is fully up to the task for functioning as a router, as exemplified by these kernel messages, time will tell:
Apr 05 17:50:30 odroid kernel: INFO: rcu_preempt detected stalls on CPUs/tasks:
Apr 05 17:50:30 odroid kernel: 1-...: (53937 GPs behind) idle=b1a/0/0 softirq=890/890 fqs=0
Apr 05 17:50:30 odroid kernel: 2-...: (116479 GPs behind) idle=8f4/0/0 softirq=645/645 fqs=0
Apr 05 17:50:30 odroid kernel: 3-...: (116479 GPs behind) idle=ed4/0/0 softirq=484/484 fqs=0
Apr 05 17:50:30 odroid kernel: (detected by 0, t=4207 jiffies, g=1365668, c=1365667, q=217)
Apr 05 17:50:30 odroid kernel: Task dump for CPU 1:
Apr 05 17:50:30 odroid kernel: swapper/1 R running task 0 0 1 0x00000000
Apr 05 17:50:30 odroid kernel: [] (__schedule) from [] (rcu_idle_enter+0x60/0x64)
Apr 05 17:50:30 odroid kernel: [] (rcu_idle_enter) from [] (cpu_startup_entry+0x198/0x218)
Apr 05 17:50:30 odroid kernel: [] (cpu_startup_entry) from [] (0x401015ac)
Apr 05 17:50:30 odroid kernel: Task dump for CPU 2:
Apr 05 17:50:30 odroid kernel: swapper/2 R running task 0 0 1 0x00000000
Apr 05 17:50:30 odroid kernel: [] (__schedule) from [] (rcu_idle_enter+0x60/0x64)
Apr 05 17:50:30 odroid kernel: [] (rcu_idle_enter) from [] (cpu_startup_entry+0x198/0x218)
Apr 05 17:50:30 odroid kernel: [] (cpu_startup_entry) from [] (0x401015ac)
Apr 05 17:50:30 odroid kernel: Task dump for CPU 3:
Apr 05 17:50:30 odroid kernel: swapper/3 R running task 0 0 1 0x00000000
Apr 05 17:50:30 odroid kernel: [] (__schedule) from [] (rcu_idle_enter+0x60/0x64)
Apr 05 17:50:30 odroid kernel: [] (rcu_idle_enter) from [] (cpu_startup_entry+0x198/0x218)
Apr 05 17:50:30 odroid kernel: [] (cpu_startup_entry) from [] (0x401015ac)
Apr 05 17:50:30 odroid kernel: rcu_preempt kthread starved for 4228 jiffies! g1365668 c1365667 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x1
Apr 05 17:50:30 odroid kernel: rcu_preempt S 0 7 2 0x00000000
Apr 05 17:50:30 odroid kernel: [] (__schedule) from [] (schedule+0x4c/0xac)
Apr 05 17:50:30 odroid kernel: [] (schedule) from [] (schedule_timeout+0x1f8/0x34c)
Apr 05 17:50:30 odroid kernel: [] (schedule_timeout) from [] (rcu_gp_kthread+0x5e4/0x948)
Apr 05 17:50:30 odroid kernel: [] (rcu_gp_kthread) from [] (kthread+0xec/0x104)
Apr 05 17:50:30 odroid kernel: [] (kthread) from [] (ret_from_fork+0x14/0x3c)
It is obvious that these problems stem from the "little" cores in the CPU:
$ journalctl | grep -C5 stalls | grep "for CPU"
Apr 04 18:55:29 odroid kernel: Task dump for CPU 3:
Apr 04 20:27:36 odroid kernel: Task dump for CPU 1:
Apr 05 01:49:18 odroid kernel: Task dump for CPU 1:
Apr 05 08:54:50 odroid kernel: Task dump for CPU 1:
Apr 05 17:50:30 odroid kernel: Task dump for CPU 1:
Apr 07 00:00:25 odroid kernel: Task dump for CPU 3:
Apr 08 00:41:53 odroid kernel: Task dump for CPU 1:
Apr 08 12:36:29 odroid kernel: Task dump for CPU 1:
Apr 09 16:48:46 odroid kernel: Task dump for CPU 1:
Apr 10 01:24:28 odroid kernel: Task dump for CPU 1:
Apr 10 20:23:02 odroid kernel: Task dump for CPU 1:
Apr 12 00:00:27 odroid kernel: Task dump for CPU 3:
Adding CPUAffinity=0-3 to /etc/systemd/system/multi-user.target.wants/sshd.service, and similarly adding CPUAffinity=4-7 to /etc/systemd/system/multi-user.target.wants/iptables.service and /etc/systemd/system/multi-user.target.wants/systemd-networkd.service does not remedy above problem.
Appendix #
Excerpt from lastb: Kinky people trying to login to the Odroid-XU4 by brute force.
xbmc ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
xbmc ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
xbian ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
xbian ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
wwwrun ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
wwwrun ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
workshop ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
workshop ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
windowse ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
windowse ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
webpop ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
webpop ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
webmaste ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
webmaste ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
webmaste ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
webadmin ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
webadmin ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
vyatta ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
vyatta ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
visitor ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
visitor ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
virus ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
virus ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
vagrant ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
vagrant ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
uucp ssh:notty 185.86.77.119 Tue Apr 4 20:37 - 20:37 (00:00)
uucp ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
users ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
users ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
user ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
user ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
user ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
user ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
user ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
username ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
user ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
username ssh:notty 185.86.77.119 Tue Apr 4 20:36 - 20:36 (00:00)
Similarly, journalctl | grep "Received disc":
Apr 04 22:57:30 odroid sshd[2812]: Received disconnect from 221.194.44.224 port 54372:11: [preauth]
Apr 04 23:00:42 odroid sshd[2815]: Received disconnect from 221.194.44.211 port 44199:11: [preauth]
Apr 04 23:11:58 odroid sshd[2818]: Received disconnect from 221.194.47.208 port 36286:11: [preauth]
Apr 04 23:12:17 odroid sshd[2820]: Received disconnect from 221.194.47.224 port 37452:11: [preauth]
Apr 04 23:16:59 odroid sshd[2824]: Received disconnect from 121.18.238.104 port 52330:11: [preauth]
Apr 04 23:18:51 odroid sshd[2827]: Received disconnect from 221.194.44.211 port 33700:11: [preauth]
Apr 04 23:26:47 odroid sshd[2830]: Received disconnect from 221.194.47.249 port 35392:11: [preauth]
Apr 04 23:45:05 odroid sshd[2836]: Received disconnect from 121.18.238.98 port 49376:11: [preauth]
Apr 04 23:45:54 odroid sshd[2838]: Received disconnect from 221.194.47.208 port 57745:11: [preauth]
Apr 04 23:54:31 odroid sshd[2841]: Received disconnect from 221.194.44.211 port 50781:11: [preauth]
Apr 05 00:04:00 odroid sshd[2932]: Received disconnect from 221.194.47.249 port 43806:11: [preauth]
Apr 05 00:15:29 odroid sshd[2964]: Received disconnect from 121.18.238.104 port 47782:11: [preauth]
Apr 05 00:17:31 odroid sshd[2966]: Received disconnect from 221.194.47.208 port 53492:11: [preauth]
Even after a few hours leaving port 22 open to the public, your log is full of this silliness.
Now that port 22 is blocked, iptables shows you how many times people attack you:
1$ iptables -t nat -t nat -L -n -v
2. . .
3 225 9840 DNAT tcp -- ethusb0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 to:127.0.0.1:15001
I.e., 225 times iptables routed port 22 to nirvana.
Similarly, DNS request from outside.
1$ iptables -L -n -v
2. . .
3 2173 792K DROP udp -- ethusb0 * 0.0.0.0/0 0.0.0.0/0 udp dpts:53:68
I.e., 2173 times people from the "real" internet asked the Odroid about DNS within a few hours. I guess these people did not have the best intentions.
Comment from Paul Alesius, 29-Apr-2017:
Great comprehensive post. I am experiencing similar issues with the scheduling as you mention in section 7.e) and the CPU affinity is an interesting observation. The kernel is unable to use swap and I’ve tried different kernel configurations with no success, I believe disabling swapping should fix it. Strangely, by writing a small program that uses memory beyond RAM, it does seem to start consuming swap, but the kernel fails to use swap automatically. It might work by disabling big.LITTLE support in the kernel, but I haven’t tried this. Let me know if you fix it somehow.On point 7.c) and shutdown -r now, I believe this is due to the eMMC requiring a special reset procedure, example: https://lists.denx.de/pipermail/u-boot/2015-January/200880.html I don’t believe this board is stable enough to remain up for more than a month or two, heavy read/write/network ends up killing mine (an USB connected 2.5″ drive is specially problematic), but works fine otherwise when mostly idle.
Reply from me, 29-Apr-2017: Thank you for your detailed comment. Indeed, I do not use swap, but not for your mentioned reasons, but rather because I never intended to run anything “large” on the Odroid.
Added 21-May-2017: Deleted text on MASQUERADE from nat-table.
1-A POSTROUTING -o eth0 -j MASQUERADE
Above rule maps real internet to Odroid IP address in the inner internet. It doesn't do any harm but hides all outer internet addresses.
Using MASQUERADE for eth0 was not initially clear to me, but makes sense in retrospect: Ingoing traffic from the "real" internet makes no sense in the internal network, so must be replaced by an internal IP address.
Added 06-Aug-2017: The router has stood the test of time. All these "stalls on CPUs/tasks" warnings are still there but apparently do not harm. The router works reliably and flawless.
Added 16-Nov-2017: Warnings on "stalls on CPUs/tasks" are completely gone with Linux kernel >= linux-odroid-xu3 (4.9.47-4).
Added 10-Jan-2019: I previously added ca. 3000 iptables rules for blocking IP address ranges which attacked me on port 22 (ssh). That many rules will deteriorate your network performance significantly. My download speed went down from 100 MBit/s to 20 MBit/s.
Added 05-Jun-2021: Decommissioning the Odroid as WiFi-router. Now the nuc will take over this duty. The Odroid proved to be very reliable. There is no failure or dissatisfaction with the router that lead to the decommissioning. It is just that the nuc is running anyway, so can take over this task, and the Odroid can be powered down to save some electricity, round about 7W.