Blog of spblinux

Environment: Tablet (Android 11, 4GB RAM, 8 cores + keyboard with touchpad),
IDE VS Code (code-server 4.7+Volar+Vetur Extended, displayed by Chrome 104) running inside Termux

Steps to get the web browser based IDE VS Code with code-server up and running:

• Install Termux from F-Droid (terminal emulator and Linux environment app)
  • due to permission issues since android 10 the Termux app on Google Play Store is outdated (reference)
  • security settings on Android: allow installation of apk
  • open Termux console and run shell command pkg update to update all components of termux
• Install NodeJS inside Termux:
  • pkg install nodejs-lts
    • in my case I got EACCESS errors in /data/data/com.termux […] cacache/tmp which were resolved by temporarily switching from nodejs-lts to nodejs and back again:
      pkg uninstall nodejs-lts
      pkg install nodejs
      • a true solution would be to chown files in tmp dir of npm to the current user:
        • npm get tmp shows the tmp directory of npm
        • id shows user and group id of current user (uid, gid)
        • chown -R 12345:12345 PATH_TO_TMP/tmp/
          chown -R 12345:12345 tmp/
      • but Termux runs as restricted user and cannot change ownership
    • code-server, currently (2022-09), requires nodejs-lts
• Install dependencies to be able to build code-server
  • pkg install build-essential binutils python libsecret (reference, reference2)
• Use npm to install on global scope code-server
  • npm install -g code-server –unsafe-permissions (reference)
  • if code-server got installed but does not run because of missing modules (minimist, yauzl, yazl) then cd into /data/data/com.termux/files/usr/lib/node_modules/code-server/lib/vscode and run npm install –legacy-peer-deps (reference)
• Run code-server
  • code-server
    • either edit $HOME/.config/code-server/config.yaml and set your password
    • or run code-server –auth none (reference)
  • open http://localhost:8080 with your web browser
  • install extensions Volar and Vetur extended from code-server web interface
• Create a NuxtJS example app (reference)
  • use second session in Termux
    • open a second terminal inside Termux (keyboard shortcuts)ctrl+alt+c creates a new session
      • ctrl+alt+p or ctrl+alt+down arrow goes to previous session
      • ctrl+alt+n or ctrl+alt+up arrow goes to next session
      • ctrl+alt+right arrow opens session menu
      • (left arrow closes it; up/down switches sessions)
  • run yarn create nuxt-app hello-nuxt
    • this command does automatically install required packages
    • when asked:
      use java-script/packager yarn/no UI framework/no modules/no linting/no testing
      render universal (SSR / SSG)/deploy static/jamstack/no version control
    • cd into project directory hello-nuxt and run yarn dev
    • (later use yarn build and yarn start for production version;
      these commands are defined in the scripts section of file package.json.)
  • open http://localhost:3000
  • open project directory $HOME/hello-nuxt with explorer of code-server
    • edit pages/index.vue and save the changes
  • these changes, when saved, are immediately shown at http://localhost:3000
  • use copy and paste to replace index.vue by code of the NuxtJS hello world example code
  • open left pane of code sandbox at hello world example code and copy code of about.vue
  • create file about.vue inside explorer of code-server, paste code and save file

simple-code-editor is another example

• goto home directory and generate the nuxt framework with command
  yarn create nuxt-app nuxt-example2
  • accept defaults when asked, just press enter, except Deployment target: use Static and no Version control
• cd into this directory, cd nuxt-example2
• run npm install simple-code-editor
• with VS Code
  • hamburger menu on top left corner: File / Open Folder
    use folder nuxt-example2
  • open nuxt.config.js and add the line
    “simple-code-editor/nuxt”,
    inside the square brackets of modules: section
  • save nuxt.config.js
  • open pages/index.vue and edit the template section:
    • replace <Tutorial/> by
      <client-only>
      <CodeEditor value=”console.log(13)” width=”95%”
      :language_selector=”true” :languages=”[[‘cpp’,’C++’], [‘c’, ‘C’]]”></CodeEditor>
      </client-only>
    • save this file
    • use right click to delete Components/Tutorial.vue
• go back to Termux console and run yarn dev
• watch result with web browser, http://localhost:3000
• to compile the final version run
  • yarn build
  • yarn generate
• to show the content of folder dist/ at http://localhost:3000 run
  • yarn start
• to Download the contents of the dist folder use VS Code
  • click on refresh in explorer “…”
  • right click on dist folder and select Download (only available with desktop browser; see ssh port forwarding below)
  • choose the target folder and confirm
• to copy the dist folder to Android Documents folder (reference)
  • run termux-setup-storage
  • cp -R dist /sdcard/Documents/
  • mv /sdcard/Documents/dist/ /sdcard/Documents/code-editor2
• this folder may be delivered by almost any simple webserver, e.g. Simple HTTP Server app on Android

Sometimes it is more comfortable to use a desktop web browser, which is possible by port forwarding with ssh.

• open Termux console and install ssh by running pkg install openssh
• edit /data/data/com.termux/files/usr/etc/ssh/sshd_config, e.g. with editor nano
  • add the line
    AllowTcpForwarding yes
• use command whoami to display the linux user name used by android, e.g. u0_a250
• set a password for this user with command passwd
• make sure the android tablet is connected to local wifi and display the ip address with ifconfig
• start sshd with command sshd (sshd of Termux listens on port 8022)
• on the desktop pc run
  • ssh -p8022 -L3000:localhost:3000 -L8080:localhost:8080 u0_a123@192.168.123.123
    (replace u0_a123 with correct username and 192.168.123.123 with correct ip address)
  • open http://localhost:8080 and http://localhost:3000 with web browser

As a customer without cloud products volume licensing was done with a “normal” account of my organization: user@mydomain.xyz. Meanwhile a work or school account (WSA) is required. Reference.

Steps required, in my case, to change the existing account into a WSA account:

• Use what is called by microsoft Internal admin takeover (reference)
• (maybe the steps below depend on my first try to create a new account user@mydomainxyz.onmicrosoft.com account)
• open https://powerbi.com/ and create an account with the same email address as the existing account
• use this newly created account to open https://admin.microsoft.com/admintakeover (if asked during login wether private or business account use business account)
• the next steps cause that the newly created account, user@mydomain.xyz, is assigned to mydomain.xyz and to the role global admin
• you are instructed to create a dns txt entry at mydomain.xyz to verify your ownership of mydomain.xyz. (This txt dns entry can be removed after verification)
• logged in at admin.microsoft.ch at settings/domains mydomain.xyz should be displayed as standard domain
• now open microsoft volume licensing center, https://www.microsoft.com/Licensing/servicecenter/default.aspx and choose sign in with your work account, user@mydomain.xyz. Single sign on should work (if already logged in at admin.microsoft.com).
• Existing licenses should be displayed. Message to upgrade account to WSA should be gone. Done!

windows 10 client: use symbolic links for Desktop, Documents and Downloads folders pointing to H: because the default install of https://www.linuxmuster.net/de/home/ uses local profile folders and network drive H: as home directory. The approach below creates this redirection of Desktop, Documents and Downloads semi automatically: all settings are made on the domain controller; if the user global-admin logs on to a workstation the symbolic links of all users which have already logged on to this workstation are created.

manage gpo by installing rsat on windows 10 (gpo and ad tools) with powershell:
Get-Command -Noun WindowsCapability
Get-WindowsCapability -Name RSAT* -Online
Get-WindowsCapability -Name RSAT* -Online | Select-Object -Property DisplayName, State
Get-WindowsCapability -Name RSAT.A* -Online | Add-WindowsCapability -Online
Get-WindowsCapability -Name RSAT.Group* -Online | Add-WindowsCapability -Online

use logon and logoff scripts (builtin gpo object sophomorix:default:school defines scripts at /var/lib/samba/sysvol/default-school/custom/windows/; create additional gpo object at global/management: sophomorix:global:management to define logon/logoff scripts for user global-admin)
User/Policies/Windows/Scripts

h: drive letter is mapped to home directory in linuxmuster, thus use
mklink /d Desktop H:\Desktop (same for Documents and Downloads). mklink requires either admin rights (elevated command prompt) or gpo policy for global-admin to allow creation of symbolic links.
Computer/Policies/Windows/Security/Local/User settings/create symbolic links

• user scripts of normal user:
  • create H:\Desktop (same for Documents, Downloads)
  • robocopy content to H:
  • rename Desktop to Desktop.bak when logged off
    (until folder Desktop has been replaced by symlink Desktop)
  • give global admin access to main user folder
  • if symlink does not yet exist, copy content from H:\Desktop to Desktop during logon
• logon script of global-admin
  • create symlink
• startup script
  • truncate log file

@echo off
rem called by gpo object in schools/default-school (builtin)
rem User/Policies/Windows/Scripts
rem ################################################################
rem #          Run custom Logon-Script                             #
rem ################################################################
echo logon.bat %USERNAME% start >>c:\tmp\log.txt
if %USERNAME%==global-admin goto :ADMIN
C:
cd \Users\%USERNAME%
rem for %%d in (Videos) do (
for %%d in (Desktop Documents Downloads) do (
 if exist %%d (
  dir |find "%%d" |find "SYMLINKD"
  if not errorlevel 1 (
   if exist %%d.bak (
    if not exist H:\%%d mkdir H:\%%d
    robocopy %%d.bak H:\%%d /e /copy:DATSO /r:1 /w:3
    if not errorlevel 9 (
      attrib -r %%d.bak
      ren %%d. %%d.unused
      attrib -r -h -s %%d.unused\* /s /d
      rd %%d.unused /s /q
    ) else (
      attrib -r %%d.bak
      ren %%d.bak %%d.unused
      attrib -r -h -s %%d.unused\* /s /d
    )
   )
  ) else (
   if exist %%d.bak (
    robocopy %%d.bak %%d /e /copy:DATSO /r:1 /w:3
    attrib -r %%d.bak
    attrib -r -h -s %%d.bak\* /s /d
    rd %%d.bak /s /q
    if exist %%d.bak ren %%d.bak %%d.error
   )
   if exist H:\%%d robocopy H:\%%d %%d /e /copy:DATSO /r:1 /w:3
  )
 ) else (
   if exist %%d.bak (
    ren %%d.bak %%d
   ) else (
    mkdir %%d
   )
   if exist H:\%%d robocopy H:\%%d %%d /e /copy:DATSO /r:1 /w:3
 )
)
goto :ENDE
:ADMIN
:ENDE

@echo off
rem called by gpo object in schools/default-school (builtin)
rem User/Policies/Windows/Scripts
rem ################################################################
rem #          Run custom Logoff-Script                            #
rem ################################################################
echo logoff.bat %USERNAME% start >>c:\tmp\log.txt
if %USERNAME%==global-admin goto :ADMIN
C:
cd \Users\%USERNAME%
icacls \Users\%USERNAME% |find "global-admin" |find "(F)"
if errorlevel 1 (icacls \Users\%USERNAME% /grant "linuxmuster\global-admin":"(NP)(F)" /c)
rem for %%d in (Videos) do (
for %%d in (Desktop Documents Downloads) do (
 if exist %%d (
  dir |find "%%d" |find "SYMLINKD"
  if errorlevel 1 (
   if not exist H:\%%d mkdir H:\%%d
   robocopy %%d H:\%%d /e /copy:DATSO /r:1 /w:3
   if not errorlevel 9 (
    attrib -r %%d
    ren %%d %%d.bak
    attrib -r -h -s %%d.bak\* /s /d
    rd %%d.bak /s /q
   ) else (
    attrib -r %%d
    ren %%d %%d.bak
    attrib -r -h -s %%d.bak\* /s /d
   )
  )
 )
)

goto :ENDE
:ADMIN
:ENDE
echo logoff.bat %USERNAME% stop >>c:\tmp\log.txt

@echo off
rem called by gpo object in global/management (not builtin)
rem User/Policies/Windows/Scripts
rem ################################################################
rem #          Run custom Logon-Script                             #
rem ################################################################
echo logon.bat %USERNAME% start >>c:\tmp\log.txt
if %USERNAME%==global-admin goto :ADMIN
goto :ENDE
:ADMIN
rem user global-admin has symlink create permission in global gpo
rem Computer/Policies/Windows/Security/Local/User permissions: create symbolic links
C:
cd \Users
rem create symlinks in user directories pointing to H:\DIRNAME **only** if
rem  1) user global admin has access to user directory
rem  2) folders Desktop, Documents and/or Downloads do not exist
rem all other work is done by logon/logoff scripts of normal users
for /d %%u in (*) do (
 dir %%u
 if not errorlevel 1 (
  cd \Users\%%u
  for %%d in (Desktop Documents Downloads) do (
   if not exist %%d (
    mklink /d %%d H:\%%d
    if not errorlevel 1 (echo mklink ok %%u\%%d >>c:\tmp\log.txt) else (echo mklink failed %%u\%%d >>c:\tmp\log.txt)
   )
  )
  cd \Users
 )
)

:ENDE
echo logon.bat %USERNAME% stop >>c:\tmp\log.txt

@echo off
rem called by gpo object in global/management (not builtin)
rem User/Policies/Windows/Scripts
rem ################################################################
rem #          Run custom Logoff-Script                            #
rem ################################################################
echo logoff.bat %USERNAME% start >>c:\tmp\log.txt
if %USERNAME%==global-admin goto :ADMIN
goto :ENDE
:ADMIN
:ENDE
echo logoff.bat %USERNAME% stop >>c:\tmp\log.txt

@echo off
rem called by gpo object in schools/default-school (builtin)
rem Computer/Policies/Windows/Scripts (running as machine account)
rem ################################################################
rem #          Run custom Systemstart-Script                       #
rem ################################################################
rem overwrite existing logfile that is, clear log at startup
if not exist C:\tmp mkdir c:\tmp
echo sysstart %USERNAME% >C:\tmp\log.txt
:ENDE

This post collects observations when using MAix GO board with Arduino IDE, call Maixduino.

Macros how special pins on board MAix GO are called, can be found in pins_arduino.h of MAix GO., like PIN_KEY_PRESS which has an external pull up resistor (see schematic).

When testing example code of libraries function Serial.begin() seems to need a little time to initialize. Console output worked after adding 300mS delay. (How much is the minimal delay required?):

Serial.begin(115200);
delay(300);

Below libraries are listed which have example code that compiles and runs with MAix GO board (maybe the code needs minimal modification like replacing BUILTIN_LED by LED_BUILTIN; use compiler error messages – Preferences/Show verbose output during compilation):

• WiFiEsp (included with Maixduino board addon of Arduino IDE)
• RingBuffer by locoduino (installable with Arduino Library Manager)
  (Maybe EspRingBuffer coming as part of of WiFiEsp bundled with Maixduino, folder src/utility, is of interest as well.)
• ArduinoJson by Benoît Blanchan (Arduino Library Manager)
• ArduinoMqtt by Oleg Kovalenk which is based on eclipse paho (Arduino Library Manager)
  Use the example ConnectEsp8266WiFiClient and replace: ESP8266WiFi.h by WiFiEsp.h (included in Maixduino), WiFiClient by WiFiEspClient, ESP.reset() by WiFi.reset() and copy WiFi setup code from example of WiFiEsp. – If not properly initialized might stop k210 cpu with illegal instruction.
• pubsubclient by Nick O’Leary (Arduino Library Manager). Same remarks as shown above (ArduinoMqtt).

According to Kendryte K210 datasheet (available here) the speed of the 4 uart interfaces of K210 can run with up to 5 MHz baud rate. Communication with esp8285 used as wifi chip runs by default with 115200 baud and seems to run with 2000000 baud as well. (Tested with sketch Examples/Maix Go/WiFiEsp/Test/EspDebug: use command AT+UART_CUR:2000000,8,1,0,1). AT command reference is here. Using the _CUR command means that after reboot the default settings get restored. Storing the settings permanently bears the risk to make esp8266 inaccessible (if not reflashed).

Source of AT command firmware is here at github, if you select tag v2.2.1 (examples/at). – Install compiler (folder dist in https://github.com/noduino/xtensa-toolchain or http://domoticx.com/sdk-esp8266-xtensa-architecture-toolchain/) and add it to path. Inside SDK folder copy folder examples/at to root folder of SDK. Run script gen_misc and choose: boot_v1.2+, user1.bin (or user2.bin), spi 40MHz, spi DOUT, 1024KB (512KB+512KB); result is in folder bin/. Uploading to esp8285 would require to shorten ground and the pad GPIO0 in the corner of the antenna connector (info from sipeed forum).

notes about using esptool.py to flash firmware (hide expanded file):

to verify size of flash
python utils/esptool.py --port /dev/cu.usbserial-143110 --baud 115200 flash_id

nodemcu board, 4MB=32000kbit, successfully flashed with
python utils/esptool.py --port /dev/cu.usbserial-143110 --baud 115200 write_flash 0x00000 bin/eagle.flash.bin 0x10000 bin/eagle.irom0text.bin 0x7E000 bin/blank.bin 0x3FE000 bin/blank.bin 0x3FC000 bin/esp_init_data_default.bin 
(tested by konsole of arduino ide; firmware has default baud rate of 115200; boot messages are sent with 74880 baud ->technical reference of esp8266)

esp8285, 1MB=4096kbit, should be flashable with
python utils/esptool.py --port /dev/cu.usbserial-143110 --baud 115200 write_flash 0x00000 bin/eagle.flash.bin 0x10000 bin/eagle.irom0text.bin 0x7E000 bin/blank.bin 0xFE000 bin/blank.bin 0xFC000 bin/esp_init_data_default.bin

To use WiFiEsp lib with 2000000 baud without setting it permanently WiFiEsp has to be patched: by default WiFi.init(&Serial1) function does call AT+RST which reboots esp8285. In patched version you can call WiFi.init(&Serial1, false) to avoid this reboot:

WiFiEsp_disable_at_rst_in_init.diff (hide expanded diff file):

diff -ur orig/WiFiEsp.cpp new/WiFiEsp.cpp
--- orig/WiFiEsp.cpp	2019-07-15 17:14:20.000000000 +0200
+++ new/WiFiEsp.cpp	2019-08-07 11:32:14.000000000 +0200
@@ -33,8 +33,14 @@
 
 void WiFiEspClass::init(Stream* espSerial)
 {
+	init(espSerial, true);
+}
+
+
+void WiFiEspClass::init(Stream* espSerial, bool run_at_rst)
+{
     LOGINFO(F("Initializing ESP module"));
-	EspDrv::wifiDriverInit(espSerial);
+	EspDrv::wifiDriverInit(espSerial, run_at_rst);
 }
 
 
diff -ur orig/WiFiEsp.h new/WiFiEsp.h
--- orig/WiFiEsp.h	2019-07-15 17:14:20.000000000 +0200
+++ new/WiFiEsp.h	2019-08-07 11:33:01.000000000 +0200
@@ -51,6 +51,8 @@
 	*/
 	static void init(Stream* espSerial);
 
+	static void init(Stream* espSerial, bool do_at_rst);
+
 
 	/**
 	* Get firmware version
diff -ur orig/utility/EspDrv.cpp new/utility/EspDrv.cpp
--- orig/utility/EspDrv.cpp	2019-07-15 17:14:20.000000000 +0200
+++ new/utility/EspDrv.cpp	2019-08-07 12:04:48.000000000 +0200
@@ -70,8 +70,15 @@
 uint8_t EspDrv::_remoteIp[] = {0};
 
 
+
 void EspDrv::wifiDriverInit(Stream *espSerial)
 {
+	wifiDriverInit(espSerial, true);
+}
+
+
+void EspDrv::wifiDriverInit(Stream *espSerial, bool run_at_rst)
+{
 	LOGDEBUG(F("> wifiDriverInit"));
 
 	EspDrv::espSerial = espSerial;
@@ -95,7 +102,7 @@
 		return;
 	}
 
-	reset();
+	reset(run_at_rst);
 
 	// check firmware version
 	getFwVersion();
@@ -114,13 +121,23 @@
 }
 
 
+
 void EspDrv::reset()
 {
+	reset(true);
+}
+
+
+void EspDrv::reset(bool run_at_rst)
+{
 	LOGDEBUG(F("> reset"));
 
-	sendCmd(F("AT+RST"));
-	delay(3000);
-	espEmptyBuf(false);  // empty dirty characters from the buffer
+	if(run_at_rst)
+	{
+		sendCmd(F("AT+RST"));
+		delay(3000);
+		espEmptyBuf(false);  // empty dirty characters from the buffer
+	}
 
 	// disable echo of commands
 	sendCmd(F("ATE0"));
diff -ur orig/utility/EspDrv.h new/utility/EspDrv.h
--- orig/utility/EspDrv.h	2019-07-15 17:14:20.000000000 +0200
+++ new/utility/EspDrv.h	2019-08-07 11:38:57.000000000 +0200
@@ -123,6 +123,9 @@
     static void wifiDriverInit(Stream *espSerial);
 
 
+    static void wifiDriverInit(Stream *espSerial, bool run_at_rst);
+
+
     /* Start Wifi connection with passphrase
      *
      * param ssid: Pointer to the SSID string.
@@ -279,6 +282,8 @@
 	static bool ping(const char *host);
     static void reset();
 
+    static void reset(bool do_at_rst);
+
     static void getRemoteIpAddress(IPAddress& ip);
     static uint16_t getRemotePort();
 

To be continued.

Server monitoring with graphical output required, but no need to run a monitoring suite like nagios? With existing debian lamp server you might want to use SmokePing.

Setup is fairly easy:

• apt-get install smokeping
• edit /etc/smokeping/config.d/General
  let cgiurl point to your server https://your_domain.tld/cgi-bin/smokeping.cgi
• edit /etc/smokeping/config.d/Targets and define the servers which should be monitored
• restart smokeping: systemctl restart smokeping (or smokeping –restart)
• open https://your_domain.tld/smokeping/smokeping.cgi

Add password protection:

• edit /etc/apache2/conf-available/smokeping.conf and add two Location sections:
  <Location /cgi-bin/smokeping.cgi>
  AuthUserFile /etc/apache2/your_password_file
  AuthName “protected”
  AuthType Digest
  AuthDigestProvider file
  require valid-user
  </Location>
  <Location /smokeping/smokeping.cgi>
  … (see above)
  </Location>
• htdigest -c /etc/apache2/your_password_file “protected” your_user
• apache2ctl configtest
  apache2ctl graceful

Credits go to Tobi Oetiker, the author of this well made tool!

some notes:

very slow disk access inside vm might be caused by balanced power plan (which is install default in server 2008r2, 2012r2 hyperv core).
use powercfg.exe -l to check powerplan and powercfg -s to set another plan, e.g. performance; check with crystalldiskmark

running some time virtual disks tend to get to small. On server 2012r2 powershell has:
get-vhd -path …, resize-vhd -path -sizebytes …GB, optimize-vhd

before moving a vhd(x) file, display acl lists using icacls (to be able to restore permissions iacl …./grant …)

… net use z:  \\1.2.3.4\sharename * /user:your_name (and net use z: /delete)

partitions inside a vhd may be resized with diskpart
– sel vdisk file=….
– attach vdisk
– sel last partition and the corresponding volume
– extend
– lis par (to check)
– detach

Excellent (!) main reference https://www.cyrius.com/debian/kirkwood/qnap/

Special case ts-221 (and probably all devices with 1GB RAM): kernel of debian stretch 4.9 (also stretch-backports 4.17) seems to fail with memory above 768M on qnap arm
(discussion: https://groups.google.com/forum/#!topic/linux.debian.ports.arm/vSaO642z8aY); dmesg shows
BUG: Bad rss-counter state mm

• use jessie installer in expert mode and install stretch (offered by jessie installer as testing release)
• before reboot
  • either switch to older kernel as described below
  • or use only 768MB of 1024MB ram to avoid this bug
    mount /dev and /proc (maybe /sys) inside installation directory and chroot into installation directory
    • copy /usr/share/doc/u-boot-tools/examples/qnap_ts119-219.config
      to /etc/fw_env.config (assuming package u-boot-tools has been installed)
    • then run fw_printenv bootargs which returns something like
      bootargs=console=ttyS0,115200 root=/dev/ram initrd=0xa00000,0x900000 ramdisk=34816
    • now use fw_setenv to add linux kernel boot parameter mem=768M by adding mem=768M to output of fw_printenv
      fw_setenv bootargs “console=ttyS0,115200 root=/dev/ram initrd=0xa00000,0x900000 ramdisk=34816 mem=768M”
      (takes some time to write to flash)
    • check with fw_printenv that bootargs still contains the original arguments (plus mem=768M)
    • reboot (check with free -h and cat /proc/cmdline; dmesg: Memory: … 0K highmem)
    • maybe, this has to be repeated after every linux kernel update (verify withc fw_printenv bootargs if mem=768M ist still there)

In case flash-kernel returns Failed to obtain MAC address, run either
ubootcfg -b 0 -f /dev/mtdblock4 -o – | grep “^ethaddr=” | sed “s/^ethaddr=//”
or fw_printenv ethaddr
which returns the mac address stored in flash; then run
iface_set_mac eth0 YOUR_QNAP_MAC_FROM_FLASH

Recovery: run this iso from qnap wiki page inside a vm or install minimal debian with isc-dhcp-server and tftpd-hpa.

To make sure that an existing harddisk gets properly reconfigured use installer option (removal of raid partitions)
guided – entire disk

• one partion with btrfs and option noatime
• in case of error see below

Workaround (ok for devices without fan): If status led remains blinking red-green create /etc/rc.local, make it executable
#!/bin/sh
qcontrol –direct statusled greenon
exit 0

Solution (fixes led, buzzer and fan): # qcontrol needs stretch backports: apt-get -t stretch-backports install qcontrol, so add to /etc/apt/sources.list

deb http://ftp.ch.debian.org/debian/ stretch-backports main

and /lib/systemd/system/qcontrold.services is broken: replaces lines
Requires=dev-input-by\x2dpath-platform\x2dgpio_keys\x2devent.device
After=dev-input-by\x2dpath-platform\x2dgpio_keys\x2devent.device

with
ConditionPathExists=/dev/input/by-path/platform-gpio_keys-event (kernel 4.9) or platform-gpio-keys-event (kernel 3.16)
and afterwards run
dpkg –configure qcontrol

In case one disk is used for the system and a second disk is used for data: mount the second disk in /etc/fstab with option noauto (else the system will not start if the data disk has been removed)

Install smartmontools and check disk with smartctl -a /dev/sdXN

Special case ts-221: does not run with kernel 4.9, so run debian stretch with kernel 3.6 -or use bootarg mem=768M as described above
(latest discussion: https://groups.google.com/forum/#!topic/linux.debian.ports.arm/vSaO642z8aY)

• first install debian jessie (as described on cyrius.com for stretch but replace stretch by jessie in download links)
  • copy /boot/*, /lib/modules/*, /usr/lib/linux-image-* into a tar archive and save it on an external storage device
• second, flash again the jesssie installer, but now run it in expert mode and install debian stretch
  • before rebooting the second last step “finish install or so” open console 2 (keys <ctrl> + <a>, then key <2>)
    • copy the saved contents of first installation (tar archive) into running installation
    • edit /usr/share/flash-kernel/functions: change force=”no” to force=”yes”
    • run update-initramfs -u -k 3.16..0-6-kirkwood (and probably you get a message to add “-t” as additional argument)
      later if kernel/initramfs get renewed, it is required to run afterwards: update-initramfs -u -k $(uname -r)

the installer does not cope well with already used disks (guided, use whole disk should work); if not, start installer in expert mode and add fdisk as installer component
(in case of running debian stretch wipefs -a removes all kind of signatures; but do double check that you will not wipe the wrong partition!)

• open shell 2 and run cat /proc/partitions
• use fdisk to create a new partition table (dos format) and create one big primary partion
• create a filesystem on this partition with mkfs.ext2
• reboot!!! And then run installer again in standard mode

Some qnap models may take more than 15 minutes until kernel and initrd are written to flash. – First reboot after installation might by succesful, even if power led keeps blinking red-green and buzzer remains silent. Led and buzzer need installation of qcontrol from backports and patched qcontrold.service file as described above.

Enable quota: btrfs quota enable /backup; btrfs quota enable /backup2
Display disk usage: btrfs sub list /backup; btrfs qgroup show /backup
unused because of to much memory usage when deleting snapshots

mkfs.btrfs shows after creation of filesystem whether incompatibilities with current kernel exist; use -O ^prop1,^prop2 if properties prop1,prop2 ar not compatible; (use wipefs to remove the newly created filesystem and recreate it using -O ^…)

Convert btrfs single partition filesystem to btrfs raid1 during debian install

• use installer to create a btrfs root partition (noatime option) on a single partition
• use installer to create on second disk a partition of same size without mountpoint
• let installer write new partitions to disk
• open second ssh connection and choose shell
  • btrfs filesystem show
  • df -h
  • btrfs device add -f /dev/PARTITION_ON_SECOND_DISK /target
  • btrfs balance start -dconvert=raid1 -mconvert=raid1 /target
  • btrfs filesystem show

Recover from failure of one drive of root filesystem on btrfs raid1

• difficulty: if / gets remounted ro because of filesystem errors reboot into installer is required
• prepare for rescue with installer
  • mkdir /var/backups/flash-installer
  • cd /tmp
  • wget http://ftp.debian.org/debian/dists/stretch/main/installer-armel/current/images/kirkwood/network-console/qnap/ts-21x/initrd
    wget http://ftp.debian.org/debian/dists/stretch/main/installer-armel/current/images/kirkwood/network-console/qnap/ts-21x/kernel-6282
  • dmesg |grep SoC
    ID=0x… displays wether kernel 6282 or 6281 is used
    cp initrd /var/backups/flash-installer/mtd2
    cp kernel-6282 /var/backups/flash-installer/mtd1 (or use kernel 6281)
  • to boot into installer run:
    cat /var/backups/flash-installer/mtd1 >/dev/mtdblock1
    cat /var/backups/flash-installer/mtd2 >/dev/mtdblock2
  • to restore installed system when running installer run:
    mount YOUR_PARTITION /mnt
    cat /mnt/var/backups/flash-kernel/mtd1 >/dev/mtdblock1
    cat /mnt/var/backups/flash-kernel/mtd2 >/dev/mtdblock2
• recovery steps (if raid1 consists of /dev/sda3 and /dev/sdb3)
  • btrfs fi show
    displays whether both raid1 drives are accessible
  • echo 1 >/dev/block/sda/device/delete
    forces /dev/sda offline (e.g. in case of too many smart errors)
  • if btrfs fi show  displays that one drive is missing, e.g. sda3, do not run other btrfs commands to avoid errors which cause transition to ro filesystem, but do immediately run
    mount -o remount,degraded /dev/sdb3 /
  • attach replacement drive and check its device name with cat /proc/partitions (od dmesg), e.g sdc3
  • remove any existing btrfs data on replacement drive partition, e.g. on sdc3 by formatting it with ext4
    mkfs.ext4 -F /dev/sdc3
  • btrfs dev add -f /dev/sdc3 /
  • btrfs fi show
    • if sdc3 is shown twice, remove sdc3 with lower id from btrfs raid1
      btrfs dev del YOUR_DEVID /
    • if it is shown, that one drive is missing run
      btrfs del missing /
  • btrfs fi show
    this should know display two drives/partitions attached
  • to check for other error btrfs scrub might be run (time consuming)
  • reboot needed to remove option degraded from /proc/mounts

Switch hard disk with root filesystem

• debian stretch on qnap sets kernel command line ROOT=”/dev/….” inside initramfs which is stored in flash. So using another hard disk requires to reflash initramfs
  • update-initramfs -u creates a new initramfs (at /boot/initrd.img-KERNEL_VERSION)
    and runs flash-kernel KERNEL_VERSION which creates /var/backups/flash-kernel/mtd2 (and mtd1) and flashes it
  • update-initramfs (/usr/share/initramfs-tools/hooks/flash_kernel_set_root) reads /etc/fstab to get the device with the root filesystem
  • use blkid YOUR_DEVICE_NAME to display the uuid of a block device
  • So to set a new root device, temporarily edit /etc/fstab (on the old root partition) to point to the new root filesystem device, run update-initramfs -u; then restore the old settings in/etc/fstab on old partition;
    verify that the new partition is correctly set in /etc/fstab on the new root partition,
  • additional step for recovery by tftp (based on instructions here)
    • backup flash partitions /dev/mtdblockN (N=0 … 5) to files mtdN
    • download installer kernel and initrd (reference)
      to determine kernel version (6281 or 6282)
      /usr/share/flash-kernel/dtb-probe/kirkwood-qnap
    • pad kernel
      dd if=kernel-628X of=kernel.pad ibs=2097152 conv=sync (with X=1 or 2)
    • create tftp img file
      cat mtd0 mtd4 mtd5 kernel.pad initrd mtd3 > F_TS-219_di
      and copy this recovery image file to another computer or storage device
  • finally reboot
  • if reboot fails, load qnap recovery live cd iso here
    • run live cd in e.g. virtualbox (use a network adapter not connected to existing lan)
    • copy recovery image file to /tftpboot/ on live cd
      • add a second network interface connected to local lan to allow upload by ssh
      • sudo su
      • ifconfig eth1 YOUR_LAN_IP
      • set a password for user ubuntu
        passwd ubuntu
      • upload img file with scp to /home/ubuntu
      • ifconfig eth1 down (because dhcp server of live cd)
      • cp img file to /tftpboot/ and replace existing img file for your qnap device
    • power off qnap and power on with reset button pressed (until double beep):
      this should reflash qnap flash partitions with debian installer

Example btrfs send, btrfs receive (btrfs volumes mounted at /backup and at /backup2;
transfer readonly subvolume rootf_ro from volume mounted at /backup to other volume mounted at /backup2)

• btrfs sub snapshot -r /backup/rootfs /backup/rootfs_ro

• btrfs send /backup/rootfs_ro |btrfs receive /backup2

Resize/shrink btrfs partition with root filesystem (e.g. openmediavault needs separate data partition);
assume that btrfs partition is mounted at /backup using /dev/sda2

• btrfs filesystem resize 32G /backup
• fdisk /dev/sda
  • p
  • d 2
  • n 2 default start +40G
    do not delete btrfs signature
  • p
  • w
  • q
• partprobe
• btrfs filesystem resize max /backup
• btrfs filesystem show
• cat /etc/fstab

Add openmediavault (version 4.x arrakis for debian stretch): instructions

• echo “deb http://packages.openmediavault.org/public arrakis main” > /etc/apt/sources.list.d/openmediavault.list
• apt-get install openmediavault-keyring
• apt-get update
• apt-get install openmediavault
• omv-initsystem

Create raid0 btrfs volume and use subvolumes as shared folders:

• mkfs.btrfs -L datafs -d raid0 /dev/sda4 /dev/sdb4
• then use webinterface filesystems to mount this volume
  (gets mounted at /srv/dev-disk-by-label-datafs; cat /etc/fstab)
• btrfs subvol create /srv/dev-disk-by-label-datafs/YourShare
• then use webinterface to define shared folder YourShare
  • use shared folder / ACL if write access fails
• (in case of macos sierra don’t) share this folder by nfs
  • currently, 2017-11-02, apply changes fails
    (tries to mount second partition included in btrfs raid0)
  • reboot from console
  • log into webinterface and apply change again: should work
  • nfs://your_ip/export/YourShare/
    (with default settings needs option insecure)
  • currently, 2017-11-02, nfs with macos sierra is awfully slow but samba sharing (activate wins server) works
    • nfs seems to have locking issues because ejecting share from macos finder fails (unless finder is forced to restart)

minidlna media player

• apt-get install minidlna
• btrfs subvol create /srv/dev-disk-by-label-datafs/Musik
• omv webinterface: add shared folder Musik on device with datafs
  which creates /sharedfolders/Musik
• edit /etc/minidlna.conf (compare with this reference)
  • inotify=no
  • friendly_name=YOUR_MEDIA_SERVER
  • media_dir=A,/sharedfolders/Musik
• chown -R minidlna:minidlna /sharedfolders/Musik
• create media file index from scratch
  • systemctl stop minidlna
  • rm /var/cache/minidlna/*
  • systemctl start minidlna
• if new mediafiles have been added
  • chown -R minidlna:minidlna
  • systemctl restart minidlna
• if minidlna does not get discovered by plugplayer (ios), then manually add server:
  http://your_ip:8200/rootDesc.xml
• to display status of minidlna use a webbrowser:
  http://your_ip:8200

OMV-Extras:

• download deb from here
• upload this file from omv webinterface section plugins
• install plugin openmediavault-omvextrasorg
• settings / system / OMV-Extras: activate wanted repository
• check (again) for available plugins (or apt-get update)

usage: some devices on internal network send diagnostic emails; keep those emails in internal network by setting up a smtp server which delivers emails directly to existing dovecot mail server using lmtp.

install postfix on qnap running dovecot
reference: https://wiki.qnap.com/wiki/Postfix

addgroup postfix
adduser -D -H -G postfix postfix
addgroup postdrop

export PATH=/opt/bin:/opt/sbin:$PATH
ln -s /opt/include /usr/include
cd /opt/src
wget –no-check-certificate https://de.postfix.org/ftpmirror/official/postfix-2.11.11.tar.gz
tar -xzf postfix-2.11.11.tar.gz
cd postfix-2.11.11

export CCARGS=’-I/opt/include -L/opt/lib -DDEF_COMMAND_DIR=\”/opt/sbin\” \
-I/opt/include/sasl -DUSE_SASL_AUTH -DDEF_SERVER_SASL_TYPE=\”dovecot\” \
-DHAS_SSL -I/opt/include/openssl -DUSE_TLS\
-DDEF_CONFIG_DIR=\”/opt/etc/postfix\” -DDEF_DAEMON_DIR=\”/opt/libexec/postfix\” -DDEF_DATA_DIR=\”/opt/var/lib/postfix\” \
-DDEF_MAILQ_PATH=\”/usr/bin/mailq\” -DDEF_HTML_DIR=\”/opt/share/doc/postfix/html\” -DDEF_MANPAGE_DIR=\”/opt/man\” \
-DDEF_NEWALIAS_PATH=\”/opt/bin/newaliases\” -DDEF_QUEUE_DIR=\”/opt/var/spool/postfix\” \
-DDEF_README_DIR=\”/opt/share/doc/postfix/readme\” -DDEF_SENDMAIL_PATH=\”/opt/sbin/sendmail\”‘

export AUXLIBS=’-lcrypto -lssl’

export LD_LIBRARY_PATH=/opt/lib
(else postconf, called by post-install, does not find libdb)

make tidy
make

replace #!/bin/sh by #!/opt/bin/bash in makedefs and post-install

edit post-install: search for chown
– replace chown root by chown admin
– in case of chown $owner (followed by chgrp $group) add these lines above the line with chown (at 2 places)
case $owner in root) owner=admin;; esac
case $group in root) group=administrators;; esac

make install

edit scripts:
/opt/libexec/postfix/postfix-script
replace #!/bin/sh by #!/opt/bin/bash
and add below
export PATH=/opt/bin:/opt/sbin:$PATH
export LD_LIBRARY_PATH=/opt/lib

postfix requires a domainname:

• either workaround: so set in /etc/hosts of mail clients
  192.168.0.123 myserver myserver.local
  if hostname of the mailserver is myserver
  • set in /opt/etc/postfix/main.cf
    myhostname = myserver.local
    mydestination = myserver, myserver.local, localhost
    mynetworks = 127.0.0.0/8 192.168.123.0/24
• or better: if you own mydomain.tld and if you have access to dns settings
  • create A record: mail.internal.mydomain.tld 192.168.0.123
  • create MX record: internal.mydomain.tld pointing to mail.internal.mydomain.tld
  • set in /opt/etc/postfix/main.cf
    myhostname = mail.internal.mydomain.tld
    mydestination = internal.mydomain.tld, myserver, myserver.local, localhost
    mynetworks = 127.0.0.0/8 192.168.123.0/24
  • use qnap web admin interface to create a normal user myname
    mailaddress is then myname@internal.mydomain.tld
    computers on 192.168.123.0/24 can send mails by smtp without authentication
    using mailserver mail.internal.mydomain.tld

for debugging: postfix logs to syslogd which does not run by default on qnap

• start syslogd: syslogd
  display messages with: tail -f /var/log/messages
  when finished: killall syslogd
• increase verbosity of postfix by editing /opt/libexec/postfix
  replace master -w by master -vvv -w
• restart postfix:
  postfix stop
  postfix start

connect by lmtp with dovecot for non virtual user setup
mailbox_transport = lmtp:unix:private/dovecot-lmtp (in main.cf)
reference https://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL
and https://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP

using /opt/var/spool/postfix/private/auth
and /opt/var/spool/postfix/private/dovecot-lmtp

after everything works non existing users can be rejected by postfix (main.cf)
smtpd_recipient_restrictions = reject_unverified_recipient

main config: service lmtp has to be enabled
if sasl auth is configured port 587 udp and tcp has to be added to etc services!
submission 587/tcp
submission 587/udp

so add to postfix start script
cat /etc/services |grep -q “[^0-9]587/tcp” || echo “submission 587/tcp” >>/etc/services
cat /etc/services |grep -q “[^0-9]587/udp” || echo “submission 587/udp” >>/etc/services
ln -s /opt/lib/libdb-4.2.so /lib
(last line because LD_LIBRARY_PATH=/opt/lib does not work for every subprocess of postfix)

problem when using dovecot with simple unix system users in default config:
postfix gives mail_user@mail_domain to lmtp but dovecot passdb of type shadow
and userdb of type passwd want mail_user (without @mail_domain suffix)

workaround: switch to passwd-file type of db for passdb and for userdb
which allows with args = username_format=%n to skip @mail_domain
Set in /opt/etc/dovecot/conf.d/auth-system.conf.ext:
service auth {
user = $default_internal_user
group = administrators
}
service auth-worker {
user = $default_internal_user
group = administrators
}

passdb {
driver = passwd-file
args = scheme=md5-crypt username_format=%n /etc/shadow
}
userdb {
driver = passwd-file
args = username_format=%n /etc/passwd
}

service auth {
unix_listener /opt/var/spool/postfix/private/auth {
mode = 0660
user = postfix
group = postfix
}
}

result: mails from local network can be sent to qnap users using smtp on port 25 without authentication

todo configure certs for tls; test smtp(s) auth

Looking for a simple web based app to take notes I discovered http://scrumblr.ca/ and a fork which added markdown tagging and import/export: https://framemo.org/.

Installation on debian stretch using apache2 as proxy allowing password protected access by ssl (described here and here):

• apt-get install nodejs npm redis-server

• mkdir -p /opt/nodejs
  cd /opt/nodejs

• adduser –no-create-home –home /opt/nodejs/scrumblr –disabled-login –gecos “Scrumblr” scrumblr

• git clone https://github.com/aliasaria/scrumblr.git

• chown scrumblr: -R /var/www/scrumblr
  cd scrumblr

• npm install
• add scrumbler extensions of ldidry and test scrumblr on the console:

  • su scrumblr -s /bin/bash

  • git remote add fork https://github.com/ldidry/scrumblr/ git fetch fork git pull fork master
  • node server.js –port 4242 –baseurl /your_url_dir
    bugfix (“RangeError…”): edit config.js and change the line redis: …
    redis: argv.redis || ‘redis://127.0.0.1:6379’
• netstat -ant reveals that scrumbler listens on all interfaces; so replace in server.js line server.listen(conf.port);
  server.listen(conf.port, ‘127.0.0.1’);
• add a service description to /etc/systemd/system/scrumblr.service (reference) and enable scrumblr.service
  (and include startup argument  –baseurl /your_url_dir)
• configure apache2 as proxy
  • enable modules: a2enmod proxy; a2enmod proxy_http
  • add a location section to virtual host config file in /etc/apache2/sites-available/
    <Location /your_url_dir>
    ProxyPass http://127.0.0.1:4242/your_url_dir
    ProxyPassReverse http://127.0.0.1:4242/your_url_dir
    </Location>
  • apache2ctl configtest; apache2ctl graceful
• Open http(s)://your_server.tld/your_url_dir/
  • mini bug: the link to demo board is an absolute url and thus ignores proxy url; to fix this remove absolute path in scrumblr/views/home.jade at line p.home!=…
    p.home!= ‘<a href=”demo”>’ + ‘demo</a>’
  • here is a reference for the use of markdown syntax (e.g. insert links and images)
• Todo: properly support editing on android and ios devices
  • ios recognizes doubleclick but cursor cannot be moved
  • android does not recognize doubleclick
  • scrumblr/client/script.js line card.children(‘.content’).editable…: if event: ‘dblclick’ gets replaced by
    event: ‘click’
    enables editing empty cards on mobile devices at the price that on the desktop dragging of a card activates edit mode
  • differentiate between ‘click’ for touch devices and ‘dblclick’ for other devices: content of diff file is here.

First install composer.

• dependencies: curl and git
• download and verify composer-setup.php as described at composer homepage section download: https://getcomposer.org/download/
• install as /usr/local/bin/composer: php composer-setup.php –install-dir=/usr/local/bin –filename=composer

Second install yii as described here: http://www.yiiframework.com/doc-2.0/guide-start-installation.html)

• create an installation directory for the new yii project one level above document root, e.g. mystuff../
• run composer inside directory mystuff:
  • composer global require “fxp/composer-asset-plugin:^1.3.1”
  • composer create-project –prefer-dist yiisoft/yii2-app-basic ./
  • If document root is defined as public_html ,and should be handled by yii framework, then run:
    ln -s yii/web/ public_html

If a database backend is used, yii code to handle tables can be auto generated using yii component gii.

• configure config/web.php according to manual: http://www.yiiframework.com/doc-2.0/guide-start-gii.html
• ensure that the web server has write (at least temporary) access to yii directory (chown -R www-data:www-data yii)
• create the database table and add comments to each table field (which will become labels for table edit form fields)
• create Model and CRUD using gii
• uncomment primary index field id in views/your_table_name/index.php

To create pdf output use plugin yii-mpdf: https://github.com/kartik-v/yii2-mpdf

• install plugin: composer require kartik-v/yii2-mpdf “1.0.1”
• add pdf config to components array in config/web.pdf
  (and do not forget to add at the beginning of web.php: use kartik\mpdf\Pdf;)
• for a view file views/your_table_name/view2.php add to controllers/yourTableNameController.php
  $pdf = Yii::$app->pdf;
  $pdf->content = $this->renderPartial(‘view2’, [‘model’ => $this->findModel($id)]);
  return $pdf->render();
• to change formatting in dataset detail view
  • reference is source code: vendor/yiisoft/yii2/widgets/DetailView.php (and vendor/yiisoft/yii2/base/Widget.php):
    variables $template = ‘<tr><th{captionOptions}>{label}</th><td{contentOptions}>{value}</td></tr>’
    and $options = [‘class’ => ‘table table-striped table-bordered detail-view’]
  • thus add in file views/your_table_name/view2.php:
    DetailView::widget([ ‘model’ => $model, ‘template’ => ‘your_settings’,  ‘options’ => [your_settings], …

User authentication: http://www.yiiframework.com/doc-2.0/guide-security-authentication.html

(Plugin which allows user registration with email confirmation etc.: https://github.com/dektrium/yii2-user
https://code.tutsplus.com/tutorials/how-to-program-with-yii2-integrating-user-registration–cms-22974)

• the standard base template of yii includes user authentication with hardcoded passwords in file
  models/User.php
• some configuration is set in config/web.php: array components, sub array user, e.g. enableAutoLogin might be set to false
• views/site/login.php defines the login page, e.g. remember me might be removed
• example code to get name of logged in user:
  use app\models\User;
  if( !(Yii::$app->user->identity===null) ){echo User::findIdentity(Yii::$app->user->id)->username;} else {echo “guest”;}
• if( !(Yii::$app->user->identity===null) && ‘100’===Yii::$app->user->id){echo “is admin”;}

Configure Mailer

• set admin email address in config/params.php
• configure mailer in config/web.php as described in
  https://code.tutsplus.com/tutorials/how-to-program-with-yii2-integrating-user-registration–cms-22974
• edit contact form views/site/contact.php