Archive for the ‘Uncategorized’ Category

some notes:

very slow disk access inside vm might be caused by balanced power plan (which is install default in server 2008r2, 2012r2 hyperv core).
use powercfg.exe -l to check powerplan and powercfg -s to set another plan, e.g. performance; check with crystalldiskmark

running some time virtual disks tend to get to small. On server 2012r2 powershell has:
get-vhd -path …, resize-vhd -path -sizebytes …GB, optimize-vhd

before moving a vhd(x) file, display acl lists using icacls (to be able to restore permissions iacl …./grant …)

… net use z:  \\1.2.3.4\sharename * /user:your_name (and net use z: /delete)

partitions inside a vhd may be resized with diskpart
– sel vdisk file=….
– attach vdisk
– sel last partition and the corresponding volume
– extend
– lis par (to check)
– detach

Excellent (!) main reference https://www.cyrius.com/debian/kirkwood/qnap/

Special case ts-221 (and probably all devices with 1GB RAM): kernel of debian stretch 4.9 (also stretch-backports 4.17) seems to fail with memory above 768M on qnap arm
(discussion: https://groups.google.com/forum/#!topic/linux.debian.ports.arm/vSaO642z8aY); dmesg shows
BUG: Bad rss-counter state mm

  • use jessie installer in expert mode and install stretch (offered by jessie installer as testing release)
  • before reboot
    • either switch to older kernel as described below
    • or use only 768MB of 1024MB ram to avoid this bug
      mount /dev and /proc (maybe /sys) inside installation directory and chroot into installation directory

      • copy /usr/share/doc/u-boot-tools/examples/qnap_ts119-219.config
        to /etc/fw_env.config (assuming package u-boot-tools has been installed)
      • then run fw_printenv bootargs which returns something like
        bootargs=console=ttyS0,115200 root=/dev/ram initrd=0xa00000,0x900000 ramdisk=34816
      • now use fw_setenv to add linux kernel boot parameter mem=768M by adding mem=768M to output of fw_printenv
        fw_setenv bootargs “console=ttyS0,115200 root=/dev/ram initrd=0xa00000,0x900000 ramdisk=34816 mem=768M”
        (takes some time to write to flash)
      • check with fw_printenv that bootargs still contains the original arguments (plus mem=768M)
      • reboot (check with free -h and cat /proc/cmdline; dmesg: Memory: … 0K highmem)
      • maybe, this has to be repeated after every linux kernel update (verify withc fw_printenv bootargs if mem=768M ist still there)

 

In case flash-kernel returns Failed to obtain MAC address, run either
ubootcfg -b 0 -f /dev/mtdblock4 -o – | grep “^ethaddr=” | sed “s/^ethaddr=//”
or fw_printenv ethaddr
which returns the mac address stored in flash; then run
iface_set_mac eth0 YOUR_QNAP_MAC_FROM_FLASH

Recovery: run this iso from qnap wiki page inside a vm or install minimal debian with isc-dhcp-server and tftpd-hpa.

To make sure that an existing harddisk gets properly reconfigured use installer option (removal of raid partitions)
guided – entire disk

  • one partion with btrfs and option noatime
  • in case of error see below

Workaround (ok for devices without fan): If status led remains blinking red-green create /etc/rc.local, make it executable
#!/bin/sh
qcontrol –direct statusled greenon
exit 0

Solution (fixes led, buzzer and fan): # qcontrol needs stretch backports: apt-get -t stretch-backports install qcontrol, so add to /etc/apt/sources.list

deb http://ftp.ch.debian.org/debian/ stretch-backports main

and /lib/systemd/system/qcontrold.services is broken: replaces lines
Requires=dev-input-by\x2dpath-platform\x2dgpio_keys\x2devent.device
After=dev-input-by\x2dpath-platform\x2dgpio_keys\x2devent.device

with
ConditionPathExists=/dev/input/by-path/platform-gpio_keys-event (kernel 4.9) or platform-gpio-keys-event (kernel 3.16)
and afterwards run
dpkg –configure qcontrol

In case one disk is used for the system and a second disk is used for data: mount the second disk in /etc/fstab with option noauto (else the system will not start if the data disk has been removed)

Install smartmontools and check disk with smartctl -a /dev/sdXN

Special case ts-221: does not run with kernel 4.9, so run debian stretch with kernel 3.6 -or use bootarg mem=768M as described above
(latest discussion: https://groups.google.com/forum/#!topic/linux.debian.ports.arm/vSaO642z8aY)

  • first install debian jessie (as described on cyrius.com for stretch but replace stretch by jessie in download links)
    • copy /boot/*, /lib/modules/*, /usr/lib/linux-image-* into a tar archive and save it on an external storage device
  • second, flash again the jesssie installer, but now run it in expert mode and install debian stretch
    • before rebooting the second last step “finish install or so” open console 2 (keys <ctrl> + <a>, then key <2>)
      • copy the saved contents of first installation (tar archive) into running installation
      • edit /usr/share/flash-kernel/functions: change force=”no” to force=”yes”
      • run update-initramfs -u -k 3.16..0-6-kirkwood (and probably you get a message to add “-t” as additional argument)
        later if kernel/initramfs get renewed, it is required to run afterwards: update-initramfs -u -k $(uname -r)

the installer does not cope well with already used disks (guided, use whole disk should work); if not, start installer in expert mode and add fdisk as installer component
(in case of running debian stretch wipefs -a removes all kind of signatures; but do double check that you will not wipe the wrong partition!)

  • open shell 2 and run cat /proc/partitions
  • use fdisk to create a new partition table (dos format) and create one big primary partion
  • create a filesystem on this partition with mkfs.ext2
  • reboot!!! And then run installer again in standard mode

Some qnap models may take more than 15 minutes until kernel and initrd are written to flash. – First reboot after installation might by succesful, even if power led keeps blinking red-green and buzzer remains silent. Led and buzzer need installation of qcontrol from backports and patched qcontrold.service file as described above.

Enable quota: btrfs quota enable /backup; btrfs quota enable /backup2
Display disk usage: btrfs sub list /backup; btrfs qgroup show /backup
unused because of to much memory usage when deleting snapshots

mkfs.btrfs shows after creation of filesystem whether incompatibilities with current kernel exist; use -O ^prop1,^prop2 if properties prop1,prop2 ar not compatible; (use wipefs to remove the newly created filesystem and recreate it using -O ^…)

Convert btrfs single partition filesystem to btrfs raid1 during debian install

  • use installer to create a btrfs root partition (noatime option) on a single partition
  • use installer to create on second disk a partition of same size without mountpoint
  • let installer write new partitions to disk
  • open second ssh connection and choose shell
    • btrfs filesystem show
    • df -h
    • btrfs device add -f /dev/PARTITION_ON_SECOND_DISK /target
    • btrfs balance start -dconvert=raid1 -mconvert=raid1 /target
    • btrfs filesystem show

Recover from failure of one drive of root filesystem on btrfs raid1

  • difficulty: if / gets remounted ro because of filesystem errors reboot into installer is required
  • prepare for rescue with installer
    • mkdir /var/backups/flash-installer
    • cd /tmp
    • wget http://ftp.debian.org/debian/dists/stretch/main/installer-armel/current/images/kirkwood/network-console/qnap/ts-21x/initrd
      wget http://ftp.debian.org/debian/dists/stretch/main/installer-armel/current/images/kirkwood/network-console/qnap/ts-21x/kernel-6282
    • dmesg |grep SoC
      ID=0x… displays wether kernel 6282 or 6281 is used
      cp initrd /var/backups/flash-installer/mtd2
      cp kernel-6282 /var/backups/flash-installer/mtd1 (or use kernel 6281)
    • to boot into installer run:
      cat /var/backups/flash-installer/mtd1 >/dev/mtdblock1
      cat /var/backups/flash-installer/mtd2 >/dev/mtdblock2
    • to restore installed system when running installer run:
      mount YOUR_PARTITION /mnt
      cat /mnt/var/backups/flash-kernel/mtd1 >/dev/mtdblock1
      cat /mnt/var/backups/flash-kernel/mtd2 >/dev/mtdblock2
  • recovery steps (if raid1 consists of /dev/sda3 and /dev/sdb3)
    • btrfs fi show
      displays whether both raid1 drives are accessible
    • echo 1 >/dev/block/sda/device/delete
      forces /dev/sda offline (e.g. in case of too many smart errors)
    • if btrfs fi show  displays that one drive is missing, e.g. sda3, do not run other btrfs commands to avoid errors which cause transition to ro filesystem, but do immediately run
      mount -o remount,degraded /dev/sdb3 /
    • attach replacement drive and check its device name with cat /proc/partitions (od dmesg), e.g sdc3
    • remove any existing btrfs data on replacement drive partition, e.g. on sdc3 by formatting it with ext4
      mkfs.ext4 -F /dev/sdc3
    • btrfs dev add -f /dev/sdc3 /
    • btrfs fi show
      • if sdc3 is shown twice, remove sdc3 with lower id from btrfs raid1
        btrfs dev del YOUR_DEVID /
      • if it is shown, that one drive is missing run
        btrfs del missing /
    • btrfs fi show
      this should know display two drives/partitions attached
    • to check for other error btrfs scrub might be run (time consuming)
    • reboot needed to remove option degraded from /proc/mounts

Switch hard disk with root filesystem

  • debian stretch on qnap sets kernel command line ROOT=”/dev/….” inside initramfs which is stored in flash. So using another hard disk requires to reflash initramfs
    • update-initramfs -u creates a new initramfs (at /boot/initrd.img-KERNEL_VERSION)
      and runs flash-kernel KERNEL_VERSION which creates /var/backups/flash-kernel/mtd2 (and mtd1) and flashes it
    • update-initramfs (/usr/share/initramfs-tools/hooks/flash_kernel_set_root) reads /etc/fstab to get the device with the root filesystem
    • use blkid YOUR_DEVICE_NAME to display the uuid of a block device
    • So to set a new root device, temporarily edit /etc/fstab (on the old root partition) to point to the new root filesystem device, run update-initramfs -u; then restore the old settings in/etc/fstab on old partition;
      verify that the new partition is correctly set in /etc/fstab on the new root partition,
    • additional step for recovery by tftp (based on instructions here)
      • backup flash partitions /dev/mtdblockN (N=0 … 5) to files mtdN
      • download installer kernel and initrd (reference)
        to determine kernel version (6281 or 6282)
        /usr/share/flash-kernel/dtb-probe/kirkwood-qnap
      • pad kernel
        dd if=kernel-628X of=kernel.pad ibs=2097152 conv=sync (with X=1 or 2)
      • create tftp img file
        cat mtd0 mtd4 mtd5 kernel.pad initrd mtd3 > F_TS-219_di
        and copy this recovery image file to another computer or storage device
    • finally reboot
    • if reboot fails, load qnap recovery live cd iso here
      • run live cd in e.g. virtualbox (use a network adapter not connected to existing lan)
      • copy recovery image file to /tftpboot/ on live cd
        • add a second network interface connected to local lan to allow upload by ssh
        • sudo su
        • ifconfig eth1 YOUR_LAN_IP
        • set a password for user ubuntu
          passwd ubuntu
        • upload img file with scp to /home/ubuntu
        • ifconfig eth1 down (because dhcp server of live cd)
        • cp img file to /tftpboot/ and replace existing img file for your qnap device
      • power off qnap and power on with reset button pressed (until double beep):
        this should reflash qnap flash partitions with debian installer

 

Example btrfs send, btrfs receive (btrfs volumes mounted at /backup and at /backup2;
transfer readonly subvolume rootf_ro from volume mounted at /backup to other volume mounted at /backup2)

  • btrfs sub snapshot -r /backup/rootfs /backup/rootfs_ro

  • btrfs send /backup/rootfs_ro |btrfs receive /backup2

Resize/shrink btrfs partition with root filesystem (e.g. openmediavault needs separate data partition);
assume that btrfs partition is mounted at /backup using /dev/sda2

  • btrfs filesystem resize 32G /backup
  • fdisk /dev/sda
    • p
    • d 2
    • n 2 default start +40G
      do not delete btrfs signature
    • p
    • w
    • q
  • partprobe
  • btrfs filesystem resize max /backup
  • btrfs filesystem show
  • cat /etc/fstab

Add openmediavault (version 4.x arrakis for debian stretch): instructions

  • echo “deb http://packages.openmediavault.org/public arrakis main” > /etc/apt/sources.list.d/openmediavault.list
  • apt-get install openmediavault-keyring
  • apt-get update
  • apt-get install openmediavault
  • omv-initsystem

Create raid0 btrfs volume and use subvolumes as shared folders:

  • mkfs.btrfs -L datafs -d raid0 /dev/sda4 /dev/sdb4
  • then use webinterface filesystems to mount this volume
    (gets mounted at /srv/dev-disk-by-label-datafs; cat /etc/fstab)
  • btrfs subvol create /srv/dev-disk-by-label-datafs/YourShare
  • then use webinterface to define shared folder YourShare
    • use shared folder / ACL if write access fails
  • (in case of macos sierra don’t) share this folder by nfs
    • currently, 2017-11-02, apply changes fails
      (tries to mount second partition included in btrfs raid0)
    • reboot from console
    • log into webinterface and apply change again: should work
    • nfs://your_ip/export/YourShare/
      (with default settings needs option insecure)
    • currently, 2017-11-02, nfs with macos sierra is awfully slow but samba sharing (activate wins server) works
      • nfs seems to have locking issues because ejecting share from macos finder fails (unless finder is forced to restart)

minidlna media player

  • apt-get install minidlna
  • btrfs subvol create /srv/dev-disk-by-label-datafs/Musik
  • omv webinterface: add shared folder Musik on device with datafs
    which creates /sharedfolders/Musik
  • edit /etc/minidlna.conf (compare with this reference)
    • inotify=no
    • friendly_name=YOUR_MEDIA_SERVER
    • media_dir=A,/sharedfolders/Musik
  • chown -R minidlna:minidlna /sharedfolders/Musik
  • create media file index from scratch
    • systemctl stop minidlna
    • rm /var/cache/minidlna/*
    • systemctl start minidlna
  • if new mediafiles have been added
    • chown -R minidlna:minidlna
    • systemctl restart minidlna
  • if minidlna does not get discovered by plugplayer (ios), then manually add server:
    http://your_ip:8200/rootDesc.xml
  • to display status of minidlna use a webbrowser:
    http://your_ip:8200

OMV-Extras:

  • download deb from here
  • upload this file from omv webinterface section plugins
  • install plugin openmediavault-omvextrasorg
  • settings / system / OMV-Extras: activate wanted repository
  • check (again) for available plugins (or apt-get update)

usage: some devices on internal network send diagnostic emails; keep those emails in internal network by setting up a smtp server which delivers emails directly to existing dovecot mail server using lmtp.

install postfix on qnap running dovecot
reference: https://wiki.qnap.com/wiki/Postfix

addgroup postfix
adduser -D -H -G postfix postfix
addgroup postdrop

export PATH=/opt/bin:/opt/sbin:$PATH
ln -s /opt/include /usr/include
cd /opt/src
wget –no-check-certificate https://de.postfix.org/ftpmirror/official/postfix-2.11.11.tar.gz
tar -xzf postfix-2.11.11.tar.gz
cd postfix-2.11.11

export CCARGS=’-I/opt/include -L/opt/lib -DDEF_COMMAND_DIR=\”/opt/sbin\” \
-I/opt/include/sasl -DUSE_SASL_AUTH -DDEF_SERVER_SASL_TYPE=\”dovecot\” \
-DHAS_SSL -I/opt/include/openssl -DUSE_TLS\
-DDEF_CONFIG_DIR=\”/opt/etc/postfix\” -DDEF_DAEMON_DIR=\”/opt/libexec/postfix\” -DDEF_DATA_DIR=\”/opt/var/lib/postfix\” \
-DDEF_MAILQ_PATH=\”/usr/bin/mailq\” -DDEF_HTML_DIR=\”/opt/share/doc/postfix/html\” -DDEF_MANPAGE_DIR=\”/opt/man\” \
-DDEF_NEWALIAS_PATH=\”/opt/bin/newaliases\” -DDEF_QUEUE_DIR=\”/opt/var/spool/postfix\” \
-DDEF_README_DIR=\”/opt/share/doc/postfix/readme\” -DDEF_SENDMAIL_PATH=\”/opt/sbin/sendmail\”‘

export AUXLIBS=’-lcrypto -lssl’

export LD_LIBRARY_PATH=/opt/lib
(else postconf, called by post-install, does not find libdb)

make tidy
make

replace #!/bin/sh by #!/opt/bin/bash in makedefs and post-install

edit post-install: search for chown
– replace chown root by chown admin
– in case of chown $owner (followed by chgrp $group) add these lines above the line with chown (at 2 places)
case $owner in root) owner=admin;; esac
case $group in root) group=administrators;; esac

make install

edit scripts:
/opt/libexec/postfix/postfix-script
replace #!/bin/sh by #!/opt/bin/bash
and add below
export PATH=/opt/bin:/opt/sbin:$PATH
export LD_LIBRARY_PATH=/opt/lib

postfix requires a domainname:

  • either workaround: so set in /etc/hosts of mail clients
    192.168.0.123 myserver myserver.local
    if hostname of the mailserver is myserver

    • set in /opt/etc/postfix/main.cf
      myhostname = myserver.local
      mydestination = myserver, myserver.local, localhost
      mynetworks = 127.0.0.0/8 192.168.123.0/24
  • or better: if you own mydomain.tld and if you have access to dns settings
    • create A record: mail.internal.mydomain.tld 192.168.0.123
    • create MX record: internal.mydomain.tld pointing to mail.internal.mydomain.tld
    • set in /opt/etc/postfix/main.cf
      myhostname = mail.internal.mydomain.tld
      mydestination = internal.mydomain.tld, myserver, myserver.local, localhost
      mynetworks = 127.0.0.0/8 192.168.123.0/24
    • use qnap web admin interface to create a normal user myname
      mailaddress is then myname@internal.mydomain.tld
      computers on 192.168.123.0/24 can send mails by smtp without authentication
      using mailserver mail.internal.mydomain.tld

for debugging: postfix logs to syslogd which does not run by default on qnap

  • start syslogd: syslogd
    display messages with: tail -f /var/log/messages
    when finished: killall syslogd
  • increase verbosity of postfix by editing /opt/libexec/postfix
    replace master -w by master -vvv -w
  • restart postfix:
    postfix stop
    postfix start

connect by lmtp with dovecot for non virtual user setup
mailbox_transport = lmtp:unix:private/dovecot-lmtp (in main.cf)
reference https://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL
and https://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP

using /opt/var/spool/postfix/private/auth
and /opt/var/spool/postfix/private/dovecot-lmtp

after everything works non existing users can be rejected by postfix (main.cf)
smtpd_recipient_restrictions = reject_unverified_recipient

main config: service lmtp has to be enabled
if sasl auth is configured port 587 udp and tcp has to be added to etc services!
submission 587/tcp
submission 587/udp

so add to postfix start script
cat /etc/services |grep -q “[^0-9]587/tcp” || echo “submission 587/tcp” >>/etc/services
cat /etc/services |grep -q “[^0-9]587/udp” || echo “submission 587/udp” >>/etc/services
ln -s /opt/lib/libdb-4.2.so /lib
(last line because LD_LIBRARY_PATH=/opt/lib does not work for every subprocess of postfix)

problem when using dovecot with simple unix system users in default config:
postfix gives mail_user@mail_domain to lmtp but dovecot passdb of type shadow
and userdb of type passwd want mail_user (without @mail_domain suffix)

workaround: switch to passwd-file type of db for passdb and for userdb
which allows with args = username_format=%n to skip @mail_domain
Set in /opt/etc/dovecot/conf.d/auth-system.conf.ext:
service auth {
user = $default_internal_user
group = administrators
}
service auth-worker {
user = $default_internal_user
group = administrators
}

passdb {
driver = passwd-file
args = scheme=md5-crypt username_format=%n /etc/shadow
}
userdb {
driver = passwd-file
args = username_format=%n /etc/passwd
}

service auth {
unix_listener /opt/var/spool/postfix/private/auth {
mode = 0660
user = postfix
group = postfix
}
}

result: mails from local network can be sent to qnap users using smtp on port 25 without authentication

todo configure certs for tls; test smtp(s) auth

Looking for a simple web based app to take notes I discovered http://scrumblr.ca/ and a fork which added markdown tagging and import/export: https://framemo.org/.

Installation on debian stretch using apache2 as proxy allowing password protected access by ssl (described here and here):

  • apt-get install nodejs npm redis-server

  • mkdir -p /opt/nodejs
    cd /opt/nodejs
  • adduser –no-create-home –home /opt/nodejs/scrumblr –disabled-login –gecos “Scrumblr” scrumblr

  • git clone https://github.com/aliasaria/scrumblr.git

  • chown scrumblr: -R /var/www/scrumblr
    cd scrumblr

  • npm install
  • add scrumbler extensions of ldidry and test scrumblr on the console:
    • su scrumblr -s /bin/bash

    • git remote add fork https://github.com/ldidry/scrumblr/ git fetch fork git pull fork master
    • node server.js –port 4242 –baseurl /your_url_dir
      bugfix (“RangeError…”): edit config.js and change the line redis: …
      redis: argv.redis || ‘redis://127.0.0.1:6379’
  • netstat -ant reveals that scrumbler listens on all interfaces; so replace in server.js line server.listen(conf.port);
    server.listen(conf.port, ‘127.0.0.1’);
  • add a service description to /etc/systemd/system/scrumblr.service (reference) and enable scrumblr.service
    (and include startup argument  –baseurl /your_url_dir)
  • configure apache2 as proxy
    • enable modules: a2enmod proxy; a2enmod proxy_http
    • add a location section to virtual host config file in /etc/apache2/sites-available/
      <Location /your_url_dir>
      ProxyPass http://127.0.0.1:4242/your_url_dir
      ProxyPassReverse http://127.0.0.1:4242/your_url_dir
      </Location>
    • apache2ctl configtest; apache2ctl graceful
  • Open http(s)://your_server.tld/your_url_dir/
    • mini bug: the link to demo board is an absolute url and thus ignores proxy url; to fix this remove absolute path in scrumblr/views/home.jade at line p.home!=…
      p.home!= ‘<a href=”demo”>’ + ‘demo</a>’
    • here is a reference for the use of markdown syntax (e.g. insert links and images)
  • Todo: properly support editing on android and ios devices
    • ios recognizes doubleclick but cursor cannot be moved
    • android does not recognize doubleclick
    • scrumblr/client/script.js line card.children(‘.content’).editable…: if event: ‘dblclick’ gets replaced by
      event: ‘click’
      enables editing empty cards on mobile devices at the price that on the desktop dragging of a card activates edit mode
    • differentiate between ‘click’ for touch devices and ‘dblclick’ for other devices: content of diff file is here.

First install composer.

  • dependencies: curl and git
  • download and verify composer-setup.php as described at composer homepage section download: https://getcomposer.org/download/
  • install as /usr/local/bin/composer: php composer-setup.php –install-dir=/usr/local/bin –filename=composer

Second install yii as described here: http://www.yiiframework.com/doc-2.0/guide-start-installation.html)

  • create an installation directory for the new yii project one level above document root, e.g. mystuff../
  • run composer inside directory mystuff:
    • composer global require “fxp/composer-asset-plugin:^1.3.1”
    • composer create-project –prefer-dist yiisoft/yii2-app-basic ./
    • If document root is defined as public_html ,and should be handled by yii framework, then run:
      ln -s yii/web/ public_html

If a database backend is used, yii code to handle tables can be auto generated using yii component gii.

  • configure config/web.php according to manual: http://www.yiiframework.com/doc-2.0/guide-start-gii.html
  • ensure that the web server has write (at least temporary) access to yii directory (chown -R www-data:www-data yii)
  • create the database table and add comments to each table field (which will become labels for table edit form fields)
  • create Model and CRUD using gii
  • uncomment primary index field id in views/your_table_name/index.php

To create pdf output use plugin yii-mpdf: https://github.com/kartik-v/yii2-mpdf

  • install plugin: composer require kartik-v/yii2-mpdf “1.0.1”
  • add pdf config to components array in config/web.pdf
    (and do not forget to add at the beginning of web.php: use kartik\mpdf\Pdf;)
  • for a view file views/your_table_name/view2.php add to controllers/yourTableNameController.php
    $pdf = Yii::$app->pdf;
    $pdf->content = $this->renderPartial(‘view2’, [‘model’ => $this->findModel($id)]);
    return $pdf->render();
  • to change formatting in dataset detail view
    • reference is source code: vendor/yiisoft/yii2/widgets/DetailView.php (and vendor/yiisoft/yii2/base/Widget.php):
      variables $template = ‘<tr><th{captionOptions}>{label}</th><td{contentOptions}>{value}</td></tr>’
      and $options = [‘class’ => ‘table table-striped table-bordered detail-view’]
    • thus add in file views/your_table_name/view2.php:
      DetailView::widget([ ‘model’ => $model, ‘template’ => ‘your_settings’,  ‘options’ => [your_settings], …

User authentication: http://www.yiiframework.com/doc-2.0/guide-security-authentication.html

(Plugin which allows user registration with email confirmation etc.: https://github.com/dektrium/yii2-user
https://code.tutsplus.com/tutorials/how-to-program-with-yii2-integrating-user-registration–cms-22974)

  • the standard base template of yii includes user authentication with hardcoded passwords in file
    models/User.php
  • some configuration is set in config/web.php: array components, sub array user, e.g. enableAutoLogin might be set to false
  • views/site/login.php defines the login page, e.g. remember me might be removed
  • example code to get name of logged in user:
    use app\models\User;
    if( !(Yii::$app->user->identity===null) ){echo User::findIdentity(Yii::$app->user->id)->username;} else {echo “guest”;}
  • if( !(Yii::$app->user->identity===null) && ‘100’===Yii::$app->user->id){echo “is admin”;}

Configure Mailer

  • set admin email address in config/params.php
  • configure mailer in config/web.php as described in
    https://code.tutsplus.com/tutorials/how-to-program-with-yii2-integrating-user-registration–cms-22974
  • edit contact form views/site/contact.php

Using a scanner often results in large files. OS X preview program allows to export to multiple formats. Choosing filetype pdf enables the option to apply quartz filters (generic export dialog; shortcut export to pdf does not show quartz filters in OS X 10.11.6). The builtin quartz filter reduce file size does create tiny documents but text might get unreadable.

But the builtin tool ColorSync allows to modify and create quartz filters (german howto with screenshots here). If a newly created filter should show up in the export dialog of the preview program it has to be copied from
/Users/your_name/Library/Filters to /Library/PDF Services
(tested in OS X 10.11.6).

Example of settings for small but (rather) readable pdf files:

  • Image: pixel per inch = 100; quality = high; all other settings empty
  • Image (gray): pixel per inch = 100; quality = high; all other settings empty
  • Image compression: jpeg; quality = slider one tick below medium position

To shrink multiple pdf files the custom quartz filter can be used from the command line:

  • Open Tools / Automator and create a new workflow
  • add action pdf / apply quartz filter
  • do not add other actions
  • save this workflow, e.g. as /yourpath/pdfreduce
  • copy all pdf files which should be converted to a new folder
  • open a terminal and cd to the folder with pdf files which should be shrinked
  • to convert one file run: automator -i yourdoc.pdf /yourpath/pdfreduce.workflow
  • multiple files: for f in *.pdf; do echo “$f”; automator -i “$f” pdfreduce.workflow; done

Nagios is a perfect server monitoring tool and a NAS runs 24 hours a day. ndo2db grapher displays realtime graphics.nagios_graph

Resources:

ndo2db grapher on exchange.nagios.org (Version 0.3.2) with a minor patch: ndographer.diff (background); installation as described on exchange.nagios.org

nagios 4.2.1 from github

nagios plugins 2.1.2 from nagios.org

ndoutils 2.1.1 from github (made by the makers of nagios): ndomod.o and ndo2db to transfer nagios data to  a mysql database.

jpgraph 3.0.7 from sourceforge

Hardware: qnap TS-119P II with Optware-IPKG app installed (gcc, automake, make, … installed); apache, php and mysql are part of qnap firmware (currently 4.2.2).

Installation notes:

Nagios:
configure options:
--prefix=/opt/nagios --enable-nanosleep --with-init-dir=/opt/etc/init.d
edit Makefile and replace

-o root -g root by -o admin -g administrators
run make all; make install; make install-init; make install-config; make install-commandmode
symlink /opt/nagios/share to /share/Web/nagios
edit /etc/config/apache/apache.conf and set Options +ExecCGI for directory /share/Web/nagios/cgi-bin
and add group nagios to qnap apache user: usermod -a -G nagios httpdusr

Nagios plugins:
configure options:
--prefix=/opt/nagios
run make all; make install (installs to /opt/nagios/sbin)
symlink /opt/nagios/sbin to /share/Web/nagios/cgi-bin

Ndoutils:
configure options:
--prefix=/opt --with-ndo2db-user=nagios --with-ndo2db-group=nagios --enable-mysql
copy ndomod-4x.o and ndo2db-4x to /opt/nagios/bin/ndomod.o and ndo2db
use phpmyadmin (installed as qnap app) to create the database and to import db/mysql.sql
(ipkg install mysql5 installs the mysql library and headers required to compile ndoutils with mysql support)
set the broker module in /opt/nagios/etc/nagios.cfg:
broker_module=/opt/nagios/bin/ndomod.o config_file=/opt/nagios/etc/ndomod.cfg
set
output=/opt/var/ndo.sock in ndomod.cfg and enter database name/user/password in ndo2db.cfg
and set lock_file=/opt/var/ndo2db.pid
run /opt/nagios/bin/ndo2db -c /opt/nagios/etc/ndo2db.cfg

Ndo2db grapher and jpgraph:
copy the tar archive content to /opt/nagios/share and create directory /opt/nagios/share/ndographer/jpgraph
patch choose_nagios.php and display_nagios_info.php using the patch file
copy the folder src contained in jgraph-3.0.7 archive  to /opt/nagios/share/ndographer/jpgraph/src
edit ndographer/nagios_graph_conf.inc.php (database name, user, password and
$cache_directory = “/opt/nagios/share$cache_htdocs”;
$pre_generated_image_path = “/opt/nagios/share/nagios-cache”;
create directory /opt/nagios/share/nagios-cache/cache and set owner to httpdusr
(and symlink /opt/nagios/share/nagios-cache to /share/Web/nagios-cache)

System configuration:

Configuration of nagios:
Either set http(s) authentication (see below) or disable authentication in /opt/nagios/etc/cgi.cfg
use_authentication=0 (warning: only if access to your_ip_adress/nagios is otherwise restricted!)
Add a notes_url line to every service which should display a graph
notes_url   /nagios/ndographer/display_nagios_info.php?name1[]=$HOSTNAME$&name2[]=$SERVICEDESC$
(displays in “service status details for a host a white icon (“view extra service notes) which opens the graphic; add other variables to set options like &max_value=20; first define settings by webinterface and check which variables are used in resulting url); to submit the changes to nagios run
/opt/etc/init.d/nagios reload

Start and stop nagios automatically:
1) enable optware start/stop scripts in /opt/etc/init.d (already described here; /dev/mtdblock5 instead of /dev/mtd5 with qnap firmware 4.2.2)
mkdir /tmp/config
mount /dev/mtdblock5 /tmp/config
– copy K010optstartstop.sh to /tmp/config (with extension ‘.sh’, not with extension.txt; partly found here and here)
chmod +x /tmp/config/K010optstartstop.sh
umount /tmp/config
2) nagios make install-init has created /opt/etc/init.d/nagios (=file daemon-init in nagios source folder)
3) create script file /opt/etc/init.d/S61nagios, make it executable and edit it:
#!/bin/sh
if [ -f /opt/etc/init.d/nagios ] ; then
[ -f /opt/nagios/etc/ssmtp.conf ] && mv /opt/nagios/etc/ssmtp.conf /opt/nagios/etc/ssmtp.conf.bak
cp /etc/config/ssmtp/ssmtp.conf /opt/nagios/etc/ && rm /opt/nagios/etc/ssmtp.conf.bak
/bin/chgrp nagios /opt/nagios/etc/ssmtp.conf
/opt/nagios/bin/ndo2db -c /opt/nagios/etc/ndo2db.cfg
/opt/etc/init.d/nagios start
fi

4) create script file /opt/etc/init.d/K39nagios, make it executable and edit it:
#!/bin/sh
if [ -f /opt/etc/init.d/nagios ] ; then
/opt/etc/init.d/nagios stop
killall ndo2db
fi

Http authentication:
All qnap apache helper binaries are in /usr/local/apache/bin
/usr/local/apache/bin/htpasswd -c /etc/config/apache/.htpasswd.nagios your_user_name
and add to /etc/config/apache/apache.conf:
<Directory “/share/Web/nagios”>
AuthType Basic
AuthName “password required”
AuthUserFile /etc/config/apache/.htpasswd.nagios
Require valid-user
</Directory>
and run
/usr/local/apache/bin/apachectl configtest
/usr/local/apache/bin/apachectl graceful
Then add your_user_name to /opt/nagios/etc/cgi.cfg:
use_authentication=1
authorized_for_system_information=nagiosadmin,your_user_name
authorized_for_configuration_information=nagiosadmin,your_user_name
authorized_for_system_commands=nagiosadmin,your_user_name
authorized_for_all_services=nagiosadmin,your_user_name
authorized_for_all_hosts=nagiosadmin,your_user_name
authorized_for_all_service_commands=nagiosadmin,your_user_name
authorized_for_all_host_commands=nagiosadmin,your_user_name
and run
/opt/etc/init.d/nagios configtest
/opt/etc/init.d/nagios reload

Configuration of sendmail on qnap:
qnap firmware installs ssmtp with config in /etc/config/ssmtp/
(and the start script S61nagios copies ssmtp.conf to /opt/nagios/etc/ and sets group nagios)
use webinterface: control panel /system settings /notification
to define the smtp server of your mail account
Then edit /etc/config/ssmtp/ssmtp.conf:
hostname=localhost
and /etc/config/ssmtp/revaliases (use same smtp server as in ssmtp.conf):
admin:you@yourdomain.org:your_smtp_server:your_smtp_port
nagios:you@yourdomain.org:your_smtp_server:your_smtp_port

and run (logged in as admin user):
echo "first message" |sendmail your_receiver@your_mailbox.org
This sends mails from admin as mails from you@yourdomain.org; (if this command runs without errors check your inbox and your spam folder).
Repeat this test as user nagios:
cp /etc/config/ssmtp/ssmtp.conf /opt/nagios/etc/; chgrp nagios /opt/nagios/etc/ssmtp.conf;
su nagios; echo "second message" |sendmail -C/opt/nagios/etc/ssmtp.conf your_receiver@your_mailbox.org; exit
Nagios notifications use the program printf which is in /opt/bin/printf (which gets installed with ipkg install coreutils)
and edit /opt/nagios/etc/objects/commands.cfg:
/usr/bin/printf has to be replaced by /opt/bin/printf
and the subject line has to be moved from sendmail to printf:
sendmail -s "........." has to be replaced by sendmail -C/opt/nagios/etc/ssmtp.conf
and printf "%b" "****** Nagios has to be replaced with printf "%b" "Subject: .........\n\n****** Nagios
Reload nagios with /opt/etc/init.d/nagios reload and try to send a forced host (or service) notification from nagios web interface: Host (or Service) state information
(Change debug level from 0 to 164 and debug_verbosity from 1 to 2 in /opt/nagios/etc/nagios.cfg to search bugs; debug logfile: /opt/nagios/var/nagios.debug; normal logfile: nagios.log)
The comments given by the user in notifications manually sent by the user may be included into the message using variable $NOTIFICATIONCOMMENT$ (reference).

Firmware update woes:

  • nagios section in /etc/config/apache/apache.config has to be reinserted
  • ipkg install procps; ipkg install inetutils: ping and ps in /opt/bin are kept
  • but nagios needs ping (and nagios initscript needs ps) non busysbox verion in /bin:
    • cd /bin; mv ping ping.bb; mv ps ps.bb; ln -s /opt/bin/ping ping; ln -s /opt/bin/ps ps

Bare metal recovery: Making the hard disk with image data bootable – update of this post; requires a licensed version of trueimage 2016 home (to be able to either download the acronis standalone iso from acronis or to create the iso with acronis media builder).

Step 1: Initialization of the disk using windows diskpart (compare this post); all data on this disk gets erased; tested with diskpart of windows 10:

  • list disk
  • select disk 1 (be careful to select the external disk which will be erased)
  • clean
  • convert gpt (automatically creates a 100 MB msr partition which remains unused)
  • create partition efi size=512
  • format fs=fat32 quick label=EFI
  • create partition primary size=2048 (bootloader and acronis)
  • create partition primary (use remaining space for image data)
  • exit

Step 2: Continue with linux; tested with ubuntu 16.04.

Step 2: Prepare and format the hard disk:

  • sudo su
  • gdisk -l /dev/sdb (be careful to select the external disk which has been prepared in step 1)
  • use df -h to show if any partitions of /dev/sdb are mounted and unmount them
  • gdisk /dev/sdb (be careful to select the external disk – replace sdb by your disk sd… in the instructions below!)
    • p
    • t
      3 <enter> (sets type of partition 3 to linux; do not set the data partition 4 to linux – else trueimage won’t allow to read/write tib files)
    • x
    • a
      3
      2 <enter> (set gpt partition flag bios bootable, used by extlinux boot loader)
    • w
  • mkfs.ext2 -L BootExtHD /dev/sdb3
  • mkfs.ext4 -L DataExtHD /dev/sdb4

Step 3: Install extlinux to make the hard disk bootable in bios mode:

  • mount /dev/sdb3 /mnt
  • mkdir /mnt/extlinux
  • extlinux -i /mnt/extlinux
  • touch /mnt/extlinux/extlinux.conf
  • mcedit /mnt/extlinux/extlinux.conf (compare these instructions on reboot.pro)
    • DEFAULT ti2016
      SAY booting TrueImage 2016 from USB disk with extlinux
      LABEL ti2016
      KERNEL /acronis/dat10.dat
      APPEND quiet force_modules=usbhid lib80211=off
      INITRD /acronis/dat11.dat,/acronis/dat12.dat
    • my machine crashes if lib80211.ko gets loaded (ti2016 build 6571)
      else parameter lib80211=off is not neccessary
  • dd if=/usr/lib/syslinux/mbr/gptmbr.bin of=/dev/sdb (add a syslinux/extlinux compatible mbr to the hard disk)

Step 4: Install grub to make the hard disk bootable in (u)efi secure boot mode; assumes that ubuntu 16.04 has been installed with grub efi boot loader and that loading of unsigned kernel (modules) has been enabled:
mokutil –disable-validation (has to be run as root, disables kernel validation at reboot; boot loader grub then complains insecure boot.)

  • mount /dev/sdb2 /mnt2
  • mkdir /mnt2/EFI
  • copy /boot/efi/EFI/ubuntu to /mnt2/EFI/ubuntu
  • edit /mnt2/EFI/ubuntu/grub.cfg
    • run: blkid /dev/sdb3
    • replace the uuid filesystem number in grub.cfg by the UUID number shown by blkid
    • remove grub hint hd0,gpt6 (in case your ubuntu (boot) partition is sda6)
    • replace set prefix=($root)’/boot/grub’
      by set prefix=($root)’/grub’
    • e.g.:
      search.fs_uuid 4164bfdd-bfbe-464a-9e7f-1ec096f84c2d root
      set prefix=($root)'/grub'
      configfile $prefix/grub.cfg
  • umount /mnt2
  • mkdir /mnt/grub
  • copy the contents of directory /boot/grub/ to /mnt/grub/
  • delete /mnt/grub/grubenv
  • edit /mnt/grub/grub.cfg
    • set default="0"
      set timeout="10"
    • taken from ubuntu:
      function load_video {
      if [ x$feature_all_video_module = xy ]; then
      insmod all_video
      else
      insmod efi_gop
      insmod efi_uga
      insmod ieee1275_fb
      insmod vbe
      insmod vga
      insmod video_bochs
      insmod video_cirrus
      fi
      }
    • taken from ubuntu (but replace filesystem UUID number ):
      font=unicode
      insmod part_gpt
      insmod ext2
      search --no-floppy --fs-uuid --set=root 4164bfdd-bfbe-464a-9e7f-1ec096f84c2d
      load_video
      gfxmode text
    • again using these instructions on reboot.pro:
      menuentry "Acronis TruImage Home 2016 on USB disk" {
      set quiet=1
      set gfxpayload=1024x768x32,1024x768
      set mbrcrcs=on
      # bug: kernel module lib80211 crashes kernel in build 6571 (ti 64bit)
      linux /acronis/dat10.dat lib80211=off quiet force_modules=usbhid
      initrd /acronis/dat11.dat /acronis/dat12.dat
      boot
      }

Step 5: Copy acronis trueimage 2016 from iso to hard disk:

  • mkdir /mnt/acronis
  • loop mount the acronis iso file (e.g. mount -o loop acronis.iso /mnt2)
  • copy the files dat10.dat, dat11.dat and dat12.dat to directory /mnt/acronis
  • umount /mnt

Step 6: enable to boot on a mac mini

  • The builtin boot selector of a mac mini expects the grub efi file inside the efi partition at it’s official place:
    EFI/boot/bootx64.efi (or EFI/boot/bootia32.efi: see step 7)
  • to configure grub as official boot loader simply copy
    MokManager.efi, shimx64.efi, grubx64.efi and grub.cfg
    from EFI/ubuntu/ to EFI/boot/ and
    rename shimx64.efi to bootx64.efi
  • Now the boot selector of OS X on a newer mac mini (e.g. late 2012) shows
    an USB disk with subtitle EFI Boot (keep alt/option key pressed on power on to start the boot selector)

Step 7 (optional): enable to boot on older mac minis (late 2007, 5.1)

  • These systems work with 32 bit efi (grub-mkimage -O i386-efi -d /usr/lib/grub/i386-efi -o grubia32.efi ...)
  • Trial & error showed that grubia32.efi created in ubuntu 16.04 (apt-get install grub-efi-ia32-bin) did load acronis true image, but keyboard and mouse remained disabled
  • Multiple sources (like this blog: My Solution How To Boot A Linux Live System) describe that grubia32.efi contained inside zip archive ISO-2-USB EFI-Booter for Mac 0.01 beta works fine
  • For some reason unknown to me this older version of grub does not block keyboard and mouse;
    but it’s command initrd does not load multiple files
  • Copy dat11.dat and dat12.dat into one file dat1112.dat:
    cp dat11.dat dat1112.dat
    cat dat12.dat >> dat1112.dat

    and in grub/grub.cfg replace
    initrd /acronis/dat11.dat /acronis/dat12.dat
    by
    initrd /acronis/dat1112.dat
  • Copy the older version of grub (local copy) to EFI/boot/bootia32.efi
  • Now it boots as described in step 6. But be prepared to wait 3 minutes until the efi32 bios has loaded the 85 MB initial ramdisk of acronis trueimage 2016

Step 8: Try to boot from this hard disk on efi and on bios systems.

  • Without step 6 on one of my systems the builtin boot selector did not show my usb disk,
    but it did show an entry boot from efi file:

    • this allowed to navigate into /EFI/ubuntu on the usb disk
    • then shimx64.efi had to be selected

 

Running a dual boot system with OS X and Windows 10 an ExFAT data exchange partition is useful.

But from time to time (after some months in my case) theExFAT partition is gone:

  • OS X and Windows 10 list a raw partition instead of an ExFAT partition

Repair in OS X using the terminal:

  • sudo su
  • diskutil list
    (this should display a “Microsoft Basic Data” partition with the size of the disappeared ExFAT partition;
    the name column is empty; the identifier like disk0s3 has to be used in the next step)
  • fsck_exfat disk0s3
    (replace disk0s3 by your ExFAT partition)
  • when asked by fsck_exfat to repair your “Main boot region” answer “y”
  • diskutil mount disk0s3
    (replace disk0s3 by your ExFAT partition)
  • now yourExFAT partition should be back in Finder
    (which can be verified on the console with: ls /Volumes)

Details taken from this discussion.

Repair attemps on Windows could use chkdsk d: /f
(replace d: with the windows drive letter of your disappeared ExFAT partition;
use <windows key> + x to open an administrative console.)

Hardware: multiple avm fritz!box router, isdn telephones attached to these routers

  • normal internet connection: router can register sip numbers
    – the router gets a public ip on the wan side
  • mobile data connection: router fails to register most voip providers
    – mobile data modems/routers get a private ip (at least in my setup)
    – private ip means listening for incoming traffic is not possible
    – smartphone clients (e.g. bria) seem to be able to use push notifications to enable incoming calls (with optional codec g729 good voice quality and rather low data rates)
  • openvpn connection through mobile data to a server with public ip:
    – router can register sip numbers and outgoing calls work
    (until the firewall of the mobile provider gets active if voip by mobile data is forbidden)
    – incoming calls are still a problem; maybe the vpn tunnel responds to slowly.
  • reliable incoming calls by usb gsm modem with voice sim card attached to the router
    – fritzbox 7270v2 and higher of voice connections by voice capable usb gsm data sticks

Openvpn tunnel server side (official howto):

  • apt-get install openvpn
  • enable ipv4 forwarding
  • set iptable rules (reference1, reference2)
    – nat routing from tunnel interface to wan interface
    – disable direct forwarding from wan to tunnel
    iptables -A FORWARD -s 10.8.0.0/24 -j ACCEPT
    iptables -A FORWARD -m state –state RELATED,ESTABLISHED -j ACCEPT
    iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE
    iptables -A FORWARD -j REJECT
  • create keys following this tutorial (and it’s openvpn sample config file) using easy-rsa
    (which is part of the debian openvpn package); no passwords except for ca signing key
    – inside easy-rsa directory, edit vars file and run
    source vars; ./clean-all; ./build-ca; ./build-key-server your_server_name;
    ./build-key-pass client_name1;
    (openssl rsa -in client_name1.key -des3 -out client_name11.3des.key for mobile clients)
    ./build-dh
    openvpn –-genkey –-secret keys/ta.key

Openvpn tunnel client side:

  • use freetz to modify the router firmware and add openvpn (plus dropbear server)
    (might be tricky with older routers with only 4MB flash; needs squashfs 128kb block size
    and many of the removal patches; be prepared to recover your router with rukerneltool)
  • detailed howto here; (64 bit linux needs sudo apt-get -y install libc6-dev-i386 lib32ncurses5-dev gcc-multilib lib32stdc++6);
  • to fetch the current stable version:
    svn co http://svn.freetz.org/branches/freetz-stable-2.0 freetz-2.0
  • make menuconfig; make
  • the freetz web interface allows to configure openvpn udp tunnel client
    – use certificates and tls auth (ta.key goes into “static key” field)
    – redirect client traffic
  • additional note: echo “clear_id 87” > /proc/tffs allows to remove the message
    unsupported changes (reference); freetz has an option to do this from web interface.

Receiving calls with usb data modem in voice mode (e.g. with huawei e1552; at command configuration): ippf forum thread

  • voice capability might have to be unlocked with dc_unlocker
    (tool shows status of usb modem (free) and is able to unlock voice (paid))
  • testing voice capability with huawei mobile partner software on windows might require editing of config\PluginsConfig.xml to enable (voice) call menu entry (reference)
  • huawei sticks with newer firmware (“hilink”) act as router and not as modem; this seems to block voice modem usage (reference)
  • alternative: raspberry pi with asterisk (RasPBX) with chan_dongle (openvpn tunnel described here)
  • to make the option gsm voice telephony appear on a fritzbox router, attach the usb data stick, open menu Internet / Mobile Connection, enter SIM PIN number (if set) and press apply; press refresh until the data stick is booked into a mobile network; then press again apply; if the data stick has (enabled) voice capability and is supported by the fritzbox router, then the menu entry gsm voice calls should appear
  • fritzbox routers with vendor branded firmware might disable gsm voice (which requires installation of generic avm firmware with rukerneltool)
  • if the firmware of the fritzbox router is modified by freetz the program minicom can be installed to display sms messages (reference)
    – ls -l /var/gsm/ displays which ttyUSBN (N=0,1,2…) is the data port (huawei e1552: USB0)
    – stty -F /dev/ttyUSB0 displays the baudrate setting of this port (which is accessed by umtsd; in my setup baudrate 9600 is used)
    – minicom -o -b 9600 -D /dev/ttyUSB0 allows to send at-commands (ctrl+a e enables local echo)
    – at+cmgl=”all” displays all SMS stored on the SIM card
    – at+cmgr=4 displays SMS with index 4 (described in detail here)
    – the commands might fail and have to be repeated if umtsd gets active at the same time