Monday, January 18, 2021

PowerBank as a UPS, for RaspberryPi

We could make use of a Standard Powerbank, as a cost effective UPS to powerup RaspberryPi. This will be particularly useful, if Pi is serving critical functions for your network 24x7, and you cannot afford any downtime.

Make a note of below important points;

a. Use a PowerBank that meets the Voltage/Current specifications of Pi.

b. The PowerBank should allow PassThrough Charging

(So you could charge your PowerBank and Pi at the same time)

c. The PowerBank should allow Advanced Current Shunt

(Even if you leave the PowerBank, with AC Power the feature will protect from Overcharging).

Moving around the Pi, While its Up and Running !

image

Sunday, January 17, 2021

Network Bandwidth Monitoring/Analysis Using RaspberryPi II

Pi could be used to Monitor the Bandwidth Usage of entire Home Network. Time series data could be visualized in different types of Charts. Historic data visualization supported (e.g Monthly Bandwidth Usage). Each Network/IP/Machine level statistics could be analyzed and viewed.

The below configuration explain about setting up such an environment. The only prerequisite would be, Make all your traffic go through Pi. i.e Change your Default Gateway from Router to Pi. This could be easily done through, updating the Router LAN settings, and putting Raspberry Pi IP-Address, in the Default Gateway box.

Now update Raspberry Pi’s Default Gateway as Routers IP, and enable IP forwarding. This will make all traffic destined for Internet will be automatically routed to the Router from Pi. Finally setup ‘ntopng’, to churn the network packets, for analysis. 

Detailed instructions are given here.

image

image

Monday, January 11, 2021

Raspberry Pi, as Network Security Appliance/AD-Blocker

In this article, we will explain how to use RaspberryPi, as a Network Security Appliance for private network, which could be used to block all ADS, and secure the network by whitelisting/blacklisting DNS names.

It could effectively improve the overall network performance as all the AD requests going out of your private network will get blocked, and never goes out of your private network. It extends the protection to all computers in your network including mobile/other devices connects to your home network.


The below configuration shows, Pi-Hole deployed in Docker containers hosted on a RaspberryPi, and managed through Portainer (A Container orchestration platform, again running inside a Docker Container)

image

 

You could find the instructions on setting up the same here.

The only change we’d, is using a new version of Pi-Hole image (pihole/pihole:v5.0-armhf)

A single day operation statistics is given below: In a single day operation, Pi-Hole has blocked 31.7% of total requests, which are of type Ads.

Since the CPU/Memory (~4.5MB) footprints are pretty low, Pi runs smooth 24x7 and meets the network performance.

image

Bypass Qemu Network Stack with Virtio-Vhost, for Near Native Performance

For virtualized environments, primary performance bottleneck could happen with below resources:

a. CPU

b. Storage

c. Network

For CPU, direct passthrough option would give a performance near to bare metal. For Storage and Network, deploying VIRTIO drivers (paravirtualization) could provide a descent performance as well. These concepts have been explained here.

With network, one more performance tuning is possible, to completely bypass the Hypervisor (Qemu) network stack, and let the GUEST directly interact with HOST’s network stack. This may give almost similar network performance for guest applications as with application running on the host.

Implementing this requires, enabling ‘vhost_net’ driver on the host, and update the VM Settings to use this driver, which is detailed here.

In an optimal setting, you will also enable the same number of IO-Queues, as with the total number of CPU Cores allotted to the Virtual Machine.

Reference

image

 

 

Wednesday, January 15, 2020

Automate IE Proxy Setting Changes on Network Adapter Changes–Windows Powershell

If you need to turn on IE Proxy settings, say if the LAN cable is plugged in and reverse the setting, once Wireless got connected, you could rely on Powershell script to automate the same in Windows environment.

This is a two step process:

1. Create the powershell script, which detects the Network Adapter Change, and update IE Settings accordingly

Sample Script here:

2. Create a new Task in “Task Scheduler”, which will be triggered on Network Changes (NetworkProfile->Event ID:10000) and System Startup

Set the above script as the Action, for the task

Eg

Program: powershell.exe

Arguments: -windowstyle hidden "<PathTo>\SwitchProxyOnNetworkChange.ps1":

Note: This will run the script in hidden mode, so that user won’t notice the powershell window – for the best experience


From now on whenever your Network get changed, the script will be triggered automatically to set IE Proxy settings accordingly.

Wednesday, November 13, 2019

Configure Console Only Login On Ubuntu 16.04+ Desktop

Occasionally we may want to login to our Ubuntu Desktop as console only mode (for eg: running our Ubuntu desktop as a bare metal hypervisor, only to launch Virtual Machines which will be connected remotely).

This will help to run the desktop with bare minimum memory (120MB RAM on Ubuntu 16.04) and CPU usage.

By default Ubuntu Desktop 16.04+ does not provide (which has Systemd than the legacy init system) this option, and you’ve to configure the same. Once configured you will get a new Grub menu entry on startup, to login as Console Mode.

  1. Copy a sample menu entry from (/boot/grub/grub.cfg) to (/etc/grub.d/40_custom)

    Sample:

    menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-0e341ce4-5a12-4a6d-a997-5336d1670353' {

           recordfail

           load_video

           gfxmode $linux_gfx_mode

           insmod gzio

           if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi

           insmod part_msdos

           insmod ext2

           set root='hd2,msdos5'

           if [ x$feature_platform_search_hint = xy ]; then

             search --no-floppy --fs-uuid --set=root --hint-bios=hd2,msdos5 --hint-efi=hd2,msdos5 --hint-baremetal=ahci2,msdos5  0e341ce4-5a12-4a6d-a997-5336d1670353

           else

             search --no-floppy --fs-uuid --set=root 0e341ce4-5a12-4a6d-a997-5336d1670353

           fi

            linux /boot/vmlinuz-4.15.0-54-generic root=UUID=0e341ce4-5a12-4a6d-a997-5336d1670353 ro  ipv6.disable=1  scsi_mod.use_blk_mq=1

           initrd /boot/initrd.img-4.15.0-54-generic

    }

     

  2. Update the copied entry as per below (highlighted text). The crux is ‘systemd.unit’ property.

    Sample:

    menuentry 'Ubuntu-TextMode' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-0e341ce4-5a12-4a6d-a997-5336d1670353' {

           recordfail

           #load_video

           #gfxmode $linux_gfx_mode

           insmod gzio

           if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi

           insmod part_msdos

           insmod ext2

           set root='hd2,msdos5'

           if [ x$feature_platform_search_hint = xy ]; then

             search --no-floppy --fs-uuid --set=root --hint-bios=hd2,msdos5 --hint-efi=hd2,msdos5 --hint-baremetal=ahci2,msdos5  0e341ce4-5a12-4a6d-a997-5336d1670353

           else

             search --no-floppy --fs-uuid --set=root 0e341ce4-5a12-4a6d-a997-5336d1670353

           fi

            linux /vmlinuz root=UUID=0e341ce4-5a12-4a6d-a997-5336d1670353 systemd.unit=multi-user.target ro  ipv6.disable=1  scsi_mod.use_blk_mq=1

           initrd /initrd.img

    }


  3. Update grub using (Sudo update-grub2)

  4. Reboot

  5. Select the menu entry “Ubuntu-TextMode’

Thursday, October 31, 2019

Enable High Performance, Block Multi Queue-IO Scheduler under Ubuntu 16.04 Desktop

Compared to Single Queue-IO Schedulers (eg. cfq, deadline etc), we can gain a high IO performance improvment through MultiQueue IO Schedulers (eg bfq, kyber). These are the new IO Schedulers built in to Linux Kernal 4.12+, but disabled until 5.0. It uses multiple IO Queue (leveraging CPU cores) to provide a high responsive system. These are specificaly designed for Desktop OS. Its architecture has been detailed here and here.

Enabling the same On Ubuntu 16.04 has been mentiond below:

1. Add both mq-deadline, bfq and kyber-iosched, to /etc/modules, to load the modules on startup

2. Append, scsi_mod.use_blk_mq=1 to the GRUB_CMDLINE_LINUX parameter in /etc/default/grub file

3. Create /etc/udev/rules.d/60-scheduler.rules, to assign bfq and kyber to block devices

eg:

# set bfq scheduler for non-rotating disks | SDD
ACTION=="add|change", KERNEL=="sd[a-z]", TEST!="queue/rotational", ATTR{queue/scheduler}="bfq"
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="bfq"

# set kyber scheduler for rotating disks | HDD
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", ATTR{queue/scheduler}="kyber"

4. Update grub using “sudo update-grub2” and reboot

5. Now your SSD would be using bfq scheduler, and HDD using kyber scheduler. Enjoy your high responsive system !


References:

https://www.omgubuntu.co.uk/2017/07/linux-kernel-4-12-released-bfq

https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mechanism_(blk-mq)

https://lwn.net/Articles/767987/

https://lwn.net/Articles/784267/

https://wiki.ubuntu.com/Kernel/Reference/IOSchedulers

https://www.stephenrlang.com/2018/01/io-scheduler-tuning/

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.2_release_notes/storage

https://www.hecticgeek.com/2016/09/supercharge-ubuntu-16-04-lts-xanmod-kernel/

https://www.cnx-software.com/2019/08/14/bfq-budget-fair-queuing-i-o-scheduler-improves-linux-systems-responsiveness-video/

https://unix.stackexchange.com/questions/375600/how-to-enable-and-use-the-bfq-scheduler

https://wiki.debian.org/SSDOptimization#Low-Latency_IO-Scheduler