Posts

Bootstrapping Alpine Linux QCow2 image

IntroductionAlpine Linux is a minimal distro with package manager (APK) that is based on busybox and musl library. Like the CirrOS, it's very lightweight, but unlike it, it's full featured. In case you don't know me, my Linux distro of choice is Fedora/CentOS, in this post I'm going to bootstrap a QCow2 cloud image of Alpine Linux on my distro of choice.


Using docker to bootstrap a working chroot Type mkdir alpine35-root docker run --rm -ti -v $PWD/alpine35-root:/data alpine:3.5 apk --arch x86_64 -X http://nl.alpinelinux.org/alpine/v3.5/main/ -U --allow-untrusted --root /data --initdb add alpine-base 
and you should get a line like this
OK: 6 MiB in 16 packages
so now we have a working alpine chroot in the directory alpine35-root
Creating Bootable QCoW2 Image Because I don't want to format my hard disk by mistake and because I know Murphy's law, I'll take those 6MB as tarball and continue on a VM.
dd if=/dev/zero of=alpine.raw bs=1M seek=511 count=1 dd if=…

Let systemd manage your running of unprivileged scripts

Image
Instructions
let's save a text file named "test.sh" having the following content


#! /bin/bash for i in `seq 50` do    echo $i    sleep 1 done

it just print number from 1 to 50 second by second (you can change 50 to any number) now as regular user type

chmod +x ./test.sh systemd-run --user --unit=my-test ./test.sh 
the above command will run the script as a user service called my-test
at any time you can trace it with systemctl and see the logs using journalctl like this

journalctl -ln 100 -f --user-unit=my-test systemctl --user status my-test
you can abort it using

systemctl --user stop my-test


Use cases
Let's assume you have a web interface that trigger something and you want to trace it later just make your unprivileged web application (written in php/python and running as regular non-root user) called "systemd-run --user" and query the status and follow the logs using systemctl and journactl
for example if you have a web interface that triggers building…

playing with "unc" the unprivileged user containers

Image
Introduction
If you are familiar with docker then you are going to love this little hack. UNC is a tool to demonstrate how to launch containers without being root.

Getting the code and compiling it git clone https://github.com/LK4D4/unc.git
cd unc
mkdir -p gopath/src/
ln -s ../.. gopath/src/unc
export GOPATH=$PWD/gopath
cd gopath/src/unc
go get
go build
cd unet
go build
cd ..
sudo cp unc unet/unet /usr/local/bin/
sudo chmod u+s /usr/local/bin/unet

What is SetUID for? "unet" is a tool used "unc" that creates veth pairs and assign one of them to container. Root user (via setuid) is needed to be able to assign a network interface for the user container.


Creating some minimal root filesystems sudo dnf install busybox # as regular unprivileged user type mkdir -p roots/busybox1/{bin,sbin,proc,dev,etc} cp /sbin/busybox roots/busybox1/sbin/ echo "root:x:0:0:root:/root:/bin/bash" > roots/busybox1/etc/passwd echo "root:x:0:" > roots/busybox1/etc/gr…

Be aware! Docker is a trap.

Image
Yes, I'm  claiming that Docker is just a hyped vendor lock-in. Don't get me wrong, Linux containers, micro-services, DevOPs,... are all very cool and they are cool for a reason. On the other hand docker is just an overrated vendor lock-in.

Linux containers is a new term for old technologies like namespace and control groups, used in production maybe for more than a decade.

LXC is one way to access those kernel feature in a boring way. OpenVZ and the newly hyped LXD is virtualization hypervisor based on Linux containers. SystemD do use containers for all of spawned services and can be used to spawn containers (using nspawn). In other words Docker did not invent containers.

I was warned about that more than a year ago but then it was not that obvious.

What's wrong with docker? So many things! At every single level! But the real problem that those problems are intentional and they are not going to be fixed.

What's the alternatives? CoreOS-backed appc and Rocket (aka. rkt

Multi-host docker cluster using OVS/VxLAN on CentOS 7

Image
Introduction

The community version of RHEL 7; Community Enterprise OS (CentOS) 7 comes with docker 1.8.x. Which does not support multi-host networking as it only support linux bridges. In other words a container on Host A can't talk to a container on Host B. If you have a cluster of 5 nodes then you can't utilize them all.

In this article I'm going to show you how to setup a multi-host docker cluster where containers on different hosts can talk to each other without NATing (Network Address Translation). I'm going to use OpenVSwitch using VxLAN tunnels (or GRE tunnels) both are Linux kernel technologies that are true and tested (they are used in production for years).

The Open Virtual Switch (OVS) is an L2 switch just like physical switch, it has ports where you plug ethernet plugs and it works at MAC address / ARP level to connect them.

Docker 1.9.x shipped in Fedora 23 do support this feature via something called overlay network but even in that case I still prefer O…

Liquid CPU Dream

Liquid CPU Dream I was holding a small cylindrical container with flat base against the light and it was just more than half-full of CPU liquid, and it was a thin liquid just like water. The flat base contains some dark squares I don't know if they were some logo or connectors.

I remember that Intel was mentioned but I did not see its logo.
I also remember that Egypt was mentioned but I can't remember why and what was the contest.

Boosting performance and concurrency in Python

Image
Python provides a base socket server that got no concurrency support by default, which can be used to create any server including HTTPServer or WSGI applications servers like the wsgiref. You can plugin concurrency support using ThreadingMixIn or ForkingMixIn  this would allow our pure-python server to handle multiple requests by forking another process or starting a new thread while the main thread in the main process keeps accepting requests.
In this post I'm going to introduce my own PooledProcessMixIn and its features over other solutions.
The concept of Pool I've taken a look at the code of those Mix-Ins and found serious performance issue with it as they allocate a new process or new thread each time a request comes to the server. Beside delaying the response waiting for the allocation, it's an open-ended approach (no re-using of those threads or processes). The pool approach is to allocate a number of threads or fork a number of processes at server initialization t…