Note: You can find all the code at every step of the process in my GitHub repository https://github.com/martinberoiz/daemon.

A daemon is a basically a background process; a program that runs continuously, most of the time waiting for external input to do specific work.

In this article I explain how to write and deploy a Linux systemd service using Python.

Linux adopted systemd as its “System and Service Manager”, replacing the previous init system.

Systemd is tasked with starting and managing several “services”, ie., background processes that provide system functionality on demand.

It is customary to add the letter d (for daemon) to programs that will run as services.

Run a Python script as a background process

So, starting from the basic, Python scripts are files that are meant to run from the command line.

For example

import time
DELAY = 10
while True:
    time.sleep(DELAY)
    print("Hello from a script!"
          " It's been {} seconds since last time.".format(DELAY))

If we run it,

$ python myscript.py

It will run forever and print a statement every 10 s.
To stop it, simply enter ctrl+c (^C) on the console and it will kill the running process.

To make the process run in the background, we can append & to the command

$ python myscript.py &

The use of the end ampersand will push myscript to the background and return the prompt for the next command. This way we have created a background process.

There are two problems with this approach.

The first one is that the process is terminated as soon as the user logs off.

The second is that there is no redirection of stdout and stderr (we could improve the example with pipes redirecting these two, but it’s beyond the scope of this article).

To remedy these, we can use the command nohup, which is a POSIX command that ignores terminal’s termination signal once we log out, so that the process can keep running.

$ nohup python myscript.py &

This last command would be enough to create a daemon, and we could finish our tutorial at this point.

Note: To terminate background processes initiated by nohup or appending ampersand, you can find the process PID with `ps -e` and terminate with `kill`

$ ps -e | grep python
...
17490 ttys000 0:00.03 python myscript.py
...
$ kill 17490

Systemd Services

We can also make use of systemd to launch the process at start-up and restart it whenever it crashes to make sure we can always have it running. Systemd also provides a unique interface to start, stop and restart services. This last one (restart) is useful if we changed any configuration file and we want to restart the service with the new one.

The process to create a systemd service is fairly simple. All we need to do, is create a .service file in /etc/systemd/system/ directory with the name of the service.

For our example we’ll use the file name mydaemond.service. Like we said before, it is customary to append a letter d after the service name.

Below is a minimal example of a service file.
We will copy the following to the file /etc/systemd/system/mydaemond.service
(Like we said before, it is customary to append a letter d after the service name.)

[Unit]
Description=My Awesome Service
After=network.target

[Service]
Type=simple
ExecStart=/home/myuser/myscript.py
Restart=on-failure

Once the file is created we can start our service invoking `systemctl`:

$ systemctl start mydaemond

And stop it:

$ systemctl stop mydaemond

---

Simplifying the installation

Installing the python module with pip

It may be convenient to provide our users with an easy way to install both our python script and to register our service.

The standard way to install packages in Python is using pip, and for that we need a setup.py file.

A minimal setup.py to install a Python module looks like this:

from setuptools import setup
setup(name='mymodule',
      version='0.1',
      description='My Awesome Python Module',
      py_modules=['myscript', ],
     )

The difference between a Python module and a Python script is that the former is intended to be imported inside other modules or scripts, while the latter is intended to be run completely from the command line.

Modules don’t usually have executable instructions in the global scope, only function definitions (and occasional variable declarations).

Scripts, on the other hand, have mostly executable instructions in the global scope.

We have written a script myscript.py so far (not a module), so this setup.py file would not help us.

We can reach a compromise between a module and a script, however, by encapsulating functionality in function definitions and only call them on the condition that it’s run from the command line.

Let’s modify our myscript.py like this:

import time
DELAY = 10
def main():
    while True:
    time.sleep(DELAY)
    print("Hello from a script!"
          " It's been {} seconds since last time.".format(DELAY))

If we run the script as it is, nothing will be done because the main loop is inside a function that is never called. On the upside, importing it will not do any harm, since the loop is not going to be executed.

Most importantly, our script can already be installed with pip:

$ pip install .
$ python
>>> import myscript
>>> myscript.main()
Hello from a script! It's been 10 seconds since last time.
Hello from a script! It's been 10 seconds since last time.
...

It will still do nothing if executed from the command line:

$ python myscript.py
$

Python modules that can be executed from command line

To fix our broken script, what we need is to add a condition at the end of the file that calls main() only if it’s being executed as a script.

How can we detect if our code has been called from the command line or imported from a module?

When we import our code, python fills the module .__name__ variable to the called module’s name

For example:

import numpy as np
print(np.__name__)

Will print the string numpy, which is the name of the imported module.

But when the module is not imported but rather called from the command line, Python will populate the __name__ variable with the string “__main__”.

So we can bifurcate the code in our script and execute main() only when it’s intended to be run as a script:

import time
DELAY = 10
def main():
    while True:
    time.sleep(DELAY)
    print("Hello from a script!"
          " It's been {} seconds since last time.".format(DELAY))

if __name__ == "__main__":
    main()

This time calling myscript.py from the command line will run forever as it is intended, but importing it as a module will not hang our computer in an infinite loop.

With this modification, our script can be installed as a module with pip as before but can be also run from the command line.

$ python myscript.py
Hello from a script! It's been 10 seconds since last time.
Hello from a script! It's been 10 seconds since last time.
...

Using pip to create an installable script

Using setuptools gives us an extra feature, which is, that now pip can create a small script from our module and place it in our $PATH. If it is installed in a virtual environment, it will be installed in the `bin/` directory, it it is installed globally, it will be installed in /usr/local/bin.

To create this script, we just have to modify our setup.py like this:

from setuptools import setup
setup(name='mymodule',
      version='0.1',
      description='My Awesome Python Module',
      py_modules=['myscript', ],
      entry_points={
          'console_scripts': [
              'mydaemon = myscript:main',
          ],
      },
     )

If we install this with pip, it will create two things.

First, it will install myscript.py as a module (so it can be imported as `import myscript`), this is of not much use for this example.

Secondly, it will create a mydaemon script, which basically calls main from myscript.

$ mydaemon
Hello from a script! It's been 10 seconds since last time.
Hello from a script! It's been 10 seconds since last time.
...

This mydaemon is the one we will use as the systemd service.

You can check that the previous functionality still works as expected.

Automating the installation of the systemd unit

We could ask our user to edit and copy himself the file mydaemond.service, but instead we will create a makefile, which is more familiar to most Linux users, and make the installation process easier.

all: mydaemon mydaemond.service
.PHONY: all mydaemon install uninstall clean

service_dir=/etc/systemd/system
awk_script='BEGIN {FS="="; OFS="="}{if ($$1=="ExecStart") {$$2=exec_path} if (substr($$1,1,1) != "\#") {print $$0}}'

mydaemon: myscript.py setup.py
    pip install .

mydaemond.service: myscript.py
# awk is needed to replace the absolute path of mydaemon executable in the .service file
    awk -v exec_path=$(shell which mydaemon) $(awk_script) mydaemond.service.template > mydaemond.service

install: $(service_dir) $(conf_dir) schedulerd.service scheduler.conf.yml
    cp mydaemond.service $(service_dir)

uninstall:
    -systemctl stop mydaemond
    -rm -r $(service_dir)/mydaemond.service

clean:
    -rm mydaemond.service

The makefile makes use of a .service template file, that we provide below.

# mydaemond.service.template
[Unit]
Description=My Awesome Service
After=network.target

[Service]
Type=simple
ExecStart=/home/myuser/myscript.py
Restart=on-failure

Now to install the service, we just need to make and make install.

$ make
$ sudo make install

The installation requires root permissions because it will copy files over /etc/ which is usually owned by root.

The makefile will basically replace the full path of the mydaemon executable into the ExecStart line using awk and then copy the file over to /etc/.

Makefile commands clean and uninstall are also provided for convenience.

I recently bought a Raspberry Pi mini-computer.

Here I offer a small tutorial for those trying to use it as a simple web server.
This will assume you already built your raspberry pi box with Raspbian installed in the SD card.

I compiled a list of recipes from different websites and pages, in the hope that this will be useful to someone with a similar setup as mine (a macbook with OS X and a wifi router at home.)

You don’t need your pi to be connected to an external monitor and keyboard to make it work. If your pi came with the Raspbian OS then it’s prepared to be accessed through SSH out of the box.

Connect your pi to your router with an ethernet cable or set up your wifi connection (you need to enable wifi first on your pi.) Then you need to find out which internal IP was the raspberry assigned to by your router, and then ssh with user pi and password raspberry. To find the IP of your pi you can check on your router settings, the DHCP Client List. Your pi should be named raspberrypi or similarly.

In principle it’s not guaranteed that the IP will be the same every time the device connects, but in my experience with my router, the IP assigned to each device is always the same.

The first time you log in you will be greeted with a setup menu. I suggest you follow the instructions on this site to get it up and running. Most importantly, change the default password to log in!

Once you know your pi’s IP (in my case was 192.168.2.3, yours may differ) you can log in normally.

ssh pi@192.168.2.3
password: raspberry

Installing Apache 2

The following are steps to install an Apache server on your raspberry pi. Apache is a popular web server in the Linux side of computers. Another emerging contender on the open source side is Nginx (pronounced ‘Engine X’), but I have no experience with it. I should probably buy another pi and try this new server; after all, one of the advantages of the pi is the relatively inexpensive hardware!

Back to Apache, installing it is relatively easy, you just type

sudo apt-get install apache2 apache2-doc apache2-utils

on the terminal and that should be enough.

Test Apache with a simple html page

You can manage several sites on your pi with Apache, all available sites are stored in files under /etc/apache2/sites-available, of those you can enable or disable the ones you like with the commands a2ensite and a2dissite (a2 is for apache2 and the rest is enable/disable site.) These commands basically create and delete symlinks into /etc/apache2/sites-enabled.

Go to /etc/apache2/sites-available. We’ll make a test page to confirm apache is running well. First we copy the default site provided by apache to have a template to work with.

sudo cp default test

Edit the test file replacing instances of “/var/www” to “/home/pi/www”
and create a simple welcome index.html in /home/pi/www

Below is an example of a simple html. If emacs is your preferred command line editor, you will have to install it now, because Raspbian doesn’t include it by default.

sudo apt-get install emacs

then you can create index.html with emacs on /home/pi/www

cd
mkdir www
emacs www/index.html

and copy the following text in it.

<!DOCTYPE>
<html>
<body>
<h1>Welcome</h1>
<p>Website coming soon</p>
</body>
</html>

Then execute the following commands to enable the site (a2ensite), disable the old site (a2dissite) and reload the sites.

sudo a2ensite test
sudo a2dissite 000-default
sudo service apache2 reload

You should see the Welcome page you set up in /home/pi/www, when you visit the pi’s IP (type http://192.168.2.3 on your favorite web browser)

If you want to access your website from outside your internal network (given by your router) you need to direct the external HTTP requests on port 80 to your raspberry pi’s port 80.

If you’re connected through Wifi like I am, then you have a wireless router and you have to configure your router to do the redirection. Select incoming port 80 to map to your pi’s internal IP’s (192.168.2.3 in my case) port 80.

Once you do that, you can access your website through your IP (the one provided by your ISP) on the web browser. You may want to hire a Domain Name Service to assign your IP number to some memorable web address. Some websites offer this service for free, noip.com is one of them.

Please leave comments if you find typos or mistakes!

For those like me that are interested in Game Development as a hobby, one of the main challenges to build a 3D game is understanding the OpenGL framework.

The distinction between the different coordinates systems is mandatory if one doesn’t want to get ultimately lost in coordinates transformations. A world coordinate system is a system of coordinates with which the scene is best described. It is the system that arises naturally from the game constrains and scene. The eye coordinate system is the system OpenGL understands, with the z coordinate sticking out from the screen and x and y running along the screen edges.

Even further, we also need a camera in our scene, that will be the point of view from where we render the scene.

It is very useful to split an OpenGL program into things our virtual objects do in the scene and how the camera moves in the scene. To achieve that split of responsibilities, it is necessary to have a Camera class that encapsulates the state of the camera at all times. The camera should know how to return its view transformation matrix to modify the point of view from which the objects will be rendered. This camera or view transformation will be the matrix that transform the vertices from the world coordinates to the eye coordinates.

For the impatient: a direct link to the github repository containing the Camera Class implementation on Objective-C can be found on this link. It can be easily modified for C++.

This article will try to explain hot to set a view transformation for a Camera class.

One way to implement a camera is to set the position and orientation of the camera in world coordinates. With this one can work out the transformation matrix to be applied to the vertices on the scene. The following will assume that we have a vector that points where the camera is looking at (lookAt), an approximate vector for what is considered “up” in the camera (upVector), and the coordinates of the camera (pos) on the scene. All three vectors in world coordinates. The upVector doesn’t necessarily must be the true upwards vector. It only must have a non-zero component along the true “up” direction.

In what follows, {i, j, k} are the basis vectors of the world coordinate system and {i’, j’, k’} are the basis vectors of the eye coordinate system. In the eye coordinate system -the only one that OpenGL cares about- the eye is always pointing in the -z direction. From the pos and lookAt vectors, we can find what is the k’ direction of the camera in world coordinates.

–k’ = normalize(lookAt – pos)

k’ = normalize(pos – lookAt)

“Up” is in the y direction in the eye coordinate system. The upVector would be j’ if it weren’t for the fact that it’s not necessarily completely upwards. Nevertheless, the cross product between upVector and k’ will be i’, as long as upVector has a component in the true j’ direction.

i’ = normalize(upVector x k’).

Finally the cross product of k’ and i’ will give us the true j’.

j’ = k’ x i’

Now that we have the relative orientation of the camera {i’, j’, k’} with respect to world coordinates {i, j, k} we can figure out the transformation (it’s a rotation) between the two frames. This can be done easily noting that i’ = (i’.i) i + (i’.j) j + (i’.k) k etc. which can be written formally (it’s not a true matrix-vector multiplication)

where means write the components of i’ in the {i,j,k} base in that row. For any other vector (x’,y’,z’) the transformation is the same. It can be seen by writing

( since it is an orthonormal matrix).

R is the rotation that takes a vector from world coordinates to eye coordinates. Its transpose will take from eye to world coordinates. To put the camera in the world we did two operations. First we rotated the camera at the origin by matrix R^t and then we translated it by vector pos.

M_cam = T(pos).R^T

The matrix we have to apply to vertices to simulate the camera motion is the inverse of this M_cam

M_v = R . T(-pos)

Implementation in OpenGL|ES:

OpenGL|ES stores the matrices in a column-major format. (We can equivalently think that it stores the transpose of the matrix in a row-major format.) To construct the matrix R, we have to fill each row with the primed direction versors in the array R[16].

 for (int i = 0; i < 3; i++) {
   for (int j = 0; j < 3; j++) {
     viewMatrix[i*4 + j] = prime[j][i];
   }
   viewMatrix[i*4 + 3] = 0.;
 }
 for (int i = 12; i < 15; i++) viewMatrix[i] = 0.;
 viewMatrix[15] = 1.;

The translation as an array T[16] is the usual one and has the form:

But since T is mostly empty we can make the multiplication R.T without storing the array T. The rotation block will remain untouched and will be replaced by a linear combination of the rotation terms and the translation terms.

In code:

 viewMatrix[12] = - position[0] * viewMatrix[0] \
                  - position[1] * viewMatrix[4] \
                  - position[2] * viewMatrix[8];

 viewMatrix[13] = - position[0] * viewMatrix[1] \
                  - position[1] * viewMatrix[5] \
                  - position[2] * viewMatrix[9];

 viewMatrix[14] = - position[0] * viewMatrix[2] \
                  - position[1] * viewMatrix[6] \
                  - position[2] * viewMatrix[10];

The whole project (WIP) can be found on this link: https://github.com/martinberoiz/CameraClass

Note: I will update this post with more details.

I found recently that Apple’s Mail.app was behaving erratically: the notification for unread emails wouldn’t disappear, even after reading all emails; junk email was properly marked as such, but refused to go into the spam folder and the rules I had set for certain senders didn’t work.

It all started after I signed up for iCloud to synch all my email accounts. That should have been my first clue to what the problem was, but it took me some time to realize what was going on. It turns out that iCloud is not smart enough (yet?) to realize that some of the synched accounts were already in place locally on my machine. It would then duplicate my locally set accounts with those from iCloud.

The problem went away once I removed those duplicates causing the problem. I still haven’t decided wether I want all my email accounts in the cloud. I don’t need all of them in all of my devices (I don’t really check all my emails on my iPhone, for example.) So, I decided to keep my most used accounts in the cloud and keep the less used ones on the MacBook that I use most frequently.

If you have the same problem, you should check your “Internet Accounts” in System Preferences and remove those you don’t want them there.