1 Install Ganeti Manager (ganetimgr)

Note: This is partly based on the instructions at https://ganetimgr.readthedocs.org/en/stable/install.html with some modifications.

1.1 Run this as root

Most of the commands below will require root access, so we'll start by going to the root user (use su or sudo)

$ sudo -s
Password:

#

1.2 Install required packages

# apt-get install git nginx mysql-server gunicorn python-gevent \
    redis-server beanstalkd

When prompted for a password for the mysql root user, pick one and remember it. Continue to install further packages:

# apt-get install python-mysqldb python-django python-redis \
    python-django-south python-django-registration python-paramiko \
    python-simplejson python-daemon python-setproctitle python-pycurl \
    python-recaptcha python-ipaddr python-bs4 python-requests \
    python-markdown
# apt-get install python-memcache memcached

1.3 Enable beanstalkd

Edit /etc/default/beanstalkd and uncomment the following line:

START=yes

Save and exit the file, then run

# service beanstalkd start

1.4 Create database

# mysql -u root -p
Enter password: <mysql root passwork picked earlier here>

mysql> CREATE DATABASE ganetimgr CHARACTER SET utf8;
mysql> CREATE USER 'ganetimgr'@'localhost' IDENTIFIED BY '<PICK_A_PASSWORD_FOR_GANETIMGR_DB_USER>';
mysql> GRANT ALL PRIVILEGES ON ganetimgr.* TO 'ganetimgr';
mysql> flush privileges;
mysql> exit

Test that the database created works

# mysql -u ganetimgr -p ganetimgr
Enter password: <password_for_ganetimgr_db_user>

mysql> exit

1.5 Check out the current version of Ganetimgr

# mkdir /srv/www
# mkdir /var/log/ganetimgr
# cd /srv/www
# git clone https://code.grnet.gr/git/ganetimgr
# cd ganetimgr
# pip install -r requirements.txt

(Note: at the time of version the 'stable' branch of ganetimgr did not include the requirements.txt file, so this is using the 'master' branch)

1.5.1 Copy the default settings for ganetimgr end edit them

# cd ganetimgr
# cp settings.py.dist settings.py
# cp urls.py.dist urls.py
# editor settings.py

The following changes will need to be made

Find the section:

ADMINS = (
    # ('Your Name', 'your_email@domain.com'),
)

Change the line above, adding and entry for the main admin user, for example:

('admin','your@email.address'),

Next, find the line:

'ENGINE' : 'django.db.backends.',

And change it to:

'ENGINE' : 'django.db.backends.mysql'

This tells the Django application framework that we'll be using MySQL as a backend.

Next, set the NAME, USER and PASSWORD to those of the database you created earlier:

'NAME' : 'ganetimgr',            # Or path to database file if using sqlite3.
'USER' : 'ganetimgr                       # Not used with sqlite3.
'PASSWORD' : '<PASSWORD_PICKED_EARLIER>', # Not used with sqlite3.

Set TIME_ZONE to your location, for instance:

TIME_ZONE = 'Africa/Nairobi'

Next, find the section:

SECRET_KEY = <CHANGE_ME>

And change it to a random string, for example:

SECRET_KEY = 'wlj84oaliehgzi48ol9qo3ijhesrbykjdzn,.38h,u4gbrg'

Please, don't copy the above key! Generate your own!

Next, find SERVER_EMAIL and DEFAULT_FROM_EMAIL and set them accordingly, for example:

SERVER_EMAIL = "no-reply@ganetimgrX.virt.nsrc.org"
DEFAULT_FROM_EMAIL = "no-reply@ganetimgrX.virt.nsrc.org"

Comment out COLLECTD_URL:

#COLLECTD_URL = "http://stats.example.com"

Find the section concerning the websockets console and change it as follows:

WEBSOCK_VNC_ENABLED = True
NOVNC_PROXY = "ganetimgrX.virt.nsrc.org:8888"
NOVNC_USE_TLS = False

(since we have no valid certificate)

Optionally, find the BRANDING section and update it with the information for your organization.

You could update the file /srv/www/ganetimgr/static/branding/logo.png if you want to change the default organization logo in the Ganeti Manager interface.

1.6 Populate the database

Next it's time to get the database populated and create a user.

# cd /srv/www/ganetimgr
# python manage.py syncdb --noinput

If everything goes well some output about tables being created will scroll past, and end with:

Not synced (use migrations):
...
(use ./manage.py migrate to migrate these)

This is normal - you now need to run:

# python manage.py migrate

Enter "yes" if asked to delete stale content.

Again, some output will flash by. It should end with:

...
 - Loading initial data for oauth2_provider.
Installed 0 object(s) from 0 fixture(s)

Now, let's create a superuser:

# python manage.py createsuperuser

Here, we'll create an admin user, so at the question:

Username (leave blank to use 'root'):

Enter admin

Next, you'll be prompted for an email address. Here, you can enter your email address, although for testing it might be better to start by using the mail address of the local user, root@ganetimgrX.virt.nsrc.org.

Finally, you'll be asked to provide a password. Don't use something too easy to guess when deploying this in production!

If all goes well, you'll see:

Superuser created successfully.

1.7 Create the admin interface page

The next command creates the pages that will be displayed in the administrative interface:

# cd /srv/www/ganetimgr
# python manage.py collectstatic

You will be prompted:

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel:

Answer yes and press <enter>.

1.8 Start the watcher process

This is responsible for tracking the state of jobs (tasks)

# ./watcher.py

Note: we'll have to remember to have watcher.py automatically be started a boot time.

1.9 Create a configuration for gunicorn

gunicorn is a python web server, which will run the Ganeti Manager application.

Create the configuration by editing the file /etc/gunicorn.d/ganetimgr, and copy paste the lines below:

CONFIG = {
    'mode': 'django',
    'working_dir': '/srv/www/ganetimgr',
    'user': 'www-data',
    'group': 'www-data',
    'args': (
        '--bind=127.0.0.1:8088',
        '--workers=2',
        '--worker-class=egg:gunicorn#gevent',
        '--timeout=30',
        '--log-file=/var/log/ganetimgr/ganetimgr.log',
    ),
}

1.10 Restart gunicorn

# service gunicorn restart

1.11 Create an Nginx configuration

Nginx, the web server, runs as the "front-end" for gunicorn.

Here's a minimal configuration get the web interface up and running:

Edit the file /etc/nginx/sites-enabled/default, erasing ALL THE LINES IN THE FILE, and copy the following lines into it:

server {
        server_name ganetimgrX.virt.nsrc.org;

        location / {
                proxy_pass http://127.0.0.1:8088;
        }

        location /static {
                root   /srv/www/ganetimgr;
        }
}

Test the configuration:

# service nginx configtest

And if all is well, proceed to restart nginx:

# service nginx restart

1.12 VNC proxy

We can now install the VNC Auth Proxy that will allow for browser-based VNC consoles!

# apt-get install python-dev
# pip install six --upgrade
# pip install service_identity
# pip install VNCAuthProxy

And start it:

# twistd --pidfile=/tmp/proxy.pid -n vncap -c tcp:8888:interface=0.0.0.0 &

(Note: this needs to be done at server startup too)

2 Create RAPI user

We now need to create a user on the Ganeti cluster nodes, that will allow Ganeti Manager to control the cluster.

Pick a random string (for example, generate it with):

head -c18 /dev/urandom | base64

... and run it through the following command:

echo -n 'ganetimgr_api_user:Ganeti Remote API:random_string' | openssl md5

Note that random string should probably not contain ':'.

Here's a sample output, which is a hashed password:

b20990f88c544cea982645e99b08d8c8

The final authentication / authorization line will look like this:

ganetimgr_api_user {HA1}b20990f88c544cea982645e99b08d8c8 write

... where write indicates that this user ganetimgr_api_user has write access to the cluster (i.e: can modify/create/delete instances!)

Now, on the master node (NOT ON THE GANETIMGR MACHINE!), create the following file (as root):

# editor /var/lib/ganeti/rapi/users

... and copy the authentication/authorization line, with the correct hashed password.

Once you've done this, distribute this file to the other nodes in the cluster with the following command:

# gnt-cluster copyfile /var/lib/ganeti/rapi/users

Note: In Debian Jessie onwards, remote access for RAPI is disabled by default. To check for this, look in /etc/default/ganeti. If the flags for the RAPI daemon include -b 127.0.0.1 then remove that part and then restart ganeti.

That's it! We are now ready to test the front-end!

3 Navigate to the ganetimgr web interface.

Open http://ganetimgrX.virt.nsrc.org/ in your browser.

You should be presented with the login screen.

Login with the admin user and password you created earlier.

Now, click on Admin in the bottom of the left menu. If your browser window isn't wide enough, you'll only see a small "Gears" icon. Click on that.

Click on Add next to Clusters under the Ganeti section.

For hostname, enter the hostname (it must be in the DNS!) of the cluster, i.e.: gntX.virt.nsrc.org.

For the "Slug", you can put "nsrc" for instance. It's best to avoid '-' in the cluster name, it seems.

For username, use the ganetimgr_api_user specified earlier.

For password, you use the randomly generated password (NOT the hashed version!) created earlier.

Leave default disk template to plain.

Now it's time to navigate back to the mainpage:

`http://ganetimgrX.virt.nsrc.org`

4 Additional notes

4.1 What's missing ?

4.2 Why we had to install VNCAuthProxy from a pip package:

The VNCAuthProxy binary package is missing the vncap module (not being built):

4.3 Issues with VNC consoles and using /etc/hosts on ganetimgr

We have the cluster node names and the ganetimgr host names in /etc/hosts as vncauthproxy uses gevent and greenlets for the event loop and connection handling / forwarding. gevent (<1.0beta) uses libevent2 and its async DNS resolver, and libevent2-dns doesn't read /etc/hosts.