msimonin@github.io

Vagrant-g5k goes parallel

Mon, 10 Oct 2016 06:30:00 +0000

Version 0.0.15 introduces the ability to boot VMs in parallel. Let’s have an example

Vagrantfile

The following Vagrantfile starts one VM per site.

SITES=['rennes', 'nancy', 'nantes', 'lille', 'luxembourg', 'grenoble', 'lyon']

Vagrant.configure(2) do |config|
    SITES.each do |site|
      config.vm.define "vm-#{site}" do |my|
        my.vm.box = "dummy"

        my.ssh.username = "root"
        my.ssh.password = ""

        my.vm.provider "g5k" do |g5k|
          g5k.project_id = "vagrant-g5k"
          g5k.site = "#{site}"
          g5k.gateway = "access.grid5000.fr"
          g5k.image = {
             "pool"     => "msimonin_rbds",
             "rbd"      => "bases/alpine_docker",
             "snapshot" => "parent",
             "id"       => "$USER",
             "conf"     => "$HOME/.ceph/config",
             "backing"  => "snapshot"
           }
          g5k.ports = ['2222-:22']
          g5k.oar = "virtual != 'none'"
        end #g5k
      end #vm
    end # each

end

Output

The output shows that the different boot sequences are interleaved.

It took 10s to have all the VMs running.

Bringing machine 'vm-rennes' up with 'g5k' provider...
Bringing machine 'vm-nancy' up with 'g5k' provider...
Bringing machine 'vm-nantes' up with 'g5k' provider...
Bringing machine 'vm-lille' up with 'g5k' provider...
Bringing machine 'vm-luxembourg' up with 'g5k' provider...
Bringing machine 'vm-grenoble' up with 'g5k' provider...
Bringing machine 'vm-lyon' up with 'g5k' provider...
==> vm-nantes: Launching the VM on nantes
==> vm-lyon: Launching the VM on lyon
==> vm-rennes: Launching the VM on rennes
==> vm-grenoble: Launching the VM on grenoble
==> vm-luxembourg: Launching the VM on luxembourg
==> vm-nancy: Launching the VM on nancy
==> vm-lille: Launching the VM on lille
==> vm-nantes: Waiting for the job to be running
==> vm-grenoble: Waiting for the job to be running
==> vm-luxembourg: Waiting for the job to be running
==> vm-nancy: Waiting for the job to be running
==> vm-lyon: Waiting for the job to be running
==> vm-rennes: Waiting for the job to be running
==> vm-nantes: Waiting for the job to be running
==> vm-lille: Waiting for the job to be running
==> vm-grenoble: Waiting for the job to be running
==> vm-nancy: Waiting for the job to be running
==> vm-luxembourg: Waiting for the job to be running
==> vm-rennes: Waiting for the job to be running
==> vm-lyon: Waiting for the job to be running
==> vm-nantes: booted @nantes on econome-8.nantes.grid5000.fr
==> vm-grenoble: booted @grenoble on genepi-33.grenoble.grid5000.fr
==> vm-nancy: Waiting for the job to be running
==> vm-luxembourg: booted @luxembourg on petitprince-9.luxembourg.grid5000.fr
==> vm-lille: Waiting for the job to be running
==> vm-rennes: Waiting for the job to be running
==> vm-lyon: booted @lyon on taurus-4.lyon.grid5000.fr
==> vm-nancy: Waiting for the job to be running
==> vm-lille: booted @lille on chimint-7.lille.grid5000.fr
==> vm-rennes: Waiting for the job to be running
==> vm-nancy: Waiting for the job to be running
==> vm-rennes: booted @rennes on parapluie-8.rennes.grid5000.fr
==> vm-nancy: Waiting for the job to be running
==> vm-nancy: Waiting for the job to be running
==> vm-nancy: booted @nancy on griffon-9.nancy.grid5000.fr

Booting Virtual Machines on Grid'5000 using Vagrant

Mon, 19 Sep 2016 10:30:00 +0000

Rationale

On the platform, having a customized environment is usually provided by the Kadepoy3 tool. It provides bare-metal provisionning of a limited number of Linux distributions (including Debian, Ubuntu) that can be adapted to the user need.

Some time ago, booting virtual machines in the production environment has been enabled.

It’s fast : we just need to get a job from the scheduler and run a kvm process inside. Usually few seconds are required to have a virtual machine ready.
It’s flexible : theorically any virtual machine can be used.

In addition to that, a ceph cluster is now available. This can be used to store virtual machine images.

Vagrant-g5k

Vagrant-g5k is a plugin for vagrant to easily deploy virtual machines on Grid’5000. From a high level point of view, each kvm process is run on a physical node after being scheduled by OAR.

The plugin is far from perfect : ideas, contribution are very welcome. Let’s take a look to its main features. An example of Vagrantfile can be found here.

Disk image

The virtual machine uses a base disk image that can be back by the frontend filesystem or by a RBD stored in the Ceph cluster. The plugin exposes different strategies to manage the disk :

copy : will copy the disk to a dedicated location. The copy will be created in your home directory on the frontend or in the same pool as the original image. This mimics the behaviour of vagrant which imports the base box in the .vagrant directory of your project.
cow : will create a Copy On Write clone of the base image. The original image will be used in read-only mode.
snapshot : will create an ephemeral Copy On Write clone of the base image.
direct : will use the base image in read-write mode.

Network

In the current implementation, network traffic from the virtual machine is NATed to the outside world. Inter virtual machines SSH communication could be added in a near future using Grid’5000 subnets. Ports can be forwarded from the virtual machine to its host machine. vagrant ssh command tunnels automatically SSH communication from your local machine to the virtual machine.

Job / virtual machine

The plugin is build based on the idea that one virtual machine correspond to one OAR job. vagrant up will submit a job to the platform. When the job is ready it will execute the kvm process. vagrant halt will delete the job. Termination signals is trapped to gracefully shutdown the virtual machine. vagrant destroy will additionally destroy the virtual machine disk image.

Currently a whole node is reserved and the virtual machine CPU/RAM is set to the maximum available. In the future a virtual machine resource demand could be map to a more complex resource selection to let the scheduler assign them.

Wish list

Automatic port forwarding to your local machine is appealing. Except for SSH, you still need to create manually a tunnel from your local machine and the physical host (ports are forwarded from the physical host to the virtual machine by kvm)
Support for parallel operation. The thread safety of the current implementation is questionnable. Adding this we should be able to start a mass of virtual machines and manage them with Vagrant.
Support for CPU/RAM customization.
Support for inter virtual machines communication
Improve code quality - be more idiomatic.

Thanks

pmorillon for (among other things) maintaining the Ceph cluster, and providing inputs to this project (especially the wrapper script around kvm)

Puppetized environment on Grid'5000

Mon, 13 Jun 2016 10:30:00 +0000

Lately, the project xp5k-openstack has been released. It deploys OpenStack on Grid’5000 and acts as a wrapper around two main components :

The Grid’5000 REST API
The Openstack Puppet modules

Under the hood it uses Rake to describe the deployment workflow.

Openstack generic deployment

The project offers to the Grid’5000 community a simple but efficient mecanism to plug different deployment scenarios. Two scenarios are shipped with the repo :

Starter Kit (all in one deployment)
Multinode

Users can then add their own scenarios to fit their needs (extra services to add, different topology, custom configuration, …)

… and beyond

The deployment process is divided in two parts :

the generic part during which a puppet cluster is deployed.
the specific part during which the specific scenario is deployed. It executes the tasks scenario:main where all the specific code resides. For instance in the multinode setup it appends some key/values to the hiera database, runs the puppet agent on the different nodes and bootstraps the installation (add cirros images, create networks, modify default security-group …)

Recently I had to deploy Mesos (more precisely to update an old deployment script) on Grid’5000. Since I wrote some puppet code to manage the deployment, I realized that I could plug this in xp5k-openstack. This releaved me from writing all the code that handle the job reservation/deployment and the puppet cluster preparation.

The mesos scenario is available : https://github.com/msimonin/xp5k-mesos

More generally xp5k-openstack should be able to manage all deployments that require puppet. Maybe we should remove openstack in the name ?

The 7 colors game

Sat, 13 Feb 2016 10:30:00 +0000

The visualisation is available here : The 7 colors game

Direct links :

Play agains the AI : Typescript version and D3js
2 player (human) game : Pure D3 implementation
Network game : Same as above + TogetherJS

Grid5000 : 10 years jobs

Tue, 02 Feb 2016 10:30:00 +0000

The visualisation is available here : grid5000-10years-jobs

My experimentation notebook (on Grid'5000)

Thu, 07 Jan 2016 10:30:00 +0000

This articles shows a simple use of iruby notebook for interacting with Grid’5000 ressources. The experimentation follows three steps :

Get some ressources on Grid’5000
Get the uptime of all the nodes of the reservation
Draw a bar plot showing the different uptime of the nodes

Check the docker image for a quick try : Docker image

The notebook

Here is the commented notebook. Full rendered notebook is also available through nbviewer service

Installation

Here comes the tricky part (no so much actually). My setup was :

Ruby 2.1.2 IPython 4.0.1 with notebook installed
The Gemfile (xp5k and restfully need a small patch each)
and … an account on Grid’5000.

source 'https://rubygems.org'

gem 'restfully',
  :git => 'https://github.com/msimonin/restfully',
  :branch => 'ripl-update'
gem 'xp5k', 
  :path => 'https://github.com/msimonin/xp5k',
  :branch => 'dsl_capture'
gem 'ffi-rzmq' 
gem 'iruby' 
gem 'nyaplot'

Docker image

A docker image is available and contains all the dependencies listed above. Please follow the instructions in https://github.com/msimonin/g5k-notebook to get started.

Moving Thymio on a grid

Sun, 01 Nov 2015 10:30:00 +0000

I explore here a way of moving Thymio on a grid composed of square tiles (25cm x 25cm). From the user point of vue, moving Thymio on this kind of grid, leads to instruct Thymio the following orders (assuming for example Thymio is on the center of a tile):

Move forward one tile
Move to the next tile on your right
Move to the next tile on your left
Go backward one tile

One way to implement this would be to translate the previous specification :

Movement		Order
Move forward one tile	—–	Move 25 cm forward
Move to the next tile on your right	—–	Turn 90° right and move 25cm forward
Move to the next tile on your left	—–	Turn 90° left and move 25cm forward
Go backward one tile	-—-	Turn 180° and move 25cm forward

But Thymio doesn’t support the instruction move exactly 25cm forward (and stop) nor turn exactly 90° left (or right). Having this kind of instruction would be great but would be very hard for the robot to handle (think of the motor calibration or the friction on the ground) and at the end would probably skew Thymio on the grid.

Since Thymio excels at following lines, maybe we just help Thymio a little bit by provoding guides on the floor. I chose the following pattern :

Assuming that Thymio is on the center of the tile (the black losange), we can translate the different movements :

Movement		Order
Move forward one tile	–—	Search for the line in front of you and follow it until the next center
Move to the next tile on your right	–—	Search for a line on your right and follow it until the next tile
Move to the next tile on your left	—–	Search for a line on your left and follow it until the next tile
Go backward one tile	–—	Make a U-turn, search for a line and follow it until the next tile

We can thus extract two main behaviours that Thymio has to learn :

Follow a black line and stop on a center of a tile
Search for a line (on left / right / bottom) from the center of a tile.

Follow the black line and stop on a center of a tile

Note : The black lines are drawn so that it can fit between the two sensors below the robot.

Thymio’s right sensor is on the line, it needs to move right a little bit. The idea is keep the angle between Thymio and Thymio small
Thymio’s left sensor is on the line, it needs to move left a little bit
Both sensors are above a black region, we stop assuming that it can be only the center of the tile. We change the internal state to reflect that Thymio is waitinga for a user interaction.

Search for on the right from the center of a tile

When the right arrow is pressed we start the procedure that search the line on the right. Thymio is instructed to turn right (rapidly). The first sensor to capture a change is the right sensor which will detect a white region.
& 3. allow the right sensor to keep track of the border of the black region
Detects the first time the left sensor is above the white region, meaning that the losange part is behind the robot and more importantly that Thymio just found the line.

Downloads

Digging openstack on Grid'5000

Wed, 30 Sep 2015 10:30:00 +0000

The visualisation is available here : digging-openstack-on-g5k