Eatmonkey 0.1.3 benchmarking

Eatmonkey has now been released for the 4th time and I started to use it to download videos from FOSDEM2010 by drag-n-dropping the links from the web page to the manager :-)

I downloaded four files and while they were running I had a close look at top and iftop to monitor the CPU usage and the bandwidth usage between the client/server (the connection between eatmonkey and the aria2 XML-RPC server running on the localhost interface).

I had unexpected results and was surprised by the CPU usage. It is very high currently which means I have a new task for the next milestone, getting the CPU footprint low. The bandwidth comes without surprises, but since the milestone will target performance where possible I will fine down the number of requests made to the server. This problem is also noticeable in the GUI in that it tends to micro-freeze during the updates of each download. So the more active downloads will be running the more the client will be freezing.

Some results as it will speak more than words:

Number of active downloads	Reception	Emission	CPU%
4 downloads	144Kbps	18Kbps	30%
3 downloads	108Kbps	14Kbps	26%
2 downloads	73Kbps	11Kbps	18%

I will start by running benchmarks on the code itself, and thanks to Ruby there is built-in support for Benchmarking and Profiling. It comes with at least three different useful modules: benchmark, profile and profiler. The first measures the time that the code necessitated to be executed on the system. It is useful to measure different kind of loops like for, while or do...while, or for example to see if a string is best to be compared through a dummy compare function or via a compiled regular expression. The second simply needs to be included at the top of a Ruby script and it will print a summary of the time passed within each method/function call. The third does the same except it is possible to run the Profiler around distinctive blocks of code. So much for the presentation, below are some samples.

File benchmark.rb:

#!/usr/bin/ruby -w

require "benchmark"
require "pp"

integers = (1..10000).to_a
pp Benchmark.measure { integers.map { |i| i * i } }

Benchmark.bm(10) do |b|
    b.report("simple") { 50000.times { 1 + 2 } }
    b.report("complex") { 50000.times { 1 + 2 - 6 + 5 * 4 / 2 + 4 } }
    b.report("stupid") { 50000.times { "1".to_i + "3".to_i * "4".to_i - "2".to_i } }
end

words = IO.readlines("/usr/share/dict/words")
Benchmark.bm(10) do |b|
    b.report("include") { words.each { |w| next if w.include?("abe") } }
    b.report("regexp") { words.each { |w| next if w =~ /abe/ } }
end

File profile.rb:

#!/usr/bin/ruby -w

require "profile"

def factorial(n)
    n > 1 ? n * factorial(n - 1) : 1;
end

factorial(627)

File profiler.rb:

#!/usr/bin/ruby -w

require "profiler"

def factorial(n)
    (2..n).to_a.inject(1) { |product, i| product * i }
end

Profiler__.start_profile
factorial(627)
Profiler__.stop_profile
Profiler__.print_profile($stdout)

Update: The profiling showed that during a status request 65% of the time is consumed by the XML parser. The REXML class is written 100% in Ruby, and that gives a good hint that the same request done with a parser written in C may present a real boost. On another hand, the requests are now only run once periodically and cached inside the pooler. This means that the emission bitrate is always the same and that the reception bitrate grows as there are more downloads running. And as a side-effect there is less XML parsing done thus less CPU time used.

Backward compatibility for Ruby 1.8

As I'm currently writing some Ruby code and that I started with version 1.9 I felt onto cases where some methods don't exist for Ruby 1.8. This is very annoying and I started by switching the code to 1.8 method calls. I disliked this when it came to Process.spawn which is a one line call to execute a separate process. Rewriting it takes around 5 lines instead.

So I had the idea to reuse something I already saw once. I write a new file named compat18.rb and include it within the sources that need it. Ruby makes it very easy to add new methods to existing classes/modules anyway, even if they exist already, so I just did it and it works like a charm.

Here is a small snippet:

class Array
        def find_index(idx)
                index(idx)
        end
end

class Dir
        def exists?(path)
                File.directory?(path)
        end
end

Update: It can happen that a fallback method from Ruby 1.8 has been totally dropped and replaced against a new method in 1.9, and in this case the older method has to be checked if it exists, and otherwise make a call to the parent.

class Array
        def count
                if defined? nitems
                        return nitems
                else
                        return super
                end
        end
end

Fed up with Moblin

I slowly begin to be fed up with Moblin, the base installation. The base system starts way too often with core-dumps (crash on mutter f.e. which also means X restarts), but mainly because of RPM. When package-kit starts to check for an update — or when you do any installation/upgrade with yum e.g. you use rpm directly or indirectly — the whole system goes unusable, the browser acts like it is frozen, it takes very long to switch between tasks, and all of this for at least a minute up to an hour if you accept to run an update. You can call this whatever you want, I call this a big fail.

This happens on an Acer Aspire One 9", where I guess they installed the cheapest SSD out there.

In fact things were getting really bad when I switched to an Xfce session, I received unbelievable long startup times. Uxlaunch, the new automatic login application on Moblin 2.1, is totally uncooperative. The Xfce session ends launching many tools and applications twice, two corewatcher-applets, two connman-applets, etc. Uxlaunch will run xfce4-session, but also executes the same desktop files — as it seems after a quick look in the code — from the autostart directory, which is a role taken by the Xfce session manager.

So I have been looking around to finally throw away some junk.

Now I have been looking close at the autostart applications since the "all-in-twice" fiasco to get this netbook fast again. Of course you have to know what you do, this kind of tasks isn't open to people without technical skills. First I changed the default "desktop" to Openbox, by downloading the RPM source package, compiling it and putting it inside the uxlaunch configuration file. Then I have been removing some base packages and manually hiding some desktop files to avoid them to autostart — I have been playing with the Hidden/NoDisplay key but it didn't have any effect on uxlaunch so it ended with a chmod 000 command.

I dropped four packages, kerneloops, corewatcher and obexd/openobex. I really don't want them around anymore. And I "dropped" seven autostart files, ofono which depends on a lot of applications, the bkl-orbiter, and the rest are Moblin panel related applications, bluetooth-panel which I don't even have on this netbook, carrick-panel as I use connman-applet which works at least for an automatic connection, two dalson applications dalson-power-applet and dalston-volume-applet, and at last moblin-panel-web.

I kept the gnome-settings-daemon although I have the Xfce settings daemon installed which I do prefer at some extends. And after all this I changed the GTK+ and icon themes through the gconf keys. And what's the conclusion? Moblin is nice, but I managed to munch it and enforce my desktop.

Update: After running under OpenBox I feel that my remark toward RPM is wrong, I don't know maybe it is the mixed use of OpenGL that makes the tasks taking ages to react. All in all, the default desktop environment is something where you must know about patience :-)