Sunday, August 14, 2011

"That is horseshit! They will not kill us!"

Hello happy or not so happy people!
This is another one of those posts where you'll read about some bullshit story happening in some bullshit town.
The company party that I told you about in the previous post went very well, it was quite big and organized in a typical for Germany way as a beer fest. They had hired a band to play which had a Bon Jovi-like lead singer and backup singer looking like Tia Carrere, in case you're wondering who is Tia Carrere please stop reading my blog post and go in IMDB to look her up.
A week after that there was a funfair type of thing in the little town where we are residing currently, which had all sorts of rides (bumping cars, Ferris wheel [?!], and other shit like that). All in all it was again organized in their beer fest style, and in the beginning I thought it was very poorly organized, but after a while I started getting used to the way these things work and it doesn't bother me anymore. I liked the beer there very much, it had a strange taste and I really don't know its type, also a good impression was that they poured it in liter porcelain glasses. So you get a liter if you want it or not :-)
All in all I started to adjust to the strange German environment and their habits some of which (closing early, no credit cards etc.) are really annoying.
But as I'm writing this it is my last day in Deutschland and hopefully I won't have to come here any time soon.
I admit that I was a little harsh in the beginning and a little more critical as usual but now I understand their way of life better, and still I prefer the UK way, even in spite of the recent events with the riots and all.
Oh and after I thought the Internet couldn't get any worse, they changed the way of providing it. They removed the proxy and now just SNAT the addresses (which is way better), BUT every 20 minutes you get disconnected from everything you're doing and sent to a captive portal to enter your username and password AGAIN. If they keep this up, I guess the next step would be to queue our web requests and wait a day or two for them to be executed, then a day or two to get the results back and in the end of the week you might open the web site you were looking for.

On the technical front I am happy with myself this week because I was able to develop a library needed for a long time. It is called IOlib as the name suggests it provides my software with general I/O interface which is really easy to use and also very fine tuned. Currently it supports: IPv4 TCP/UDP, IPv6 TCP/UDP, FILE I/O, UNIX Sockets, BPF Sockets. Also it has its own write buffer management subsystem, so you only have to configure it. It will work under Linux, *BSD and most Unix variants (supporting kqueue, epoll, poll and select).
The TODO list includes:
Timers (implemented as I/O events)
Multi-threading (it is done with this in mind, probably a thread per protocol used)
Dynamic loading of new protocols (almost done)
Preliminary tests show very good results, I was able to implement a new daemon with simple protocol and a client for it in few lines and they communicate without any issues. The library will be released as open source too because it will be used to replace the current networking code in all my other software.
Also I'm in the middle of writing ROMFS implementation for FreeBSD (not only user-land tool, but kernel too). I may release it too after a while.

Here's an interesting fact in the past few months 2 keyboards were flooded with beer (1 standard and 1 in laptop) and both have the same problems with 1 key presses 2 or more other keys. I think production quality has gone way down, I have an old keyboard which has been flooded many times with different drinks and still works well after a clean up.
Oh and another one I found an error in "Linear Algebra" book by Georgi Shilov (translation in English and published by Dover). The error consists in them calculating the determinant of a matrix with wrong terms, having 2 entries from the same row in a single term. I have written to them to fix it for future prints, hopefully they read their emails :-) Other than that the book is very well written and I like the contents, it goes very smoothly from undergraduate material to postgraduate material.
Oh I forgot to write about my farewell to Germany, with over 7 different types of beer, hot salsa dip and tortilla chips :-) These guys have good beer that's for sure. My trip home will include the following route: Arnstorf (by taxi, because buses don't work on Bank holiday Monday of course!) -> Landau (by train) -> Munich (by train) -> Frankfurt (by plane) -> Sofia. All in all I will go out on the 15th and arrive in Bulgaria on the 16th.

For the ones wondering the title of this post is from the cult movie Aliens (1986).

And a goodbye quote which I like:
"kqueue is older than epoll. I think Linux should simply have implemented
the kqueue API instead of inventing epoll, but the Linux people insist on doing
all the mistakes of the other people again. For example, the epoll guy initially
thought he could get away without level triggering."
- Felix von Leitner in Scalable Network Programming (

Wednesday, July 27, 2011

"Strangers don't last long here."

Hey guys, since my last post we've moved ourselves to Germany, and this post will be mainly about that.
Continuing from the last post, after almost 12 hours of travelling from Leicester to the small village in Germany
(coming by plane) we ended up in a place with 6000 population, looking like industrial zone, no pretty hills or sites, only smell of shit and look of grey everywhere.
As we entered the "apartment" that was "prepared", it was apparent that it is in line with the look & smell. It looked like it wasn't cleaned for a very long time, everything was very dusty and dirty, there were no cutlery, toilet paper, cleaning tools or sheets. The bed was broken, it is one of those matrices over wooden planks which fall all the time and some are already broken. To top it off the view of the building is one of the walls of a big factory which starts working early in the morning (lots of noise..). We cannot clean this place because the few tools for cleaning that are left are dirtier than the place itself.
The Internet - we were given access to a proxy which filters 50 % of the Internet, even has policies after some hours it starts filtering all sorts of sites, and it doesn't allow CONNECT method, meaning only web. We had to run Tor tunnel through it so we can connect to other ports outside, after all I can't do my job without Internet access, and when we called them and asked for real Internet access, they asked us - can you open ? As if browsing is Internet access, they also didn't know what SSH is. After they understood what we mean, they just told us that it can not be done. So now we are using Tor tunnel through squid proxy via wireless connection... You can imagine how much that sucks!
Eating cold sandwiches and the occasional under-cooked meal in such a place is not the way a person should live. A month is feasible but it is definitely not healthy.

Now about Germany, I was every enthusiastic about coming here and now it is the opposite. I think that for a big country of such type in this age you _must_ be able to accept at least debit (and credit) cards. Which is not the case here, they work mainly with EC cards only, and even the places which support those are not so many. In contrast with UK where I can get in the WC with a card, here I can't shop with it even at some stores at the airport or big super markets. I've been to Poland, Netherlands, Belgium and Austria, and they all accept cards everywhere. Although Germany has very nice autobahns (?!), its public transportation service is not very well organized. For example there are no buses Saturday and Sunday from/to the village. There are no trains from a big city nearby to Munich as well, etc. etc. Everything closes at 6 p.m., the only place we found that works late (23h!) is a brewery near by (5 km).
On the bright side I can say they have excellent bikeways, and a lot of them as well. Too bad we had our bikes stolen :-)
All in all Germany is definitely not a country where I can live. Now I started appreciating UK a lot more.

There will be a company party where my girlfriend works, and probably that will be interesting given that it is a 4000 employee company with 800 million of revenue, it should be a pretty big event. We'll see how that will play out :-)

Last few days I have few ideas going through my head, about the GRaph router. Firstly the just-in-time compiled code naturally arises in these applications, that is clear to everyone. What I don't like about it is the compiler optimizations that are missing (if done wrong) and also the ability to insert code into the kernel, ain't that pretty, needs a lot of thought to be done safely. Anyway I was thinking of using LLVM to generate node code and insert it into the kernel afterwards, it has great potential and will probably work well, so we'll not miss much of the compiler optimizations. But I also think that if we do a greatly optimized C version of the same code, the difference will not be more than 10 %, even in some cases it will be slower, because of the compiler and cache usage. One of the main uses of JIT (in this application) is to avoid going to memory so much, but as new architectures get faster and faster memory, and get optimized for other types of code/use, it is may be a better idea in the long run to make it in statically compiled language.
 On the other hand I was thinking about porting it to other operating systems, like Linux and the others from the BSD family. This task should be relatively easy, as much of the router is self-sustainable, the shared code is mostly in memory management, driver connection and upper layer connection (mostly for applications' sake). All that can be easily adapted to any operating system, so that will definitely be one of the goals. I am quite interested in porting it to minix 3, but there are no drivers (as of yet) for high performance ethernet adapters there. The worst part will be the user-space software (blah).

This weekend I had one of those Sunday-early-morning calls about a machine which has frozen (a central storage server), they are very pleasing, especially when you've been to a brewery the night before :-)
Anyway at first sight it looked like hardware issue, but eventually the most likely fault was in the software. So I did the long needed upgrade (it had very big uptime, so it wasn't touched for a while) of controller firmware, operating system kernel etc. etc. and now we're seeing noticeable change in performance (the raid controller works better). Hopefully it will stay that way for a long time, in contrast the other most important machine which distributes the work for the nodes has over 360 days of uptime, and it does 1000 things :) The nodes have stayed strong as well with over 360 days of uptime, which is very gratifying. Too bad soon there will be a huge upgrade to operating system and our software and restarting them will be imminent :-)

'til next time! Who knows maybe we'll end up in a small unknown African village, what is more frightening is that it might not be worse :-)))

Sunday, July 17, 2011

"Hello, Oompa Loompas of science!"

Hello again, some time passed and I forgot about this little place on the net :)
First off I have to say that I had my graduation ceremony which went fairly well in a nice place called Founder's Building. There were the hats, togas and a big ceremony in the chapel, I think the ceremony was for the Mathematics department and some other science (phylosophy ?) that day. Afterwards it continued with a reception in the yard and some hat throwing, so in the end it's official I am a master of science in Mathematics :)
But there's long way to go there (I have a pile of books to read in order to feel comfortable with this qualification). After that we went back to Leicester (we did Leicester -> London -> Egham -> Windsor -> London -> Leicester trip). It was a really interesting trip, I managed to get Amazon Kindle 3G (more on that later) on the way back and we had to run across the London underground in order to get in time for our train back, we barely managed to get there 5 minutes before leaving by cutting through a shorter underground line which saved us couple of stops. The next day we went in the National Space Centre in Leicester which is a really nice place, it deserves all the awards it had won most definetely. We had guests and got the chance to visit it again, this time we had time to do everything we missed last time :) and it was good fun, where will you get the chance to ride micro gravity simulator which uses air to simulate it.

Another interesting news is probably the fact that I have started rewriting some of the software I've written over the years in order to release it to the public (BSD license I guess), so it wouldn't gather dust on the drives :)
Since most of it is written for *BSD, it would take some effort to be ported to Linux or some other OS.
I am thinking to release these parts:
 - Lightweight multi-threaded DHCP server
 - Multi-threaded IGMP (multicast) relay software (currently used to offer TV in over 1000 VLANs)
 - SMTP Spam scanner (used to filter messages from a huge network)
 - DHCP/MAC logger (extremely efficient)
 - GRaph router :) (without the drivers, but with XML interfaces and some CLI when I get to write it)

I think that's about enough, there are few other small tools that will be released, but all of this depends on how  much time I'll have to prepare the packages and the software for that matter, because it has to be re-written in order to be released. All in good time I guess and volunteers are always welcome ;)

Before I got to UK I was able to attend the concert Sofia Rocks (went for Whitesnake and Judas Priest mostly), which really sucked. It was organization issues mainly, the scene was rather small, the beer shops were very few and worked even slower (no one was in a hury apparently to service the huge queue in front, we had to wait over 40 minutes to get a beer!). Whitesnake didn't do a nice show, they took a break after every song and while waiting there were few solos from members of the current band (?) but they were mostly weak, especially the guitar, and also there were some new songs but I think the public wanted the old great ones. Afterwards came Judas Priest and rocked the place with a great show. The End.

About the Kindle I was able to get (got it for my graduation as a present to myself :), it is REALLLY great device, probably my best buy for some time now. I am able to read everywhere, and with the 3G I can check/browse anything pretty much anywhere. It is very nice to have free internet connection in over 100 countries. And it is actually like paper, but the pictures look great too (being black/white with some gradients and all). I am really delighted by the device and will be reading a lot on it :) The PDF render nicely, you just have to switch it in landscape mode and they're read perfectly. I am like a little child in a candy shop and can't choose which books to read, so I have started 3 books in parallel :)
I was actually able to order another Kindle for a friend from my Kindle while away from home. The browser does great job given the circumstances. All in all it is a truly great device.

In the spirit of logically disconnected thoughts tonight I'll continue with another news, tonight we are flying to Germany (Munich), and will be there for some time (a month to few months), it will be interesting to see how everything is done over there, I am kind of a fan of Germany (can you believe that, given that I studied mathematics and most of my lecturers were German ?). I will definetely try to learn some German while I'm there, although we'll be in a small village near Munich. Good news is that it is near many countries and some very nice places like Vienna, which will be visited. And of course to try the infamous German beers will be one of the main goals :)
I have also a lot of work  (and reading for that matter) to do, so I hope I'll be able to catch up in Germany (it will be peaceful and that really helps me, I can't focus well otherwise). I am working on a hardware switch project which is going well, my friend who makes the hardware was able to give a test run to the first prototype and it switched (at least), so from here on I'll be busy writing its firmware to use the chip's capabilities which are impressive for something that small. Another aspect is that I am re-writing a BGPv4 package to work with my graph router, and also making a new version of the graph router itself, so I can't complain that I don't have anything to do :) Sleepless nights are ahead!

P.S. Hopefully soon I'll have some pictures from the graduation ceremony, and guys please stop sending me Google+ invitations, thank you but I will most definetely not enter any new social networks, the ones that I'm visiting are even more than I need, and most of all I don't want to rely on Google for everything.

Have fun!

Friday, June 24, 2011

"Good morning, Vietnam"

"Adrian Cronauer: Goooooooood morning, Vietnam! Hey, this is not a test! This is rock and roll! Time to rock it from the Delta to the D.M.Z.!"

Hello critters!
Long time, no see, the past few days I've been a bit busy but here is the next little thought dump.
Firstly on the agenda is a long standing issue with the mathematics surrounding computer science (or computer science using mathematics anyway you prefer), some guy wrote a book arguing that modern computer scientists don't need mathematics or that, and I quote,
"The notion of the algorithm, simply does not provide conceptual enlightenment for the questions that most computer scientists are concerned with." 
Needless to say I am STRICTLY AGAINST that point (and generally that idea), and I will try to summarize & generalize my arguments in a few brief points:

  1. The most basic notion and idea that comes to mind first is that you cannot truly grasp a concept about any subject without understanding the underlying principles and methods that lead to the creation and support of that idea. 
  2. One cannot show adequately and fully an idea which builds, or somehow relies, upon those principles. 
  3. Algorithm proofs & analyses. (not so general)
  4. The statement that mathematics isn't needed in computer science in itself implies that one limits oneself to only use some small part of the arsenal that is presented to him, or called another way one becomes a coding monkey. (Thanks to the CAD monkeys for the term :)
  5. The aforementioned limit also applies to mental development in those fields.
  6. etc. etc. 

Now don't get me wrong, I don't say that a computer scientist should be able to prove the Four colour theorem, but graph theory  for example is very heavily used in all aspects of computer science and having a more in-depth knowledge in it can prove vital in many situations, it could turn a good solution into a great one, but then proving so would also require mathematics, but then showing its limits as well and so forth. There are many likewise examples that could be given for other mathematics branches involved in computer science. There are a few related links in the end.

Second on the agenda is shorter and more technical, an interesting branchless solution to the integer comparison problem. There are numerous solutions on-line for it, but this one is a bit different and goes like this:
 Let's have two integers X and Y, then we want to be able to take 2 different "paths" in our program based on  the following criterion: if X <= Y -> path 1, otherwise -> path 2, but we want to do it branchless (no conditionals).
Solution: Define a 2 element jump array (or any other type of array needed based on the criterion), lets call it jump_table[2]. Then just execute the following: 
jmp jump_table[((X-Y)>>31)&0x1];
Simple proof (that it works): X-Y in integer notation would be either positive, zero or negative (trichotomy), 
 1. if it is positive or zero: bit 32 == 0 (and we take path 1) 
 2. if it is negative: bit 32 == 1 (and we take path 2)
After testing it against the conditional version for 1000000 iterations it proved to be ~ 16 % faster overall. But the most important property is that it is branchless, and in some cases that matters much more.

Third on the agenda I've been really agitated with the multi-core craze that's been going on lately. Now people started counting their cores, instead of the clock speed (as was the previous misdirection). I have always disliked (that's me being polite) people who try to show-off without having any valid reason to do so (aka posers/fakes). I think that humility is a virtue, and even in case one has reason to show-off it should be left to the public to decide so. Bragging shows simplicity and eagerness for recognition rather than eagerness for the subject itself. Now to the point, multiple cores is not real multiprocessing, because they have to share the same bus, memory lanes and of course socket. It _WILL NOT_ provide linear speed-up in anything for those (and many other) simple reasons, more to the point more cores doesn't mean more "speed", higher "speed" or any of the like. Such notions are more complicated and actually depend on many variables on multiple levels (hardware design, software design etc.). If manufacturer X has announced 16 core chip it doesn't necessarily mean it will perform better than provider Y's 4 core chip. More cores can better obturate a bus if used/designed correctly of course, and can provide very good speed-up for CPU intensive applications (which again depends on the software/hardware design working together). I don't want to get more technical, I think you should get the point by now. In my next post I'll probably write down a few thoughts about virtualization and the "cloud". 

'til next time!

P.S. In the next posts I'll give a brief overview of my new way for load-balancing between multiple peers which uses  relatively simple notions from combinatorics. One day it should become a paper.. (laziness)
If there are any mistaken/illogical sentences - I couldn't get a good night sleep, get over it!

The links:
Computer Science Mathematics - Some of the mathematics background required for computer science.
Theoretical Computer Science Talk - Given by Akamai's chief of technology, Tom Leighton
Want to be a computer scientist? Forget maths (Article) - About the book I talked about earlier.
From rigor to rigor mortis - A bonus. Take a look inside, you may like it.

Wednesday, June 15, 2011

Day hard.

Offspring on (check)
Green tea with brandy (check)

So hello ineffaceable people! It was an amusing day, after a sleepless night (once again), as most of you have experienced such feeling after a long long (looong...) day, and night for that matter, of work followed by "early" wake up. It is like you are in one of those movies where the main character just wakes up, everything is blurry and there's almost no recollection of what happened last night. Hit pretty fast by the strong cup 'a' coffee you've just made for yourself with the ingenious idea that it will straighten you, but alas you find that concentration and focus are not even remotely present. (by the way did you know that there are missiles called ALAS ? I sure did not).
 So lets get to it then, shall we ? (Parts a bit technical, skip if bored to death.)
There are 2 parallel stories that developed, one involving my iPhone doing things that probably even Apple won't believe and the second one involving one nefarious bug in a small software.
Lets begin with the iPhone story: I'll try to structure it as much as I can since I think that the states the phone went through could be easily represented by a graph which could logically explain why/what made it go POO.
 I started by breaking free my girlfriend's phone and thought "hmm, why should mine be still locked ? Only because I have an iPad baseband on my iPhone and a twisted version of iOS which was barely put together shouldn't deny me of the pleasure of playing with Cydia, oh and yes - my wi-fi is out of order (hardware problem) too.." Of course everything broke down, iTunes wouldn't recognize that there was a device connected, as a matter of fact none of the jailbreak/recovery mode/DFU mode programs would recognize it.. But lets not get our hopes down, I started fiddling with the bootstrap software and was finally able to reconfigure it to boot into Recovery mode again, afterwards iTunes started spilling errors (1600 - unknown error [?!]), and because of the iPad baseband I couldn't just update/fix it through iTunes so I had to fix it using the "pirate" software (you think of another term at this hour..), it had already loaded 2 different exploits which should've allowed jailbreaken firmware to be uploaded, so I went on a limp and did the iPad baseband reinstall procedure with a new custom firmware (making use of the new exploits) and was able to go from Dead mode -> DFU Mode -> Recovery Mode -> Standard firmware -> DFU Mode -> Jailbroken/custom firmware -> Incompatible Cydia/iOS and packages on top as a result.. The plan was to use some tool called Cyder to install Cydia software through the PC (without the need for wi-fi), but of course with incompatible versions it all went to a very dark place very fast. I'll stop here because I started to bore myself with my own story and continue with the second one which developed in parallel (in the next window on the taskbar)..

 The bug with the software was trivial, but manifested in a rather interesting way. Here's the quick overview: a multi-threaded program which services some descriptors, a long forgotten static descriptor variable (unused) and a select syscall. If this doesn't ring any bells yet, here's what happens next: since the software doesn't handle the select syscall well and upon receiving a descriptor with id = 0 it just selects it, since it doesn't use STDIN, if you've started it in background and still logged on there is no problem, BUT once you quit and the descriptor gets closed select should return EBADF (according to the manual of select(2), and /usr/include/errno.h says error 9 == EBADF). Now put that in a room with improper select return value handling and you've got yourself a software which behaves normally when started but upon exit goes to 100 % (good it's only 1 thread doing the selects) CPU usage. Of course easily "debuggable" issue, if you even get to a debugger, the cause is pretty logical, but it's rare to hit such bug nowadays, which makes it fun :-) Moral of the story ? Don't leave unused variables behind or one day they'll haunt some programmer dealing with your mess (or you for that matter) !

 'nough said, going to glue together some parts for a design show model now,
Haunt you soon!

P.S. For the sleeping beauties Day hard is mildly related to Die hard, now the question is have you watched it ? If not, you are not of age to read this blog, so shoo!

Tuesday, June 14, 2011

Hello, world! (and goodbye privacy :)

Hello, generally I am against these types of communication/logging, unless one has something really new/important to say. So I'll try 'n stick to this simple guideline, hopefully it won't be a complete waste of time as I expect it to be.
There are times when one just needs to express his frustration/admiration about something he has experienced, so I guess this is as good place as any.
Enough said, be back soon. Goodnight!

P.S. I guess you can consider this as another way to write the infamous "Hello, World!". Fancy, isn't it ?