Wednesday, August 13, 2014

"0 zero must equal 100%. Good Luck."

Another eventful year passed by since my last post but I've been so busy that I completely forgot about this little place. Anyhow now it's time to give an approximately correct summary of all that happened. I guess I'll miss a lot of events, but it is what it is.
  I cannot start with the same thunder as last time, we haven't moved to another country this time so we're still living in Brno but we did switch apartments and I guess that's as good start as any for this post. We were fed up with the little space we had in our previous apartment and our stuff kept piling up so we got lucky and caught an apartment advertisement for a place which was twice as big and only 3 minutes away! Fortunately for us, and probably unfortunately for them, we had some friends visiting at the time of the move so they kindly helped us with all the stuff. Before one starts to actually move, one has a very distorted idea about all the stuff that needs to be packed and moved, or at least in my case I had it very wrong because in my mind it looked very quick and compact, but when we started packing and moving I realized how many things we've gathered and in fact started wondering where did we keep it all before that in the small apartment, if we had to go back I wouldn't think it'd be possible to stuff it in again :-)
  This year we chose to spend new year's eve in Brno and see the fireworks, some friends visited again and it was a nice celebration, we didn't make it until very late/early in the morning but we did see a lot of fireworks, I think it was like an hour of non-stop booming and lights, and of course a lot of smoke so much smoke in fact that we couldn't see the other end of the center square. A few months after that DevConf happened here in Brno which was quite the event, a lot of speakers from around the world visited and there were many interesting talks. As always we went out during the night to show them around and visited some new pubs, we had a lot of fun and it was very interesting to meet all those talented people. The event itself was very well organized with a huge party held in the largest disco place in Brno which was reserved only for us with enough free food and drinks and entertainment to make it exciting and visited by almost everyone who was on the conference. Looking forward to next year's conference!
  On the professional front a lot has happened both here and in the ISP in Sofia. I'll start with Sofia as it's the "bad news" unfortunately as it happens with most good companies there comes a person with a lot of money who makes it impossible to continue the good work and wants to buyout and restructure everything, so that pretty much happened and it was more hostile takeover than necessary, I've quit by myself because I couldn't in my right mind work for the new guy no matter the amount of money, as I've said repeatedly money is only one factor for me and it's not the leading/most important one, never has been and I hope it'll never be. So I've been explaining how things work to various new people who were hired to replace me, they got a fraction of it and keep breaking stuff that I need to fix but I think it'd be enough to continue operating all my software that drives the company after all I've written everything from the shapers/routers to the DHCP server and the BGP daemon. The good news is that the software rights are not owned by the company because I've done that in my time and also it wasn't included in my official work characteristic, so I didn't have to give them any sources. I've arranged for them to make a contract for support and updates with a company that I'll consult if they want to continue using my software, so they're not left in the dark. It's very painful for me because I've invested a lot of time developing that company technologically in the past 8 years and 6 months (man time flies..) and now I'm forced to leave it all to someone who doesn't care about anything else but money. Anyway I have new projects related to my software that are in the works and if they work out as I expect things might look up in Sofia. Also I got a new version of my router in the works which brings even bigger performance improvements (e.g. 100 000 nodes can be created and configured for < 0.5 sec and one of our shapers/routers usually has ~ 40 000 nodes in total, so it should be able to load the whole thing in 1-2 seconds from 0 to full configuration with ~ 5000 clients and full routing table, shapers and packet inspection in place since we keep ~ 5000 users per machine). I also decoupled it from the driver so it can actually be hooked to any network card now and also made it very portable, it easily runs on all FreeBSDs from 8.x to the latest 10.x.
  About RH/here I've had a crazy year, a lot of bug fixes and developments, I also found a remotely triggerable kernel crash in the Linux network subsystem which now has a CVE assigned (2014-0100) and have posted a fix for it, it was a quite complicated race condition in the inet fragmentation code and it was very difficult to debug at first, but it's all fixed now. I've converted the bonding to use RCU locking and improved its performance in all modes, fixed a ton of bugs in it and introduced a new shiny option handling API which came quite nice in the end and is now used for all bonding options, some day I may make it more generic so it can be reused in other parts of the kernel. I also fixed a bug with the mac_len calculation in the core network stack that caused GSO failures with 802.1q packets and brought the performance back to its normal levels (250 Mbps -> 5 Gbps in my tests). Recently I worked with Florian in the making of the patches that moved the inet fragmentation evictor from softirqs to a workqueue which coupled with a few more changes (most important of which was the LRU list removal) also improved performance and made the code quite cleaner, I'm still cooking some patches for the inet frag code that will build on these but I have also a lot of work inside RH and time is never enough :-) I think these are the highlights though there were many other fixes and improvements that I did over the year. Other than that I think Red Hat has been getting bigger and bigger over the year, and I expect to see a lot of interesting new projects come out, I hope I'll be responsible for at least some of them of course ;-)
  On the personal side we recently visited Bali, Indonesia for our holiday this summer. It was a crazy long flight to get there from Brno, I'll try to summarize the path:
2 hours Brno -> Vienna Airport, 1 hour Vienna -> Frankfurt airport -> ~ 12 hours Kuala Lumpur -> 3.5 hours Bali, Indonesia total: 18.5 hours
And this time is without the waiting in the airports, with the waiting the time goes to 24+ hours. Of course on the way there our suitcase was lost and we arrived on the other side of the world only with our clothes on our backs and 2 backpacks. The good side is that they recovered our suitcase and brought it to us 2 days later. About Bali, it's a very beautiful destination I must say it has so many things to see, try and visit that our 2 weeks were barely enough. We were fortunate to hire the best guide for 3 full days to shows around the most interesting places, of course with a driver and a car with A/C since the humidity there is so high that it makes it difficult to move for a long time during the day. We stayed in a 5* hotel which was also one of the oldest native hotels (not a chain or some other box-like standard hotel) and it was very good. We were very pleased with the service and everything in the hotel overall, of course it being right on the beach helped a lot too! Half the time we stayed under the palm trees enjoying the ocean breeze, occasionally getting a massage and the other half we spent going around the island. It's quite the adventure to go around Bali because in the populated areas everyone's driving (almost no walkers) and the driving regulations were pretty much missing, for example when our driver was about to get ahead of another car he'd use the horn to notify them and also when he was taking a turn he'd use the horn to notify anyone coming from the other side. The roads were pretty tight, mostly having 2 lanes (1 in each direction) and with all the traffic that normally leads to a lot of jams, it could take you 2+ hours to travel 40 kilometers that's why we would go out at 8 a.m. and go sightseeing with our guide and driver until 8 p.m. each day. Their culture is quite unique and even though Indonesia is mostly Muslim, Bali is mostly Hindu. They are very good in the arts, especially wood working, painting and rock carving, they use volcanic rocks and turn them into beautiful pieces of art like statues and ornaments. Every house has a mini Hindu temple inside it and each house is very unique because it usually has hand-made ornaments, so the architecture is quite spectacular. Their temples are also extraordinary, beautifully built and always in the best locations, for example the Tanah Lot temple which is built on a very small piece of land in the ocean or the Uluwatu temple which is on the edge of a high cliff with the ocean beneath.. unique view! We also got to feed wild monkeys which live around the Uluwatu temple :-) We tried almost every local thing we could find and explored as much as we could. I got to finally try the infamous Kopi Luwak coffee in a spice garden where they produce it locally and also produce a ton of other interesting things like vanilla, cocoa and all sorts of spices. I also brought some back with me and a full pack of spices and real chocolate :-)
The day before the last we went to a award-winning Japanese restaurant where the chef cooked in front of us which was more of a show than a dinner as it was incredible and he was so good, of course as a result the meal was awesome, experience definitely recommended! I'd say Bali isn't a place where you should just lie all day, if you want to feel and see the real culture and all the exceptional places you must take the time and visit all the beautiful sites that it offers (and there are many!). If you're the lazy type and would like only to lie down all day under the sun I'd say the Maldives is a better destination, but I realize now that for me such destination would get boring very fast, in Bali you could do something new every day or try something new :-)
I'll cut the story here as I can go on for days about Bali and in conclusion I'd like to say that it is a unique experience, definitely worth every trouble and penny to get there! Recommending it with both hands.
  We also got a new TV set when we got back, chose a 3D big LED panasonic. I actually enjoy the 3D it's fun, but I still think the future is in the VR sets like Oculus, that smells like a game changer to me and I hope to be able to acquire one soon, unfortunately they're not yet in the stores but the moment they hit them I'm getting one.
  On a ranting note we're constantly getting the Aygo fixed. Every month there's something new, first we had to change the battery, then the breaks, break pads and some other stuff, then our alternator belt snapped which turned out to be caused by the water pump so now we're changing both alternator belt and a water pump, this is my rant for this post I guess :-) Oh ironically the belt snapped when we were taking our coffee machine back from being repaired.

Enough!

P.S. The title is from Terry Gilliam's new movie "The Zero Theorem", interesting flick totally in Gilliam's style

Sunday, March 10, 2013

"Gentlemen, you had my curiosity. But now you have my attention."

It's been almost a year, but what year it was! I couldn't find time (or motivation) in order to write anything. Today I decided to resurrect my blog. So let's see where I left off last time, I'll try to be chronological.
  Firstly we moved to live in the Czech Republic, in Brno to be specific, because I got an offer from Red Hat for a kernel developer position which I couldn't pass up. They "moved" us to Brno and for the past 8 months we've been living there. The "moving" itself was very interesting because we made it into a small EU trip, coming by car and going through many countries on the way. We slept in Budapest which I have to say is one of the most astonishing looking places I've ever been to. I really liked it there and will definitely repeat that. Since it was during the summer, the weather was perfect and we had a lot of fun while moving. Why Brno you might ask ? Well, it turns out that many IT companies are residing here and are building huge support/development centres. I think that Red Hat's development centre here is the largest one they have. By the way it was fun when we were speaking on the phone with my future team because everyone kept asking me about why I would want to work on the Linux kernel (since most of my projects, and 90 % of the kernel ones, are on BSD-based OS), but I'm not some zealot who blindly believes in the supremacy of one operating system, it is kernel development and it is one of the primary subjects I'm interested in, that is enough for me. By the way in case you didn't know - everything that gets developed for the RHEL kernel has to be included in the official upstream kernel first, so Red Hat gives back a lot to the community (and the various communities are in the heart of RH's operations). I think there was a slogan in RH saying "Innovation through collaboration", and since we were named the 4th most innovative company by Forbes this is actually working out pretty well. I won't divulge much into technical details - for now I'm working on SolarFlare's 10G drivers, QLogic's netxen 10G drivers, Marvell's sky2 gigabit adapters and most importantly (and most fun for me) on the bonding module. Of course in the meantime there're many other smaller issues in other areas that I have to work on.
 Anyway next came up my birthday and I turned a square age which I celebrated accordingly with many friends and family members (I think we made it up to 50 people on the party), it was great! On the other front in the ISP in Sofia, there's a new IPTV distribution system based on BSD which is SMP-aware and I'm very pleased with the performance, I've been looking into porting some of my routing/QoS software to the Linux kernel in my free time, but we'll see where that goes. I also wrote a small Linux kernel module which is used for chain debugging (netdev, inetaddr, inet6addr), I've used it actively while working on multiple patches related to the chains and had to trace events. It is publicly accessible at https://github.com/newbg/chaindbg where I decided to create an account finally. I've got few other small projects which I will upload in the coming months (ifwatch - interface statistics/flags monitoring software which supports ncurses/text modes and works on Linux and *BSD and some ported software I mentioned earlier).
 After spending a couple of months here an interesting position opened up in RH for a technical writer with very good English skills, and I obviously thought of Yoana as she's got Cambridge's proficiency certificate and recommended her. The ECS team liked her and since last December we're colleagues :-)
I think she's enjoying the work a lot, she's always learning new stuff and going to courses organized by co-workers on various IT topics. She even went to Python programming classes! Her team's great I got to say, we've been going out for couple of months with them now and it's been quite fun.
 Last week we were at DevConf 2013, which is one of the largest open source conferences in EU I think, it was very well organized! One of the most pleasant surprises was that they were giving away Red Hat gloves with conductive threads so you could work on touch devices in the cold outside. That was a great idea, and let's not forget the conference party after the first day which was also very well organized!
 On the phone front (saga still not over!) I moved to HTC One S last summer, and now I've migrated to Nexus 4 which I have to say is the best Android handset I've seen so far, I'm very pleasantly surprised. I hope to stay with this one for a long time. Although on the tablet front I had to get an iPad because nothing comes closer to usability and battery life, and I wanted to turn it into a detachable laptop (Zaggfolio's keyboards are awesome!). I've jailbroken it and with the camera usb kit one can open USB flash storage, and has the freedom to do whatever one wishes. I've got even git and vim installed on the iPad now, so here's my mini-laptop which goes 14 hours on battery and has resolution higher than 99 % of the devices out there. Intel announced an ultrabook which has the same idea a couple of days ago (with detachable screen and all) but it is running Windows 8 and I don't know how compatible it would be with anything else, I guess time will tell.
 Oh yeah, I almost missed it, we spent new year's eve in Vienna with some friends from Bulgaria in a Bulgarian restaurant there (of course when it got close to midnight we went to the centre). I really enjoyed it and we stayed until 5 a.m. this time I think. Although the fireworks weren't very good, the show was nice and we got ourselves hats with pigs with piercings, it was good that we brought a bottle of whisky there with us and it turned into a decent party :-) I have to thank Radush for organizing everything.

Well, I guess that's all folks! I've probably missed a lot but I'll sum it up and say that we've been exploring the pubs in Brno (of which there are many) and they all have something unique to offer which is enough to go back for.

P.S. The title is a quote from the new Tarantino movie called "Django Unchained", needless to say the movie is very good in its own way :-)

Friday, April 27, 2012

"Why do you people always question? Why ask why, when *how* is so much more fun?"

    It's been awhile again since my last post, but as always there were many issues to deal with along the way. I would have to start with the technical changes (surprise, surprise). The diskless OpenBSD IPv6 router that I was blogging about in my previous post is obsolete now, replaced by a diskless FreeBSD version because as we saw when certain circumstances were present the OpenBSD would just crash (kernel coredump upon NFS failure for example) which is not tolerable so I rebooted it under FreeBSD (since I have a few OSes ready for netboot) and all the problems are gone. You can see some basic stats about our IPv6 connectivity and usage at http://IPv6.onlineDirect.bg. We also added new project to the mirrored ones called Sagemath and you can access it at http://Sagemath.Igor.onlineDirect.bg as usual. Continuing with the tradition of providing new services for our clients I decided to run our own Counter-Strike servers since most of the public ones are extremely crappy, I've added the minimum add-ons so the game can be enjoyable and also optimized the server a lot so the latency should be minimal. You can play CS Classic at cs.onlinedirect.bg:27015 and CS Italy2 Respawn at cs.onlinedirect.bg:27016.
    Another new development would be the new advanced DPI engine for my router which extremely accurately is able to differentiate between numerous protocols and if configured apply QoS based on some portion (or portions for that matter, it can be composed of few different parts) of the packet. We are using it to throttle "bad" traffic in favor of the "normal" user traffic so the service we provide would be optimal all the time. Currently I've implemented it as hybrid system doing the analysis in user-space and the matching/QoS in my router in kernel-space. I had to modify heavily the tree structures that we use because of the space requirements (there can be millions of concurrent matches per single machine) so they are as compact as possible. My tests show that currently for a little over 1 Mpps (that's our international traffic) about 4 - 6 % of the CPU is used for analysis (of course if a stream is analyzed it is cached etc) with superb accuracy and the improvement of our service is very noticeable. I've also added new user-space tools so one can track how the DPI engine works and see some statistics about it in real-time. Oh and the router now supports new load-balancing algorithm which can use up to 10 x 1/10 gbps network adapters in a single trunk.
   In the spirit of improvements I also decided to finally fix a few little issues with my BGPv4 implementation and in the process I also developed new peer balancing algorithm (which is very simplified version of the one I was writing paper about "Using combinatorics to load-balance Internet traffic") and results have jaw dropping effect. The Internet traffic got balanced evenly between all of our peers with difference margin of at worst 15 % (which means extremely well). I've added various triggers to the BGPv4 software which go off upon changes in the state of the peer and the balancing operation is fully automated. Another huge improvement in the BGPv4 software is the better use of tree structures to minimize CPU-time and now when a peer "dies" or is connected you can't see the BGP on the "top" process list for more than a second (even then taking 5 % of CPU time). I've added locking for the asynchronous events since I found a bug there which in very rare cases caused it to crash. All in all after all the changes I haven't had to log on to the machine that it's running on for any reason other than monitoring it.
    Oh wait! There's even more! :-) There's some news about the IOlib I mentioned in some of my previous posts. I've been actively developing it to add cross-platform compatibility and the results are very satisfying as the current state is:
OSes tested: *BSD (kqueue / poll), Linux (epoll / poll), Solaris (/dev/poll, poll)
IPv4 TCP/UDP sockets
IPv6 TCP/UDP sockets
Unix domain sockets
FILE I/O
SSL sockets (using OpenSSL)
I've easily managed to make a daemon that sustained over 200 000 active connections transferring over 500 MB/s with very little effort (most of which went into optimizing kernel tunables). When I add all the protocols and functionality that I wish for such a library I will release it as open source (with no license - simply free).
    On a personal note I got myself an SSD disk (OCZ Vertex 3) and I'm in heaven right now! This is by far the best upgrade one can make. As my CPU stayed idle because of my old slow HDD, now I can really feel the speed (I upgraded my PC to Sandy Bridge Core i5-2500 with 1600 MHz DDR3 memory). Boot time is little over 10 seconds without optimizing the operating system start-up (only removed few things from the bootloader), everything just pops up immediately. Since my work is connected with a lot of kernel recompilation the Linux kernel with default config from Debian compiled for about 10 minutes, bonnie++ showed 700 MB/s read speed and  nearly 500 MB/s write speed when tested. The productivity went through the roof with this upgrade and working on the PC is a lot more pleasant :-)
   But wait! There's more, I moved all my sources in a git repository that I setup on my storage. I'm pleasantly surprised by how well it works and how easy it is to develop software by using it. I've configured different SSH keys for various access types and also added active backup. In this spirit (it is a Bulgarian saying) I decided to switch to a more centralized identification, authorization and accounting system using OpenLDAP, Kerberos and the ldap pam module. I also played around with the Single Sign On idea using Kerberos, I even locked myself out of the machines one time so it's been fun :-) but I prefer more paranoid approaches to security.
   Some good news about the ISP is that we won the project we had applied for and now let there be PON service :-) In my next post I'll divulge more about this one, there would be some really big and nice new toys to play with. I'm hoping for 10G NetFPGA dev boards!
   It was about time I generate myself a PGP keypair :-)
My key id is: E01113B1
Keyserver: pgp.mit.edu
I also updated my CV and started using Dropbox (very convenient by the way) so you can find my CV in PDF format at http://dl.dropbox.com/u/75462401/CV.pdf

That's it for now! (since there're always a lot of ideas before writing and while writing they all disappear)
 
P.S. One interesting project that I'm working on is an ARC cache implementation for a VM subsystem :-)

P^2.S. The title is a quote from the cult movie "Spawn"

P^3.S. You can access my FreeBSD's loader.conf & sysctl.conf these are used on one of the most loaded servers in terms of jails, services & traffic. The machine is performing great (it has network and file i/o load). You can see info about the machine at http://Igor.onlineDirect.bg.

Monday, February 13, 2012

"Mein F├╝hrer! I can walk!"

Hey guys, another chill morning after another blizzard and so another post comes :-)
It's been quite cold the last few weeks, some temperature records broken (and I was trying to figure out why a temperature record broken would be a good thing..) and half of the country is in despair.
  First things first in January we went to the RIPE training in Skopje, Macedonia. The weather was in its infancy but still was able to snow so much as not to be able to see where the road ends and the field begins, the snow plows were not released to clean yet, so on one sharp turn we couldn't make it and went in the snow on the side of the road. It wasn't very bad, all of us were fine and the car seemed OK, the only problem was that it was floating freely as the tires didn't touch the ground :-) After waiting for an hour and a half some man with a truck pulled us out and we continued to Skopje. I have to say this was on the Bulgarian side of the road, on the Macedonian side the snow was cleaned very well and the roads were fine.
After we got to the hotel which had to be 4-star (I would give it at most 2 stars), it was like pulled from the 60s and not in the good way. The rooms were cold, there was very loud wind. Anyway the next day we started the training session at 9 am. The session itself was divided in various different topics, there was information about PI space, IPv4 exhaustion, the end game (after there are no IPv4 networks), IPv6 deployment technologies, LIR's involvement in RIPE NCC rules and discussion about some RIPE NCC rules, various interesting projects at RIPE NCC like Atlas.ripe.net (I'll discuss this one later), Labs.ripe.net, Stat.ripe.net and much more.
The session continued from 9 am to 17 pm with a small lunch break. The next day was all about IPv6 :-)
Afterwards we were on our way back to Sofia. On the next day of work I decided to ask for IPv6 allocation from RIPE NCC and we got our 2a00:87c0::/32 network. And all hell broke loose!
I wanted to start using it right away, so I called one of our international providers and arranged for another BGP session just for IPv6. So far so good, then I decided to route it with OpenBSD since we no longer have any OpenBSDs at the datacenter (after Igor many machines were taken out). Of course it had to be more interesting than that - I wanted to run the OpenBSD on a diskless machine. I have to be honest - on the Internet there are manuals and tutorials to run OpenBSD diskless either 1. from another OpenBSD or 2. to install OpenBSD diskless (which proved to be very easy).
But I had to run OpenBSD diskless from FreeBSD. The process is basically the same (as with running it from another OpenBSD) just the details of the implementation differ. I'll try to give very brief overview of what has to be done in order for OpenBSD to run diskless:
1. Get OpenBSD sources and recompile the kernel with:
"config        bsd root on nfs swap on nfs"
2. You must have a configured RARP daemon because that's how it gets an IP address at boot.
3. You must have DHCP daemon for the pxeboot process mainly
4. You must have BOOTPARAM daemon because it gets its NFS parameters through it.
5. As 4 suggests you need NFS exports :-)
6. You MUST populate OpenBSD's /dev in the right way!
(This one could really make you sweat if you're not doing it from OpenBSD!)
Easiest way - install ksh and execute the MAKEDEV script (with small rearrangements of some command arguments) you'll be ok. Also a way to generate password for OpenBSD from the bsd.rd (the ramdisk install):
run httpd on the dhcpd, put mount_nfs there and then use OpenBSD's "ftp" to download it and mount the nfs, chroot to it and then use "encrypt -b 6 <password>" and useradd's -p argument to supply the encrypted password on the command line. You can boot bsd.rd directly (just with DHCP running for the pxeboot).
And after all this we have IPv6 dedicated server, the BGP session is running since. There are about 8000 routes right now :-) I also ran DNS64 implementation just for testing purposes. There are already some clients on IPv6 and we are routing it amongst client VLANs. It's a fresh look on the world I have to say, you read all about the protocols beforehand but the actual implementation and practical running still feels new and very different. I love having such huge address space, I could do any addressing scheme for the clients with VLANs/MAC addresses integrated in the IPv6 address or any combination of those. Oh and we have IPv6-only DNS server running as well.
   A little sidenote: We also volunteered for RIPE NCC's Atlas probe and they approved us. We are waiting for the probe to arrive and we'll be able to assign it IPv4 and IPv6 addresses. I think to make it public one.
   I also ordered Arduino Uno Rev.3 board with some sensors, resistors and a breadboard. I've been looking at various MCUs all weekend and wondered if I could make my own board with ATmel's Xmega MCU (it has AES/DES support), although I found software AES implementation for ATmega328. I also got permission to implement a project about RFID for our data center. I've ordered RFID reader and a couple of RFID tags which will be assigned to the people who have access. I'll connect it to an Arduino board with ethernet shield and will make a check-in system. I also decided to add few sensors for the data center (movement, temp. and humidity). I was thinking if we were to put more RFID devices at various points in the data center we could do triangulation as well, but I guess the same effect can be obtained by using appropriate sensors with single Arduino board. By the way I think to try and implement IPv6 on the ATmega328 and use it over the ethernet shield (although it's quite limited).
   Looks like this is it for now! On the mathematics front I have some news but I'll wait some more before sharing :-)

P.S. For your sake don't be wondering where the title is from and go watch          the movie straight away!

Sunday, January 15, 2012

"The fear of offending is stronger than the fear of pain"

Hey long time no see :-)
                                            The song for this post will be: Rammstein - Fuhre Mich
I have forgotten about this little place under the net. Also I had a lot to do in the past months (6 I think since my last post). So I'm gonna start from that point on and see where that takes me.
We've moved back to Sofia, Bulgaria from UK/DE for now at least. My girlfriend is looking into the possibility for masters degree in the UK, so nothing is final yet :)
   What bothers me the most is probably that I haven't released/fixed the software I was talking about way back just some minor additions/fixes. Although with the router I've made huge progress and I had the chance to play with 10G SFP+ interfaces at last, making them line-rate with millions of packets per second was a delight. Of course such generated traffic is far from the truth, it is still nice to see some new options and problems. Also I had the chance to migrate the master router (edge) of the ISP to the new xeons based on nehalem architecture and I can say without even optimizations for this particular architecture the results are staggering - about 10x better performance for the same load with the router (it is basically standing on 98 % idle while 800 MB/s / 1 Mpps are going through it and it shapes also!). Imagine now a better software version of that and with optimizations for that particular architecture, it will be a beast of different nature. Oh I had the pleasure of trying netmap for the purpose of getting the router out of kernel- and in user- space but I find the project still immature. I've reported few bugs and also asked about when some new features will be available (or should I write them myself), but the idea of me supporting another set of patches/software which is concurrently being developed - no thank you :) So I'll wait it out and probably use it in some future release.
By the way the uptime of the shapers is almost 2 years now :-)
   There were also some changes about Igor, I've removed the FreeBSD FTP mirror (left only the CVSup mirror which is up-to-date and syncs every 3 hours) because it was taking over 900 GB and no one uses it, also there was no feedback from the mirrors mailing list of FreeBSD because the official Bulgarian mirror isn't working for a long time. I've added few new mirrors: official LibreOffice mirror(which is quite active by the way), PC-BSD official mirror and Calculate Linux official mirror. Also I'm still looking for projects to mirror so expect to see some new ones soon as well. Oh lets not forget about the official Bulgarian Freenode IRC server which is running from Igor - Hitchcock.Freenode.net.You can find me there under the nickname Raz- in #FreeBSD, #Math, #Crypto and #Linux-BG.
   There should probably be a post about my phone adventures alone but I'll try to sum them up in a paragraph here. I decided to switch to something new after being 3 years with my iPhone (which is still working by the way) so I got myself an Android phone, actually the best hardware-wise Android phone I was able to find - Samsung Galaxy S II and then all hell broke loose. It was the beginning of my headaches, first my coverage went down considerably (and with bugs, the phone wouldn't recover network coverage, also it wasn't showing the wi-fi signal strength properly loosing it from time to time) after that the chaos that Android is having adware, viruses and so on in the Market. I had a GSM with anti-virus software just in case... Also the fact that it is linux based is tickling all the wrong places. The input lag which is considerably higher than iPhone/Windows Mobile/BlackBerry 9900. I really don't like the idea of the need for dual-core CPU and 1G of RAM so my phone to run slightly normal. That is against everything I believe in software-wise, as is Java and the likes. I tried the original firmware, then rooted the device and tried optimized firmwares - in the end my problems still persisted. I saw demos of Android ICS and the input lag is still there it is just more eye-candy and the menus are better arranged compared to the total chaos. So I decided to solve my problem in a more cardinal way, I got rid of the bloody disaster and got myself a nice BlackBerry 9900 (not QNX, but still very nice look and feel). A really well-built device which works flawlessly and has very nice features that help me with my work a lot but most of all doesn't irritate me. I've spared some details here so I'll leave the story here with a happy ending after all.
   Some other news on the university front - it is possible to start doing a Ph.D. this year in BAS (Bulgarian Academy of Sciences). I hope it will be in abstract algebra or coding theory, we'll see there are still many variables :) I got an innovation certificate for the router and related software as well.
   In the end of January I'll be attending RIPE's meeting in Skopje, Macedonia related to IPv4/IPv6 resource allocation and object manipulation in the RIPE database. After we had an audit from RIPE which went well given that we exchanged some 20 mails or something like that. I really have to start making graphs for the usage of each address pool (handling a /17 divided in so many pools ain't pleasant) if I'm to request new allocations from RIPE.
   Also we are preparing a new massive project involving many new technologies where innovation is still to come and good ideas to be implemented, but I'll give more details on this some later time when all is ready :-)
   My birthday went very well, friends came from different cities and it was very pleasant evening. I would like to thank everyone who could come and hope to see them next time too :)
   On new year's eve we were in a friend's new apartment where we could see the fireworks from all of Sofia (very nice view) and it was amazing. All went very well, there was just one irritating person (some self-proclaimed zen- master) who said that mathematics limits you in your thinking (trying to re-create the phrase in English), I decided that talking to such a person is a waste of time so he was simply ignored by me all night :-)

Oh and an interesting solution to the leap year check:
printf("%s\n", ((((-(year % 4))>>31)&0x1) + (1 - ((-(year%100))>>31)&0x1)*(((-(year%400))>>31)&0x1)) > 0 ? "Not Leap" : "Leap");


So continuing with my mathematics studies and work is pretty much eating all of my time.

That's all folks!

P.S. The quote this time is from a new movie called The Girl with the Dragon Tattoo (2011), a very good movie which I enjoyed thoroughly.

P.P.S. Happy FreeBSD 9.0-RELEASE! Very nice new features and also it is dedicated to the memory of Dennis M. Ritchie:
The FreeBSD Project dedicates the FreeBSD 9.0-RELEASE to the memory of Dennis M. Ritchie, one of the founding fathers of the UNIX[tm] operating system. It is on the foundation laid by the work of visionaries like Dennis that software like the FreeBSD operating system came to be. The fact that his work of so many years ago continues to influence new design decisions to this very day speaks for the brilliant engineer that he was.
May he rest in peace.

Sunday, August 14, 2011

"That is horseshit! They will not kill us!"

Hello happy or not so happy people!
This is another one of those posts where you'll read about some bullshit story happening in some bullshit town.
The company party that I told you about in the previous post went very well, it was quite big and organized in a typical for Germany way as a beer fest. They had hired a band to play which had a Bon Jovi-like lead singer and backup singer looking like Tia Carrere, in case you're wondering who is Tia Carrere please stop reading my blog post and go in IMDB to look her up.
A week after that there was a funfair type of thing in the little town where we are residing currently, which had all sorts of rides (bumping cars, Ferris wheel [?!], and other shit like that). All in all it was again organized in their beer fest style, and in the beginning I thought it was very poorly organized, but after a while I started getting used to the way these things work and it doesn't bother me anymore. I liked the beer there very much, it had a strange taste and I really don't know its type, also a good impression was that they poured it in liter porcelain glasses. So you get a liter if you want it or not :-)
All in all I started to adjust to the strange German environment and their habits some of which (closing early, no credit cards etc.) are really annoying.
But as I'm writing this it is my last day in Deutschland and hopefully I won't have to come here any time soon.
I admit that I was a little harsh in the beginning and a little more critical as usual but now I understand their way of life better, and still I prefer the UK way, even in spite of the recent events with the riots and all.
Oh and after I thought the Internet couldn't get any worse, they changed the way of providing it. They removed the proxy and now just SNAT the addresses (which is way better), BUT every 20 minutes you get disconnected from everything you're doing and sent to a captive portal to enter your username and password AGAIN. If they keep this up, I guess the next step would be to queue our web requests and wait a day or two for them to be executed, then a day or two to get the results back and in the end of the week you might open the web site you were looking for.

On the technical front I am happy with myself this week because I was able to develop a library needed for a long time. It is called IOlib as the name suggests it provides my software with general I/O interface which is really easy to use and also very fine tuned. Currently it supports: IPv4 TCP/UDP, IPv6 TCP/UDP, FILE I/O, UNIX Sockets, BPF Sockets. Also it has its own write buffer management subsystem, so you only have to configure it. It will work under Linux, *BSD and most Unix variants (supporting kqueue, epoll, poll and select).
The TODO list includes:
Timers (implemented as I/O events)
Multi-threading (it is done with this in mind, probably a thread per protocol used)
Dynamic loading of new protocols (almost done)
Preliminary tests show very good results, I was able to implement a new daemon with simple protocol and a client for it in few lines and they communicate without any issues. The library will be released as open source too because it will be used to replace the current networking code in all my other software.
Also I'm in the middle of writing ROMFS implementation for FreeBSD (not only user-land tool, but kernel too). I may release it too after a while.

Here's an interesting fact in the past few months 2 keyboards were flooded with beer (1 standard and 1 in laptop) and both have the same problems with 1 key presses 2 or more other keys. I think production quality has gone way down, I have an old keyboard which has been flooded many times with different drinks and still works well after a clean up.
Oh and another one I found an error in "Linear Algebra" book by Georgi Shilov (translation in English and published by Dover). The error consists in them calculating the determinant of a matrix with wrong terms, having 2 entries from the same row in a single term. I have written to them to fix it for future prints, hopefully they read their emails :-) Other than that the book is very well written and I like the contents, it goes very smoothly from undergraduate material to postgraduate material.
Oh I forgot to write about my farewell to Germany, with over 7 different types of beer, hot salsa dip and tortilla chips :-) These guys have good beer that's for sure. My trip home will include the following route: Arnstorf (by taxi, because buses don't work on Bank holiday Monday of course!) -> Landau (by train) -> Munich (by train) -> Frankfurt (by plane) -> Sofia. All in all I will go out on the 15th and arrive in Bulgaria on the 16th.

For the ones wondering the title of this post is from the cult movie Aliens (1986).

And a goodbye quote which I like:
"kqueue is older than epoll. I think Linux should simply have implemented
the kqueue API instead of inventing epoll, but the Linux people insist on doing
all the mistakes of the other people again. For example, the epoll guy initially
thought he could get away without level triggering."
- Felix von Leitner in Scalable Network Programming (http://bulk.fefe.de/scalable-networking.pdf)

Wednesday, July 27, 2011

"Strangers don't last long here."

Hey guys, since my last post we've moved ourselves to Germany, and this post will be mainly about that.
Continuing from the last post, after almost 12 hours of travelling from Leicester to the small village in Germany
(coming by plane) we ended up in a place with 6000 population, looking like industrial zone, no pretty hills or sites, only smell of shit and look of grey everywhere.
As we entered the "apartment" that was "prepared", it was apparent that it is in line with the look & smell. It looked like it wasn't cleaned for a very long time, everything was very dusty and dirty, there were no cutlery, toilet paper, cleaning tools or sheets. The bed was broken, it is one of those matrices over wooden planks which fall all the time and some are already broken. To top it off the view of the building is one of the walls of a big factory which starts working early in the morning (lots of noise..). We cannot clean this place because the few tools for cleaning that are left are dirtier than the place itself.
The Internet - we were given access to a proxy which filters 50 % of the Internet, even has policies after some hours it starts filtering all sorts of sites, and it doesn't allow CONNECT method, meaning only web. We had to run Tor tunnel through it so we can connect to other ports outside, after all I can't do my job without Internet access, and when we called them and asked for real Internet access, they asked us - can you open google.com ? As if browsing is Internet access, they also didn't know what SSH is. After they understood what we mean, they just told us that it can not be done. So now we are using Tor tunnel through squid proxy via wireless connection... You can imagine how much that sucks!
Eating cold sandwiches and the occasional under-cooked meal in such a place is not the way a person should live. A month is feasible but it is definitely not healthy.

Now about Germany, I was every enthusiastic about coming here and now it is the opposite. I think that for a big country of such type in this age you _must_ be able to accept at least debit (and credit) cards. Which is not the case here, they work mainly with EC cards only, and even the places which support those are not so many. In contrast with UK where I can get in the WC with a card, here I can't shop with it even at some stores at the airport or big super markets. I've been to Poland, Netherlands, Belgium and Austria, and they all accept cards everywhere. Although Germany has very nice autobahns (?!), its public transportation service is not very well organized. For example there are no buses Saturday and Sunday from/to the village. There are no trains from a big city nearby to Munich as well, etc. etc. Everything closes at 6 p.m., the only place we found that works late (23h!) is a brewery near by (5 km).
On the bright side I can say they have excellent bikeways, and a lot of them as well. Too bad we had our bikes stolen :-)
All in all Germany is definitely not a country where I can live. Now I started appreciating UK a lot more.

There will be a company party where my girlfriend works, and probably that will be interesting given that it is a 4000 employee company with 800 million of revenue, it should be a pretty big event. We'll see how that will play out :-)

Last few days I have few ideas going through my head, about the GRaph router. Firstly the just-in-time compiled code naturally arises in these applications, that is clear to everyone. What I don't like about it is the compiler optimizations that are missing (if done wrong) and also the ability to insert code into the kernel, ain't that pretty, needs a lot of thought to be done safely. Anyway I was thinking of using LLVM to generate node code and insert it into the kernel afterwards, it has great potential and will probably work well, so we'll not miss much of the compiler optimizations. But I also think that if we do a greatly optimized C version of the same code, the difference will not be more than 10 %, even in some cases it will be slower, because of the compiler and cache usage. One of the main uses of JIT (in this application) is to avoid going to memory so much, but as new architectures get faster and faster memory, and get optimized for other types of code/use, it is may be a better idea in the long run to make it in statically compiled language.
 On the other hand I was thinking about porting it to other operating systems, like Linux and the others from the BSD family. This task should be relatively easy, as much of the router is self-sustainable, the shared code is mostly in memory management, driver connection and upper layer connection (mostly for applications' sake). All that can be easily adapted to any operating system, so that will definitely be one of the goals. I am quite interested in porting it to minix 3, but there are no drivers (as of yet) for high performance ethernet adapters there. The worst part will be the user-space software (blah).

This weekend I had one of those Sunday-early-morning calls about a machine which has frozen (a central storage server), they are very pleasing, especially when you've been to a brewery the night before :-)
Anyway at first sight it looked like hardware issue, but eventually the most likely fault was in the software. So I did the long needed upgrade (it had very big uptime, so it wasn't touched for a while) of controller firmware, operating system kernel etc. etc. and now we're seeing noticeable change in performance (the raid controller works better). Hopefully it will stay that way for a long time, in contrast the other most important machine which distributes the work for the nodes has over 360 days of uptime, and it does 1000 things :) The nodes have stayed strong as well with over 360 days of uptime, which is very gratifying. Too bad soon there will be a huge upgrade to operating system and our software and restarting them will be imminent :-)


'til next time! Who knows maybe we'll end up in a small unknown African village, what is more frightening is that it might not be worse :-)))