benchmarks; Xen vs OpenVZ 23 March 2007 @ 08:02 pm
Kir Kolyshkin
k001
[openvz]

There is a somewhat interesting article at the Infoworld blog, discussing the VMware vs. Xen and Xen vs. VMware benchmarks. It appears VMware did a not-quite-good job comparing their ESX to Xen, so Xen came back and presented another comparison, where it is either on par or a bit better than ESX.

From my experience working as Virtuozzo QA team leader (a few years ago) doing all sorts of performance and stress tests for Virtuozzo kernel, I know that there are very many factors influencing the results. Consider this: if you happen to run your test while cron is running daily jobs like slocate's update, log rotation routines etc., your performance could be 10 to 50 per cent slower. This was a very simple and obvious example -- just disable cron daemon before you do your testing. A trickier example is when networking performance increase by 10 to 15% if you bound a NIC interrupt to a single CPU on an two or four-way SMP box.

So, my suggestion is to take those benchmarks and comparisons with a grain of salt. Better yet, do your own comparison using your hardware and your workloads -- and make sure you understand all the results. So if something is slow -- find out why. If something is faster than it should be -- find out why, find out what you did wrong. Perhaps this part -- results analysis -- is the most complex part in the performance testing field.

Having said that, I'd like to point out a Xen vs. OpenVZ comparison, done by a German student Björn Gross-Hohnacker who I met at last year's LinuxWorld Cologne. Björn graciously allowed us to publish his results, so we have translated part of it into English.

Here is the bottom line summary: IPC and disk I/O performance is better (or much better) for OpenVZ than Xen, CPU-intensive tasks are about the same for both, networking is a bit better in OpenVZ. Conclusion: for homogeneous (i. e. Linux-only) environments, OpenVZ is way better -- as it was designed to be.

You are taking this with a grain of salt, aren't you? ;)

Tags: , , , , ,
Previous | Mem | Next
Permanent Link8 comments | Leave a comment
Post a new comment
[info]anyher on March 29th, 2007 - 10:07 am
Yep, i knowing about cron(well, in my case windows scheduler) tricks first-hand.
About results of OpenVz vs Xen - i think it's something that is looks realistic to me. I can't smell something completely rotten in that benchmarks.
(Anonymous) on May 14th, 2007 - 05:30 pm
IPC, disk IO.. it is not very required info.. I think, that best way - real benchmarks with mysql/apache with different configs (python,php,perl)/mongrel/nginx and other software =(
(Anonymous) on August 30th, 2007 - 04:17 am
Xen vs OpenVZ
HP has a "real world" app test here: http://www.hpl.hp.com/techreports/2007/HPL-2007-59.html
with similar conclusions...
(Anonymous) on November 15th, 2007 - 08:12 pm
Thanks
I was about to move my openVZ account to xen until I say your article.
Now I am having second thought after reading your article.

I read somewhere that java server performs better on xen (in terms of memory requirement). What is opinion on this?

Thanks,

surajz (www.himalayansouvenirs.com)
(Anonymous) on January 5th, 2008 - 12:07 pm
how about memory usage?
i just read here:
http://blog.craz8.com/articles/2006/12/25/openvz-redefines-ram/

how you need twice ram more than xen for webhosting vps application.
is that true?

tia.
Kir Kolyshkin[info]k001 on January 5th, 2008 - 02:03 pm
Re: how about memory usage?
how you need twice ram more than xen for webhosting vps application.
is that true?


No, it is not.

In fact, OpenVZ requires less memory than Xen, which is shown, for example, in HP labs report comparing OpenVZ and Xen, see my blog posting about it here, and please read the full report itself.

The main influence of RAM usage is density, i.e. the more RAM you use for an application, the less applications you can run on a server. HP labs tell in their report Xen was able to run 4 instances of a particular application they tested, while OpenVZ was fine with at least 6 instances of that app (they haven't tried running more). This means that OpenVZ density is at least x1.5 times higher.

Speaking of the blog post that you refer to -- yes, it is a problem, but rather than OpenVZ problem, this is a problem of hosting service providers which either oversell (i.e. putting way too much VPSs on a single system) or do not understand various limits and guarantees provided by OpenVZ's user beancounters subsystem. To mitigate that, we wrote a lot of detailed documentation regarding OpenVZ UBC, see http://wiki.openvz.org/UBC. Note that we never state that privvmpages is RAM -- rather than that we explain each UBC parameter in great details.

So, yes, maybe OpenVZ UBC is a bit complex, and it certainly reqiuires a bit of expertise to operate. You see, it's like a car with a manual gearbox -- you have to learn when and how to change gears in order to drive efficiently.
(Anonymous) on July 21st, 2010 - 12:56 pm
Dated
This report is way out of date now. As events have evolved - Xen has become vastly superior to OpenVZ in almost every way possible. Also, who gives a fuck about disk performance when OpenVZ's memory performance is so horrible?
Kir Kolyshkin[info]k001 on July 21st, 2010 - 01:41 pm
Re: Dated
First, the entry you are replying to was posted back in 2007.

Second, if you want to say what you are saying, please back it up with figures, otherwise it's crap, oh, sorry, your own subjective opinion that can or can not have something in common with the real world situation.

Come on, give us the results of yours (or someone else's) performance comparison and we'll discuss it. You know, just saying that "A has become vastly superior to B in almost every way possible" doesn't really proves anything. And saying "B's memory performance is so horrible" should also be backed by some facts.

Of course you don't have to do anything from the above list if the sole purpose of your comments is to troll.