Posts Tagged ‘ performance ’

joshsimons

Hadoop Performance on vSphere 5.1

April 19, 2013
By
Hadoop Performance on vSphere 5.1

We’ve just published a third Hadoop performance paper, written by VMware performance expert Jeff Buell, which looks in detail at the relative performance of a bare-metal 32-node Hadoop cluster compared to a range of virtual clusters with up to 128 VMs. The executive summary is that while we saw a 13% performance degradation in a head-to-head comparison of a 32-node physical cluster against a 32-VM virtual cluster (one VM per host) running on the same hardware and running the same tests, virtualized performance can be increased significantly — to the point where virtualized Hadoop actually runs a bit faster than physical — by increasing the number of VMs per host. We’ve seen this effect before with Hadoop...

Read more

joshsimons

Paravirtual RDMA for Low Latency and Flexibility

December 27, 2012
By

The Office of the CTO has been exploring how to best enable application access to RDMA for those applications requiring the ultimate in high bandwidth, low-latency communication, which includes many HPC MPI applications as well as many scale-out databases and BigData approaches. Passthrough mode is the most straightforward way to enable guest-level RDMA. With passthrough (which we call VM DirectPath I/O), a physical PCI device can be made directly visible to the guest operating system running within the virtual machine. We published a research note showing that this approach delivers very good InfiniBand latencies (under 2us) and excellent bandwidths over a wide range of message sizes. There is a downside, however: Punching through the virtual machine abstraction in this way disables several...

Read more

joshsimons

2013 Predictions: When Worlds Collide

December 18, 2012
By
2013 Predictions: When Worlds Collide

It’s time again to dust off the crystal ball, cast the yarrow sticks, read the tea leaves, and share some thoughts about where the IT world is heading over the next twelve months and beyond. Because my focus at VMware is on High Performance Computing (HPC), I’ll confine my prognostications primarily to the growing area of overlap between mainstream Enterprise Computing and HPC — places where each world can benefit from the other in important ways. Before I start, let’s get one no-brainer prediction out of the way: 2012 will be remembered for Nicira , and not Nibiru . Yes, software-defined networking ( SDN ) will trump planetary collisions and the...

Read more

joshsimons

Centralizing HPC Compute Resources using Private Cloud

December 5, 2012
By

I met recently with a group of academic CIOs and IT Directors in Denver to talk about how virtualization and private cloud technologies can help lower the barriers to adopting a centralized approach to delivering HPC resources in an academic environment. Of course, the barriers and concerns — losing control of resources, fear of not getting ones “fair share” in a shared facility, being forced into a standardized and sub-optimal software environment, etc.  – are relevant outside of academia as well. Because the topic is of broad interest, I’ve published the slides in SlideRocket and added an audio track that explains why virtualization should not be viewed simply as a tax one pays to move into a cloud environment, but rather as an approach...

Read more

joshsimons

HPC Performance in the Cloud: Status and Future Prospects

October 1, 2012
By
HPC Performance in the Cloud: Status and Future Prospects

I spoke last week at ISC Cloud 2012 in Mannheim, Germany about the performance of HPC applications in the cloud, citing results from several studies. I have summarized the talk below and my PDF slide deck is available here . For full details of ISC Cloud 2012, I recommend the coverage at  HPC in the Cloud . I gave my talk to share some specific data about HPC performance in virtual environments, but I first described what new capabilities can be gained by virtualizing an HPC environment. I felt this was necessary because in HPC discussions it is often the case that...

Read more

joshsimons

High Performance Computing with vSphere 5.1

August 29, 2012
By
High Performance Computing with vSphere 5.1

We just announced a pile of new vSphere features at VMworld 2012 in San Francisco this week so I thought I’d take a few moments to describe several of those capabilities that will be of particular interest to High Performance Computing customers.

Read more

joshsimons

Virtualization, Italian Style

July 11, 2012
By

I’m in Fiuggi, Italy this week teaching a four-lecture course on Virtualization and High Peformance Computing as part of the Eighth International Summer School on Advanced Computer Architecture for High Performance and Embedded Systems ( ACACES 2012 ). I have about 70 students in the class, primarily from European countries.

Read more

joshsimons

The Case for IT Convergence

July 6, 2012
By

I gave a talk recently at the HPC Advisory  Council Workshop at the International Supercomputing Conference ( ISC ’12) in Hamburg. The main purpose of the presentation, which was titled Ultra-low Latency in the Cloud: How low can we go?,  was to share our InfiniBand performance results with the HPC community to begin to address the question of whether cloud computing can be useful for more demanding types of applications than the throughput-oriented technical workloads that are known to run well in a virtualized cloud environment. (The short answer is we believe it can.) Before I presented our performance results, I spent the first part of the talk explaining what I see as an...

Read more

joshsimons

Ultra-low latency on vSphere at VMworld 2012: Voice your Interest!

June 4, 2012
By

Low-latency communication is one area we’ve been exploring within the Office of the CTO as part of our efforts to enable new workloads to run well on the platform, including High Performance Computing (HPC) applications and scale-out Enterprise applications.  If you are interested in hearing about this topic at VMworld 2012, consider casting a vote for the two proposals listed below during the public voting period, which ends on June 8th. Vote here (free registration required). 1514 Ultra-low latency on vSphere: How low can we go? Low-latency applications are becoming more important in the Enterprise datacenter and for emerging workloads like High Performance Computing. As a response to this, the Office of the CTO is exploring several options to...

Read more

joshsimons

vSphere 5: vNUMA paper now available

October 27, 2011
By

I promised in the blog post vNUMA: What it is and why it matters to share the full research paper with you when it became available. It can now be downloaded here . If you find the paper useful, please click to give it a star rating on that same page. As a reminder, here is the paper abstract: A major obstacle to virtualizing HPC workloads is a concern about the performance loss due to virtualization. We will demonstrate that new features significantly enhance the performance and scalability of virtualized HPC workloads on VMware’s virtualization platform. Specifically, we will discuss VMware’s ESXi Server performance for virtual machines with up to 64 virtual CPUs as well as support for exposing virtual NUMA topology...

Read more

Where to Find Us

IPDPS 2013
May 20-24
Boston, MA
Josh Simons
vForum Germany
May 27-28
Frankfurt, Germany
Paul Strong
International Supercomputing Conference 2013
June 16-20
Leipzig, Germany
Josh Simons
VMworld US
August 26-29
San Francisco, CA
VMworld EMEA
October 15-17
Barcelona, Spain

Other VMware Blogs