A few days after I wrote about the performance benchmark bag of tricks, EMC was the first to fire the first salvo at NetApp’s SPECSfs2008 world records on NFS IOPS.
EMC is obviously using all its ammo to deflate NetApp chest thumping act, with Storagezilla‘s blog. Mark Twomey, who is the alter ego of Storagezilla posted several observations about NetApp apparent use of disk short stroking to artificially boost its performance numbers. This puts NetApp against the wall, with Alex MacDonald (who incidentally is SNIA NFSv4 co-chairman) of the office of the CTO responding hard to Storagezilla’s observation.
The news of this appeared in The Register. Read all about it.
With no letting up, the article also mentioned EMC Isilon’s CTO, Rob Pegler, adding more fuel to the fire.
I spoke about short stroking as some of the tricks used to gain better numbers in benchmark. And I also mentioned that these numbers have little use to the real work and I would like to add that these numbers are just there for marketing reasons. So, for you readers out there, benchmark is really not big of a deal.
Have a great weekend!
A very interesting report surfaced in front of me today. It is Information Week’s IT Pro ranking of Data Deduplication vendors, just made available a few weeks ago, and it is the overview of the dedupe market so far.
It surveyed over 400 IT professionals from various industries with companies ranging from less than 50 employees to over 10,000 employees and revenues of less than USD5 million to USD1 billion. Overall, it had a good mix of respondents. But the results were quite interesting.
It surveyed 2 segments
- Overall performance – product reliability, product performance, acquisition costs, operations costs etc.
- Technical features – replication, VTL, encryption, iSCSI and FCoE support etc.
I wanted to sign off early tonight but an article in ComputerWorld caught my tired eyes. It was titled “EMC to put hardware into servers, VMs into storage” and after I read it, I couldn’t help but to juxtapose the articles with what I said earlier in my blogs, here and here.
It is very interesting to note that “EMC runs vSphere directly on the storage controllers and then uses vMotion to migrate VMs from application servers onto the storage array, ..” since the storage boxes have enough compute power to run Virtual Machines on the storage. Traditionally and widely accepted, VMs should be running on servers. Contrary to beliefs, EMC has already demonstrated this running of VMs capability on their VNX, Isilon and Symmetrix.
And soon, with EMC’s Project Lightning (announced at EMC World in May 2011), they will be introducing server side PCIe-based SSDs, ala Fusion-IO. This is different from the NetApp PAM/FlashCache PCIe-based card, which sits on their arrays, not on hosts or servers. And it is also very interesting to note that this EMC server-side PCIe Flash SSD card will become a bridge to EMC’s FAST (Fully Automated Storage Tiering) architecture, enabling it to place hot, warm and cold data strategically on different storage tiers of the applications on VMware’s VMs (now on either the server or the storage), perhaps using vMotion as a data mover on top of the “specialized” link created by the server-side EMC PCIe card.
This also blurs the line between the servers and storage and creates a virtual architecture between servers and storage, because what used to be distinct data border of the servers is now being melded into the EMC storage array, virtually.
2 red alerts are flagging in my brain right now.
- The “bridge” has just linked the server back to the storage, after years of talking about networked storage. The server is ONE again with the storage. Doesn’t that look to you like a server with plenty of storage? It has come a full cycle. But more interesting and what I am eager to see is what more is this “bridge” capable of when it comes to data management. vMotion might be the first of many new “protocol” breeds to enhance data management and mobility with this “bridge”. I am salivating right now of this massive potential.
- What else can EMC do with the VMware API? This capability I am writing right now is made possible by EMC tweaking VMware’s API to maximize much, much more. As the VMware vStorage API is continually being enhanced, the potential is again, very massive and could change the entire landscape of cloud computing and subsequently, the entire IT landscape. This is another Pavlov’s dog moment (see figures below as part of my satirical joke on myself)
Sorry, the diagram below is not related to what my blog entry is. Just my way of describing myself right now. 😉
I am extremely impressed with what EMC is doing. A lot of smarts and thinking go into this and this is definitely signs of things to come. The server and the storage are “merging again”. Think of it as Borg assimilation in Star Trek.
Resistance is futile!
I have been receiving a lot of email updates from Texas Memory Systems for many months now. I am a subscriber to their updates and Texas Memory System is the grand daddy of flash and DRAM-based storage systems. They are not cheap but they are blazingly fast.
Lately, more and more vendors have been coming out with all SSDs storage arrays. Startups such Pure Storage, Violin Memory and Nimbus Data Systems have been pushing the envelope selling all SSDs storage arrays. A few days ago, EMC also announced their all SSDs storage array. As quoted, the new EMC VNX5500-F utilizes 2.5-in, single-level cell (SLC) NAND flash drives to 10 times the performance of the hard-drive based VNX arrays. And that is important because EMC has just become one of the earliest big gorillas to jump into the band wagon.
But does it make sense? Can one justify to invest in an all SSDs storage array?
At this point, especially in this part of the world, I predict that not many IT managers are willing to put their head on the chopping board and invest in an all SSDs storage array. They would become guinea pigs for a very expensive exercise and the state of the economy is not helping. Therefore the automatic storage tiering (AST) might stick better than having an all SSDs storage array. The cautious and prudent approach is less risky as I have mentioned in a past blog.
I wrote about Pure Storage in a previous blog and the notion that SSDs will offer plenty of IOPS and throughput. If the performance gain translates into higher productivity and getting the job done quicker, then I am all for SSDs. In fact, given the extra performance numbers
There is no denying that the fact that the industry is moving towards SSDs and it makes sense. That day will come in the near future but not now for customers in these part of the world.
The writing’s on the wall and the relationship has been on the rocks since Mr. Black decided to take on 2 new wives (one in 2007 and one in 2010) and Miss Purple had a good run when things were hot.
Why Black and Purple? For a while within the local circle of EMC Malaysia, Dell’s EMC CLARiiONs were known as “Black” while EMC’s own CLARiiON was “Purple”. They were the colours of the bezels of each respective storage box. And the relationship, which Dell signed with EMC in 2003, was supposed to last 10 years but today, Dell has decided to end that relationship 2 years early. Here is one of the news at eWeek.com.
The “divorce” was inevitable. Gaps started showing up in the relationship when Dell acquired EqualLogic in 2007 and this relationship went to a point of no return when Dell started pursuing 3PAR back in 2010. Dell eventually lost 3PAR to HP and got Compellent instead. It was bound to happen, sooner or later.
Storage is becoming a very important strategy for Dell. As server virtualization grows, the demand for Dell servers wanes but storage demand kept growing. That is why it makes sense for Dell to have their own storage techonology. In addition to Compellent and EqualLogic, Dell has also acquired Exanet and Ocarina Networks in 2010.
It has been a good run for both companies, especially EMC, who was able to make use of Dell’s aggressive sales force to increase their market penetration for CLARiiON. And given the market dynamics, it is crucial that a company like Dell, with little innovation in the past, change their approach of reselling other people’s products and start owning and developing their own technology.
Europe is the worst hit region in this present economic crisis. We have seen countries such as Greece, Portugal and Ireland being some of the worst hit countries and Italy was just downgraded last week by S&P. Last week was also the release of the 2Q2011 External Disk Storage Systems figures from IDC and the poor economic sentiments are reflected in the IDC figures as well.
Overall, the factory revenue for Western Europe grew 6% compared to the year before, but declined 5% when compared to 1Q2011. As I was reading a summary of the report, 2 very interesting trends were clear.
- The high-end market of above USD250,000 AND the lower-end market of less than USD50,000 increased while the mid-end market of between USD50,000-100,000 price range declined
- Sentiments revealed that storage buyers are increasingly looking for platforms that are quick to deploy and easy to manage.
As older systems are refreshed, larger companies are definitely consolidating into larger, higher-end systems to support the consolidation of their businesses and operations. Fundamentals such as storage consolidation, centralized data protection, disaster recovery and server virtualization are likely to be the key initiatives by larger organization to cut operational costs and maximizing of storage economics. This has translated to the EMEA market spending more on the higher-end storage solutions from EMC, IBM, HDS and HP.
NetApp, which has been always very strong in the mid-end market, did well to increase their market share and factory revenue at IBM’s and HP’s expense because their sales were flattish. Dell, while transitioning from its partnership with EMC to its Dell Compellent boxes, was the worst hit.
The lower-end storage solution market, according to IDC figures, increased between 10-25% depending on the price ranges of USD5,000 to USD10,000 to USD15,000. This could mean a few things but the obvious call would be the economic situation of most Western European SMBs/SMEs. This could also mean that the mid-end market could be on the decline as many of the lower-end systems are good enough to do the job. One thing the economic crisis can teach us is to be very prudent with our spendings and I believe the Western European companies are taking the same path to control their costs and maximizing their investments.
The second trend was more interesting to me. The quote of “quick to deploy and easy to manage” is definitely pushing the market to react to more off-the-shelf and open components. From an HP stand point of their Converged Infrastructure, the x86 strategy for their storage solutions is making good sense, because I believe there will be lesser need for proprietary hardware from traditional storage vendors like EMC, NetApp and others (HP included). Likewise, having storage solutions such as VSA (Virtual Storage Appliance) and storage appliance software that runs on the x86 platforms such as Nexenta and Gluster could spell out the next wave in the storage networking industry. To have things easy, specialized appliances which I have spoke much of lately, hits the requirement of “quick to deploy and easy to manage” right on the dot.
The overall fundamentals of the external disk storage systems market remain strong. Below is the present standings in the EMEA market as reported by IDC.
I was reading the news from Oracle OpenWorld and a slew of news about specialized appliances are on the menu.
Oracle added Big Data Appliance and Oracle Exalytics Business Intelligence Machine to its previous numero uno, Exadata Database Machine. EMC, also announced its Green Plum Data Computing Appliance and also its VNX Unified Storage for Oracle.
The EMC VNX Unified Storage for Oracle is a VNX system that has Oracle installed in a VMware vSphere virtual machine environment. The system is meant to unify all Oracle environments--database over Oracle Direct NFS, application servers over NFS, and testing and development over NFS--resulting in less disk space used and faster testing. EMC says this configuration was made because 50% of Oracle customers are virtualizing their systems today. The VNX Unified Storage for Oracle includes EMC's Fully Automated Storage Tiering (FAST) technology, which migrates most frequently used data between a primary Fibre Channel drive and solid state drives and migrates less frequently used data to Serial ATA (SATA) drives and its FAST Cache. In an Oracle environment, FAST is well-suited to database applications that generate a large number of random inputs-outputs, that experience sudden bursts in user query activity, or a high number of user loads and where the entire working set can be contained in the solid state drive cache. Based on testing carried out on an Oracle Real Application Clusters (RAC) 11g database that was configured to access the VNX7500 file storage over the Network File System (NFS), using the Oracle Direct NFS (dNFS) client, results showed an 100% improvement in transactions per minute (TPM), 170% improvement in IOPS, and a 79% decrease in response time, the company said.
As for GreenPlum, EMC quoted:
The company also is showing off the EMC Greenplum Data Computing Appliance(DCA) for Big Data Analytics configuration, which provides a new migration path to Greenplum for Oracle Data Warehouse. This system includes the Greenplum Data Computing Appliance, EMC's Global Data Warehouse, and EMC's IT Business Intelligence Grid infrastructure. The EMC Greenplum DCA consists of 8 to 16 segment servers running Red Hat Enterprise Linux. Each segment server contains 96 to 192 processor cores, with 384 GB to 768 GB of memory per segment server. The DCA includes 12 600-GB Serial Attached SCSI (SAS) 15K RPM drives for a total useable and compressed capacity of 73 TB to 144 TB. The DCA competes with Oracle's Exadata Database Machine. In tests performed with this server/storage configuration and a 15-TB Oracle Data Warehouse, the DCA processed a 99 million rows query in less than 28 seconds vs. seven minutes in a traditional Oracle environment and data loads decreased from six days to 29 minutes
It is getting pretty obvious that specialized appliances are making waves at Oracle OpenWorld but what’s more interesting is the return of a combined and integrated environment of compute and storage as I have mentioned in my previous blog. And I forsee that these specialized appliances will be one of the building blocks of cloud computing together with general purposes platforms such as x86, JBODs and the glue to all these, virtualization, notably VMware.
News of Joe Tucci quitting EMC at the end of 2012 is abuzz tonight. Here’s one from The Register.
He is one of the longest serving CEO in the storage industry and since he took over the helm in 2001, he has brought EMC to where it is today. Like him or loathe him, you cannot deny that he is one of the best out there. Having gone thru at least 3 economic downturns, he has turned EMC into an industry giant, a juggernaut.
The next question is who will succeed him? There are many candidates from long-serving senior staff to the new ones that EMC has recruited in the recent years. It will only be end of 2012 when Joe finally leaves EMC but the search for his successor will be an interesting one. We shall soon know.
When VMware 5.0 was launched last month, I heard the feature called Virtual Storage Appliance (VSA) was finally out and is now being offered as an SMB/SME “storage” solution. In my mind, alarm bells were ringing because in its own stealthy manner, VMware had just become a storage player.
What VMware is offering is “Hey! If you don’t have money to buy your enterprise storage array, don’t worry. Make your own shared storage with our very own VMware VSA“. VSA utilizes the internal disks of the ESX/ESXi host as its shared storage.
VSA is nothing new. For years, LeftHand Networks had one for its engineers to do demo and show the functionality of their solution. EMC had it too, and recently I found out that NetApp has its own VSA, but only resell through its partner, Fujitsu. I am not 100% sure about the NetApp thing and I need a NetApp guy to verify this.
Smaller players, but not insignificant, such as Nutanix, Nexenta and Tintri are already offering their own versions and implementation of VSA to their customers, each with its own uniqueness and differences. With the release of the VMware VSA into the open, we shall see all the big storage players offering their VSAs to VMware, like natives offering sacrifices to VMware God. Or perhaps, it has already begun. It is ala-Nexus 1000v all over again.
VMware has become a huge juggernaut and it is merely using its advantage to consolidate the storage component under its control. When VMware version 4.0 came out, vStorage API was introduced along with VAAI (vStorage API for Array Integration). VAAI was created to enhance the storage experience by offloading specific storage operations to the native features of that supported storage platform. That’s all I know about VAAI at this moment, but with this feature, the storage array is tightly integrating its platform to VMware, or should I say … quietly ensnared by VMware tentacles of doom! (Evil laugh in the background! Mua ha ha ha ….!)
In the recently past VMworld, this storage story is slowly being unfurled even more to the world. VASA (vStorage API for Storage Awareness) was recently announced and EMC’s COO Pat Gelsinger spoke about the tighter integration (that word again!) that blurs the administration domain of the VMware admin and the storage admin. Below is a video of Pat Gelsinger talking about VASA below (this is long 55 minute video – Click only if you have the time).
Mind you, the entire vStorage API is still evolving as VMware 5.0 rolls out but here’s the thing. VMware has come out and say that the storage world about LUNs, RAID groups and mount points are a level below what the VMware admin should be concerned about. VMware admins handles their storage at the VM level or as VMDK and therefore, anything below it is of little significance to them. Again, you can see that VMware is using its muscle to say “If you guys want to play, you have to play by my rules“.
So, some new announcements came out from VMworld for storage such as Capacity Pools, I/O Multiplexer, and Storage DRS (Storage Distributed Resource Management) and also an enhanced version (probably more storage resilient) SRM (Site Recovery Manager). All these are being managed at a level above the traditional storage admin level and VMware has said that the VMware admin would be able to carve out a VM volume with its own set of default storage properties, defined snapshot retentions, replication and perhaps even compression and deduplication. But all these will be happening at the VM volume or VMDK level, not a level below that.
Details are still sketchy at this point in time and we probably won’t see these GA until probably VMware version 6.0. But the inertia has been rocked quietly and the VMware storage momentum will gain strength as time passes by. We could see that VMware would just need JBOD (just a bunch of disks) because it has its own enterprise storage features through its vStorage APIs or its future storage specifications. We have seen it happening in VSA with VMware offering its own storage.
From the similar news, what surprised me was what was quoted as shown below.
The presenters said VMware developed the APIs with EMC, NetApp, Dell, IBM and Hewlett-Packard,but they began the session with a disclaimer that none of those vendors has committed to support the APIs in their arrays.
Why the hell would EMC, NetApp, Dell, IBM and HP do something like that?!! Don’t they know that this could contribute to their insignificance in the future?
I am still perplexed but as the whole thing is still evolving, VMware seems to be only obvious winner here.
After the IDC report a couple of weeks back, Gartner released their Worldwide External Controller-Based (ECB) Disk Storage Market report last week. The Gartner reports mirrors the IDC report, which confirms the situation in the storage market, and it’s good news!
Asia Pacific and Latin America are 2 regions which are experiencing tremendous growth, with 27.9% and 22.4% respectively. This means that the demand of storage networking and data management professionals is greater than ever. I have always maintained that it is important for professionals like us to enhance our technical and technology know-how to ride on the storage growth momentum.
So from the report, there are no surprises. Below is a table to summarizes the Gartner report.
As you can see, HP lost market share together with Dell, Fujitsu and Oracle. Oracle is focusing its energies on its Exadata platform (and it’s all about driving more database license sales), and hence their 7000-series is suffering. Despite Fujitsu partnership with NetApp and EMC, and also with its Eternus storage, lost ground as well.
Dell seems to be losing ground too, but that could be the after effects of divorcing EMC after picking up Compellent early this year. Dell should be able to bounce back as there are reports stating that Compellent is picking up a good pace for Dell. One of the reports is here.
The biggest loser of the last quarter is HP. Even though it has a 0.3% of a market drop, things does not seem so rosy as I have been observing their integration of 3PAR since the purchase late last year. No doubt they are firing all cylinders, but 3PAR does not seem to be helping HP to gain market share (yet). The mid-tier has to be addressed as well and having the old-timer EVA at the helm is beginning to show split ends. Good for the hairdresser; not good for HP. IBRIX and LeftHand complete most of HP storage line-up.
HDS is gaining ground as their storage story is beginning to gel quite well. Coupled with some great moves consolidating their services business and also their Deal Operations Center (DOC) in Kuala Lumpur, simplifies the customers doing business with them. Every company has its challenges but I am beginning to see quite a bit of traction from HDS in the local business scene.
IBM also increased market share with a 0.2% jump. Rather tepid overall but I was informed by an IBMer that their DS8000s and XIVs are doing great in the South East Asia Region. Kudos but again IBM still has to transform its mid-tier DS4000/5000 business, which IBM OEMs the storage backend from NetApp Engenio.
EMC and NetApp are the 2 juggernauts. EMC has been king of the hill for many quarters, and I have been always surprised how nimble EMC is, despite being an 800 pound gorilla. NetApp has proven its critics wrong. For many quarters it has been taking market share and that is reflected in the Gartner Half Year Report below:
There you have it folks. The Gartner WW ECB Disk Storage Report. Again, I just want to mention that this is a wonderful opportunity for us doing storage and data management solutions. The demand is there for experienced and skilled professionals but we have to be good, really good to compete with the rest.