Configuring EMC DD Boost with Veeam Availability Suite

This article provides a tour of the configuration steps required to integrate EMC Data Domain System with Veeam Availability Suite 9 as well as provides benefits of using EMC DD Boost for backup application.

Data Domain Boost (DD Boost) software provides advanced integration with backup and enterprise applications for increased performance and ease of use. DD Boost distributes parts of the deduplication process to the backup server or application clients, enabling client-side deduplication for faster, more efficient backup and recovery. All Data Domain systems can be configured as storage destinations for leading backup and archiving applications using NFS, CIFS, Boost, or VTL protocols.

The following applications work with a Data Domain system using the DD Boost interface: EMC Avamar, EMC NetWorker, Oracle RMAN, Quest vRanger, Symantec Veritas NetBackup (NBU), Veeam and Backup Exec. In this example, we will be using Veeam Availability Suite version 9.

Data Domain Systems for Service Provider

Data Domain Secure Multitenancy (SMT) is the simultaneous hosting by a service provider for more than one consumer (Tenant) or workload (Applications, Exchange, Standard VMs, Structured Data, Unstructured Data, Citrix VMs).

SMT provides the ability to securely isolate many users and workloads in a shared infrastructure, so that the activities of one Tenant are not apparent or visible to the other Tenants. A Tenant is a consumer (business unit, department, or customer) who maintains a persistent presence in a hosted environment.

Basic Configuration requirements are:

  • Enable SMT in the DD System
  • Role Based Access Control in DD Systems
  • Tenant Self-Service in the DD Systems
  • A Tenant is created on the DD Management Center and/or DD system.
  • A Tenant Unit is created on a DD system for the Tenant.
  • One or more MTrees are created to meet the storage requirements for the Tenant’s various types of backups.
  • The newly created MTrees are added to the Tenant Unit.
  • Backup applications are configured to send each backup to its configured Tenant Unit MTree.

Prerequisites:

  1. Backup Server

Physical Server- Fibre Channel or iSCSI

OR

Virtual Server- Fibre Channel with N-Port Virtualization or NPIV or Pass-through Storage or iSCSI

  1. Backup Software

Backup Application, DD Boost Library, DD Boost-over-FC Transport

  1. Storage Area Network

Fibre Channel or iSCSI

  1. Data Domain System

DD Boost Service

DD Boost-over-FC Server

SCSI Commands over FC

SCSI Processor Devices

  1. Virtual Infrastructure

Hyper-v Server cluster & System Center Virtual Machine Manager OR

VMware vCenter with vSphere Hosts

Designing DD Boost for resiliency & availability

The Data Domain System broadcast itself to the backup server using one or more path physically or virtually connected. The design of entire systems depend on the Data Domain sizing on how you connect Data Domain with backup server(s), how many backup jobs will be running, size of backup, de-duplication, data retention and frequency of data restore. A typical backup solution should include the following environment.

  • Backup server with 2 initiator HBA ports (A and B)
  • Data Domain System has 2 FC target endpoints (C and D)
  • Fibre Channel Fabric zoning is configured such that both initiator HBA ports can access both FC target endpoints
  • Data Domain system is configured with a SCSI target access group containing:
  • Both FC target endpoints on the Data Domain System
  • Dual Fabric for fail over and availability
  • Multiple physical and logical Ethernet for availability and fail over

Examples of Sizing

To calculate the maximum simultaneous connection to Data Domain Fibre Channel System (DFC) from all Backup servers. DFC device (D) is the number of devices to be advertised to the initiator of the backup server(s). Lets say we have 1 backup server, single data domain systems, the backup server is running 100 backup jobs.

DFC Device Count D= (minimum 2 X S)/128

J=1 Backup Server x 100 Backup Jobs=100

C= 1 (Single DD System)

S=JXC (100X1)=100

D=2*100/128 = 1.56 rounded up 2

Therefore, all DFC groups on the Data Domain system must be configured with 2 devices.

Step1: Preparing DD System

Step2: Managing system licenses

  1. Select Administration > Licenses> Click Add Licenses.
  2. On the License Window, type or paste the license keys. Type each key on its own line or separate each key by a space or comma (DD System Manager automatically places each key on a new line)
  3. Click Add. The added licenses display in the Added license list.

OR

  1. In System Manager, select Protocols > DD Boost > Settings. If the Status indicates that DD Boost is not licensed, click Add
  2. License and enter a valid license in the Add License Key dialog box.

Step3: Setting up CIFS Protocol

  1. On the DD System Manager Navigation>click Protocols > CIFS.
  2. In the CIFS Status area, click Enable.

Step4: Remove Anonymous Log on

  1. Select Protocols > CIFS > Configuration.
  2. In the Options area, click Configure Options.
  3. To restrict anonymous connections, click the checkbox of the Enable option in the

Step4: Restrict Anonymous Connections area.

  1. In the Log Level area, click the drop-down list to select the level number 1.
  2. In the Server Signing area, select Enabled to enable server signing

Step5: Specifying DD Boost user names

The following user will be used to connect to DD boost from backup software.

  1. Select Protocols > DD Boost.
  2. Select Add, above the Users with DD Boost Access list.
  3. On the Add User dialog appears. To select an existing user, select the user name in the drop-down list. EMC recommends that you select a user name with management role privileges set to none.
  4. To create and select a new user, select Create a new Local User and Enter the password twice in the appropriate fields. Click Add.

Step6: Enabling DD Boost

  1. Select Protocols > DD Boost > Settings.
  2. Click Enable in the DD Boost Status area.
  3. Select an existing user name from the menu then complete the wizard.

Step7: Creating a storage unit

  1. Select Protocols > DD Boost > Storage Units.
  2. Click Create. The Create Storage Unit dialog box is displayed.
  3. Enter the storage unit name in the Name box e.g. DailyRepository1
  4. Select an existing username that will have access to this storage unit. EMC recommends that you select a username with management role privileges set to none. The user must be configured in the backup application to connect to the Data Domain system.
  5. To set storage space restrictions to prevent a storage unit from consuming excess space: enter either a soft or hard limit quota setting, or both a hard and soft limit.
  6. Click Create.
  7. Repeat the above steps for MonthlyRepository1 each Data Domain Boost-enabled system.

Step8: Encrypting Communication between Backup Server and Data Domain (Optional)

Generate an advanced certificate from Active Directory Certificate services and install into the Data Domain DD Boost. You must install the same certificate into the backup servers so that both data domain and data domain client which is backup server can talk to each via encrypted certificate.

  1. Start DD System Manager on the system to which you want to add a host certificate.
  2. Select Protocols > DD Boost > More Tasks > Manage Certificates….
  3. In the Host Certificate area, click Add.
  4. To add a host certificate enclosed in a .p12 file, Select I want to upload the certificate as a .p12 file. Type the password in the Password box.
  5. Click Browse and select the host certificate file to upload to the system.
  6. Click Add.
  7. To add a host certificate enclosed in a .pem file, Select I want to upload the public key as .pem file and use a generated private key. And Click Browse and select the host certificate file to upload to the system.
  8. Click Add.

DD Boost client access and encryption

  1. Select Protocols > DD Boost > Settings.
  2. In the Allowed Clients section, click Create. The Add Allowed Client dialog appears.
  3. Enter the hostname of the client. This can be a fully-qualified domain name (e.g. Backupserver1.domain.com) or a hostname with a wildcard (e.g. *.domain.com).
  4. Select the Encryption Strength. The options are None (no encryption), Medium (AES128-SHA1), or High (AES256-SHA1).
  5. Select the Authentication Mode. The options are One Way, Two Way.
  6. Click OK.

Step9:Configuring DD Boost over Fibre Channel

  1. Select Protocols > DD Boost > Fibre Channel.
  2. Click Enable to enable Fibre Channel transport.
  3. To change the DD Boost Fibre Channel server name from the default (hostname), click Edit, enter a new server name, and click OK.
  1. Select Protocols > DD Boost > Storage Units to create a storage unit (if not already

created by the application).

  1. Install the DD Boost API/plug-in (if necessary, based on the application).

Step10: Configuring storage for DD Extended Retention (Optional)

Before you proceed with Extended Retention you must add required license on the DD System.

  1. Select Hardware > Storage tab.
  2. In the Overview tab, select Configure Storage. In the Configure Storage tab, select the storage to be added from the Available Storage list.
  3. Select the appropriate Tier Configuration (or Active or Retention) from the menu.
  4. Select the checkbox for the Shelf to be added.
  5. Click the Add to Tier button. Click OK to add the storage.

Step11: Configure a Veeam backup repository

  1. To create an EMC Data Domain Boost-enabled backup repository, navigate to the Backup Infrastructure section of the user interface, then select Backup Repositories and right-click to select Add Backup Repository.

DDBoost

  1. The next step is to select the repository type, De-duplicating storage appliance. Type the Name of the DD Systems, Choose Fibre Channel or Ethernet Option, add credentials to connect to DD System and Gateway to connect to DD System. To be able to connect Veeam Backup server to the DD System using Fibre Channel you must add DD System & Veeam Backup server in the same SAN zone. You also need to enable FC on the DD System. To be able to connect Veeam Backup Server using Ethernet Veeam backup Server and DD System must be in same VLAN or for multi-VLAN you must enable unrestricted communication between VLANs.
  2. On the next screen, select the Storage Unit of the DD System to be used by the Veeam Server as repository, leave concurrent connection as default
  3. On the Next screen, enable vPower NFS, complete the wizard

Step12: Configure Veeam Backup Job & Backup Copy Job

The critical decision on backup jobs will be whether to do an active full backup or leverage synthetic full backups. Veeam Backup Job Creation GuideVeeam Backup Copy Job Creation Guide

Here is short business case of backup type.

Veeam Backup Options:

  1. Active Full- Financial or health sector prefer to keep a monthly full backup of data and retain certain period of time for corporate compliance and satisfying external auditor’s  requirement to keep data off-site for a period of time.
  2. Synthetic Full- A standard practice to keep synthetic full at all time to reduce storage cost and recovery time objective for any organization.

Sythetic

  • For most environments, Veeam recommends to do synthetic full backups when leveraging EMC Data Domain Boost. This will save stress on primary storage for the vSphere and Hyper-V VMs and the Boost-enabled synthesizing is very fast.
  • For a Backup Copy job using GFS retention (Monthly, Weekly, Quarterly and/or Annual restore points), the gateway server must be closest to the Data Domain server, since the Backup Copy job frequently involves an offsite transfer. When the Data Domain server is designated in the repository setup, ensure that consideration is given to the gateway server if it is being used off site.
  • Backup job timed out value must be higher than 30 minutes to be able to retry the job if it is to fail for any reason

DD System Option:

  • A virtual synthetic full backup is the combination of the last full (synthetic or full) backup and all subsequent incremental backups. Virtual synthetics are enabled by default.
  • The synthetic full backups are faster when Data Domain Boost is enabled for a repository
  • DD Boost reduces backup transformation time by less than 80% of total time if DD Boost was not used.
  • The first job has the bulk of the blocks of the vSphere or Hyper-V VM on the DD Boost Storage Unit, it will only need to transfer metadata and any possible changed blocks. This can be a significant improvement on the active full backup process when there is a fast source storage resource in place.
  • With DD Boost, multi-link provides fail over & resiliency. DD Boost also provides parallel processing of concurrent jobs to DD Boost Storage unit.
  1. To display the DD Boost option settings, select Protocols > DD Boost > Settings >Advanced Options.
  2. To change the settings, select More Tasks > Set Options. Select or deselect any option to be enabled.
  3. Click OK.

Dell Compellent: A Poor Man’s SAN

I have been deploying Storage Area Network for almost ten years in my 16 years Information Technology career. I have deployed various traditional, software defined and converged SANs manufactured by a global vendor like IBM, EMC, NetApp, HP, Dell, etc. I was tasked with the deployment of Dell Compellent in my previous role for several clients. I was excited about the opportunities and paused after reading the documentation presented to me. I could not co-relate implementation of a SAN and expected outcome desired by clients. When over wild sales pitch is sold to businesses with high promises, then there will always be hidden risks that come with this sales pitch. Lesson number one never trusts someone blindly although they have a very decent track record, resellers are often after a quick sale and get out. Lesson number two make sure you know who to trust as your partner in the transition to have a new SAN. Decide what to procure based on your business case, ROI, workload analysis, capacity planning, lession learnt and outcome of requirement analysis. Consider current technology trend, where you are at now, a technology road map and where you want to be in future, e.g. Google Cloud, AWS or Azure. Capital investment can be the one off exercise these days before you pull the plug off on the on-premises infrastructure and fork-lift to Azure, Amazon or Google Cloud. Consider aligning technology stream with the business you do. I have written this article to share my own experience and disclose everything I learnt through my engagement on Dell Compellent deployment projects so that you can make a call by yourself. I will elaborate each feature of Dell Compellent and what exactly this feature does when you deploy a Compellent. FYI I have no beef with Dell. Let’s start now… “Marketing/sales pitch” vs “practical implication.”

Target Market: Small Business

Lets not go into detail, that will be a different topic for another day. Please read Dell’s business proposition “Ideally suited to smaller deployments across a variety of workloads, the SC Series products are easy to use and value optimized. We will continue to optimize the SC Series for value and server-attach.”

Management Interface: Dell Compellent Storage Center has a GUI designed to be accessible allegedly ease of use. Wizards offer few everyday tasks such as allocation, configuration, and administration functions. Compellent Storage Center monitoring tools provide very little insight on how storage backend is doing. You have to engage Dell remote support for diagnostic, and monitoring tools with alert and notification services. Storage center is not as granular as the competitor NetApp and EMC. Storage center has little information on storage performance, bottle neck and backend storage issues. Compellent is by design thin provisioned storage. There is no option in management center to assign as thick provisioned volume. IOPS and latency are calculated in volume and IOPS and latency are calculated in disks are far too different than real IOPS. You may see little IOPS in volume but click at drive level IOPS you will see storage controller is struggling to cope with the IOPS. Management center does not provide any clues who is generating this much IOPS.

Contact technical support they will say RAID scrub is killing your storage. Your standard request to tech support that stops the RAID scrub in a business hour. “You cannot do it” another classic reply by tech support. If you go through Compellent management center, you will find nothing that can schedule or stop RAID scrub.

Data Progression: In theory, Data Progression is an automated tiering technology that should have optimised the location of data, both on a schedule and on demand as prompted by a storage profile. Compellent’s tiering profiles streamline policy administration by assigning tier attributes based on the profile. On-demand data progression in a business hour will drive Compellent into crazy. If you are Citrix VDI mainstream than your workload is pretty much dead until data progression is complete.

A side effect of this technology is storage controller struggle to maintain on demand data progression and IO request at the same time hence there will be queue depth, and longer seek time in backend storage. In this situation, storage seek time is higher than normal.

Storage Profile: Storage profile in lay man’s terms is segregating expensive and cheap disk and profiling them in tier 1 (SSD RAID 10), tier 2 (15K Fibre Channel RAID 10, RAID 5, RAID 6) and tier 3 (7.2K SATA RAID 5, RAID 6). The storage profile determines how the system reads and writes data to disk for each volume as they are known in Compellent terms and how the data ages over time a feature called Data Progression. For example, random read request goes to tier 1 where you kept hot data, and a year old emails go to tier 3.

Storage Profiles supposed to allow the administrator to manage both writable blocks and replay blocks for a volume. It is fundamentally a tiering of storage in a controlled way. In theory, it supposed to be in a controlled environment. However, in reality, it does add extra workload to Dell Compellent controller. Let’s say you have tiered your storage according to your read and write intense IO. What happens when to READ and WRITE intense volume gets full?. Storage controller automatically triggers an on demand data progression from upper tier to lower tier to store data. Hence a WRITE intense IO is generated in lower tier what you wanted to avoid in the first place that’s why you profiled or tiered your storage. Mixing data progression with storage tiering defeats whole purpose of storage profiling.

Compellent Replay: Replay is essentially a storage snapshot in Dell terms. Dell Compellent Data Instant Replay software creates point-in-time copies called Replays. With Data Instant Replay Dell Compellent storage Replays at any time interval with minimal storage capacity. But here is the catch you will be most likely to run storage replay during the daily backup window. Backup generates lots of READ IOPS and Replays generate lots of READ and WRITE IOPS at the same time which is a daily backup window. Hence your backup is going to be dead slow. You will run out of the backup window and never be going to finish backup before the business hours. It will be a nightmare to fulfil data retention SLA and restore of any file systems and sensitive applications.

IOPS & Latency: Input/Output per second is a measurement unit of any hard disk and storage area network (SAN). This is a key performance matrix of a SAN regardless of manufacture, and this matrix remains unchanged. If you are to measure a SAN, this is where you begin. Never think that you have a bounce of virtual machines and it’s okay to buy SAN without IOPS consideration. There is the difference between a virtualised DHCP server and virtualised SQL server. A DHCP server may generate 20 IOPS but a SQL server can generate 5000 IOPS depends on what you are running on that SQL server. Every query you send to a SQL server or the application depends on the SQL server generate IOPS both read and write IOPS. For a Citrix XenApp and XenDesktop customer, you have to take into consideration that every time you launch a VDI session, open an word document, you generate IOPS, once you click save button on a word document, you generate write IOPS. Now you multiply the IOPS of each VDI session by the number of users, number of applications, number VDI and users inputs to estimate your real IOPS.

Now think about latency, in plain English, latency is the number of seconds or milli seconds you wait to retrieve information from a hard disk drive. This is calculated in round-trip between your request and the hard disk serve your request. Now you think millions of requests are bombarded on the storage area network. A SAN must sustain those requests and serve application requests, again it depends on what sort of workload you are running on a SAN. For example, file servers, Citrix profile, Citrix VDI, Exchange Server and SQL servers need low latency SAN.

In Dell Compellent, you may see volume IOPS e.g. 2000 but if you view disks hosting the same volume, then you might see 5000 IOPS. Then you must ask question how-come 5000-2000=3000 IOPS are generated automatically. Does Compellent has any tools to interrogate storage controller to see how additional workloads are generated? No it doesn’t. Your only bet is Dell support telling you the truth if you are lucky. The answer is automated RAID scrub is generating extra workloads on storage i.e. 3000 IOPS which could have been utilized for real workloads.

To co-relate this analysis with an all flash array storage, e.g. Dell Compellent, the SAN must be able to offer you the major benefits of a storage area network. If this storage cannot provide you low latency and high IO throughput for sensitive applications and workloads then you need to go back to drawing board or hire a consultant who can analyse your requirements and recommend you the options that match your need and budget. For further reading find Citrix validated solutions, storage best practices recommended by VMware and Microsoft. There are many tooling available in the market for you to analyse workload on applications, on a virtual or a physical infrastructure.

RAID Scrub: Data scrubbing is an error correction technique that uses a background task to inspect storage for errors periodically, and then correct detected errors using redundant data in the form of different checksums or copies of data. Data scrubbing reduces the likelihood that single correctable errors will accumulate, leading to reduced risks of uncorrectable errors.

In NetApp, you can schedule a RAID Scrub that suits your time and necessity however in Dell Compellent you cannot schedule a RAID Scrub through GUI or Command line. Dell technical support advised that this is an automated process takes places every day to correct RAID groups in Dell Compellent. There is a major side effect running automatic RAID scrub. RAID scrub will drive your storage to insane IOPS level, and latency will peak to high causing production volume to suffer and under perform. Performance of virtualisation will be degraded so badly that production environment will struggle to serve IO request. Dell advised that Dell can do nothing about RAID scrub because RAID scrub in SCOS operating systems is an automated process.

Compellent Multipathing: By implementing MPIO solution you eliminate any single point of failure in any physical path (s) and logical path(s) among any components such as adapters, cables, fabric switches, servers and storage. If one or more of these elements fails, causing the path to fail, multipathing logic uses an alternate path for I/O so that applications can still access their data. Each network interface card (in the iSCSI case) or HBA should be connected by using redundant switch infrastructures to provide continued access to storage in the event of a failure in a storage fabric component. This is the fundamental concept of any storage area network AKA SAN.

New generation SANs are integrated with multipath I/O (MPIO) support. Both Microsoft and VMware virtualisation architecture supports iSCSI, Fibre Channel and serial attached storage (SAS) SAN connectivity by establishing multiple sessions or connections to the storage array. Failover times may vary by storage vendor, and can be configured various way but the logic of MPIO remains unchanged.

New MPIO features in Windows Server include a Device Specific Module (DSM) designed to work with storage arrays that support the asymmetric logical unit access (ALUA) controller model (as defined in SPC-3), as well as storage arrays that follow the Active/Active controller model.

The Microsoft DSM provides the following load balancing policies. Microsoft load balance policies are generally dependent on the controller design (ALUA or true Active/Active) of the storage array attached to Windows-based computers.

Failover
Failback
Round-robin
Round-robin with a subset of paths
Dynamic Least Queue Depth
Weighted Path

VMware based systems also provide Fixed Path, Most Recently Used (MRU) and Round-Robin Configuration which is the most optimum configuration for VMware virtual infrastructure.

To explain ALUA in simple terms is that Server can see any LUN via both storage processors or Controller or NAS Head as active but only one of these storage processors “owns” the LUN. Both Storage Processor can view logical activities of storage using physical connection either via SAN switch to the server or via direct SAS cable connections. Hyper-v or vSphere ESXi server knows which processor owns which LUNs and sends traffic preferably directly to the owner. In case of controller or processor or NAS Head Failure Hyper-v or vSphere server automatically send traffic to an active processor without loss of any productivity. This is an essential feature of EMC, NetApp and HP products.

Let’s look at Dell Compellent now. Dell Compellent does not offer true Active/Active Controllers for any Storage. Dell Controllers Explained! Dell Verified Answer. Reference from Dell Forum….

“In the Compellent Architecture, both controllers are active. Failover is done at either the port or controller level depending on how the system was installed. Volumes are “owned” by a particular controller for mapping to servers. Changing the owning controller can be done – but it does take a volume down.”

I can confirm that this is exactly Dell Customer support advised me when I called them. Dell Compellent can take up to 60~90 seconds to failover from one controller to another. Which means entire virtual environment will go offline for a while and get back online. To update firmware or to replace a controller you have to bring everything down then bring everything back online which will cause a major outage and productivity loss for the entire organisation.

Multipathing options supported by the Host Utilities

Multipath I/O Overview

Multipathing Considerations

Dell Compellent is not an ALUA Storage Array

Performance Issue:  To identify Dell Compellent bottleneck for a virtualisation platform hosted in Compellent. Run the following in Windows perfmon in a virtual machine or a physical machine where a volume of Compellent storage is presented via HBA or iSCSI initiator. Use Windows perfmon, create a data collector set of the below attributes and generate a report using PAL tools. Extract seek time, latency, IOPS and queue depth in the Compellent storage. You will see a bottleneck in every area of storage you can expect. Read further on Windows Performance Monitoring Tools

\LogicalDisk\Avg. Disk Sec/Read

\LogicalDisk\Avg. Disk Sec/Write

\LogicalDisk\Disk Bytes/Sec

\LogicalDisk\Disk Reads/Sec

\LogicalDisk\Disk Writes/Sec

\LogicalDisk\Split IO/sec

\LogicalDisk\Disk Transfers/sec

Use the following Tools to analyse workloads and storage performance in your storage area network: 

Capacity planning & workload analysis tools

Multi-vendor storage performance and capacity monitoring

RVTools 

Windows Perfmon

PAL Analaysis Tools

Storage load generator / performance test tool

Dell EqualLogic Storage Management Pack Suite for SCOM

Monitoring EMC storage using SCOM 2012 SP1 with ESI Management Packs

IBM Storage Management Pack for Microsoft System Center Operations Manager

Cost Compare:

The cost of each gigabyte of storage is declining rapidly in every segment of the market. Enterprise storage today costs what desktop storage did less than a decade ago. So why are your overall costs increasing when buying storage? Let’s make it simple! Ask yourself questions?

How much will the storage cost? How much will the SAN cost to implement? How much will the SAN cost to operate? Now use the below tools to calculate the real cost of the owing black box?

Amazon EBS Pricing

Google Cloud Platform Pricing Calculator

Azure Storage Pricing

IBM TCO Calculator

vSAN Hybrid TCO and Sizing Calculator

HPE Business Value Calculator

Microsoft TCO Calculator

So what will you be looking in a SAN? 

  • Lower TCO
  • Storage Performance
  • Scale-to-Fit
  • Quality of Service
  • Uncompromised Availability and uptime
  • Cloud Ready
  • Reduction of Data (de-duplication)
  • Reduction of backup
  • Analytics and automation
  • Reduction of Data Centre footprint 

Summary: Dell Compellent makes a compelling argument for all-flash performance tiers. Yes, this argument is in sales pitch not in reality. A price conscious poor man who needs just any SAN and has a lower IO environment can have Compellent. For mainstream enterprise storage, Dell Compellent is a bad experience and can bring disaster to corporate Storage Area Network (SAN).

I had no doubt when Compellent introduced all flash arrays it was innovative but Compellent’s best days are gone. Just shop around, you will find better all-flash, converged, hybrid and virtual arrays which are built on better software, controllers and SSDs. There are flash arrays in the market which run clever codes and algorithm within the software to produce high IO, low latency and performance for sensitive applications.

Related Articles: 

EMC Unity Hybrid Storage for Azure Cloud Integration

Pro Tips For Storage Performance Testing

Storage Top 10 Best Practices

SQLIO download page

SQLIOSim tool KB article

SQL Server Storage Engine Team Blog

SQL Server I/O Basics

SQL Server I/O Basics – Chapter 2

Predeployment I/O Best Practices

Disk Partition Alignment Best Practices for SQL Server

EMC Symmetrix DMX-4 Enterprise Flash Drives with Microsoft SQL Server Databases

Implementing EMC CLARiiON CX4 with Enterprise Flash Drives for Microsoft SQL Server 2008 Databases

Microsoft. “Lab Validation Report: Microsoft Windows Server 2012 with Hyper-V and Exchange 2013

Microsoft TechNet. “Exchange 2013 Virtualization.”

Microsoft TechNet. “Failover Clustering Hardware Requirements and Storage Options.” Aug 2012.

Dell Compellent Storage to be discontinued after Dell-EMC merger

DELL is buying EMC. This is an old news. You already know this. There are many business reasons Dell is buying EMC. EMC is the number one storage vendor and a big cat of NASDAQ. One key business justification is to get into Enterprise market with enterprise class product lines and second big reason is break into cloud market utilising dominant presence of EMC. Have you rationalised your opinion on what Dell storage product line likely to be once merger is complete. There are many argument in for and against of various combination of storage line Dell will come up. Let’s look at current product lines of Dell and EMC.

Dell Current Product Line:

  • Network Attached storage based on Dell 2U rack servers.
  • Direct Attached Storage
  • iSCSI and FCoE SAN solution such as PowerVault MD, EqualLogic, Compellent

EMC Product Line:

  • EMC XtremIO – the XtremIO all-flash array—ideal for virtual desktop infrastructure (VDI), virtual server, and database
  • EMC VMAX enterprise class storage- Mission critical storage for hyper consolidation and delivering IT as a service.
  • EMC VNX/VNXe – hybrid flash storage platform, optimized for virtual applications.

Software Defined Storage

Dell

  • Software defined storage such as Dell XC Series powered by Nutanix

EMC

  • EMC Isilon – High-performance, clustered network-attached storage (NAS) that scales to your performance and capacity requirements.
  • EMC ScaleIO – Hyper-converged solution that uses your existing servers, and turns them into a software defined SAN with massive scalability, 10X better performance and 60% lower cost than traditional storage.
  • EMC Elastic Cloud Storage (ECS) – Cloud-scale, geo-federated namespace, multi-tenant security and in-place analytics ECS is the universal platform
  • EMC ViPR Controller – deliver automation and management insights from your multivendor storage.
  • EMC Service Assurance Suite – Service Assurance Suite delivers service-aware software defined network management that optimizes your physical and virtual networks, increases operational efficiency by ensuring SLA’s, and reduces cost by maximizing resources.
  • EMC ViPR SRM – optimize your multivendor block, file & object storage tiers to application service levels you’ll maximize resources, reduce costs and improve your return on investment.

Other Partnership and Products of EMC

EMC Vblock Systems – VCE is a technology partnership which EMC plays major role to deliver converged cloud solutions for midrange to enterprise client. Converged Infrastructure technology that provide virtualization, server, network, storage and backup, VCE Converged Solutions simplify all aspects of IT.

EMC Hybrid Cloud- Federation Enterprise Hybrid Cloud for delivering IT-as-a-Service. With thousands of engineering hours, the Federation brings together best-in-class components from EMC, VCE, and VMware to create a fully integrated, enterprise-ready solution.

VMware Partnerships-EMC Corp plans to keep its majority stake in VMware Inc. EMC, which owns about 80% of VMware, bought the company in 2004 for $700 million. VMware accounted for about 22 percent of EMC’s revenue of $23.2 billion in 2013. EMC and VMware share a cloud vision. Through joint product development, solutions, and services, EMC is the number one choice for VMware customers for storage, backup, security, and management solutions.

RSA Information Security division- RSA info security offers data protection and identity management.

Pivotal- EMC and VMware partnership to manufacture software and big data solutions.

Virtustream- EMC and VMware joint $1.2B acquisition of this brand to provide public cloud services.

Dell to discontinue Compellent after merger–make sense

There are too many eggs on the busket already. Would Dell continue to sell identical products in different name or stream line all products. Dell is after streamlining all products. It is well known by Dell loyal customer that EqualLogic will disappear from Dell product line at 2018. We learnt that in Dell partner’s conference. Then question will remain what will happen to Compellent? In current market place, VNX directly compete with Dell Compellent. But VNX has more customer base than Compellent. VNX is in market for almost 20 years and still growing fast. Compellent is in new shape and in market with SC series product line. But Compellent has frustrated its customer with poor performance and poor customer support. A very poor explanation and guidelines from Dell presales team on how to align Dell storage with business requirement.

Dell can offer customer will both VNX and Compellent knowing Compellent did not work from the beginning or Dell streamline its product and kill Compellent all together. Then promote VNX as it worked past 20 years and has a proven track record. Killing Compellent will disturb few already unhappy customer who simply wanted cheap SATA disks. But killing VNX will disturb wide range of customers and annoy them once and for all. Consequence of that would be losing customers to HP and NetApp which dell desperately wants to avoid and gain control of storage market. This way Dell-EMC will retain undisputed title of EMC as a number one storage vendor. This make sense for any non IT savvy walking on the street. I am certain and believe that Dell will discontinue Compellent serries all together.  Protecting $67 billion dollar acquisition of EMC is more important than protecting $960 million acquisition of Compellent. It would obviously make sense for Michael Dell to kill Compellent and promote VNX as a sole mid range storage.

References:

Dell Compellent: A Poor Man’s SAN

EMC plans to keep stake in VMware, despite investor pressure: source

EMC-VMware Alliance

EMC Cloud

Dell Product Lines

EMC Product Lines

VCE Converged Systems

Dell buying EMC

Dell-EMC: What Storage Customers Should Do

Gartner’s verdict on mid-range and enterprise class storage arrays

Previously I wrote an article on how to select a SAN based on your requirement. Let’s learn what Gartner’s verdict on Storage. Gartner scores storage arrays in mid-range and enterprise class storage. Here are details of Gartner score.

Mid-Range Storage

Mid-range storage arrays are scored on manageability, Reliability and Availability (RAS), performance, snapshot and replication, scalability, the ecosystem, multi-tenancy and security, and storage efficiency.

mid1

Figure: Product Rating

mid2

Figure: Storage Capabilities

mid3

Figure: Product Capabilities

mid4

Figure: Total Score

Enterprise Class Storage

Enterprise class storage is scored on performance, reliability, scalability, serviceability, manageability, RAS, snapshot and replication, ecosystem, multi-tenancy, security, and storage efficiency. Vendor reputation are more important in this criteria. Product types are clustered, scale-out, scale-up, high-end (monolithic) arrays and federated architectures. EMC, Hitachi, HP, Huawei, Fujitsu, DDN, and Oracle arrays can all cluster across more than 2 controllers. These vendors are providing functionality, performance, RAS and scalability to be considered in this class.

ENT1

Figure: Product Ratings (Source: Gartner)

Where does Dell Compellent Stand?

There are known disadvantages in Dell Compellent storage array, users with more than two nodes must carefully plan firmware upgrades during a time of low I/O activity or during periods of planned downtime. Even though Dell Compellent advertised as flash cached, Read SSD and Write SSD with storage tiering, snapshot. In realty Dell Compellent does its own thing in background which most customer isn’t aware of. Dell Compellent run RAID scrub every day whether you like it or not which generate huge IOPS in all tiered arrays which are both SSD and SATA disks. You will experience poor IO performance during RAID scrub. When Write SSD is full Compellent controller automatically trigger an on demand storage tiering during business hour and forcing data to be written permanently in tier 3 disks which will literally kill virtualization, VDI and file systems. Storage tiering and RAID scrub will send storage latency off the roof. If you are big virtualization and VDI shop than you are left with no option but to experience this poor performance and let RAID scrub and tiering finish at snail pace. If you have terabytes of data to be backed up every night you will experience extended backup window, un-achievable RPO and RTO regardless of change block tracking (CBT) enabled in backup products.

If you are one of Compellent customer wondering why Garner didn’t include Dell Compellent in Enterprise class. Now you know why Dell Compellent is excluded in enterprise class matrix as Dell Compellent doesn’t fit into the functionality and capability requirement to be considered as enterprise class. There is another factor that may worry existing Dell EqualLogic customer as there is no direct migration path and upgrade path have been communicated with on premises storage customers once OEM relationship between Dell and EMC ends. Dell pro-support and partner channel confirms that Dell will no longer sell SAS drive which means IO intense array will lose storage performance. These situations put users of co-branded Dell:EMC CX systems in the difficult position of having to choose between changing changing storage system technologies or changing storage vendor all together.