From the Blogosphere

Cloud Computing and Platform-Based Vulnerabilities

Sat, 11 Feb 2012 05:00:00 EST

What do these two vulnerabilities have in common? Apache Killer. Post of Doom. Right, they’re platform-based vulnerabilities. Meaning they are vulnerabilities peculiar to the web or application server platform upon which applications are deployed. Mitigations for such vulnerabilities generally point to changes in configuration of the platform – limit post size, header value sizes, turn off some value in the associated configuration. But they also have something else in common – risk. And not just risk in general, but risk to cloud providers whose primary value is in offering not just a virtual server but an entire, pre-integrated and pre-configured application deployment stack. Think LAMP, as an example, and providers like Microsoft (Azure) and VMware (CloudFoundry), more commonly adopting the moniker of PaaS. It’s an operational dream to have a virtual server pre-configured and ready to go with the exact application deployment stack needed and offers a great deal of value in terms of efficiency and overall operational investment, but it is – or should be – a security professional’s nightmare. It’s not unlike the recent recall of Chevy Volts – a defect in the platform needs to be mitigated. The only way to do it, for car owners, is to effectively shut down their ability to drive while a patch is applied. It’s disruptive, it’s expensive (you still have to get to work, after all), and it’s frustrating for the consumer. For the provider, it’s bad PR and negatively impacts the brand. Neither of which is appealing.

Who's Managing Your PaaS Apps?

Fri, 10 Feb 2012 09:00:00 EST

PaaS v2.0 should be more open than the current implementations, and cultivate tools communities. But the focus on open development stacks is ignoring the second aspect of PaaS - the management of live applications after they are built. PaaS providers need to allow for communication of SLA and business process requirements by consumers, and cloud management tools can help. We are also seeing the realization of the application-centric approach to cloud management. That takes me to the part that no one is really talking about when it comes to PaaS - the management aspect. Up to this point, we have more often talked about the development side (pre-production). But, PaaS then hosts the application created. How will organizations dictate the SLAs to be in place when the application is live for others to consume (i.e. scale-up, scale-down, etc.)? How will they relay their DR requirements, or moreover how will PaaS providers allow for different RTO/RPO strategies?

Getting Ready for Big Data

Fri, 10 Feb 2012 07:30:00 EST

IT departments and data centers are used to seeing demand for resources surge. In recent years, this has been especially evident in the area of data storage. No matter what you want to call it – “data explosion,” or something else – you can’t deny the fact that organizations simply have a greater need for capacity today than ever before. Add in emerging technologies such as real-time predictive intelligent solutions, and you can begin to see how the demand for data is going to continue to explode. Fortunately, technology is moving to keep pace. There is a technology coming down the pipe known as “big data,” and it’s going to change the way that IT and data centers handle all of that excess demand.

Enterprise Cloud Services – SAP

Fri, 10 Feb 2012 05:00:00 EST

As the name suggests one of the key factors of ‘Enterprise Cloud’ is that it’s intended for the enterprise market, in particular the enterprise applications that they use such as SAP, Oracle and JD Edwards amongst others. Where Cloud Computing overlaps with this sector is ‘Enterprise Cloud Computing‘, and you can follow this topic via … Continue reading »

NSF Releases Cloud Computing Report

Thu, 09 Feb 2012 11:30:00 EST

The National Science Foundation released their report on cloud computing. It can be found here. The intent of this report is to provide information that guides funding programs. The NSF used NIST’s guidance on cloud computing to inform their research and decision making. This report will be instrumental in informing Federal decision makers and cloud investment. It is important to pay attention to such documents as they can help businesses align with Federal government priorities. The NSF has identified the following areas as research areas worthy of financial support from the federal government:

Enterprise 2.0 Cloud Computing – DevOps for Everyone

Thu, 09 Feb 2012 05:30:00 EST

Although it can feel like you’re playing an intense game of Buzzword Bingo, the key way to approach new technologies like Cloud Computing is to marry them up with other hot topics, like social media and big data. Typically these aren’t entirely different domains more so simply different perspectives on to the same system, and so it’s helpful to throw them all in a pot and discuss: DevOps, IaaS, PaaS, web 2.0, enterprise 2.0, .. etc.

EMC VFCache Respinning SSD and Intelligent Caching (Part 2)

Wed, 08 Feb 2012 16:15:00 EST

EMC vFCache respinning SSD and intelligent caching (Part II)

By Greg Schulz

This is the second of a two part series pertaining to EMC VFCache, you can read the first part here.

In this part of the series, lets look at some common questions along with comments and perspectives.

Common questions, answers, comments and perspectives:

Why would EMC not just go into the same market space and mode as FusionIO, a model that many other vendors seam eager to follow? IMHO many vendors are following or chasing FusionIO thus most are selling in the same way perhaps to the same customers. Some of those vendors can very easily if they were not already also make a quick change to their playbook adding some new moves to reach broader audience. Another smart move here is that by taking a companion or complimentary approach is that EMC can continue selling existing storage systems to customers, keep those investments while also supporting competitors products. In addition, for those customers who are slow to adopt the SSD based techniques, this is a relatively easy and low risk way to gain confidence. Granted the disk drive was declared dead several years (and yes also several decades) ago, however it is and will stay alive for many years due to SSD helping to close the IO storage and performance gap.

Data center and storage IO performance capacity gap (Courtesy of Cloud and Virtual Data Storage Networking (CRC Press))

Has this been done before? There have been other vendors who have done LUN caching appliances in the past going back over a decade. Likewise there are PCIe RAID cards that support flash SSD as well as DRAM based caching. Even NetApp has had similar products and functionality with their PAM cards.

Does VFCache work with other PCIe SSD cards such as FusionIO? No, VFCache is a combination of software IO intercept and intelligent cache driver along with a PCIe SSD flash card (which could be supplied as EMC has indicated from different manufactures). Thus VFCache to be VFCache requires the EMC IO intercept and intelligent cache software driver.

Does VFCache work with other vendors storage? Yes, Refer to the EMC support matrix, however the product has been architected and designed to install and coexist into a customers existing environment which means supporting different EMC block storage systems as well as those from other vendors. Keep in mind that a main theme of VFCache is to compliment, coexist, enhance and protect customers investments in storage systems to improve their effectiveness and productivity as opposed to replacing them.

Does VFCache introduce a new point of vendor lockin or stickiness? Some will see or place this as a new form of vendor lockin, others assuming that EMC supports different vendors storage systems downstream as well as offer options for different PCIe flash cards and keeps the solution affordable will assert it is no more lockin that other solutions. In fact by supporting third party storage systems as opposed to replacing them, smart sales people and marketeers will place VFCache as being more open and interoperable than some other PCIe flash card vendors approach. Keep in mind that avoiding vendor lockin is a shared responsibility (read more here).

Does VFCache work with NAS? VFCache does not work with NAS (NFS or CIFS) attached storage.

Does VFCache work with databases? Yes, VFCache is well suited for little data (e.g. database) and traditional OLTP or general business application process that may not be covered or supported by other so called big data focused or optimized solutions. Refer to this EMC document (and this document here) for more information.

Does VFCache only work with little data? While VFCache is well suited for little data (e.g. databases, share point, file and web servers, traditional business systems) it also able to work with other forms of unstructured data.

Does VFCache need VMware? No, While VFCache works with VMware vSphere including a vCenter plug in, however it does not need a hypervisor and is practical in a physical machine (PM) as it is in a virtual machine (VM).

Does VFCache work with Microsoft Windows? Yes, Refer to the EMC support matrix for specific server operating systems and hypervisor version support.

Does VFCache work with other unix platforms? Refer to the EMC support matrix for specific server operating systems and hypervisor version support.

How are reads handled with VFCache? The VFCache software (driver if you prefer) intercepts IO requests to LUNs that are being cached performing a quick lookup to see if there is a valid cache entry in the physical VFCache PCIe card. If there is a cache hit the IO is resolved from the closer or local PCIe card cache making for a lower latency or faster response time IO. In the case of a cache miss, the VFCache driver simply passes the IO request onto the normal SCSI or block (e.g. iSCSI, SAS, FC, FCoE) stack for processing by the downstream storage system (or appliance). Note that when the requested data is retrieved from the storage system, the VFCache driver will based on caching algorithms determinations place a copy of the data in the PCIe read cache. Thus the real power of the VFCache is the software implementing the cache lookup and cache management functions to leverage the PCIe card that complements the underlying block storage systems.

How are writes handled with VFCache? Unless put into a write cache mode which is not the default, VFCache software simply passes the IO operation onto the IO stack for downstream processing by the storage system or appliance attached via a block interface (e.g. iSCSI, SAS, FC, FCoE). Note that as part of the caching algorithms, the VFCache software will make determinations of what to keep in cache based on IO activity requests similar to how cache management results in better cache effectiveness in a storage system. Given EMCs long history of working with intelligent cache algorithms, one would expect some of that DNA exists or will be leveraged further in future versions of the software. Ironically this is where other vendors with long cache effectiveness histories such as IBM, HDS and NetApp among others should also be scratching their collective heads saying wow, we can or should be doing that as well (or better).

Can VFCache be used as a write cache? Yes, while its default mode is to be used as a persistent read cache to compliment server and application buffers in DRAM along with enhance effectiveness of downstream storage system (or appliances) caches, VFCache can also be configured as a persistent write cache.

Does VFCache include FAST automated tiering between different storage systems? The first version is only a caching tool, however think about it a bit, where the software sits, what storage systems it can work with, ability to learn and understand IO paths and patterns and you can get an idea of where EMC could evolve it to, similar to what they have done with recoverpoint among other tools.

Evolving data access patterns and life cycles (more retention and reads)

Does VFCache mean all or nothing approach with EMC? While the complete VFCache solution comes from EMC (e.g. PCIe card and software), the solution will work with other block attached storage as well as existing EMC storage systems for investment protection.

Does VFCache support NAS based storage systems? The first release of VFCache only supports block based access, however the server that VFCache is installed in could certainly be functioning as a general purpose NAS (NFS or CIFS) server (see supported operating systems in EMC interoperability notes) in addition to being a database or other other application server.

Does VFCache require that all LUNs be cached? No, you can select which LUNs are cached and which ones are not.

Does VFCache run in an active / active mode? In the first release it is active passive, refer to EMC release notes for details.

Can VFCache be installed in multiple physical servers accessing the same shared storage system? Yes, however refer to EMC release notes on details about active / active vs. active / passive configuration rules for ensuring data integrity.

Who else is doing things like this? There are caching appliance vendors as well as others such as NetApp and IBM who have used SSD flash caching cards in their storage systems or virtualization appliances. However keep in mind that VFCache is placing the caching function closer to the application that is accessing it there by improving on the locality of reference (e.g. storage and IO effectiveness).

Does VFCache work with SSD drives installed in EMC or other storage systems? Check the EMC product support matrix for specific tested and certified solutions, however in general if the SSD drive is installed in a storage system that is supported as a block LUN (e.g. iSCSI, SAS, FC, FCoE) in theory it should be possible to work with VFCache. Emphasis, visit the EMC support matrix.
What type of flash is being used?

What type of nand flash SSD memory is EMC using in the PCIe card? The first release of VFCache is leveraging enterprise class SLC (Single Level Cell) nand flash which has been used in other EMC products for its endurance, long duty cycle to minnimize or eliminate concerns of wear and tear while meeting read and write performance. EMC has indicated that they will also as part of an industry trend leverage MLC along with Enterprise MLC (EMLC) technologies on a go forward basis.

Doesnt nand ssd flash cache wear out? While nand flash SSD can wear out over time due to extensive write use, the VFCache approach mitigates this by being primarily a read cache reducing the number or program / erase cycles (P/E cycles) that occur with write operations as well as initially leveraging longer duty cycle SLC flash. EMC also has several years experience from implementing wear leveling algorithms into the storage systems controllers to increase duty cycle and reduce wear on SLC flash which will play forward as MLC or Enterprise MLC (EMLC) techniques are leveraged. This differs from vendors who are positioning their SLC or MLC based flash PCIe SSD cards for mainly write operations which will cause more P/E cycles to occur at a faster rate reducing the duty or useful life of the device.

How much capacity does the VFCache PCIe card contain? The first release supports a 300GB card and EMC has indicated that added capacity and configuration options are in their plans.

Does this mean disks are dead? Contrary to popular industry folk lore (or wish) the hard disk drive (HDD) has plenty of life left part of which has been increased by being complimented by VFCache.

Various SSD locations, types, packaging and usage scenario options

Can VFCache work in blade servers? The VFCache software is transparent to blade, rack mount, tower or other types of servers. The hardware part of VFCache is a PCIe card which means that the blade server or system will need to be able to accommodate a PCIe card to compliment the PCIe based mezzaine IO card (e.g. iSCSI, SAS, FC, FCOE) used for accessing storage. What this means is that for blade systems or server vendors such as IBM who have a PCIe expansion module for their H series blade systems (it consumes a slot normally used by a server blade), PCIe cache cards like those being initially released by IBM could work, however check with the EMC interoperability matrix, as well as your specific blade server vendor for PCIe expansion capabilities. Given that EMC leverages Cisco UCS for their vBlocks, one would assume that those systems will also see VFCache modules in those systems. NetApp partners with Cisco using UCS in their FlexPods so you see where that could go as well along with potential other server vendors support including Dell, HP, IBM and Oracle among others.

What about benchmarks? EMC has released some technical documents that show performance improvements in Oracle environments such as this here. Hopefully we will see EMC also release other workloads for different applications including Microsoft Exchange Solutions Proven (ESRP) along with SPC similar to what IBM recently did with their systems among others.

How do the first EMC supplied workload simulations compare vs. other PCIe cards? This is tough to gauge as many SSD solutions and in particular PCIe cards are doing apples to oranges comparisons. For example to generate a high IOPs rating for marketing purposes, most SSD solutions are stress performance tested at 512 bytes or 1/2 of a KByte or at least 1/8 of a small 4Kbyte IO. Note that operating systems such as Windows are moving to 4Kbyte page allocation size to align with growing IO sizes with databases moving from the old average of 4Kbytes to 8Kbytes and larger. What is important to consider is what is the average IO size and activity profile (e.g. reads vs. writes, random vs. sequential) for your applications. If your application is doing ultra small 1/2 Kbyte IOs, or even smaller 64 byte IOs (which should be handled by better application or file system caching in DRAM), then the smaller IO size and record setting examples will apply. However if your applications are more mainstream or larger, then those smaller IO size tests should be taken with a grain of salt. Also keep latency in mind that many target or oppourtunity applications for VFCache are response time sensitive or can benefit by the improved productivity they enable.

What is locality of reference? Locality of reference refers to how close data is to where it is being requested or accessed from. The closer the data to the application requesting the faster the response time or quick the work gets done. For example in the figure below L1/L2/L3 on board processor caches are the fastest, yet smallest while closest to the application running on the server. At the other extreme further down the stack, storage becomes large capacity, lower cost, however lower performing.

What does cache effectiveness vs. cache utilization mean? Cache utilization is an indicator of how much the available cache capacity is being used however it does not give an indicator of if the cache is being well used or not. For example, cache could be 100 percent used, however there could be a low hit rate. Thus cache effectiveness is a gauge of how well the available cache is being used to improve performance in terms of more work being done (IOPS or bandwidth) or lower of latency and response time.

Isnt more cache better? More cache is not better, it is how the cache is being used, this is a message that I would be disappointed in HDS if they were not to bring up as a point of messaging (or rebuttal) given their history of emphasis cache effectiveness vs. size or quantity (Hu, that is a hint btw ;).

What is the performance impact of VFCache on the host server? EMC is saying greatest of 5 percent or less CPU consumption which they claim is several times less than the competitions worst scenario, as well as claiming 512MB to 1GB of DRM on the server vs. several times that of their competitors. The difference could be expected to be via more off load functioning including flash translation layer (FTL), wear leveling and other optimization being handled by the PCIe card vs. being handled in the servers memory and using host server CPU cycles.

How does this compare to what NetApp or IBM does? NetApp, IBM and others have done caching with SSD in their storage systems, or leveraging third party PCIe SSD cards from different vendors to be installed in servers to be used as a storage target. Some vendors such as LSI have done caching on the PCIe cards (e.g. CacheCaid which in theory has a similar software caching concept to VFCache) to improve performance and effectiveness across JBOD and SAS devices.

What about stale (old or invalid) reads, how does VFCache handle or protect against those? Stale reads are handled via the VFCache management software tool or driver which leverages caching algorithms to decide what is valid or invalid data.

How much does VFCache cost? Refer to EMC announcement pricing, however EMC has indicated that they will be competitive with the market (supply and demand).

If a server shutdowns or reboots, what happens to the data in the VFCache? Being that the data is in non volatile SLC nand flash memory, information is not lost when the server reboots or loses power in the case of a shutdown, thus it is persistent. While exact details are not know as of this time, it is expected that the VFCache driver and software do some form of cache coherency and validity check to guard against stale reads or discard any other invalid cache entries.

What will EMC do with VFCache in the future and on a larger scale such as an appliance? EMC via its own internal development and via acquisitions has demonstrated ability to use various clustered techniques such as RapidIO for VMAX nodes, InfiniBand for connecting Isilon nodes. Given an industry trend with several startups using PCIe flash cards installed in a server that then functions as a IO storage system, it seems likely given EMCs history and experience with different storage systems, caching, and interconnects that they could do something interesting. Perhaps Oracle Exadata III (Exadata I was HP, Exadata II was Sun/Oracle) could be an EMC based appliance (That is pure speculation btw)?

EMC has already shown how it can use SSD drives as a cache extension in VNX and CLARiiON servers ( FAST CACHE ) in addition to as a target or storage tier combined with Fast for tiering. Given their history with caching algorithms, it would not be surprising to see other instantiations of the technology deployed in complimentary ways.

Finally, EMC is showing that it can use nand flash SSD in different ways, various packaging forms to apply to diverse applications or customer environments. The companion or complimentary approach EMC is currently taking contrasts with some other vendors who are taking an all or nothing, its all SSD as disk is dead approach. Given the large installed base of disk based systems EMC as well as other vendors have in place, not to mention the investment by those customers, it makes sense to allow those customers the option of when, where and how they can leverage SSD technologies to coexist and complement their environments. Thus with VFCache, EMC is using SSD as a cache enabler to discuss the decades old and growing storage IO to capacity performance gap in a force multiplier model that spreads the cost over more TBytes, PBytes or EBytes while increasing the overall benefit, in other words effectiveness and productivity.

Ok, nuff said for now, I think I see some storm clouds rolling in...

Cheers Gs

Greg Schulz - Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)

twitter @storageio

Amazon S3’s Exponential Growth

Wed, 08 Feb 2012 13:47:00 EST

In a recent Amazon Web Service Blog, it was quoted that Amazon S3 has reached over 762 Billion objects at the end of 2011. We have been following Amazon S3’s growth closely. As usual, we will plug the numbers in an Excel spread-sheet and see its growth in a chart. As shown in the chart, you can see the numbers fit into an exponential trend line pretty well. Worked out the math, the Amazon S3 objects double every 8 months. If you never used Amazon S3 before, It is OK because you may never need to directly interface Amazon S3. You may be using Amazon S3 indirectly through some online storage services, where quite many are powered by Amazon S3.

API Management – Infrastructure vs SaaS

Wed, 08 Feb 2012 13:15:00 EST

The Enterprise is buzzing with API initiatives these days. APIs not only serve mobile applications, they are increasingly redefining how the enterprise does B2B and integration in general. API management as a category follows different models. On one hand, certain technology vendors offer specialized infrastructure to handle the many aspects of API management. On the other, an increasing number of SaaS vendors offer a service which you subscribe to, providing a pre-installed, hosted, basic API management system. Hybrid models are emerging, but that’s a topic for a future post. Before opting for a pure SaaS-based API management solution offering, consider these below.

An Analysis of a NASA Dbase Hack-and-Dump

Wed, 08 Feb 2012 09:30:00 EST

[Editor's note: this analysis predates any official announcements by NASA] Recently, some news of a NASA hack-and-dump passed my twitter deck. I decided after watching a few of my friends re-tweet the news that it might be worth checking out. At least I’d see if I could perform some password analysis on any dumped credentials, or even test out the new Crucialpoint Cloudera Hadoop password cracker on any leftover hashes. What follows is a step-by-step methodology of how I put together the pieces on the hack, and what I think likely happened based on my experience as a web application hacker/programmer.

Three Buzzwords That Every CIO Hears but One They Should Listen To

Wed, 08 Feb 2012 05:00:00 EST

Anyone that's managing an enterprise IT with aging or outdated client/server systems is starting to feel the heat. Soon, their systems and applications will be obsolete and unsupported. At the same time, the industry as whole is seeking the fastest gateway to the latest .NET, HTML5 and mobile deployments including SaaS models. Right about now, there will be a knock on your door with team members offering their advice as to what you should do to ‘keep up' with the latest trends and supported architectures. Here are three of the buzzwords they will have probably used, and what they actually mean.

OpenStack Cloud Pulls Plug on Microsoft Hyper-V

Wed, 08 Feb 2012 04:08:00 EST

Despite Microsoft's best efforts to "engineer compatibility" between its Hyper-V virtualisation software and the OpenStack open source cloud project since October 2010, it appears that may have run out for the Redmond software giant. Adrian Bridgwater

Getting the Best Out of Amazon EC2 Micro Instances

Tue, 07 Feb 2012 22:25:00 EST

One of the popular instance types supported by Amazon EC2 is the Micro Instance. In November 2010, Amazon announced the free tier and started offering 750 hours of Micro Instance usage free per month for the first one year. Initially this offer was available only to Amazon Linux instances but last month AWS announced that they are extending this to Microsoft Windows Server instances. Looking at the technical specifications, the Micro Instance type doesn’t have the muscle required for heavy lifting. It comes with burst CPU capacity that can go up to 2 Elastic Compute Units (ECU). That means the CPU performance is not predictable. The main memory offered is 613MB! This is just not sufficient for running any serious workloads. Of course, storage can be added through EBS and the free tier covers up to 30GB of storage.

EMC VFCache Respinning SSD and Intelligent Caching (Part I)

Tue, 07 Feb 2012 17:00:00 EST

This is the first part of a two part series covering EMC VFCache. EMC formerly announced VFCache (aka Project Lightning) an IO accelerator product that comprises a PCIe nand flash card (aka Solid State Device or SSD) and intelligent cache management software. In addition EMC is also talking about the next phase of the flash business unit and project Thunder. The approach EMC is taking with vFCache should not be a surprise given their history of starting out with memory and SSD evolving it into an intelligent cache optimized storage solution.

Software Defined Networks (SDNs) - A History Lesson

Tue, 07 Feb 2012 10:15:00 EST

I'm a big fan of history, especially technology history, because it gives us such wonderful guidance about what to expect with each new "technology disruption". Just like the history of mankind, technology tends to follow repeatable trends, actions and mistakes. The latest trend that I've been carefully watching is around Software Defined Networks (SDNs). Today's SDN discussions are primarily focused on how new paradigms will change the architecture of IP networks and how network-level services are delivered and managed. This shift is being led by companies like Big Switch, Nicira, Embrane

Desktop VDI May Be Ready for Prime Time but Is the Network?

Tue, 07 Feb 2012 08:45:00 EST

Considering the innate differences between just the two most popular mobile operating systems – Android and iOS – gives rise to understanding how costly and complex an infrastructure might need to be to support both. It’s not at all unlike the issues with server virtualization. Management and delivery architectures require different solutions depending on the platform, so despite potentially costly investments to scale, organizations are often staying single-vendor with respect to its virtualization platform strategy.

How Fast Does Technology Change?

Tue, 07 Feb 2012 07:30:00 EST

How fast does technology change? What is its impact? It is something like the hour hand of a clock. If you keep staring at it you would feel that it always remains stationary. If you go away and come back after sometime, you will see that it has moved a lot. The same thing is true with technology. “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.” – Roy Amara was a researcher, scientist and past president of the Institute for the Future.

Cross-Platform Mobile Code Generator – a Tool Comparison

Tue, 07 Feb 2012 05:19:00 EST

This is the fourth post for cross-platform mobile development tool comparison. There are two more to come. For convenience of analysis, I had divided the tools into five categories (here is an overview). Mobile Web (JavaScript-CSS library), (here is the detail review) Visual Tool (No access to Code), (here is the detail review) App Generator (Native application for multiple platforms), Hybrid App (Leverages embedded browser control) and Game Builder. The classification is somewhat arbitrary and for some tools it becomes little difficult to classify but here is my logic.

How Quickly Will Software Vendors Move to the Cloud?

Tue, 07 Feb 2012 05:15:00 EST

There’s an excellent discussion going on over on the Cloud Computing Google Group about the pace of migration of traditional software to a SaaS model. Here I recently went into some of the very real reasons why the migration is slower than some would like, but didn’t really talk about the pace of adoption. There are some numbers that make for some interesting analysis. According to PwC, in 2009, the top 100 software vendors (traditional non-SaaS) generated 3.7% of their revenues from SaaS in the US; and 1.1% of their revenues from SaaS in Europe. In the same report, the US has a 44% market share and Europe has 36% market share by revenue (License, Maintenance and Support). According to Gartner, in 2010, the WW installed enterprise software market grossed about $104 Billion. So, roughly, we could say that installed software vendors (US & EU only) brought in nearly $5 Billion in revenues in 2010. So nearly 5% of revenues since the inception of SaaS (not including ASP)?

Organizing the Private Cloud: Coming Solutions

Tue, 07 Feb 2012 04:30:00 EST

There was a time in the data center when we dedicated specific infrastructure and specific administrators to specific projects. This was a period of tremendous inefficiency and underutilization. This approach created a huge overhead in terms of administration, and project cycles that seemed to go on and on.

Today, however, cloud computing and virtualization technologies have change things for the better. Data centers are much more likely to pool resources – computing, storage, and network resources – into a private cloud configuration.

Private cloud challenges

While these virtualized private cloud solutions don’t have the administrative headaches that the old approach had, they do present some unique challenges. There are a number of components in the private cloud that need to be correctly managed, including:

• Data center automation

• Virtualization management

• Performance monitoring

• Automated provisioning

• Chargebacks

Each of these areas requires a separate product or solution. In some ways, this recreates some of the inefficiencies of the pre-virtualization world. Each component acts independently, and it’s up to data center personnel to try to fit the pieces together and make sure each solution is working well with the others.

Coming solutions

Fortunately, solutions providers are beginning to recognize this problem. Solutions are emerging that look at the private cloud from the bird’s eye view, and that manage all of the various moving parts at once.

A good example of this is OpenStack. OpenStack is an open source project that is designed to provide a core set of management tools to help organize the private cloud. Some of those tools include things like management of virtual machines, storage management, and even image servicing.

OpenStack was originally developed by NASA along with Rackspace, but is now an open source project with more than 130 companies participating. These are big names in cloud computing, too, and include companies like Cisco, Citrix, HP, NEC, and others.

Making the private cloud even better

In the long run, these solutions are going to offer an even greater degree of efficiency and management capability to the private cloud. Once they take hold and start to gain third party support, they’ll become viable solutions for just about any data center.

Advanced Load Balancing for Developers

Mon, 06 Feb 2012 09:45:00 EST

It has been a while since I wrote an installment of Load Balancing for Developers, and now I think it has been too long, but never fear, this is the grad-daddy of Load Balancing for Developers blogs, covering a useful bit of information about Application Delivery Controllers that you might want to take advantage of. For those who have joined us since my last installment, feel free to check out the entire list of blog entries (along with related blog entries) here, though I assure you that this installment, like most of the others, does not require you to have read those that went before.

ZapNGo! Is still a growing enterprise, now with several dozen complex applications and a high availability architecture that spans datacenters and the cloud. While the organization relies upon its web properties to generate revenue, those properties have been going along fine with your Application Delivery Controller (ADC) architecture.

Now though, you’re seeing a need to centralize administration of a whole lot of functions. What worked fine separately for one or two applications is no longer working so well now that you have several development teams and several dozen applications, and you need to find a way to bring the growing inter-relationships under control before maintenance and hidden dependencies swamp you in a cascading mess of disruption.

With maintenance taking a growing portion of your application development manhours, and a reasonably well positioned test environment configured with a virtual ADC to mimic your production environment, all you need now is a way to cut those maintenance manhours and reduce the amount of repetitive work required to create or update an application. Particularly update an application, because that is a constant problem, where creating is less frequent.

With many of the threats that your ZapNGo application will be known as ZapNGone eliminated, now it is efficiencies you are after. And believe it or not, these too are available in an ADC. Not all ADC’s are created equal, but this discussion will stay on topics that most ADCs can handle, and I’ll mention it when I stray from generic into specific – which I will do in one case because only one vendor supports one of the tools you can use, but all of the others should be supported by whatever ADC vendor you have, though as always, check with your vendor directly first, since I’m not an expert in the inner workings of every one.

There is a lot that many organizations do for themselves, and the array of possibilities is long – from implementing load balancing in source code to security checks in the application, the boundaries of what is expected of developers are shaped by an organization, its history, and its chosen future direction. At ZapNGo, the team has implemented a virtual test environment that as close as possible mirrors production, so that code can be implemented and tested in the way it will be used. They use an ADC for load balancing, so that they don’t have to rewrite the same code over and over, and they have a policy of utilizing a familiar subset of ADC functionality on all applications that face the public.

The company is successful and growing, but as always happens in companies in that situation, the pressures upon them are changing just by virtue of their growth. There are more new people who don’t yet have intimate knowledge of the code base, network topology, security policies, whatever their area of expertise is. There are more lines of code to maintain, while new projects are being brought up at a more rapid pace and with higher priorities (I’ve twice lived through the “Everything is high priority? Well this is highest priority!” syndrome while working in IT. Thankfully, most companies grow out of that fast when it’s pointed out that if everything is priority #1, nothing is). Timelines to complete projects – be they new development, bug fixes, or enhancements are stretching longer and longer as the percentage of gurus in the company is down and the complexity of the code and the architecture it runs on is up.

So what is a development manager to do to increase productivity? Teaming newer developers with people who’ve been around since the beginning is helping, but those seasoned developers are a smaller and smaller percentage of the workforce, while the volume of work has slowly removed them from some of the many products now under management. Adopting coding standards and standardized libraries helps increase experience portability between projects, but doesn’t do enough.

Enter offloading to the ADC. Some things just don’t have to be done in code, and if they don’t have to be, at this stage in the company’s growth, IT management at ZapNGo (that’s you!) decides they won’t be. There just isn’t time for non-essential development anymore.

Utilizing a policy management tool and/or an Application Firewall on the ADC can improve security without increasing the code base, for example. And that shaves hours off of maintenance projects, while standardizing on one or a few implementations that are simply selected on the ADC. Implementing Web Application Acceleration protocols on the ADC means that less in-code optimization has to occur. Performance is no longer purely the role of developers (but of course it is still a concern. No Web Application Acceleration tool can make a loop that runs for five minutes run faster), they can allow the Web Application Acceleration tool to shrink the amount of data being sent to the users’ browser for you. Utilizing a WAN Optimization ADC tool to improve the performance of bulk copies or backups to a remote datacenter or cloud storage… The list goes on and on.

The key is that the ADC enables a lot of opportunities for App Dev to be more responsive to the needs of the organization by moving repetitive tasks to the ADC and standardizing them. And a heaping bonus is that it also does that for operations with a different subset of functionality, meaning one toolset gives both App Dev and Operations a bit more time out of their day for servicing important organizational needs. Some would say this is all part of DevOps, some would say it is not. I leave those discussions to others, all I care is that it can make your apps more secure, fast, and available, while cutting down on workload.

And if your ADC supports an SSL VPN, your developers can work from home when necessary. Or more likely, if your code is your IP, a subset of your developers can. Making ZapNGo more responsive, easier to maintain, and more adaptable to the changes coming next week/month/year. That’s what ADCs do. And they’re pretty darned good at it.

That brings us to the one bit that I have to caveat with F5 only, and that is iApps. An iApp is a constructed configuration tool that asks a few questions and then deploys all the bits necessary to set up an ADC for a particular application. Why do I mention it here? Well if you have dozens of applications with similar characteristics, you can create an iApp Template and use it to rapidly bring new applications or new instances of applications online. And since it is abstracted, these iApp templates can be designed such that AppDev, or even the business owner, is able to operate them Meaning less time worrying about what network resources will be available, how they’re configured, and waiting for operations to have time to implement them (in an advanced ADC that is being utilized to its maximum in a complex application environment, this can be hundreds of networking objects to configure – all encapsulated into a form). Less time on the project timeline, more time for the next project. Or for the post deployment party. One of the two. That’s it for the F5 only bit.

And knowing that all of these items are standardized means less things to get mis-configured, more surety that it will all work right the first time. As with all of these articles, that offers you the most important benefit… A good night’s sleep.

Technorati Tags: Application Delivery Controllers,VPN,Security,Applicaiton Development,Acceleration,WAN Optimization,Encryption,F5 Networks,Load Balancing For Developers,Don MacVittie,blog

Connect with Don:	Connect with F5:

Has Cloud Finally "Crossed the Chasm" in IT?

Mon, 06 Feb 2012 06:15:00 EST

Every year, our friends at ESG post results of their annual Spending Intentions Survey, indicating where many businesses are likely to spend their IT dollars over the coming year. Recently Steve Duplessie posted an article on his blog entitled Cloud – The Cost Containment Strategy that concludes cloud has finally “crossed the chasm” in IT. According to preliminary data, cloud represents the largest % projected spending increase for 2012 IT initiatives– a very exciting turn. Truth is, cloud storage addresses long-standing IT priorities, with three of these priorities topping the list nearly every year.

Installed Software Leverages Cloud Computing, Not Replaced by It

Mon, 06 Feb 2012 05:30:00 EST

Explore why not just revenue streams and business models are slowing the demise of installed software. And see how independent software vendors and their customers are leveraging the cloud....

Cloud Management: The Agentless Approach

Mon, 06 Feb 2012 03:00:00 EST

Automating the deployment and runtime management of large application deployments running on multiple cloud resources across various cloud providers is a challenging problem. Managing remote resources is not a new problem the big difference is that in cloud the resources are added and removed dynamically and at a larger scale. Before cloud people have built management systems using two approaches:

Agent
Agentless

Historically the major trade-off between the agent and the agentless approach has been control vs. rollout time and maintenance costs. You usually get more control with the agent approach compared to the agentless approach. Whereas, the agentless approach is easy to deploy and manage as it doesn’t require rolling out new agents and maintaining their versions.

At Kaavo for deploying software and performing runtime configuration we choose to use SSH instead of requiring proprietary agents to be installed on the managed cloud resources. Some of the design considerations for our decision are:

Easier Rollout and Ongoing Maintenance: Using SSH for management gives you same or more control and security compared to an agent based approach without the overhead of deploying and maintaining proprietary agents on all images across all supported datacenters.
Better Security: Both the agent approach vs. the agentless (SSH) requires communication between the group of manager servers and the servers/resources being managed. As a result you have to manage the firewall rules on the communication ports (for incoming and outgoing packets) properly on the cloud servers/resources to avoid holes for the intruders to exploit. However, since SSH protocol has been around for a while and has been well tested on a large scale it is less likely to have any unknown security vulnerability compared to writing your own proprietary agent or protocol.
Greater Control and Flexibility: In agent based approach the agent code that is executed on the servers or the server side scripts can’t be changed on the fly in case there is any unexpected change in the environment. Whereas in our agentless approach we generate the configuration scripts and files just in time and send them to the servers for execution. This on-demand just-in-time generation of scripts gives us greater flexibility and control in managing cloud resources.

To rollout a scalable agentless approach for cloud management we had to solve some interesting complex problems like managing firewall rules automatically for managed servers in a dynamic cloud environment, handling distributed event queues, and handling execution order dependencies for parallel processes. We solved these hard problems because we didn’t want to take the easy way out by using configuration agents. If you have any further questions about this or want to learn more about why we choose agentless approach for cloud management please contact us.

Getting Started with EMC Atmos Backup

Fri, 03 Feb 2012 22:20:00 EST

EMC Atmos is used in many places now. It is used by well know service providers such as AT&T Synaptic Storage or Peer 1 CloudOne. As we learned from our customers recently, it can also be used by private enterprise to backup data from employee’s local C drive. The scenario is simple, there are many employees that need to do backup from their desktops. EMC Atmos becomes the center piece where the backup data goes to. Since it is cloud storage software-capable, it can be used from both inside the company and also from over the Internet. Who doesn’t want an easy backup solution like that, where the IT Admin of the company has full control of the data and all the backup data is contained in one Atmos instance? This article documents an easy setup of using Gladinet Cloud Desktop as the desktop backup solution connecting into the on-site EMC Atmos Storage.

Cloud Needs Application Architects to Understand IaaS

Fri, 03 Feb 2012 11:30:00 EST

Application development has been moving in the direction of platform abstraction. That is, the need for developers to have detailed knowledge of the infrastructure that the application was being deployed on was becoming less important with increasing sophistication of the application platform for which they were developing. Cloud computing is now reversing this course of action, at least in the short term. Actually, the platform abstraction is a bit of a misnomer since the implementation resulted in operations struggling to tweak the infrastructure to meet performance requirements. Additionally, most applications typically had their own dedicated hardware allowing for specialization to meet the needs of the applications deployed on that hardware.

Marketing Cloud Computing to Non-Believer CIOs

Thu, 02 Feb 2012 19:16:00 EST

Cloud computing is a pretty big deal. It's one of the top priorities for many CIOs. So, it should be a pretty easy process to build a B2B content strategy that helps CIOs make a purchasing decision. Or is it? I subscribe to a lot of newsletters and have a ton of Google Alerts to help me mine for information I use to learn more about the buyers my clients sell to. In my Inbox today was a bonanza of a resource. SearchCIO sent me a link to: Quick take: Why iRobot's CIO doesn't like enterprise cloud computing This resource is a quick interview with Jay Leader, CIO of iRobot, in which he clearly distills exactly why he's not sold on cloud computing. He shares the questions he has and the obstacles he'd need to overcome to change his mind. This kind of stuff is uber helpful to constructing nurturing programs, as topics for blog posts and providing seed ideas for all types of content.

BPM on Demand – Fantasy or Fast Track to Agility?

Thu, 02 Feb 2012 14:00:00 EST

The automation of processes is a key enabler of the Cloud phenomena – without process the Cloud remains a passive environment that undoubtedly saves you money and removes some of the operational headaches, but does little else. The Cloud without process cannot deliver on the promise of Business Technology or the Service-Oriented Enterprise. All of the thoughts and ideas around assembling applications quickly to support a business imperative simply will not happen without process technology. However we need to be very clear – process management in the cloud is not just about BPM Suites on demand. Indeed, the term BPM on Demand is beginning to take on a new meaning when used in conjunction with cloud computing.

Performance in the Cloud: Business Jitter Is Bad

Thu, 02 Feb 2012 10:30:00 EST

One of the benefits of web applications is that they are generally transported via TCP, which is a connection-oriented protocol designed to assure delivery. TCP has a variety of native mechanisms through which delivery issues can be addressed – from window sizes to selective acks to idle time specification to ramp up parameters. All these technical knobs and buttons serve as a way for operators and administrators to tweak the protocol, often at run time, to ensure the exchange of requests and responses upon which web applications rely. This is unlike UDP, which is more of a “fire and forget” protocol in which the server doesn’t really care if you receive the data or not.

Like Cars on a Highway

Thu, 02 Feb 2012 10:00:00 EST

Every once in a while, as the number of people following me grows (thank you, each and every one), I like to revisit something that is fundamental to the high-tech industry but is often overlooked or not given the attention it deserves. This is one of those times, and the many-faceted nature of any application infrastructure is the topic. While much has changed since I last touched on this topic, much has not, leaving us in an odd inflection point. When referring to movies that involve a lot of CGI, my oldest son called it “the valley of expectations”, that point where you know what you’d like to see and you’re so very close to it, but the current offerings fall flat. He specifically said that the Final Fantasy movie was just such a production. The movie came so close to realism that it was disappointing because you could still tell the characters were all animations. I thought it was insightful, but still enjoyed the movie.

The Cloud API Is Pseudo-Consolidation of Infrastructure

Thu, 02 Feb 2012 09:00:00 EST

In most cases, the use of the term “consolidation” implies the aggregation (and subsequently elimination) of like devices. Application delivery consolidation, for example, is used to describe a process of scaling up infrastructure that often occurs during upgrade cycles. Many little boxes are exchanged for a few larger ones as a means to simplify the architecture and reduce the overall costs (hard and soft) associated with delivering applications. Consolidation. But cloud has opened (or should have opened) our eyes to a type of consolidation in which like services are aggregated; a consolidation strategy in which we layer a thin veneer over a set of adjacent functionalities in order to provide a scalable and ultimately operationally consistent experience: an API. A cloud API consolidates infrastructure from an operational perspective. It is the bringing together of adjacent functionalities into a single “entity.” Through a single API, many infrastructure functions and services can be controlled – provisioning, monitoring, security, and load balancing (one part of application delivery) are all available through the same API. Certainly the organization of an API’s documentation segments services into similar containers of functionality, but if you’ve looked at a cloud API you’ll note that it’s all the same API; only the organization of the documentation makes it appear otherwise.

Big Data and Cloud Data-Center Analytics: Solace Systems

Thu, 02 Feb 2012 06:45:00 EST

There is a growing theme of enterprise applications not only moving to the Cloud but also becoming part of a Big Data environment. Furthermore there is also a fever brewing around PaaS (Platform as a Service) being the hot topic for 2012, and this is all about standardizing key middleware software into the Cloud. This field will overlap with the growing focus on Big Data as analytics vendors who specialize in data-centres redefine their products accordingly.

How to Build a Private Cloud Using Microsoft Technologies

Thu, 02 Feb 2012 06:00:00 EST

***This is a LIVING document*** I will be updating this article from time to time as things like Release Updates, Hotfixes, Service Packs and other updates come into being. Please check back often to get the latest information. I will keep the revision list updated at the bottom so you know what changes have been made. This is also *NOT* a complete Private Cloud solution. there are many, many pieces to creating and managing a private cloud including things like - rapid deployment via templates, elasticity and scalability, high availability and redundancy, virtual machine mobility, automation, service management, usage based charge back and more. i recommend you check out http://www.microsoft.com/privatecloud to see the full story. This document will guide you through the process of setting up the bare minimum components to demo a Private Cloud environment using current release versions of Microsoft products and technologies. It is NOT meant for nor is it an ideal configuration for use in a production environment. If you have a Technet or MSDN subscription then you have all the software you need already. Otherwise you can download FREE TRIAL versions of all the necessary components from the Microsoft Technet Evaluation Center.

Trends in Cloud Computing Adoption – 2012

Thu, 02 Feb 2012 06:00:00 EST

What can we expect from cloud computing in 2012? Where will cloud computing be one year from now? If you look back at the important cloud computing events you will find that nothing of much significance had happened in 2010. The same can be said for the 2011 and I suspect that 2012 will not be any different. But, one thing has changed during the 2011. Neither cost saving nor flexibility is the primary driver for cloud adoption There is clear indication that mobility has become the prime reason for cloud adoption.

DevOps PaaS – Platform for Web Evolution

Thu, 02 Feb 2012 05:30:00 EST

The distinction between IaaS and PaaS, and why and how customers will use these services, is very effectively explained through the Elections Canada RFP from last year. I summarized their plans and requirements in a white paper, called ‘Web Business Evolution‘ because this was the term they themselves used to quantify the nature of the transformation they wanted to achieve, and I thought it was exactly right too. The real needs of customers, to a very detailed level of specification, is documented in these RFPs and so they’re extremely illuminating to identify the market demand patterns.

Vulnerability Assessment with Application Security

Wed, 01 Feb 2012 08:45:00 EST

Protecting web applications is an around-the-clock job. Almost anything that is connected to the Internet is a target these days, and organizations are scrambling to keep their web properties available and secure. The ramifications of a breach or downtime can be severe: brand reputation, the ability to meet regulatory requirements, and revenue are all on the line. A 2011 survey conducted by Merrill Research on behalf of VeriSign found that 60 percent of respondents rely on their websites for at least 25 percent of their annual revenue. And the threat landscape is only getting worse. Targeted attacks are designed to gather intelligence; steal trade secrets, sensitive customer information, or intellectual property; disrupt operations; or even destroy critical infrastructure. Targeted attacks have been around for a number of years, but 2011 brought a whole new meaning to advanced persistent threat. Symantec reported that the number of targeted attacks increased almost four-fold from January 2011 to November 2011.

Is Your Integration Platform a Relic?

Tue, 31 Jan 2012 06:30:00 EST

As companies increasingly adopt more and more SaaS/Cloud based applications, as more and more data are cloud-based, and as social networking data becomes increasingly critical to sales, marketing and customer satisfaction applications, the "old style" integration stacks that were originally created to run on PCs or On-Premises Servers will be a non-optimal solution. This is the normal evolution of things. The mainframe-centric nature of computing gave way to Client/Server which in turn gave way to Internet and then SaaS/Cloud. Just like Mainframe-based applications could never really be retrofitted to be good client/server applications, or how non-relational databases never really worked well with SQL front-ends slapped on, Older architectures (Server or LAN-based application integration suites) can't simply been retrofitted or wrappered to become sky-based.

Canadian Government Cloud Computing – Best Practices

Tue, 31 Jan 2012 05:45:00 EST

Here is the first version of our new white paper : Best practices in Canadian Government Cloud Computing. Download : 14-page PDF The objective of this paper is to document all the standards and technologies that go into defining a framework for public sector Cloud Computing in Canada, covering key topics such as: Understanding the … Continue reading »

The Strategic Potential of Collaboration Software

Tue, 31 Jan 2012 05:00:00 EST

Those who follow the HyperOffice blog know that in our last series of posts, we have been focusing on the business problem of collaboration, beyond a narrow technology focus. “Collaboration” in its broad sense, is what every organization is ultimately involved in – people working together to achieve organizational goals. In our last post, the “three pillars of collaboration” we had emphasized that to be truly collaborative, organizations need to get three ducks in a row – policies/processes, technology and culture – rather than depending solely on technology. But that is not to say that technology is completely subordinate to the other two “pillars of collaboration”. Quite often, technology creates previously unthinkable possibilities. To illustrate, before collaborative mobile and internet technologies – there was simply no way to keep field workers on the company’s information grid. But now employees can be kept on the grid wherever they are, which creates fantastic new opportunities in terms of how organizations can operate and structure themselves. This is an example of technology profoundly impacting the organizational bottom line.

Analyzing the PaaS Landscape – Force.com

Mon, 30 Jan 2012 22:03:00 EST

I am helping one of my customers choose the right development and deployment platform. They are building an extremely data-centric and process-driven application. In the process of choosing the right development and deployment platforms, I evaluated existing Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) offerings. Deploying an application on IaaS will involve building everything from the scratch. Since the application requires functionality that is already available in the form of 3rd party components, we may have to deal with the licensing and compatibility issues of these components. So, it becomes the responsibility of the developer to write the core business logic that includes the integration of multiple 3rd party components. Since we are dealing with IaaS, we also need a solid DevOps strategy to deploy and manage the application. Once the application goes live, we need to define the scaling policy and monitor the infrastructure needs regularly. This requires investment not just in the Cloud infrastructure but also in highly skilled developers and administrators to develop and deploy the application that my customer is not ready to invest. The plan is to have a lean and mean team that is highly efficient.