Executive Summary With an array of new regulations on the horizon, governing where, how and when firms store their data, hedge fund managers are scrambling to ensure that their data storage configuration remains compliant, while also striking a balance between security, cost and remaining at the forefront of the latest technological trends. HFM’s Data Centres Insights report draws…
October 2017

Report Overview

Executive Summary

With an array of new regulations on the horizon, governing where, how and when firms store their data, hedge fund managers are scrambling to ensure that their data storage configuration remains compliant, while also striking a balance between security, cost and remaining at the forefront of the latest technological trends. HFM’s Data Centres Insights report draws on data gathered from our quarterly survey and interviews with hedge fund CTOs, COOs and heads of IT to present actionable insights for decision makers re-evaluating their data storage needs.

The debate over whether to store data internally or externally continues to rage and yet the signs are that external storage is pulling ahead as managers and investors alike become more comfortable with the notion and firms seek to avoid the huge costs involved in buildings one’s own infrastructure in-house. From issues including third-party risk to power outages and connectivity, the report’s first section examines the rewards and pitfalls of storing one’s data in-house versus external rack space. Are hedge funds still reluctant to trust third-party providers? Yes, but our research suggests that this is on the wane, with hedge funds increasingly comfortable with the concept of third-party storage, whether physical or cloud, not least due to the cost benefits.

The second section builds on this by investigating the considerations managers weigh up when choosing between renting rack space in a third-party data centre and adopting cloud-based systems. We found that quantitative investment managers were the group most likely to use a range of data storage methods, in part due to their greater need for functionality and ‘compute’, while younger firms were at the head of the pack in terms of cloud adoption. Although the top three factors when deciding on which data storage solution to opt for remain the same across cloud and physical rack space – namely security, reputation and cost – the picture became more nuanced as we dug deeper, revealing a range of secondary considerations specific to each type.

Finally, the third section explores the process of migrating to the cloud. Here, our research uncovers four key hurdles facing firms in making the transition; the psychological, the regulatory, the technical and the operational. We break down these challenges, provide suggestions on how managers might ease the transition process and look at the potential rewards on offer for those firms who do manage the transition successfully. For more tenured managers, upgrading from years of accumulated infrastructure is expensive and cumbersome, whereas for newer managers the question is how they migrate, not if. We conclude that, not only will cloud uptake continue at an exponential rate, but that cheaper public cloud providers will continue to grow their market share.


Key Findings

Most managers expect to need to upgrade their data storage solution within the next five years

More than 60% of our survey respondents expect to update their data storage configuration within the next five years, with 11% expecting to do so within the year. Crucially, it is managers established since 2010, those who have greater cloud adoption, who are most confident that their solution will last them. HFM expects that – as older managers re-evaluate their data storage needs, new firms are founded and the cloud permeates all aspects of our daily lives – investors and funds alike will become not only more comfortable with the concept of the cloud, but more comfortable with the idea of storing their data with a public cloud provider.

Uptake from emerging managers will soon make cloud storage the industry’s go-to solution

Physical data storage is still the most popular solution among hedge fund managers, but cloud usage is catching up. More than one-third of managers surveyed used either on-site racks, off-site racks, or a combination of the two, compared to just 12% who only used the cloud. However, younger managers are driving change, with 26% of firms established post-crisis using only the cloud, compared to just 3% of firms founded prior to 2008. Many of the CTOs the team spoke to said that they were either looking at, or in the process of, adopting a cloud solution and HFM expects the cloud to become the primary means of data storage within the next five years.

The adoption of public cloud solutions would be faster still were it not for concerns about service levels

One of the most important considerations for firms when choosing a cloud provider was the level of service provided. Managers were worried about maintaining 24/7 access to their public cloud platform and for those experiencing technical difficulties instant help does not come cheap. One popular public cloud solution provider quotes $200 per month for a 48-hour response time to technical issues and $15k per month for a 15-minute response time. Given that a loss of access to one’s systems can lead to huge potential losses for the firm, particularly those funds running systematic strategies, this seemed to be one of the main barriers to public cloud adoption.

Data centre storage is a ‘buyer’s market’ and one particularly suited to brand-name managers

In major ‘hub’ cities such as London and New York the data centre market is most certainly on the side of the buyer. Data centres are keen to bring on board new clients and hedge fund firms can use this to their advantage, extracting concessions from their provider. Such concessions may include a dedicated member of staff to look after one’s rack space, the removal of penalty clauses from contracts and price discounts. Managers should also consider the prestige that their business brings to the data centre, as many data storage firms will look to gain a toehold in a certain sector, such as the financial services industry, by bringing on board a marquee client.

The complementary benefits of external and internal solutions will result in more hybrid models

The advantages of internal and external data storage solutions are myriad and complementary. Internal infrastructure offers maximum control and customisation, while the best external data centres are at the cutting edge of developments, security and expertise. Our survey found that quantitative hedge fund managers had the highest uptake of both types, eager to utilise the full range of benefits. The future, therefore, likely lies in a hybrid model. Not only will managers have the expertise provided by an external vendor alongside the control and customisability of an internal system, but they will also have a system where a recovery centre is easily accessible.

Some professionals are struggling to overcome a ‘psychological hurdle’ with regards third-party storage

Despite the strides made in recent years many managers remain wary about adopting third-party data storage facilities and cloud platforms. Several professionals the team spoke with registered concerns about the greater possibilities of a breach and about the relative maturity of cloud technology. However, research shows that younger professionals are less affected by these concerns, suggesting the block is largely a mental one. The focus should be on ensuring a proper data governance framework is in place and procedures such as data encryption are routine. Overcoming this psychological hurdle may be anathema to some, but is the first step to migrating to the cloud.

Data centres take security seriously but their ‘large surface area’ have some managers worried

Whether it is a natural disaster, terrorist attack or a rogue individual gaining entry to a data facility, it could be easy in this age of cyber threats to overlook physical security. Security staff and key fobs will reduce the chances of receiving unwanted guests, as will staff education. Some data centres have taken matters to extremes by setting up shop in former military bunkers. However, because larger centres are more likely to be targeted by hackers, some managers believe that an internal system is ultimately more secure: its smaller ‘surface area’ being a stronger pull than a centre’s expertise and cutting-edge technology.

Incoming data legislation in Europe will likely change managers’ priorities when choosing storage solutions

Although managers are acutely aware of the need to pick a reputable cloud provider, few appear to have given significant consideration the regulatory and compliance aspect of their data storage set-up. Few of our survey respondents raised it as a top-three consideration when choosing either a cloud provider or physical data centre. One firm whose cloud-based data storage solution was breached, with the leak of personally identifiable information on over 100,000 individuals, was named and shamed by the SEC and fined $75k. With the impending implementation of GDPR, Mifid II and NISD, this must surely be an area of greater focus for European fund managers.

Most firms don’t test their key systems through their back up centres during working hours

A similar proportion of managers run their key systems through their back up centre during working hours as those that never do, even on weekends. Also of note, however, is how controversial the topic was for some interviewees, with one Switzerland-based professional calling those firms who run their systems via a back-up centre during the working week “very brave”. A CTO at one London-based manager, meanwhile, plans to start such tests at the end of 2017 having recently invested in a DR set-up. Overall, 60% of managers avoid testing their key systems during the times that matter most, i.e. business hours. This may be worth rethinking.


The findings in this report were based on three primary sources: research interviews conducted in person and over the telephone, a proprietary survey, and analysis of HFM and third-party data. Research was gathered between July and August 2017. In total, more than 60 firms operating in the hedge fund industry contributed to our research. These were mainly hedge fund and asset managers, as well as service providers, regulators and allocators.

Many of the data exhibits are based on the findings of the HFM Insights Technology Survey Q3 2017, a proprietary survey of senior hedge fund technology and operations staff. The 50 respondents to this survey were based largely in the US and the UK as well as mainland Europe and Asia, and included quantitative and discretionary managers. As a group, the firms represented more than $150bn in combined assets under management.