Flash - Aha - Saviour Of The Universe?

Rarely has the world of IT, regardless of the role within, been more united over the understanding that the IT landscape – from endpoint to data/application repository – has been changing out of all recognition over the past few years, and is continuing to do so.

It started with “Everything as a Service”, taking in virtualisation/containers, that cloudy thing and the fundamental “flattening” of the architectural landscape, combined with consolidating what were multiple elements of an IT topology, into a single service or system. We’ve talked about such examples in this ‘ere blog before, such as with respect to combining networking and security in the form of SASE, and now – after conversations with VAST Data – it’s clear that the “collapsed backbone of the 90s networks” equivalent is storage. And, wow, did it need it. I recall testing storage and backup solutions from the very early 90s onwards and there were more tiers involved than a Mayfair wedding cake. And, so often within IT, tiers lead to tears – that’s weeping tears, as the admin attempts to get their head around the management headache that online, nearline, offline, archived, SSD, magnetic, tape etc tiers and layers presented when trying to perform an incredibly complex task such as, er, restoring some allegedly backed up database.

Oh, how we used to laugh when that valuable data “backed up” the week before simply refused to restore itself. Or maybe not… Refreshingly, VAST talks about tier-less universal storage and a completely reworked data and application delivery architecture (DASE – Disaggregated Shared Everything) for enabling effectively unrestricted scalability. DASE essentially separates the storage media from the CPUs that manage that media and provide storage services. This “disaggregated” storage, including all the system metadata, is shared by all the VAST Servers in the cluster. We talked recently, wrt FileCloud, who I’ve been carrying out a lot of testing with, about “the power of metadata” and its access requirements. In the VAST world, DASE allows users to scale the capacity of a storage cluster independently from the compute resources of the cluster – i.e., enclosures for capacity and servers for performance – without either acting as a bottleneck for the other.

Data exists to be accessed. Data access is therefore all about 24×7 availability. Key to this is speed of access, which is why VAST solutions are entirely flash-based. Any Black Adder fans of old will recall “flash by name and flash by nature” (RIP Rik Mayall) but the performance differences between a completely flash-based system and a HDD-based system are like comparing a Tesla with a first generation moped. The catch has always been the economics aspect – flash has been very expensive compared to magnetic alternatives. But here too, VAST is looking to transform those economics to make it a viable technology for the masses – hence the term “universal storage”.

Combining those elements of ease of access, performance and scalability really does lead to a very positive reimagining of the data landscape – and therefore the IT landscape itself and – most importantly – the user experience. Creating a flatter architecture equally reduces the management brief spectacularly. It makes automation of repeat tasks far easier, while massively increased performance means the AI elements powering that automation are far more efficient – i.e., they will actually work 😊.

On the AI/ML front – and surely these have to be the most overhyped terms in IT for the past couple of years, often with no meaning – there are many examples of why automation IS the only way forward; manual alternatives are simply too slow. I have clients in all kinds of areas from security to network fault detection and even application migration who can show automation is the only solution to Digital Transformation. But, on a general basis, cost of infrastructure has historically been the limiting factor for advancing AI/ML/modelling, in that it needs a lot of resource, hence the massive importance here, of making flash-based storage economically viable, beyond specialist HPC requirements.

So, much to explore here going forward – as they say, “watch this space”.