A data warehouse is essentially a system that collates data into a central repository from a wide range of sources for analytical and reporting purposes. Traditionally, data warehouses have existed as on-premises systems, but over the last few years, data warehouse architecture has begun to change, shifting towards cloud-based systems. But what are the differences and benefits of a modern data platform vs a traditional one? Let’s find out.
1. Open data access
Traditional on-premises data warehouses rely on some form of structured query language on an SQL-type database. This means data has to be organised in a structured set which is perfectly adequate for processing data such as documents or line of business applications but you’ll quickly hit a sticking point if you want to start generating business intelligence analysis when you’re collating data from weather feeds, footfall from a shop floor or sentiment data from social media feeds. The problem with this data is that the format starts to vary massively. A modern data platform, on the other hand, is able to consume data in any format, from anywhere and process it in a central location.
2. Logical construct
If you’ve got a traditional data warehouse, you’ll be used to throwing data at a physical server and leaving it there until you’re ready to analyse it. But with a cloud data warehouse, the data doesn’t necessarily have to reside permanently in one place within the platform. The beauty of a modern data warehouse is that it’s atomic – it can pull data from various data sources, in different formats at different times and it can stream data in and drop it as soon as it’s finishes analysing it. This means the process is entirely fluid and constantly changing so as a result you effectively get a virtualisation layer that consolidates all of these different pots of data into one logical construct. And today, this is more important than ever because that’s what gives us the advanced functionality to move from reactive analytics to predictive analytics.
3. It’s metadata driven
Metadata is a set of data that describes and gives information about other data. Most traditional data warehousing models don’t contain metadata. This means IT teams have to rely on manual processes to bring data into their enterprise data warehouse. Over many years of updates and code changes – without structure and standards in place – maintenance costs become a burden and reports are bottlenecked by the time-consuming work required to massage data into new formats.
But today’s digital businesses don’t have time to waste. They want to take advantage of fleeting opportunities by integrating and analysing data in real time to quickly exploit perishable insights. To do this, they need to tag data with metadata which allows us to classify a set number of rules and index it so we can identify what data sets we have in order to mature our data capability.
With the proliferation of data in today’s world it’s absolutely vital to have this capability because otherwise you could end up with data sprawl and the cost and size of your data environment could end up spiraling out of control.
4. Highly orchestrated The term ‘highly orchestrated’ has virtually become synonymous with cloud. In this context a highly orchestrated modern data warehouse refers to the optimal utilisation of data at minimal cost. With metadata in place, we can start to automate the lifecycle of data. We can govern it automatically and we can start to optimise the utilisation of that data. Say for instance you’ve got data that you’ve already analysed – you still need it but it’s stale. If you’re in a traditional data platform, where data can’t be moved, it might be costing you 3p per gb per month, but if you’re in a modern data platform, you can move that data onto a tier that costs 0.5p per gb per month to drive cost savings.
5. Highly secure One of the biggest differences, and biggest benefits, of a modern data platform is the ability to share data easier and faster.
Say for instance a local council wanted access to an NHS trusts data or vice versa, if you’re operating a traditional data platform, you would have to give them access to the full data store. However, with a modern data platform, you can start to apply object level security and compliance. An object is a specific data set, so for example, an allergy, or an NHS number. So if a local council only wanted to know the residents with asthma, you could share that specific data quickly and easily because you have granular level security which essentially makes collaboration easier and faster.
Ultimately, having a data warehouse that resides in the cloud improves the overall value of the data warehouse. It means that business intelligence and other applications can deliver faster, smarter insights to the business since the access, performance, flexibility and security are far better.
To see how Salford City Council is using a smart data analytics platform to improve revenue streams and make informed decisions around how to facilitate business growth in Salford, read the press release here.