Big data is actually a very descriptive term, as it is a term which describes large volumes of data.

Large volumes of data is actually an understatement. We should actually say, enormous or gigantic amounts of data. Let me explain with an example using the Square Kilometre Array telescope why I mean by enormous amounts of data.

SKA will produce over 10x today's global web traffic. Need to quickly transmit, process & store huge constant data flow...

Quoted from Twitter

Let's put the data that will be produced by the SKA in perspective.

What is the average capacity of a Computer Hard drive.

As technology progressed over the years, the average size of a computer hard drive grew tremendously as explained in the paragraph below.

Over the years the average hard disk drive capacity has been increasing. It started at about 1 GB in the 1990s and has now reached about 1 to 6 TB per drive. The average capacity per drive was about 1.70 TB at the start of 2017 then went up to 1.90 TB by the start of 2018 and 2.20 TB by the end of 2018. The average capacity per drive in the first quarter of 2019 was about 2.50 TB, which was then brought down to 2.40 TB in the second quarter and third quarter and then raised again to 2.70 TB in the last quarter of 2019. The capacity was increased again in 2020 to 2.90 TB in the first quarter, 3.30 TB in the second quarter and is expected to go up to 4.10 TB by the third quarter of 2020.
Source

Taken the above into account, we should all get the bigger picture.

During the first phase of development of the SKA, it will produce around 3TB of data per second, which will be transmitted and processed by the central server.

If we think about it, that is around the amount of data of an average computer hard drive every second. That is big data.

How will Big Data assist us in the future

There are many examples of how Big Data will influence our lives in the future, but let's use an example in our every day lives.

As I mentioned in my The Internet of Things explanation more and more IOT devices are used by various service providers. In the event where we have IOT devices linked to our electricity supply, the service provider will have the ability to constantly monitor our usage in real-time.

Image Credit

Due to the fact that the supply is monitored in real time, there is an enormous amount of data that flow through there data centers. This data is then processed and we will have the ability to analyse the usage at a very detailed level.

The analysis will build up statistics of our everyday usage and ultimately the service provider will have the ability to understand through the data exactly what our consumption is literally by the second. When a service provider have the ability to monitor at the required levels, they will also have the ability to see exceptions to our regular usage. This is very important information as the exceptions might mean that the service provider might be able to make very important assumptions.

For example: Let say a household use x amount of electricity everyday and the patterns tell us that between 07:00 and 07:30, we use a specific amount of data as this is the time we make a quick breakfast before the kids go to school.

At 07:30 the usage as per the norm for the specific household, usually goes down significantly as the oven is not used anymore. The service provider is also integrated with Google IOT network and they notice that the family is already moving on their usual road to drop off the kids at school before the parents go to the office.

The Service Provider now have the ability to phone the family with very important information, as the data will show that the family has forgotten to turn of the oven.

This examples shows a good example how Big Data can help to prevent dangerous incidents by early warning , but on the other hand, we have the problem of the protection of personal information where various laws and human rights will prevent this type of service providers to actually use the data. This leaves a lot to think about.

Please feel free to comment below.

Happy Blogging!