Big Data explained including characteristics
Big Data: this article provides a practical explanation of Big Data. The article starts with the definition, followed by the connection of this concept to the Internet of Things (IoT) and its characteristics. You will also read about the various tools and method that are being used to make sense of the gigantic piles of data that are obtained in todays world. Enjoy reading!
What is Big Data?
The definition of Big Data
Big Data (BD) is a term used to describe large quantities of data that are obtained and processed in, for example, the business world.
Packages of data, both structured and unstructured, are often so big that it is difficult to process and analyse them. However, the business world benefits a lot from analysing Data, so the term Big Data in this context is mainly about the question of how data can be used to come to better decisions, improved insights, and strategic business movements.
BD can be applied in almost every situation where an extreme amount of information is obtained. Users leave behind a lot of information on the Internet, which may subsequently be sold to retailers to identify behavioural trends.
This is then used to align their campaigns more precisely with the market, change prices, or adjust stock. For instance, smart energy meters send user data to energy suppliers to predict disruption or stimulate efficient usage. Oil companies sometimes track thousands of sensors in their machines. Information on the output of these machines is also stored. The result of this are gigantic data sets, Big Data.
BD is so big and complex that conventional technologies, skills, and data analyses techniques are no longer sufficient. For this reason, it uses technologies and initiatives that allow for these new possibilities. BD must meet several requirements. These can be remembered by the so-called three V’s.
The Internet of Things (IoT) and Big Data
The term Big Data has been in use since the nineties of the previous century. Some people attribute this to John Mashey, who popularised the term. However, the origins of the term can be traced back much earlier; to the sixties and seventies, in fact. This is when the first data centres were created and the concept of databases was developed in greater detail.
After the introduction of the Internet, huge quantities of new information became available–information which people did not exactly know how to handle. Disregarding intentionally uploaded photos and videos on social media platforms such as Facebook etc., people began to realise how much data users left behind.
It wasn’t long before smart devices came along, either. These devices are connected to the so-called Internet of Things (IoT). Through these devices, companies can collect data about user habits and product performance.
Although these BD are often unstructured and difficult to trace to a specific person, the data is often of a sensitive nature. As a result, the concept of the Internet of Things is often criticised and is said to infringe on people’s privacy.
Characteristics of Big Data
One defining factor is its volume, or scope. In order to process all these amounts, new tools are continuously being developed. The four V’s originated as a way to describe the overwhelming amounts of new information:
Figure 1 – The 4V’s of Big Data
Velocity (speed)
Clicks on the Internet and ad views record user behaviour with millions of activities per second. The stock exchange has created algorithms that show market changes within microseconds, and machine processes exchange data between millions of devices. These data streams take shape with unprecedented speed and are often available in real-time. This information is not only analysed in detail, but also provides real-time access to certain websites or makes credit card verification possible. Thanks to BD technology, these data are analysed without them having to be stored in a database, something that is certainly undesirable when it comes to sensitive credit card information, for example.
Volume
The V of volume indicates the immense amounts of data that are generated every second via social media, videos, photos, sensors, cars, and other smart devices. The amounts are now so large that they can no longer be stored centrally. Instead, distributing systems are used. These systems store sections of the information in different locations. Software then collects them all together upon request.
In 2000, an ordinary PC had about 10GB of storage space. These days, a Boeing 737 generates 240 Terabyte of flight data during a short flight alone. This clearly demonstrates the explosive increase in volume over the years.
Variety
BD can take many shapes and does not always consist of numbers. The different types of data that can be used nowadays are different from the data in the past. 21st century data are often unstructured. In fact, about 80% of BD. These data are difficult to process.
Traditional database systems are designed with the intention to easily manage small quantities of structured information packages, implement fewer updates, and guarantee the predictable consistent structure of data.
This became more complex as technologies were being developed with which increasingly large quantities of user data were being generated. For this reason, businesses began to use the relational database more often. This database makes it possible to structure unstructured information and thus make it more valuable.
Veracity (value and accuracy)
Veracity is the final V. In the context of BD, veracity refers to the quality and value of data and how accurate they are. The collection of so much information is of little value if its reliability and accuracy can’t be determined. The more reliable and accurate the information, the more valuable for the business and the more accurate the analysis.
Application of Big Data
BD affects organisations in almost every sector. When using BD correctly, businesses can save money, generate revenue, and reach other goals. Uses of this include:
Research & Development
Smart organisations use the generated data to anticipate consumer demands. Netflix, for example, builds predicting models in order to provide suggestions for the user, such as a new film or series. Procter & Gamble uses data and its analyses to plan, produce, and launch new products. The manufacturing world also eagerly uses Big Data. Manufacturers increase the quality and output and minimise waste at the same time (Lean). This is especially important in the competitive market.
Maintenance
Technology businesses can deduce factors from Big Data that can predict mechanical failures. This way they can dig deep into the structured data and they can solve potential problems before they occur by, for example, using maintenance more effectively and maximising employability of parts and devices.
Customer Experience
Having a clear overview of a customer’s demands, increases the chances of creating a loyal customer. With the large quantities of data, a profile for each individual customer can be created more easily than ever. Consumers leave data behind on social media, websites, logbooks, and other sources, which businesses can then use to improve experiences and maximise value. Concrete examples of this are personalised offers or active customer contact.
Innovation
BD can help businesses to innovate by studying inter-dependencies between organisations, individuals, entities, and processes, and then make them comprehensible. These data insights are used to base decisions on.
BD also stimulates the development of new applications. Businesses collect billions of pieces of real-time data and immediately use them to optimise customer experience. Major American cities, for example, use BD in combination with MongoDB—an open source database—in order to decrease criminal behaviour and improve municipal services.
Now it’s your turn
What do you think? Are you familiar with the explanation of Big Data? Do you ever use Big Data in your own organisation or work environment? What is your opinion on Big Data when it comes to privacy and sensitivity? Do you have any tips or additional comments?
Share your experience and knowledge in the comments box below.
More information
- Zikopoulos, P. C., Eaton, C., DeRoos, D., Deutsch, T., & Lapis, G. (2012). Understanding big data: Analytics for enterprise class hadoop and streaming data (p. 176). New York: Mcgraw-hill.
- Labrinidis, A., & Jagadish, H. V. (2012). Challenges and opportunities with big data. Proceedings of the VLDB Endowment, 5(12), 2032-2033.
- Madden, S. (2012). From databases to big data. IEEE Internet Computing, 16(3), 4-6.
How to cite this article:
Janse, B. (2019). Big Data. Retrieved [insert date] from Toolshero: https://www.toolshero.com/innovation/big-data/
Original publication date: 04/09/2019 | Last update: 05/02/2023
Add a link to this page on your website:
<a href=”https://www.toolshero.com/innovation/big-data/”>Toolshero: Big Data</a>
Ben Janse
Ben Janse is a young professional working at ToolsHero as Content Manager. He is also an International Business student at Rotterdam Business School where he focusses on analyzing and developing management models. Thanks to his theoretical and practical knowledge, he knows how to distinguish main- and side issues and to make the essence of each article clearly visible.
