Processing Tier

“In God we trust. All others must bring data.”

First of all, data is ingested and collected then the role of data processing comes into the picture.

Data processing generally is the collection and manipulation of items of data to produce meaningful information, it can be considered a subset of information processing.

The third tier of the big data pipeline is Data processing. First of all, raw data is collected and to be processed so that we can get useful pieces of information, then we store the data and can be shared with the people who can use it to get actionable insights.

Basically, we use CAP’s theorem where C stands for Consistency, means data should be consistent and every read receives the most recent write or all the servers in the system have same data, A stands for Availability, wiz. every request receives a non-error response – without the guarantee that it contains the most recent write and P stands for Partition Tolerance, it means the system continues to operate despite an arbitrary number of messages being dropped by the network between nodes.

Picture referance:- https://www.researchgate.net/figure/CAP-Theorem_fig4_221462089

1.CASSANDRA:

Apache Cassandra is a free and open source distributed NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no point of failure.

It basically has a keyspace and replication factor, while creating keyspaces, replication factor is defined and also class is defined by strategy.
syntax:- create keyspace with replication ={‘class’:”,’replication factor’:}

There are many components in Cassandra like-

1.Partitions
2.Clustering columns
3.Drivers
4.Node
5.RIng
6.peer-to-peer
7.Gossip
8.Snitch
9.Vnodes
10.Replication
11.Consistency
12.Hinted Handoff
13.Write-Path
14.Read-Path
15.Compaction.

Image result for apache cassandra

2.MongoDB:

MongoDB is an open source database that uses a document-oriented data model. MongoDB is one of several database types to arise in the mid-2000s under the NoSQL banner. Instead of using tables and rows as in relational databases, MongoDB is built on an architecture of collections and documents. Image result for mongo database

3.MySQL:

MySQL is an open source Relational Database Management System(RDBMS). As using RDBMS we can manage and process data.MySQL is written in C and C++. Its SQL parser is written in yacc, but it uses a home-brewed Lexical Analyzer.

Image result for MySQL