4.11 Big Data
Big Data
Content |
Additional information |
---|---|
Know that 'Big Data' is a catch-all term for data that won't fit the usual containers. Big Data can be described in terms of:
|
Whilst its size receives all the attention, the most difficult aspect of Big Data really involves its lack of structure. This lack of structure poses challenges because:
Machine learning techniques are needed to discern patterns in the data and to extract useful information. 'Big' is a relative term, but size impacts when the data doesn’t fit onto a single server because relational databases don’t scale well across multiple machines. Data from networked sensors, smartphones, video surveillance, mouse clicks etc are continuously streamed. |
Know that when data sizes are so big as not to fit on to a single server:
Know what features of functional programming make it easier to write:
|
Functional programming languages support:
|
Be familiar with the:
|
Each fact within a fact-based model captures a single piece of information. |