• Breadth
• Depth
• Scale
• Target

Volume

• The size of data

Velocity

• Latency of data processing to interactivity

Variety

• The diversity of sources, formats, quality, structures

• Sharing
  • Concurrent access to multiple users
• Data model enforcement
  • Make sure all applications see clean, organized data
• Scale
  • work with datasets too large to fit in memory
  • Complexity hiding interface
• Flexibility
  • use the data in new, unexpected ways

• How is data physically organized on disk?
• What kinds of queries are efficient on this organization?
• How hard it is to update/add new data? (Organization for reading data might not be good for writing data fast)
• what happens with Adhoc queries?

• Activities of users at terminals and most application programs should remain unaffected when the internal representation of data is changed and even when some aspects of the external representations are changed.

• Key idea: Programs that manipulate tabular data exhibit an algebric structure allowing reasoning and manipulation independently of physical data representation. Physical data independence

Operationally

• In the past: Works even if data doesn't fit in memory
• Now: Can use 1000s of cheap computers to operate

Algorithmically

• In the past: Find a polynomial time algorithm that requires no more than \(N^m\) operations
• Now: If you have N data items, require no more than \(\frac{N^m}{k}\) operations for very large \(k\)

Convert all images from TIFF to PNG

Run 1000s of simulations.

Most frequent word in each document.

Histogram of words in each document.

• A function that maps TIFF into PNG images.
• A function that maps parameters to simulation.
• A function that maps a document to its most common word.
• A function that maps a document to word frequencies.
• what if we want to compute the frequencies across all documents?

Computing frequencies across all documents.