What is Map Reduce in Plain English?

First of all it is not a new thing.
It is a design pattern from 60s.
It is a little surprising me that another popular talk of nowadays, MVC is from 70s…
What is going on? :)

Nearly every company’s data is increasing speedy.
Doing aggregations, sorts, searches on big data needs some specialized algorithms and platforms.
Certanly you need paralel processing for quicker results.

Map/Reduce is an efficent way of doing paralel calculations
and it is a developer friendly functional programming approach.

You have just two functions.
One snipet to map, one snipet to reduce…

Google has the most famous distributed implementation of mapreduce.
And after that obvious success, others would also like to implement similar platforms… (Hbase, Hypertable … )

These solutions can make a difference when you need scalability,
if you are just playing at home (like me :) ) or with your small community,
probably you will not see the big difference…
but you must understand the passion behind learning big things :)

I want to show it with this easy schema…

map reduce example schema

Map finds what you are looking for…
Reduce consolidates the data at the end.
Removes duplicates, sorts and aggreagates (sums, gets average etc..)
You can have multiple reduces for more scalibility.

Please read these artciles for more detail information;
ayende.com/blog/4436/what-is-map-reduce-for-anyway
ayende.com/blog/4435/map-reduce-a-visual-explanation
nosqltapes.com/video/understanding-mapreduce-with-mike-miller
mongodb.org/display/DOCS/MapReduce
cookbook.mongodb.org/patterns/unique_items_map_reduce/

Be Sociable, Share!

    Category: NoSql - Comments Off on What is Map Reduce in Plain English?

    Comments are closed.

    Back to top