It is tempting. Why not load everything into a database or even into memory if it fits, process the data and store the complete result? It is simple, and quite fast considering how much data there was to process. Now how does it scale? Well we can gi...