minogully's Journal: Parallel Memoization

What if memoization was done on all calculations by a separate processing unit and memory?

So you'd have the main CPU that would do calculations, and the Memoization Processing Unit (MPU) that would look to see if the calculation was already done, and if so, return the result, and if not, wait for the result from the main CPU. If the MPU is able to return a result faster than the CPU, than the CPU cancels the calculation. If the CPU returns the result faster than the MPU can set up a new hash lookup, then the MPU stops where it is and starts with the next calculation (we don't need to record something that the CPU can do faster). The MPU would have it's own dedicated memory so there isn't an issue with using up system RAM. The MPU's memory should be flash so that it won't lose results when powered off, but due to the speed reduction in using flash vs. RAM, this could be supplemented with RAM in some configuration to get a best of both worlds situation.

The aim is for the MPU to save everything and to deal with the limit on the memory, the MPU can discard low priority items, which it would prioritize based on the length of time it took to do the original calculation, and also how long it has been since the result was accessed.

The inspiration for this idea is how I perceive our brains to work. We don't do complicated calculations in our heads when we, say, make an estimate on whether or not we have enough time to cross the road before the car that's coming will hit us. No, instead, we've seen objects moving towards us before and based on past experience we can estimate how long it will take to arrive at our position from how fast it looks like it's going. So our brains are constantly matching up current experiences with past ones, and when our brains find a match we know what to expect out of the current experience. This isn't doing calculations, this is recalling the result from a previous experience.