That's all interesting speculation, but it's way beyond the mainstream understanding of the contribution of biology to intelligence, which is to say, not much.
Also, IQ tests intentionally discriminate between ability and academic achievement. Speed of arithmetic computation can reduce performance, but it can never increase it. Have you ever taken an IQ test? Reducing this to absurdity, you can train a parrot on multiplication tables, but it will never score above chance on an IQ test.
I can't say it's impossible that intelligence can be improved or reduced with motivation or environment. Clearly environment has some role, and as I said before, individual IQ scores can be adversely affected by poor preparation or imperfect tests. Children may pick up test-taking skills at different rates, but by adulthood those skills become more difficult to pick up quickly. It is telling, then, that a recent study found parent to young child IQ heritability to be 50%, while parent to adult child heritability was closer to 80%. Telling, because parents have more opportunity to tip the scales for young children's performance.
Your hypothesis fails to explain the results of twin studies. If all humans have the same capacity for intellectual performance, but their development is completely moderated by environmental factors, then separated twins would have no correlation. While there is informed disagreement about how heritable intelligence is, there is no reasonable argument that it is not heritable at all.