Wednesday, 20 July 2011
I'm on a computing course this week, learning about data processing in a streaming environment. For anyone who doesn't know what that means: basically, instead of writing code to process everything in a database (or a file) that already exists, with streaming data you have to be prepared for new pieces of the puzzle to appear at any time.
Much as learning a new language involves subtle mental shifts (I think of it as looking at the world through a different filter), I find that I have to learn a new way of thinking every time I try to pick up a new style of programming. You have to shape the problem you want to solve according to the available tools (in this case, programming languages and patterns).
The necessary mental shift doesn't always come naturally: it took me a couple of years of object-oriented programming to wean myself off the 'goto' statement. They say that a man with a hammer treats every problem as a nail... this is more like learning how to use a screwdriver.
The exercises tend to deal with small examples of precise points, and are easy enough to solve - but what I'm really trying to learn this week is a new pattern of thinking, and that can't quite be taught.
So far as I can tell with streaming data, the tricky part is in deciding what to grab from the stream of data flowing past, what to calculate, what to keep, and how to update it. It's certainly a fun challenge, and I'm looking forwards to applying it to real problems.