The Evening Read™

Favorable conditions will never come, and the scholar must learn to seek knowledge under unfavorable conditions. That is self-control. No one has time to finish the tasks of life, so the scholar must in faith entrust the future to God. No one will escape death, so the scholar must accept the human condition soberly and yet humbly offer the life of learning to God.

Heck, Joel. Learning In War-Time. C. S. Lewis Blog. 2 Feb. 2008. Web. 23 July 2010.

Yes, there’s a slew of posts lacking about the wedding and honeymoon in Jamaica, but I just discovered UNdata at lunch today.

This is some cool stuff, and I’m trying to think of some “data” questions I had previously. It’s reminiscent of Wolfram Alpha, which was created by the folks who made Mathmatica.

Lately, I’ve been doing some heavy reading into the future of parallel programming, particular in the realm of data management. The hottest things these days¹ is to implement Google’s MapReduce model. There’s multiple vendors who have taken PostgreSQL and manipulated it to add MapReduce functionality for scale-out OLAP data warehouses.

In thinking about how this can be applied to my core competency, Microsoft SQL Server, I did some research. I hope that under the hood, Microsoft is thinking about this model for the next release, but it could probably be implemented now (if you have a strong understanding of the engine) via CLRs. A few Google searches later, and I discovered the Data Miners Blog and their take on it.

Emulating it in raw SQL is very ugly. But in terms of MapReduce’s goal, parallelization, the quesiton is wether this is synchronous or asynchronous processing. SQL Server already has Service Broker, which is designed for a more asynchronous distribution of the work load. My understanding of MapReduce, is that it’s a synchrnous operation. We’d still be waiting for all of the map() funcitons to complete before reduce() can run to sort or aggregate.

Then the obvious applicaiton for a MapReduce applicaiton then is as an SSIS data flow! I don’t have the tools available to me to attempt to implement it logically, but it’d be interesting to attempt

¹ Monash, Curt; DBMS2; MapReduce Sound Bites

I’ve been doing a fair amount of technical research lately, thinking about my career. Let’s face it, it’s a Microsoft world, but when it comes to big iron, it’s usually a *NIX derivative. All said, MySQL is great, but PostgreSQL is the new heavyweight contender. And Perl or Python apparently.

So here’s my quick list (including some personal pet projects):

• OpenBSD – for all of my pf needs
• OpenSolaris – because ZFS is amazing and this site runs on it
• PostgreSQL – it’s the new thing
• Python – I’ve got friends at Google and they swear by it!
• Perl – this is going to take much more work
• CUBRID – Need to run it on a *NIX box for replication testing, but there’s that annoying Java dependency
• CouchDB – Its not SQL, it’s MapReduce

“The conclusion is inescapable, that to be in the presence of even the meanest, lowest, most repulsive specimen of humanity in the world is still closer to God than when looking up into a starry sky or at a beautiful sunset.”

Mike Mason, The Mystery of Marriage, pg. 46