IndianGeek

Let’s say that you and I decide to play pool. We agree to play eight-ball, best of five games. Our perception is that what follows is a contest to see who will do something called winning.

But I don’t see it that way. I always imagine the outcome of eight-ball to be predetermined, to about 95% certainty, based on who has practiced that specific skill the most over his lifetime. The remaining 5% is mostly luck, and playing a best of five series eliminates most of the luck too.

I found this really insightful. It seems obvious when you read it but for some reason it doesn’t seem to occur naturally to me.

Pomegranate is a novel distributed file system built over distributed tabular storage that acts an awful lot like a NoSQL system. It’s targeted at increasing the performance of tiny object access in order to support applications like online photo and micro-blog services, which require high concurrency, high throughput, and low latency. Their tests seem to indicate it works:

We have demonstrate that file system over tabular storage performs well for highly concurrent access. In our test cluster, we observed linearly increased more than 100,000 aggregate read and write requests served per second (RPS). 

Rather than sitting atop the file system like almost every other K-V store, Pomegranate is baked into file system. The idea is that the file system API is common to every platform so it wouldn’t require a separate API to use. Every application could use it out of the box.

The features of Pomegranate are:

• It handles billions of small files efficiently, even in one directory;
• It provide separate and scalable caching layer, which can be snapshot-able;
• The storage layer uses log structured store to absorb small file writes to utilize the disk bandwidth;
• Build a global namespace for both small files and large files;
• Columnar storage to exploit temporal and spatial locality;
• Distributed extendible hash to index metadata;
• Snapshot-able and reconfigurable caching to increase parallelism and tolerant failures;
• Pomegranate should be the first file system that is built over tabular storage, and the building experience should be worthy for file system community. 

Very cool technology. This reminded me of a distributed filesystem Google Tech Talk (http://www.youtube.com/watch?v=3xKZ4KGkQY8) on Wuala (http://www.wuala.com/) that I found fascinating for all the little problems they had to overcome to make this work.

Well written article on the role of Linkers, covers *nix and Windows systems.

While searching yet again for the SQL to find a particular piece of text inside SQL server proc definitions I came across this cool Addin project that does exactly that and also has other cool features like generating the data insert scripts for a table.

perl -wle ‘print “Prime” if (1 x shift) !~ /^1?$|^(11+?)\1+$/’ [number]

This is a nice trick to check if a number is prime using regular expressions, I am going to try and see how this performs compared to regular methods and see if anything interesting comes up.

To understand why NoSQL is important to you as an app developer, let’s consider the use cases for some of these features:

• Frequently-written, rarely read statistical data (for example, a web hit counter) should use an in-memory key/value store like Redis, or an update-in-place document store like MongoDB.
• Big Data (like weather stats or business analytics) will work best in a freeform, distributed db system like Hadoop.
• Binary assets (such as MP3s and PDFs) find a good home in a datastore that can serve directly to the user’s browser, like Amazon S3.
• Transient data (like web sessions, locks, or short-term stats) should be kept in a transient datastore like Memcache. (Traditionally we haven’t grouped memcached into the database family, but NoSQL has broadened our thinking on this subject.)
• If you need to be able to replicate your data set to multiple locations (such as syncing a music database between a web app and a mobile device), you’ll want the replication features of CouchDB.
• High availability apps, where minimizing downtime is critical, will find great utility in the automatically clustered, redundant setup of datastores like Casandra and Riak.

Despite all the use cases described above, there will always be a place for the highly normalized, transactional, ad-hoc-query capabilities of SQL databases. We’re adding new tools to our toolbox, not removing old ones.

Nice summary on the Heroku blog about use cases for non-relational storage services.

And just because of the simple fact that you are human, you find yourself heir to an inherent unsatisfactoriness in life which simply will not go away. You can suppress it from your awareness for a time. You can distract yourself for hours on end, but it always comes back–usually when you least expect it. All of a sudden, seemingly out of the blue, you sit up, take stock, and realize your actual situation in life.

There you are, and you suddenly realize that you are spending your whole life just barely getting by. You keep up a good front. You manage to make ends meet somehow and look OK from the outside. But those periods of desperation, those times when you feel everything caving in on you, you keep those to yourself. You are a mess. And you know it. But you hide it beautifully. Meanwhile, way down under all that you just know there has got be some other way to live, some better way to look at the world, some way to touch life more fully. You click into it by chance now and then. You get a good job. You fall in love. You win the game. and for a while, things are different. Life takes on a richness and clarity that makes all the bad times and humdrum fade away. The whole texture of your experience changes and you say to yourself, “OK, now I’ve made it; now I will be happy”. But then that fades, too, like smoke in the wind. You are left with just a memory. That and a vague awareness that something is wrong.

But there is really another whole realm of depth and sensitivity available in life, somehow, you are just not seeing it. You wind up feeling cut off. You feel insulated from the sweetness of experience by some sort of sensory cotton. You are not really touching life. You are not making it again. And then even that vague awareness fades away, and you are back to the same old reality. The world looks like the usual foul place, which is boring at best. It is an emotional roller coaster, and you spend a lot of your time down at the bottom of the ramp, yearning for the heights.

So what is wrong with you? Are you a freak? No. You are just human. And you suffer from the same malady that infects every human being. It is a monster inside all of us, and it has many arms: Chronic tension, lack of genuine compassion for others, including the people closest to you, feelings being blocked up, and emotional deadness. Many, many arms. None of us is entirely free from it. We may deny it. We try to suppress it. We build a whole culture around hiding from it, pretending it is not there, and distracting ourselves from it with goals and projects and status. But it never goes away. It is a constant undercurrent in every thought and every perception; a little wordless voice at the back of the head saying, “Not good enough yet. Got to have more. Got to make it better. Got to be better.” It is a monster, a monster that manifests everywhere in subtle forms.

Very nice article on Buddhist philosophy and the power of meditation.