I think that the method which Mr. Spencer uses in his Data of Ethics will help to fix our opinion. Ideality in conduct is altogether a matter of adaptation.
~ William James (in The Varieties Of Religious Experience)

0. Intro 🐘

Setting The Stage

What a singular book it was—the masterpiece entitled The Varieties Of Religious Experience by the younger brother of Henry James—in how it managed, along with The Principles of Psychology, to do nothing less than launch the enterprise itself of American psychology. But we are after a different kind of enterprise today, one having to do with the far more immediate varieties (of experience) that squarely revolve around the new currency of our society—data.

Enter the varieties of data experience.

Data, of course, comes in all kinds: Big data, streaming data, data in tables, you name it. (There’s a “data” variant for that.) And today we get to have fun with questions such as these:

• Is NoSQL really a prequel to SQL? (To jog your memory, “SQL” is pronounced “sequel” in most quarters.)
• Is it true that all your joins—both inner and outer—are actually belong to streaming?
• Of the three biggies in the world of distributed data—Consistency, Availability and Partition-tolerance (aka CAP)—which two shall it be today?

Plenty there to get us going, eh.

Getting On The Stage

First, though, and in a nod to William James’ subtly-named masterwork The Varieties Of Religious Experience, be forewarned that we are about to take a decidedly pictorial walk through the contours of the landscape that makes up (a big part of) the data experience.

Why “pictorial”?, you ask.

Why not “pictorial”?, I say.

Heh. Let me level with you: That’s so because, in all seriousness, and speaking here to the “pictorial” aspects, I am inviting you to take in the aforementioned jaunt (through the data landscape) coming right up.

Behold “Exhibit A”: A picture I snapped with my iPhone on a recent morning walk down a rustic trail, one that was replete with magnificently speckled cows in the far distance. (Hence, a speck, or two)

1. It All Began One Fine Texas Morning ⛷

Look, the sinuous trail above sure was an inviting one. So I took the plunge—well, not literally, as a dive into the concrete would’ve been a bit too jarring for my liking—and got going on a brisk walk, though not quite a dash. (We’ve done that elsewhere.)

Lo and behold, the varieties of the Texas flowers in bloom that greeted me was staggering. It got me thinking to the equally rich varieties in which data come bundled our way. We technologists love to get our hands on reams and reams of tables populated with rich data; streaming data has always been fair game.

But what about the tables—yeah, those fabled tables—in the picture coming right up?

2. Of Tables And Streams And Such 📜

Yes, Akram, what about the tables in the pic above?

Well, I will reveal, in full candor, what’s up with that at the very end (in the tradition of Bertrand Meyer’s….) Surely you can wait? No?

A long time ago—some time after the dinosaurs stopped roaming our planet and a bit before the arrival of the Internet—we had this thing called The Yellow Pages, where one could purportedly find the listings of businesses. What a radical idea (for its age.) And for readers of a certain (youthful) age, I’m not making this up. Honest.

The reason I bring that up is the lovely jingle that went with The Yellow Pages phenom:

“So goodbye yellow brick road…”

Wait, and excuse moi. Not exactly that. It was actually—here goes take two now—more like:

“Let your fingers do the walking.“

And today, we go one step further: Yep, leaving behind us all that talk of how one should “Let your fingers do the walking”, we’re going to let our pictures do the talking. Is that radical or what?

You decide because, from here on out, it’s (mostly) you and the pictures—my (muffled) musings make some solitary appearances. But that’s about it.

Here we go.

3. Look Me Straight In The Eyes 🗿

And now we get up-close-and-personal—I’m talking eye-level gaze here—with those tables. (Remember, as always, that we’re on terra firma, we’re squarely in Programming Digressions territory, so everything’s on the table—including the physical ones!)

Where have we seen that intrepid sleuth before, you know the one on the “mezzanine” table above, walking straight toward us, with an orange suitcase in one hand and an overcoat in the other?

Anyone?

While you figure that one out, a little bird has told me that the sleuth’s mission this time is to get to the bottom of relativity, as applied to data. For example

• Should tables right be considered as containers—to accumulate and be observed over time—for data at rest?
• Is the metaphor of streams as being one of data in motion—for encoding a quantized view of the evolution of a table over time—at just the right level?

That little bird also told me to look up: Chirpily good stuff.

4. An Assembly-Line Or Something? 🏭

Will the right-sized data—I don’t care if it’s Big Data, or Streaming Data—please stand up now and make yourself known? Ah, therein lies the rub: Where is the sweet spot wherein lies the habitat of that oh-so-right-sized data?

But first, check this delightful writeup: Streaming Analytics.

It is quite extraordinary: Seldom do I see quality shine through in writing as I’ve seen in that writeup! To pick just one example of how “relatable” the narrative is, I have but to look to this lead-in sentence:

Data is a perishable commodity: It holds the most value at the time it’s produced or captured.

What I have in mind—in my remarking on that writeup being splendidly “relatable”—is simply this: Others go round and round (the mulberry bush) without so much as even trying to bring the subject down anywhere remotely close to Earth. But that’s not Streaming Analytics. Good stuff. Enjoy.

Meanwhile, back at the ranch here, it’s time to ponder: I’ve done my part—I’ve got all the ducks in a row. Now it’s up to you (and your imagination.)

5. Those Lovely Transactions! 🚧

Transaction processing, good old transaction processing: Where would we be without such a crucial abstraction—one where we group together several reads and writes into a logical unit to simplify error handling and concurrency issues—at our side (or at least in our computer systems.)

This all goes to prove that, sometimes at least, you can get what you want. (Why, thank you, Mr. Jagger.)

6. Of ACID And BASE 💰

Time to brush up on our high school Chemistry, amirite? (In particular that pesky pH scale for disambiguating or something between acidity and alkalinity.)

Relax.

We have a transaction (or two) to complete first. And this reeks of déjà vu, but oh well: See those dark brown tables, almost nested one inside the other? I have a feeling that this may be tables all the way down. But let’s not rush; we’ll wait, and find out for sure.

Oh, nearly forgot: We were going to see about—surely you haven’t forgotten that we had a transaction (or two) to complete—this fabled parable of distinguishing between acidity and alkalinity.

(If this talk of high school Chemistry is causing you to break in cold sweat, we’ve got you, too, covered—Think of it this way: Tell acid from base is easier than navigating these COVID-19 pandemic times, now isn’t it?)

So this transaction—immutable or otherwise—was going to happen at a reception where we were gonna meet a connection, Mr. Jagger, amirite?

Why yes, and here’s the scoop, complete with links for your enlightenment:

ACID and BASE.

There you have it. And we are moving right on.

Allow me tell you—and this is especially for all those still inseparably glued to the Ahmad Tea box in the pic above—that tea time is now officially over.

We got some buildings to stand up now.

Let’s stand up and deliver.

7. The Architecture Metaphor 🏰

Check the edifice above, at once delicate and sturdy, laden with meaning: Is that déjà vu or what? I mean, for one thing, 

• It’s those tables again, nested one inside another like Matryoshka dolls.
• Then there’s that serpentine, pliable ruler—”S Ruler” I think it’s called.
• And finally, on the lowermost of those tables sits a glorious dandelion in full bloom—a gift for me from my mom—encased ethereally, for ever and ever, with its wondrousness intact, unsullied.

Anyone see the data leitmotif in there? Here’s a hint from the play book of a sleuth—and it’s not Tintin this time:

“Data! Data! Data!” Sherlock Holmes cried impatiently. “I can’t make bricks without clay.“
~Arthur Conan Doyle (in The Adventure of the Copper Beeches)

You see, it’s elementary: It’s about relationships—between streams and tables—which permeate and power our data-rich industry. Consider how

• Non-grouping operations consume streams and produce those dainty streams as “sinks”.
• Grouping operations, on the other hand, consume streams and spit out tables as—you guessed it!—”sinks”.

That little bird—you know that one with which we had crossed paths earlier—also told me to look up, one more time: Chirpily good stuff.

Onward (and upstream, to be sure.)

8. It’s Nearly Curtain Time 🎬

Behold the motley crew that populates the topmost table above…

Now that’s what I call one big, happy family: Transaction processing rubbing shoulders with—more like wedged-in between (1) a SQL cookout guide, and (2) a tome on streaming data goodness. (And that’s only the background!)

One more time: It’s those tables. Again.

Man, this is too weird for words. So what’s the unstated message? Is it that “Families that play together stay together“? (Actually, judging by the supremely cozy and ragtag “family” assembled in the pic above, that would be a reasonable assessment.)

You’re getting close, but not quite there, yet.

What is the secret, then?

Hint: The wondrously nestled tables are miming a message, amirite?

9. Hey, You On Top Of That Soap-Box 📣

This is where Tintin gets to stand on his soap-box. Wait. That’s no soap-box! That’s a nifty Ahmad Tea-box…

While our pal Tintin ascends to the Pantheon, we mere mortals have plenty of coolness right here on our hands: I’m telling you, that guy E.F. Codd was a genius, way ahead of his times. He accomplished nothing less than lay the foundation for a declarative language—SQL—that powers databases the world over.

And exactly how did it spread like wildfire? In two words: Network effects—think network effects.

10. It’s Tables All The Way Down 🐢

Tintin descends, his mouth still wide agape with astonishment at having solved yet another mystery. (Where have we seen that exact phenom before?!)

Well, I leave you with that pointer (for further explorations.) Meanwhile, we get to rejoice in the heartwarming sight of the assemblage above: Now that’s what I call a big, happy family. That ain’t no diaspora; that won’t get me a patent anytime soon either, but that’s okay because I am satisfied in the knowledge of having sparked your imagination—I did now, didn’t I?—to go forth and explore.

Ah, the punchline: The recurring pattern—we know a thing or two about that sort of thing, amirite?—in the bedecked tables has (hopefully) told the tale it was meant to. So in the spirit of software pioneer Bertrand Meyer’s grand finale “Afterword” to his classic book—recall how a whole generation was introduced to object technology through this book—entitled Object-Oriented Software Construction, we reveal in full candor that

It’s tables all the way down.

Till next time, I leave you with this thought: Do you like “pictorial” essays—such as this one—more than those that have less pictures (and more, um, words)?

Either way, our jaunt through the data landscape has officially come to a close. Stay safe, especially so during these COVID-19 pandemic times.

And remember that voting is democratic and patriotic: Please vote.