Reveling In The Glory Of Software (On A Stormy Night!)

0 fTo iterate is human; to recurse, divine.
~ Peter Deutsch

Well begun is half done.
~ Aristotle

Preamble 🌒


Whoa, Help Us Get Our Bearings!

It was a dark and stormy night… The pitch black sky remained sunk under mounds of darkness, only occasionally lighting up from the sporadic dazzling displays of lightning strikes. The stark storm bolts reminding me of the awesome power of nature; for all practical purposes, it was only during those fleeting moments of stormy lights that I was able to maintain my bearings on the desolate hillside where I found myself that night in the rustic countryside  ⛈

3 f

As I stood there, gingerly balanced on the parapet of the castle on the hill overlooking a deep gorge—peering intently into the deep gloom fore and aft which had swallowed up the night—an  especially ominous storm bolt seared the nighttime sky… 🏰

Wait, wait, cut! (Part I: Apache Storm) 🎬

A Storm Bolt Kicks Off The Drama…

Did someone say something about… a Storm bolt, amirite? 💫

How could it be, far removed from civilization as I was at that time, that my geek psyche—upon the mere mention of a storm bolt—had reflexively jump-started my mind into high gear with the recent adventure with, what else, but a Storm bolt, of course.

4 f

All Alone In The Supercomputing Lab

It was a dark and stormy night… There I was, all alone in the supercomputing lab, with the power of practically limitless computing heft at my disposal: My fingers were flying across the keyboard as I wrestled with, what else, but a Storm bolt of course. Scheherazade-style, I immediately set about crafting the telling of the story in one of the languages I know best: the language of computers! 💻

2 f

Recursion And Automagic, Oh My!

Recursion writ large… Anyhow, here was the problem I was trying to solve (should any of my software savvy sisters and brothers wish to give me a helping hand, mostly for a software postmortem at this time) at the witching hour, all alone; supercomputers haven’t quite achieved sentience, yet, so I still feel confident that my earlier statement about my lonesomeness holds water 👻

Ah, yes, I was going to tell you about the vexing problem with the Storm bolt: If you all recall the beauty—so the adage that beauty is in the eye of the beholder is doubly true when it comes to us troublesome and troubled techie types—of using the Java class named BaseBasicBolt as our base class is that it “automagically” give us programmers two lovely gifts: (1) anchoring, and (2) acking. Storm does them both—anchoring and acking—equally well, swimmingly well! 🏊

5 f

Brave New World Lies Yonder

(Fair warning: Not that it’s a bad thing at all—in all fairness though to my non-technical readers—but the next two paragraphs contain technical content… I straddle the two worlds, the technical and the non-technical, caring deeply for each in equal parts, because that’s my thing: crafting essays at the intersection of culture, software, technology, and engineering. In fact, it was a scant few minutes ago that I heard from one of my dear, non-technical readers—she was providing feedback on an earlier draft of this essay—where she had the following poignant observation to share with me, far too valuable to miss sharing with you: “If George Orwell or Aldous Huxley were living today, what would they have to say about today’s computers and the way they speak in tongues? It’s the Tower of Babel all over again, as the uninitiated grope around blindly in a brave new world.” Ah, for one thing, it sure rings true with me, and for another, now if only I could write like she does; maybe one day, though I wouldn’t hold my breath; you’re pretty much stuck with me for now!) 👺

  1. When it comes to anchoring, all you have to remember is this: Within the execute() method of the BaseBasicBolt implementation, I’ll be emitting a tuple to be passed on to the next bolt. At this point of emitting, the out-of-the-box BasicOutputCollector will assume the responsibility of anchoring the output tuple to the input tuple. In the other—complementary—bolt, we simply emit the result. What happens is that this outgoing order tuple will be “automagically” anchored to the incoming complementary tuple. As simple as ABC; no rocket science going on here! ⚓
  2. And when it comes to acking, all you have to remember is this: Whenever the execute() method of the BaseBasicBolt implementation has done its thing, the tuple that was sent to it will be “automagically” acked. Should there be a failure within the execute() method, the way to handle that is to notify BaseBasicBolt by throwing a FailedException or perhaps another suitable Exception. From there onwards, the BaseBasicBolt will take care of marking that tuple as failed. How simple is that? How elegant is that, now? ⏰
Eureka! I was onto my moment of enlightenment, woohoo! The lightbulb went off in my head; everything was clear, even as I pleaded the heavens to make it just so… Oh, to be unshackled from the chains of human bondage, all the while profusely apologizing to the hovering spirit of Somerset Maugham out there, somewhere, perhaps in the vicinity of where the caged bird sings… I know it now: the caged bird soared heavenward. I saw it with my own eyes.

21 f

All Your Dreams Are Strange, Love Comes Walking In…

Woohoo, problem solved, so it must be time for yours truly to go home, amirite? Well, not so fast, buster…

It slowly dawned on me that everything that I’ve finished narrating at this point—my adventure of deriving a unique solution all alone in the supercomputing lab—was merely a dream. Rats! A gloomy night like the one in which I found myself enshrouded, right out of something from the dark imagination of Edgar Allen Poe, sure can do strange things to one’s consciousness… 🎥

As I braced myself to absorb the full import of this realization, I took an ill-advised step on the castle’s parapet where, as you’ll recall, I was gingerly balanced all along, and then… I slipped!

10 f

Hurtling Down Into Empty Spaces

Next thing I knew, I was hurtling through space (barely half a foot away from the stony wall of the castle) but luckily, on making impact with the steep embankment at the foot of the wall, I fell—miraculously enough—such that my back rammed flush against the thick grassy slope, and whose cushion proved to be a lifesaver and a half!

Nonetheless, states change, shift happens, and I kept sliding down at breakneck speed down the grassy slope and headed straight for a Kafkaesque cave that threatened to swallow me whole 🙈

Remember how Neo—in the first movie of The Matrix trilogy—had perilously slid down the chute, thanks in large part to the rescue mission headed by Trinity, and commandeered, of course, by Morpheus? (Or perhaps like Michael Douglas and Kathleen Turner both riding the mudslide somewhere in Central America, in the fun movie Romancing the Stone!) I found myself in an eerily similar predicament… 🙉
Oh. My. God.

7 f

And Into The Kafkaesque Cave!

For a few moments—for all I know it might have been at eternity—I think I lost consciousness. When I came to, I found myself not in a Kafkaesque abyss, but rather what looked like the maw of a gloomy cave… Stalactites and stalagmites all over the place, no matter in which direction I don’t my head. It sure felt like being in a spooky cave… A handful of sonar detection-savvy bats screeched past my ears and confirmed as much! Oh my, out of the fire, and into the frying pan, oh my! 🐌

Mercifully enough, though, I had somehow managed to careen down the perilous slope largely unscathed other than some ragtag bruises along my back and legs. How that Kafkaesque cave came to be at the foot of the castle, or how that castle came to be at the top of the hillside above the cave, I will perhaps never find out… At that point In time, frankly, I couldn’t have cared less for such epistemological concerns!

Wait, wait, cut! (Part II: Apache Kafka) 🎬 🎬

Careening Into Cloud Computing

Did someone say something about… Kafka, amirite? 💧
It was a dark and stormy night… There I was, all alone in the basement of a Menlo Park startup in California, with the power of practically unlimited computing heft—thanks to the marvels of cloud computing—at my disposal: My fingers were flying across the keyboard as I grappled with, what else, but a Kafka spout of course (I don’t think I was dreaming this time, or was I?) 👓

8 f

A Smashing Time With Cloud Computing

Needless to say, this particular Kafka spout—a programming conceptual construct of sorts—was the conduit out of the Storm topology and into my Apache Kafka topic! And there I was: I sipped cool lemonade from a tall tumbler standing next to my keyboard as I listened to a tune by Mark Knopfler (the song Golden Heart, if you really must know!) Life was good.

(Another fair warning: Not that it’s a bad thing—in all fairness though to my non-technical readers—but the next two paragraphs contain a fair amount of technical material… No rocket science for you techie types; just saying…) 🚀
So here was the problem—not half as vexing as the one involving the Storm topology which I had dreamily wrestled with earlier on, in the supercomputing lab as you’ll recall, but an interesting problem nonetheless—that I had set about solving with the Kafka topics at my beck and call: Remember good old compaction? Why yes, you know the routine where Kafka will normally store incoming messages for fixed periods of time, afterwards purging messages older than whatever the retention period happens to be…

Will You Change, Or Will You Stay The Same?

More interestingly, and closer to the topic at hand (pun intended even though we are generally chatting about Kafka topics), a software application can blithely use Kafka to store its current state. Anytime—make that every time—the state of the application changes, the application simply writes out the new state into a Kafka topic. And this is where the resilience guarantee of Kafka comes in: all that an application needs to recover (its latest state) is to reads those messages from Kafka to recover from a crash, and you’re done! 🚑

9 f

Alas, Reality Intrudes…

But here is the rub, the fly in the ointment, reality intruding into our serene tub: Remember what we read above about good old—make that bad old—compaction? Indeed. So there I was, having a blast with handling the power to change the retention policy on a topic from “delete” (recall how it deletes events that are older than retention time to “compact”, which, in turn, only stores the most recent value, per-each-key, in the topic)?

Ooh la la,  here, then, was the dastardly fly in the ointment—fly-swatter anyone?—cheekily staring me in the eyes! Darn, my Kafka topic contains null keys, and you know what happens in such situations, right? Yep, compaction will fail, miserably so. Yoohoo, Tony Hoare, now did you really have to make that billion dollar mistake, did you, really? Arghh… (Someone hand me Maalox, and that, too, by the bowls so I may ingest the antacid into my bowels!) 😭

Maalox f

The Joys Of Eventual Consistency

Hmm…  I wonder, I thought to myself, if perhaps Apache Kafka will allow me to change the retention policy on a topic from “delete” (which, as you might know, will delete events older than the retention time) to “compact” (which, as you might also know, will store only the recent-most value for every topic key available). I wonder…

Indeed, such are the joys of eventual consistency 😉

Just as I was getting warmer—not in the physical sense since it was a dark and stormy night, for crying out loud, dear well-wishers of mine—by the minute, realizing that I was on to a fabulous solution, I was awakened by another batch of stealthy bats screeched away, whooshing barely inches from my ears… 😳

12 f

I Make It To The Lip Of The Cave

Groaning and positively nonplussed that my magnificent Kafka solution would have to wait for another day, I creakily stood up, my back still hurting from the long skid along the hillside by the castle, and made it to the lip of the cave 😵

A1 fDarn, I Feel Dwarfed

Feeling small and lowly—like the boy Jack in the palm of the outstretched hand of his nemesis The Giant—there I was, all tattered and torn, broken and bereft. Thus it was that I looked up into the sky and saw… a spark! 🌟

Surely this couldn’t be happening, I thought to myself, rubbing my eyes in disbelief. My head reflexively jerked forward—the full impact of having been hit by the proverbial wall of bricks was only now beginning to make itself known to my body—and then, just as suddenly, my head jerked backward and I looked up again…

It was still there, that spark, hovering barely five feet above my head, suspended like a hummingbird in its selfsame incandescence!

13 f

Wait, wait, cut! (Part III: Apache Spark) 🎬 🎬 🎬

Rescuing Machine Learning

Did someone say something about… Spark, amirite? 🔥

It was a dark and stormy night… There I was, all alone in the Miami penthouse, with the raw power of an extensive server farm of brawny computer at my beck and calling: My fingers were flying across the keyboard as I put together a Machine Learning (ML) solution with, what else, but a Spark topology of course 👕

Miami f

Only For You

Due to its confidential nature—it’s for your eyes only and on a strictly need-to-know-basis—I can’t talk much about the monumental significance (both in scale and sophistication) of the ML solution I was carefully crafting in the luxurious Miami penthouse overlooking the soft and sandy moonlit beach which was frequented at this time by only the occasional crab or crustacean going out for their midnight snack… 🐙

Recurse f

Of Infinite Loops And Recursion

Surely I’m not dreaming this time, or am I? 😂

Immersed in my suspended state of recursion-writ-large (you know the routine about sometimes-non-terminating babushka doll inside babushka doll inside…), just as I was contemplating the lack of macros in the otherwise awesome language that is Scala—my ML solution sure could have used a Lisp style macro-driven boost right at that time—a gust of wind brazenly blew straight into my face and woke me up! Darn it!! 😴

Emerge f

Yet There I Was…

I let out an audible groan this time—darn those oh-so-elusive tantalizing dreams—doubly reverberating my  earlier contemplation of how I had, and not too long ago either, creakily stood up, my back still hurting from the long skid along the hillside by the castle, and made it to the lip of the cave. But there I was, standing above the cave’s lip 🚧

I looked up and momentarily saw—thanks wholly to the light from yet another lightning streak in the  nighttime skies above—a winding staircase carved into the rocky hillside merely a hundred feet away from where I stood. Yet again, as had happened just a bit earlier in the Kafkaesque cave, my head reflexively jerked down—the physical agony of having been hit by the wall of bricks was increasingly making itself known to my body—and then I looked up again…

15 f

Hope Springs Eternal In The Human Breast!

Woohoo! This would be my ticket, my escape from the sordid mess in which I found myself—you know the Kafkaesque abyss and the equally Kafkaesque cave where we had been spelunking. Truth be told, I wanted to give that staircase a bear hug right then and there (Hmm… On second thoughts, that might prove a tad too painful, you know, hugging the staircase and all!) 🐻

6 f

The Word In Spanish, I Mean, In Italian?

All the same, there it was, the loveliest staircase on which any human had cast their eyes. So lovely, in fact, that I got to thinking about the Italian word for a staircase: Scala!

Wait, wait, cut! (Part IV: Scala) 🎬 🎬 🎬 🎬

It Still Was A Dark And Stormy Night All Right

Did someone say something about… Scala, amirite? 🏃

It was a dark and stormy night… There I was, all alone in the Austrian Alps, listening to the sound of music softly playing all around me, surrounded even as I already was in fields of bright flowers that stretched in every direction until I just couldn’t make out the horizon 🌻 🌻 🌻 🌻

16 f

Somewhere In The Austrian Alps Now…

By the way, the transatlantic flights of late and cheap restaurant food have not done wonders for my health; in particular, my throat was hoarse as hell. So a word to the wise—especially for those of you into home remedies—that as I sat there squatting on a tuffet somewhere in the Austrian Alps (and hey, I wasn’t eating my curds and whey!) with my trusty MacBook balanced on my lap, I was gently sucking on some Ricola throat lozenges… (Good stuff, I’m telling you, that Ricola).

Dude, like, what was I doing all alone in the Alps? Frankly, I couldn’t care less how I had got there; all I knew—and frankly cared for—was that I found myself in a paradise-on-earth in the middle of springtime! On top of all that, I was hacking Scala code for crying out loud (If this be a foretaste of things to come, I’m toast!) Woohoo, life was good… 🏄

14 f

Let’s Get The Loveliness Of Macros Into Scala!

So here was the problem that I was trying to solve: implement and add the capability into the Scala programming language for users to create their own macros (Lisp-style). Oh yes!! (Out of thin air, the solution suggested itself to me; it came to me unbidden, so I really can’t even take credit for it). And did I mention that I make it a point to carry with me during my travels—transatlantic and otherwise—a vellum-bound notebook, Leonardo da Vinci-style, in which I write especially significant results and observations? For sure, my MacBook remains trusty as ever; it’s just that using a vellum-bound notebook on special occasions makes them, those occasions that is, well, special occasions! 🎨

Notebook f

Why, Hello Kitty

All excited, I sat down to scribble the magnificent solution—we programmers tend to think that something on the order of implementing and adding the capability to the Scala programming language to empower users with creating their own macros (Clojure-style) would qualify as a special occasion—in my vellum-bound notebook. But wait, did I hear a cat’s soft meow, almost a purr? Surely this wasn’t true; I mean, what would a true-blue member of the feline species be doing in the Austrian Alps, for crying out loud?! 🐱

11 f

In Honor Of Fermat’s Last Theorem

But hey—being the cat-lover that I am—I needed to get to the bottom of where the tantalizing feline sound was coming from… But before getting up, I scribbled something in the margin of my notebook only this much, even as the siren sounds of a cat somewhere in time beckoned wildly to me. I wrote: “Today I have discovered a truly remarkable software design (and implementation) for macros in the Scala programming language which, alas, this margin is too small to contain. Sigh.

Then I went a-hunting for cats and, as they say, the rest is history… 📓

Right at that moment, a particularly stiff breeze blew into my face and woke me up 😴

Darn, waking up at the most momentously wrong moment—times number four, for crying out loud—is no fun! Dang!!

I let out yet another mournful groan—darn those oh-so-elusively tantalizing dreams—on realizing that Scala programmers hankering for macros (myself prominently included among them) would have to wait for another day. Oh well ⛵

Freedom f

Back At The Ranch

Meanwhile, back in the real word, things were looking up: the storm had mysteriously subsided into nothingness, into nothingness in the blink of an eye, and I found myself looking up into a clear nighttime sky. Much more importantly—for someone like me looking to lift himself out of the Kafkaesque furnace—the staircase ahead, a mere hundred feet away, was going to be my ticket to freedom 👟

So I took a step in that direction… Everything went well—and this is primarily for those dear readers who will settle for nothing less than the full gory details of our story in its winding down (denouement aka unraveling) phase—as I climbed up the staircase back to be reunited with civilization, I met an individual sure looked like he had been a sailor at one time or another in his life.

18 f

Popeye The Sailor Man, Toot Toot!

Shiver Me Timbers! There he was, standing upright, busily puffing away as pipe, evidently thinking deep thoughts, and only occasionally would he look up. That lonesome soul had clearly not seen a fellow human in ages because his eyes lit up on spotting a fellow human: me, having just climbed up the final step of the staircase.

He uttered to me (almost as if the desolate soul thought that I was perhaps a fellow seafaring chap) the following fateful words as best as I can recall: “It was a dark and stormy night and the captain said to the first mate, ‘mate tell us a story,’ so the mate began ‘it was a dark and stormy night.’”

19 f

Back To Square One…

Oh. My. God.

No sooner had the crusty sailor uttered those fateful words that the sky suddenly darkened—just as suddenly as I had witnessed it clear up—and it was a dark and stormy night… all over again!
It’s been said that, All’s well that ends well. Would that that were true for yours truly at that juncture, hey, even half-true, amirite?

Yikes, back to square one…
20 f

Look into my eyes.
Can’t you see they’re open wide?
Would I lie to you, baby?
Would I lie to you?
Don’t you know it’s true?
Girl, there’s no one else but you.
Would I lie to you, baby, yeah?

~ Donny Osmond (Lyrics from Would I Lie To You)

Shipwreck f

Through the storm we reach the shore
You give it all but I want more
And I’m waiting for you
With or without you
With or without you
I can’t live
With or without you
And you give yourself away
And you give yourself away
And you give
And you give
And you give yourself away
My hands are tied
My body bruised, she’s got me with
Nothing to win and
Nothing left to lose

~ U2 (Lyrics from With Or Without You)

A f

Oh, she takes care of herself
She can wait if she wants
She’s ahead of her time
Oh, and she never gives out
And she never gives in
She just changes her mind
She will promise you more
Than the Garden of Eden
Then she’ll carelessly cut you
And laugh while you’re bleedin’
But she’ll bring out the best
And the worst you can be
Blame it all on yourself
Cause she’s always a woman to me

~ Billy Joel (Lyrics from She’s Always A Woman)

Do you not draw me into existence, and into yours I draw you?
This is the last straw, why would I then claw you…

Shall we remains locked in our recursive scaffolding?
Where you draw me, and I draw you…

Can the frost freezing our resonance not melt away?
You thaw me, and I thaw you…

Does mutually assured destruction serve either one?
I bleed when you saw me, and you bleed when I saw you…

Let’s wash our hands of old sins, shall we?
Where you would fault me, and I
would flaw you…

Enough is enough, roughness and lacerating gruff, no?
You bludgeon my armament, and I raw you…

Shall we will cease-and-desist, love and build each other up?
Where you laud me, and I laud you…

What are these voices from the past, they wail to us now?
Saying that you lost me, and I lost you…

Is forgiveness giving up all hope for a better past?
If so, let’s remember how you first saw me, and how I
first saw you…

~ Akram Ahmad (“The Trampoline Of Our Lives” 💔 💕 💘 💝 💞 — A random poem of fiction by a writer, blogger, software craftsman, son, husband, father, brother, and friend)

B f

For in reverie you cannot rise above your achievements nor fall lower than your failures.
And take with you all men:
For in adoration you cannot fly higher than their hopes nor humble yourself lower than their despair.
And if you would know God, be not therefore a solver of riddles.
Rather look about you and you shall see Him playing with your children.
And look into space; you shall see Him walking in the cloud, outstretching His arms in the lightning and descending in rain.
You shall see Him smiling in flowers, then rising and waving His hands in trees.

~ Kahlil Gibran (from his lyrical poetry-prose-mishmash gem entitled The Prophet)

C f

One comment

  1. – Every now and then (and with an annoying frequency!), my readers keep running into problems with posting comments on the Blogger platform, which of course is the one which hosts this blog (The Blogger platform is quite good on the whole, it does have its weak spots, and the inability to post comments, for a frustratingly large number of readers, is the biggest one, causing much frustration for my readers and, in turn, for me!)

    – So from time to time, I'll continue posting comments (such as this one right here) on behalf of fellow readers.

    Speaking of which, here's a comment I received (on this essay) in the past 24 hours, and I have their permission to post it on their behalf:

    I love it! Your witty presentation style, with illustrations, headings, and links, is very reassuring as I find myself groping through this dark and stormy night in hopes of finding a rainbow at the end.

    – Thank you so much, dear Reader! Delighted that you enjoyed this fanciful excursion into the land of recursion.

    My bottom line is this: More and more, I have gravitated toward writing the kinds of essays that I myself would love to read! Makes me so happy when these choices intersect in a happy union 🙂 🙂

Your Comment Here!

This site uses Akismet to reduce spam. Learn how your comment data is processed.