To iterate is human; to recurse, divine.
~ Peter Deutsch
Well begun is half done.
Whoa, Help Us Get Our Bearings!
It was a dark and stormy night… The pitch black sky remained sunk under mounds of darkness, only occasionally lighting up from the sporadic dazzling displays of lightning strikes. The stark storm bolts reminding me of the awesome power of nature; for all practical purposes, it was only during those fleeting moments of stormy lights that I was able to maintain my bearings on the desolate hillside where I found myself that night in the rustic countryside ⛈
Wait, wait, cut! (Part I: Apache Storm) 🎬
A Storm Bolt Kicks Off The Drama…
Did someone say something about… a Storm bolt, amirite? 💫
All Alone In The Supercomputing Lab
It was a dark and stormy night… There I was, all alone in the supercomputing lab, with the power of practically limitless computing heft at my disposal: My fingers were flying across the keyboard as I wrestled with, what else, but a Storm bolt of course. Scheherazade-style, I immediately set about crafting the telling of the story in one of the languages I know best: the language of computers! 💻
Recursion And Automagic, Oh My!
Recursion writ large… Anyhow, here was the problem I was trying to solve (should any of my software savvy sisters and brothers wish to give me a helping hand, mostly for a software postmortem at this time) at the witching hour, all alone; supercomputers haven’t quite achieved sentience, yet, so I still feel confident that my earlier statement about my lonesomeness holds water 👻
Brave New World Lies Yonder
(Fair warning: Not that it’s a bad thing at all—in all fairness though to my non-technical readers—but the next two paragraphs contain technical content… I straddle the two worlds, the technical and the non-technical, caring deeply for each in equal parts, because that’s my thing: crafting essays at the intersection of culture, software, technology, and engineering. In fact, it was a scant few minutes ago that I heard from one of my dear, non-technical readers—she was providing feedback on an earlier draft of this essay—where she had the following poignant observation to share with me, far too valuable to miss sharing with you: “If George Orwell or Aldous Huxley were living today, what would they have to say about today’s computers and the way they speak in tongues? It’s the Tower of Babel all over again, as the uninitiated grope around blindly in a brave new world.” Ah, for one thing, it sure rings true with me, and for another, now if only I could write like she does; maybe one day, though I wouldn’t hold my breath; you’re pretty much stuck with me for now!) 👺
- When it comes to anchoring, all you have to remember is this: Within the execute() method of the BaseBasicBolt implementation, I’ll be emitting a tuple to be passed on to the next bolt. At this point of emitting, the out-of-the-box BasicOutputCollector will assume the responsibility of anchoring the output tuple to the input tuple. In the other—complementary—bolt, we simply emit the result. What happens is that this outgoing order tuple will be “automagically” anchored to the incoming complementary tuple. As simple as ABC; no rocket science going on here! ⚓
- And when it comes to acking, all you have to remember is this: Whenever the execute() method of the BaseBasicBolt implementation has done its thing, the tuple that was sent to it will be “automagically” acked. Should there be a failure within the execute() method, the way to handle that is to notify BaseBasicBolt by throwing a FailedException or perhaps another suitable Exception. From there onwards, the BaseBasicBolt will take care of marking that tuple as failed. How simple is that? How elegant is that, now? ⏰
All Your Dreams Are Strange, Love Comes Walking In…
Woohoo, problem solved, so it must be time for yours truly to go home, amirite? Well, not so fast, buster…
It slowly dawned on me that everything that I’ve finished narrating at this point—my adventure of deriving a unique solution all alone in the supercomputing lab—was merely a dream. Rats! A gloomy night like the one in which I found myself enshrouded, right out of something from the dark imagination of Edgar Allen Poe, sure can do strange things to one’s consciousness… 🎥
Hurtling Down Into Empty Spaces
Next thing I knew, I was hurtling through space (barely half a foot away from the stony wall of the castle) but luckily, on making impact with the steep embankment at the foot of the wall, I fell—miraculously enough—such that my back rammed flush against the thick grassy slope, and whose cushion proved to be a lifesaver and a half!
Nonetheless, states change, shift happens, and I kept sliding down at breakneck speed down the grassy slope and headed straight for a Kafkaesque cave that threatened to swallow me whole 🙈
And Into The Kafkaesque Cave!
For a few moments—for all I know it might have been at eternity—I think I lost consciousness. When I came to, I found myself not in a Kafkaesque abyss, but rather what looked like the maw of a gloomy cave… Stalactites and stalagmites all over the place, no matter in which direction I don’t my head. It sure felt like being in a spooky cave… A handful of sonar detection-savvy bats screeched past my ears and confirmed as much! Oh my, out of the fire, and into the frying pan, oh my! 🐌
Wait, wait, cut! (Part II: Apache Kafka) 🎬 🎬
Careening Into Cloud Computing
A Smashing Time With Cloud Computing
Needless to say, this particular Kafka spout—a programming conceptual construct of sorts—was the conduit out of the Storm topology and into my Apache Kafka topic! And there I was: I sipped cool lemonade from a tall tumbler standing next to my keyboard as I listened to a tune by Mark Knopfler (the song Golden Heart, if you really must know!) Life was good.
Will You Change, Or Will You Stay The Same?
Alas, Reality Intrudes…
But here is the rub, the fly in the ointment, reality intruding into our serene tub: Remember what we read above about good old—make that bad old—compaction? Indeed. So there I was, having a blast with handling the power to change the retention policy on a topic from “delete” (recall how it deletes events that are older than retention time to “compact”, which, in turn, only stores the most recent value, per-each-key, in the topic)?
The Joys Of Eventual Consistency
Hmm… I wonder, I thought to myself, if perhaps Apache Kafka will allow me to change the retention policy on a topic from “delete” (which, as you might know, will delete events older than the retention time) to “compact” (which, as you might also know, will store only the recent-most value for every topic key available). I wonder…
Indeed, such are the joys of eventual consistency 😉
I Make It To The Lip Of The Cave
Groaning and positively nonplussed that my magnificent Kafka solution would have to wait for another day, I creakily stood up, my back still hurting from the long skid along the hillside by the castle, and made it to the lip of the cave 😵
Darn, I Feel Dwarfed
Feeling small and lowly—like the boy Jack in the palm of the outstretched hand of his nemesis The Giant—there I was, all tattered and torn, broken and bereft. Thus it was that I looked up into the sky and saw… a spark! 🌟
It was still there, that spark, hovering barely five feet above my head, suspended like a hummingbird in its selfsame incandescence!
Wait, wait, cut! (Part III: Apache Spark) 🎬 🎬 🎬
Rescuing Machine Learning
Did someone say something about… Spark, amirite? 🔥
Only For You
Due to its confidential nature—it’s for your eyes only and on a strictly need-to-know-basis—I can’t talk much about the monumental significance (both in scale and sophistication) of the ML solution I was carefully crafting in the luxurious Miami penthouse overlooking the soft and sandy moonlit beach which was frequented at this time by only the occasional crab or crustacean going out for their midnight snack… 🐙
Of Infinite Loops And Recursion
Surely I’m not dreaming this time, or am I? 😂
Immersed in my suspended state of recursion-writ-large (you know the routine about sometimes-non-terminating babushka doll inside babushka doll inside…), just as I was contemplating the lack of macros in the otherwise awesome language that is Scala—my ML solution sure could have used a Lisp style macro-driven boost right at that time—a gust of wind brazenly blew straight into my face and woke me up! Darn it!! 😴
Yet There I Was…
I let out an audible groan this time—darn those oh-so-elusive tantalizing dreams—doubly reverberating my earlier contemplation of how I had, and not too long ago either, creakily stood up, my back still hurting from the long skid along the hillside by the castle, and made it to the lip of the cave. But there I was, standing above the cave’s lip 🚧
Hope Springs Eternal In The Human Breast!
Woohoo! This would be my ticket, my escape from the sordid mess in which I found myself—you know the Kafkaesque abyss and the equally Kafkaesque cave where we had been spelunking. Truth be told, I wanted to give that staircase a bear hug right then and there (Hmm… On second thoughts, that might prove a tad too painful, you know, hugging the staircase and all!) 🐻
The Word In Spanish, I Mean, In Italian?
All the same, there it was, the loveliest staircase on which any human had cast their eyes. So lovely, in fact, that I got to thinking about the Italian word for a staircase: Scala!
Wait, wait, cut! (Part IV: Scala) 🎬 🎬 🎬 🎬
It Still Was A Dark And Stormy Night All Right
Did someone say something about… Scala, amirite? 🏃
It was a dark and stormy night… There I was, all alone in the Austrian Alps, listening to the sound of music softly playing all around me, surrounded even as I already was in fields of bright flowers that stretched in every direction until I just couldn’t make out the horizon 🌻 🌻 🌻 🌻
Somewhere In The Austrian Alps Now…
By the way, the transatlantic flights of late and cheap restaurant food have not done wonders for my health; in particular, my throat was hoarse as hell. So a word to the wise—especially for those of you into home remedies—that as I sat there squatting on a tuffet somewhere in the Austrian Alps (and hey, I wasn’t eating my curds and whey!) with my trusty MacBook balanced on my lap, I was gently sucking on some Ricola throat lozenges… (Good stuff, I’m telling you, that Ricola).
Let’s Get The Loveliness Of Macros Into Scala!
So here was the problem that I was trying to solve: implement and add the capability into the Scala programming language for users to create their own macros (Lisp-style). Oh yes!! (Out of thin air, the solution suggested itself to me; it came to me unbidden, so I really can’t even take credit for it). And did I mention that I make it a point to carry with me during my travels—transatlantic and otherwise—a vellum-bound notebook, Leonardo da Vinci-style, in which I write especially significant results and observations? For sure, my MacBook remains trusty as ever; it’s just that using a vellum-bound notebook on special occasions makes them, those occasions that is, well, special occasions! 🎨
Why, Hello Kitty
All excited, I sat down to scribble the magnificent solution—we programmers tend to think that something on the order of implementing and adding the capability to the Scala programming language to empower users with creating their own macros (Clojure-style) would qualify as a special occasion—in my vellum-bound notebook. But wait, did I hear a cat’s soft meow, almost a purr? Surely this wasn’t true; I mean, what would a true-blue member of the feline species be doing in the Austrian Alps, for crying out loud?! 🐱
In Honor Of Fermat’s Last Theorem
But hey—being the cat-lover that I am—I needed to get to the bottom of where the tantalizing feline sound was coming from… But before getting up, I scribbled something in the margin of my notebook only this much, even as the siren sounds of a cat somewhere in time beckoned wildly to me. I wrote: “Today I have discovered a truly remarkable software design (and implementation) for macros in the Scala programming language which, alas, this margin is too small to contain. Sigh.“
Then I went a-hunting for cats and, as they say, the rest is history… 📓
Darn, waking up at the most momentously wrong moment—times number four, for crying out loud—is no fun! Dang!!
Back At The Ranch
Meanwhile, back in the real word, things were looking up: the storm had mysteriously subsided into nothingness, into nothingness in the blink of an eye, and I found myself looking up into a clear nighttime sky. Much more importantly—for someone like me looking to lift himself out of the Kafkaesque furnace—the staircase ahead, a mere hundred feet away, was going to be my ticket to freedom 👟
Popeye The Sailor Man, Toot Toot!
Shiver Me Timbers! There he was, standing upright, busily puffing away as pipe, evidently thinking deep thoughts, and only occasionally would he look up. That lonesome soul had clearly not seen a fellow human in ages because his eyes lit up on spotting a fellow human: me, having just climbed up the final step of the staircase.
Back To Square One…
Oh. My. God.
Yikes, back to square one…
Can’t you see they’re open wide?
Would I lie to you, baby?
Would I lie to you?
Don’t you know it’s true?
Girl, there’s no one else but you.
Would I lie to you, baby, yeah?
~ Donny Osmond (Lyrics from Would I Lie To You)
Through the storm we reach the shore
You give it all but I want more
And I’m waiting for you
With or without you
With or without you
I can’t live
With or without you
And you give yourself away
And you give yourself away
And you give
And you give
And you give yourself away
My hands are tied
My body bruised, she’s got me with
Nothing to win and
Nothing left to lose
~ U2 (Lyrics from With Or Without You)
Oh, she takes care of herself
She can wait if she wants
She’s ahead of her time
Oh, and she never gives out
And she never gives in
She just changes her mind
She will promise you more
Than the Garden of Eden
Then she’ll carelessly cut you
And laugh while you’re bleedin’
But she’ll bring out the best
And the worst you can be
Blame it all on yourself
Cause she’s always a woman to me
~ Billy Joel (Lyrics from She’s Always A Woman)
Do you not draw me into existence, and into yours I draw you?
This is the last straw, why would I then claw you…
Shall we remains locked in our recursive scaffolding?
Where you draw me, and I draw you…
Can the frost freezing our resonance not melt away?
You thaw me, and I thaw you…
Does mutually assured destruction serve either one?
I bleed when you saw me, and you bleed when I saw you…
Let’s wash our hands of old sins, shall we?
Where you would fault me, and I
Enough is enough, roughness and lacerating gruff, no?
You bludgeon my armament, and I raw you…
Shall we will cease-and-desist, love and build each other up?
Where you laud me, and I laud you…
What are these voices from the past, they wail to us now?
Saying that you lost me, and I lost you…
Is forgiveness giving up all hope for a better past?
If so, let’s remember how you first saw me, and how I
~ Akram Ahmad (“The Trampoline Of Our Lives” 💔 💕 💘 💝 💞 — A random poem of fiction by a writer, blogger, software craftsman, son, husband, father, brother, and friend)
For in reverie you cannot rise above your achievements nor fall lower than your failures.
And take with you all men:
For in adoration you cannot fly higher than their hopes nor humble yourself lower than their despair.
And if you would know God, be not therefore a solver of riddles.
Rather look about you and you shall see Him playing with your children.
And look into space; you shall see Him walking in the cloud, outstretching His arms in the lightning and descending in rain.
You shall see Him smiling in flowers, then rising and waving His hands in trees.
~ Kahlil Gibran (from his lyrical poetry-prose-mishmash gem entitled The Prophet)