Perl Before Swine

Return of the Regex (App)

Hello again, stranger! Today, I’m back with an update on the first project I ever wrote about here. The app was initially created to streamline a small bit of my daily work, but alas, its home remained on a third party site, rather than having a cosy, Devops managed spot within my current company’s code base.

I set out to change that this summer.

And what an adventure it turned out to be! As an operations-centric employee, my formal exposure to the lands of Devops was limited to… well, none. I had heard the names of our internal and third party tools in passing, but that was the extent of my knowledge. I determined to change that, but I wasn’t sure where to start. Naively, I figured I’d just dive into our documentation.

In a revelation that will surprise no one, it wasn’t as easy as that. Philosophies on documentation are many and varied. One I recently learned to admire and emulate is TAGRI – short for They Ain’t Gonna Read It. In short, the intent is to write useful and readable documentation, rather than necessarily thorough and extensive documentation. After all, it’s meant to be read, right? And most probably it’s being written for the benefit and accessibility of the reader.

For the majority of cases, I see the strong value in this approach. If a developer needs to deploy a new service and gets stuck along the way, why force them to skim through pages and pages of how-tos and step-by-steps? Why not simply create a short and easy-to-find page on their specific issue? Awesome. Except when a brand newbie like me wades into the endlessly interconnected net of pages, all of which seem to hint at a unified and easy approach that remains tantalizingly out of reach. Where do I even start? For third-party tools, I can put my Google-Fu to work on unfamiliar terminology, but what about internally built infrastructure?

After a few weeks of increasingly discouraged sifting through the same stack of what was clearly a very helpful set of documentation to those who have had the benefit of an introduction to the tools and their use, I wasn’t much closer to an internal deploy than when I started. That’s when I realized the key piece I had missed. What was stopping me from joining the ranks of the aforementioned users who have had the benefit of an introduction? One wouldn’t have had nearly the same success pre-introduction as post- in Victorian England, and neither was I. But that didn’t mean I had to remain so!

In a serendipitous turn of events, my company happened to be rolling out a mentorship program right around the time I came to this conclusion. I’ll skip past the details of mentor matching and meeting and right to the immeasurable value of my mentor’s contribution to my problem. By virtue of having worked with our Devops ecosystem in their core role, my mentor was able to give me the high-level knowledge I needed to get started, as well as directing my attention to the appropriate and harder to document areas of our deployment tools (such as where on the blocks of console logs I should look to see why my build might have failed.)

In the end, my mentor helped me accomplish in a mere couple of hours the feat that I had been stuck on for weeks of documentation wading and reading. I’ve always been a believer in trying to help oneself first before asking others to help you – otherwise, how might you learn to find the answers to a similar but non-identical issue in the future? An expert’s time is valuable, no matter what they’re an expert on. The adventure with my internal deploy showed me the other side of this coin.

I learned that there’s a place, a time, and likely a time-limit on self-directed searching. When returns have diminished beyond usefulness, it may be time to seek out a mentor. I was fortunate to have a formal avenue to do so, and I highly encourage other businesses to streamline such pairings too, but it doesn’t have to be quite so formal – sometimes just posting on a Slack channel for help will get you there, and you don’t have to feel like you’re wasting others’ valuable time to do it. After all, your time is valuable, too! My next aspiration is to be as helpful to a newbie in a field I’m confident about as my mentor was to me.

Why is my Javascript returning undefined?

I’m writing this today to share a solution to a problem my Google-fu couldn’t resolve. Maybe putting this out there will put it in someone else’s search result.

Which terms did I try, you might ask. First of all, behold the title of this post for a hint. Beyond that, a bunch of variations on the below:

“Chrome extension not running javascript correctly for some users”
“Chrome.tabs returns undefined”
“Javascript not running correctly for all users”
“Small usergroup undefined values returned”
“Undefined values returned from previously working code”

Some of you might see where this is going, and to you, I say – post more on StackOverflow! For the rest of you, who may also be stuck in this situation, I have a very simple word: AdBlock. AdBlock. Or is that two words? Ad Block? Anyway.

In a brief and depressing overview of my issue, I’ll say that a Chrome extension I’d been working on to replace a Google form ticket submission process was in final testing stages, and a small group of users were running into errors. The oddest part to me was that it was only a few, and those few hadn’t had issues previously – only in this latest version. What had I added? What might have changed on their end? Was it their Chrome version? No, they had the same ones as those of other testers… I was ready to tear my hair out. That’s when one of the users randomly pinged me with the revelation – “I added the domain to AdBlock whitelist and now it works fine.”

So I can’t really even say that I resolved the issue – just one dedicated tester trying random things until something worked. In summary – if your question is “why is my Javascript returning undefined?” the answer just might be “AdBlock.”

This is the part where you roast me for my poor Google-fu, and my soapboxing of obvious answers. I’m ready.

Working with Other People’s HTML – a Reflection

Get ready for another stream of consciousness.

In a recent work project, I had the task of creating two Chrome extensions. Each one would start at a specific origin page, scrape some data from said origin page, collect form input from the user, and then open a pre-populated Jira ticket template in a new tab using Atlassian’s documentation.

That’s where the similarities ended. I won’t bore you with too many of the differences regarding why I needed two different extensions in the first place, since they fundamentally do the same thing. Let’s just wave our collective hands at it and say “permissions, etc.”

And now, on to the reflection. The first extension I built relied on a pervasive internal tool that was built about a decade ago. Although this tool is integral to the daily process of many of my company’s client-facing staff (among other departments), its functionality and layout was built more like Rome than like Rome’s children. That is to say – the core hasn’t been changed in as long as the tool has been around, and new features, UI changes, and other tweaks, took place right on top of the tangled ball of code yarn that is the internal workings of the tool.

As a result of the tangled alleys and ad-libbed features of this tool, data scraping felt like the most Macgyver’d, jerry-rigged process. None of the data I worked with had any labels aside from the text that happened to precede it. And the oddest challenge turned out to be white space – did you know that node.nextSibling returns text elements too? I discovered this while pulling my hair out wondering why the contents of node.nextSibling returned either white space or undefined in my javascript! Not even StackOverflow uncovered this little nugget for me – I reached all the way to this thread on ars technica’s forum to get to the bottom of it. (Shout out to user Liam’s post from 2001 that recommended creating my own “getNextValidSibling” function!)

I slogged my way through creating a separate function to scrape each datum I needed, as they all worked just a bit differently. Some were hidden in anchor tags, some were inexplicably surrounded by parentheses or unexpected white space, and some just hid among preceding text with annoyingly similar names. No IDs in sight. Walking out of that coding adventure, I found myself feeling like I had just wandered out of the woods, having followed no path. It was almost unexpected when I realized “hey, that’s the last function, and I’m pretty sure it’s doing what I want!” Phew, what an journey.

And now, on to the next extension. It relies on scraping data from a much more recently created tool, with robust documentation, QA, and systemic/structural changes where needed. You might already know where I’m going with this. The new tool, while displaying similar information to the previous one, had its HTML laid out like a manicured garden. Not an attribute-less <td> tag in sight! Each item I needed to scrape had a neatly titled “data-qa” attribute, so I was able to resolve my scraping needs with just one function, receiving the value of the data-qa attribute to return the needed text. It was like Marie Kondo had been through there – nothing unnecessary and everything in its place.

What I’m getting at here is pretty intuitive. (In Russia, we might say “Поймет не только взрослый, Но даже карапуз.” – it’s understandable by not only an adult, but even by a toddler.) Working with neat and tidy code is a pleasure! Pass it on to the next reviewed if and when you’re able, to save them wandering in the forest. That, or document your API, whichever you like best.

Disclaimer: My code is still hideously un-navigable, and I promise I a working on it. At least there’s a few comments in there, right? …Right?

Footnote for the initiated: enjoy this blast from the past. Don’t stand, don’t jump, don’t sing, don’t dance, in construction zones or where there’s a hanging load.

And so it begins

Today’s the day I write my very first blog post. This is much more intimidating than I had anticipated.

I write to you, dear reader, to both show my pride in completion of my very first self-designed programming project, and to underscore the challenges and frustrations I faced in getting this thing live.

I’m talking, of course, about the Regex Converter. This no doubt soon-to-be-famous, simple web app was born out of a two-fold need. The first driving force behind it was obligation. I needed to come up with, design, write, build, and deploy some… thing to present as a final project to complete a little (not so little) online course called CS50. Or, more formally, Harvard CS50’s Introduction to Computer Science. We’ll save the second deciding factor for later.

I started CS50 because a colleague of mine took a proactive approach to finding growth and career development opportunities, instead of waiting for them to come to him. Not only did he look for these opportunities for himself, he also ventured forth to build an after-work-hours study group of like-minded work mates to work through and study together. Thanks, Ryan – I wouldn’t be writing this blog right now if not for your enthusiastic plugs of how fun and challenging it was to make silly games in Scratch.

I started the course in April of 2019 and finished on the exact deadline – December 31st of the same year. Boy, was it stressful right down at the end there. During the course of CS50, we journeyed from the light-hearted Scratch to the hair-pulling world of resizing .jpgs in C, and on toward the abstraction-rich tool that is Python. From there, we integrated Python with Flask and Jinja to create our very own, somewhat functional web apps! It is upon the skeleton of the C$50 Finance problem that I built my Regex app.

Now, let’s get back to reason number two of this thing’s birth. As part of my role at work, I analyze raw text data using a SQL-like query language that doesn’t offer a case insensitivity option. We got around this issue with Regex, and we wrote the Regex slowly, by hand. For example, let’s say we’re looking to find rows in a table that contain the word “pear,” but we don’t know the case mix to expect, and we want all case version represented. This required us to manually write .*[Pp][Ee][Aa][Rr].* – just typing that four letter word just now took me a good 25 seconds, and I made some errors. It. Was. Ridiculous. There must be a better way!

So, rather than continuing to fruitlessly request a case insensitivity option for our query language, I figured I could at least automate the Regex writing process. After all, it’s incredibly repetitive and predictable – exactly what automation is made for. Best of all, I figured I could do so with my nascent Python skills. My algorithms may not be elegant, but they get the job done right every time! In fact, I’m so unashamed of them, I’m willing to share my source code, right here, right now! (Note – I accept constructive criticism on both the contents of the repository and on this presentation thereof.)

“I don’t feel like opening all that,” I hear you say. That’s cool too – you’ll get some screencaps of both the UI and the output near the bottom of the post, here.

So anyway, I’m sure my pride shows through fine. On to the challenges, as promised. Without getting into the fact that CS50 is the most challenging online course I’ve ever come across, let alone undertaken, I’ll jump right into deployment on Heroku. CS50 is generous enough to provide a fully functional and robust IDE to both write and run code with, so right up until the end of the course, I had it easy. Then, the real world knocked me squarely on my bottom.

Now, I will impart on to you two critical facts that I learned during the course of the deployment process. The first is that, to deploy a Python app on Heroku, you need a procfile. Heroku does a better job of explaining what exactly that is here. The second critical fact is that, if you mess up your procfile the first time, it’s really hard to convince Heroku to overwrite it with new versions. Like, really hard. Or at least it was for me! I spent a good 5-6 hours troubleshooting just this one aspect, and in a way it was a blessing in disguise because I sure learned a lot about how not to procfile.

So, if you take nothing else from this post (unlikely, given the glowing wonder of the Regex app linked herein,) know this: Write your procfile in a program that doesn’t append newlines to the end of your text (like a code editor or just command line) and get it right the first time. If not, you’ll be stuck either hunting through your gitignore file, or if you’re a stubborn goat like I am, making a whole new repository just to get a clean start. In fact, if you looked at the code I linked above, you’ll notice my -m’s literally say “fresh start.” Although I’ll never get those hours of my life back, the joy I felt in finally seeing my app living free and in the wild was immense!

Behold! What style, what finesse, what elegant design! Ok, maybe that’s pushing it a little bit, but at least you can read the text and it doesn’t flow off the page. Gotta start somewhere, right?

I’m particularly pleased with my output samples. Because how would you describe the output in words? It’s quite obscure, but for all that, exactly what I needed to ease my regex and case sensitivity woes.

Assuming you’re curious about all 5 options, (and who wouldn’t be?) I’m happy to share the output below for your perusal (and formatting judgment.)

Did I mention earlier I accept constructive criticism? I’m really on the verge of demanding it with output formatted like that.

And now, on to the next challenge – deploying the very same app internally. It sounded all sunshine and daisies until I realized the maze of requirements I’ll be wading through to make this happen. It’ll be procfile-esque troubleshooting for hours on end again, I just know it.

Look at you! You made it all the way to the bottom of the post. Did you skip the middle? Did you painstakingly read every word? Either way, I’m grateful for your attention to this first rant, and I look forward to sharing more of my experiences as I fall into proverbial pits and climb back out – hopefully the stick I placed across the pitfall will help at least one of you tiptoe across without falling in too!

Thanks for looking at the very first installment of LPT! Learn. Practice. Teach.