Sign in Subscribe
Salesforce

My First Project: Setting Up a Dynamic Tracker for Salesforce's Trailhead

Matthew Monroe

• 2 min read

Okay, so maybe I was a little too ambitious for my first non-work related project. Let me start by saying—Wow. This was and is a project that I almost regret committing to. I'm used to small-scale scripts - scripts intended to serve singular and simple purposes, such as automating linear tasks. This.. This turned into quite the endeavor.

I'm a man of many focuses. Call it ADHD, call it boredom—I call it diversity. I've often been referred to as a jack-of-all-trades, and I take pride in that. Knowledge is my driving force, and I've long dreamed of becoming an academic scholar. My five-year plan includes earning a bachelor's degree in Computer Science, followed by diving into Mechatronics. From there, who knows? Maybe I'll finally chase that Mechanical Engineering degree I’ve been fantasizing about since my teens.

One of my passions is Salesforce, the massive CRM platform used by thousands of companies worldwide. Over the past eight years, I’ve had the privilege of working with both Salesforce Lightning and Classic. Its robustness and flexibility have always amazed me. In fact, I loved working in Salesforce so much that I began taking courses on Trailhead, Salesforce’s education platform. Before programming became my main focus, I considered becoming a Salesforce administrator—a challenging feat I still hold in high regard. The certification exam is notoriously difficult; as Chris Gardner put it, “~65% of people fail their first certification exam.” But I digress—what better way to kickstart my portfolio than by building a tracker for my own Trailhead progress? Or so I thought...

Inspiration Hits (Sort Of)

Today, I learned one thing for sure: dynamic web scraping is far trickier than I initially imagined. My journey started with a burst of inspiration—oddly enough, after an intense VR Drums Rock session. Exhausted, sweaty, and out of breath, I thought, “This is the perfect time to start!” And so, I did.

Having some experience with HTML scraping, I thought I had a solid plan:

  1. Install BeautifulSoup.
  2. Ensure the target data is public and doesn’t require authentication tokens.
  3. Design the script structure.

These initial steps were quick and straightforward. With ChatGPT assisting, I sketched out a basic script and read it over for errors. However, I soon realized that this project would be much more complex than anticipated.

Discovering GraphQL

HTML scraping requires knowing the class names of the data elements you’re targeting, but when I inspected the Salesforce profile page with developer tools, I couldn’t find the necessary class information. After some back-and-forth with ChatGPT, I learned that Salesforce retrieves profile data using GraphQL, a concept I was entirely unfamiliar with.

Exploring the developer tools’ network section, I located the GraphQL API calls and realized that scraping wouldn’t work. Instead, I needed to find a way to access and use these API calls directly.

Chromedriver and Selenium Adventures

Assuming the HTML was populated with JavaScript, I decided to use Chromedriver and Selenium to dynamically render the page and extract the data. Setting up Chromedriver wasn’t too bad, though navigating its download page was a bit clunky. Even so, this approach led to roadblock after roadblock.

Frustrated but determined, I scoured the internet for solutions. Finally, I stumbled upon a GitHub repository with scripts that automated similar tasks. By exploring the code, I found the GraphQL variables I needed: earnedPointsSum and earnedBadgesCount. Victory! Or so I thought.

A Long Road to Success

While finding the right variables felt like a breakthrough, it was just the beginning. After hours of trial-and-error cycles—test, fail, debug, repeat—I eventually decided to scrap my work and start over.

This time, armed with what I’d learned, I focused on accessing the API directly rather than relying on scraping. At last, I successfully retrieved the data I wanted: badge count, points, completed trails, and more from my public Trailhead profile. “Victorious, at last!” I thought.

But little did I realize—I wasn't even halfway there.