The advantages of going deep on architecture, recording significant events, and opening up the doodle bottleneck

Sanity Check • No. 021

Oct 22, 2023

I’m Ben, and welcome to Sanity Check. The newsletter for tips, stories, doodles, and even some questions about working in the data field. Glad you’re here.

QQ: QUICK QUOTES

Here’s what’s new with me:

🎤 dbt Lab’s Coalesce conference was this week. I did not get to go in person, but there was still plenty to be excited about with the online talks. More on those talks once they go live on YouTube.

🚢 I’ve joined a challenge to post 30 mini-articles in 30 days. I’m already on day nine without breaking the streak. Follow along on Twitter/X to get all of it.

🗓️ Speaking of streaks, my publishing streak for this newsletter ended when I missed last week! Previously it took me ten months to restart my weekly writing. This time, I’m back after a week. Here’s to getting back on the horse!

FRESH FEATURE

3 Advantages Data Engineers gain from going deep on Analytics Architecture

It is easier than ever to stand up an analytics stack. Do not let the lack of difficulty translate to a lack of understanding. A firm grasp of what is happening under the hood will give you an advantage in three ways.

1) Quick Problem Resolution

When your data stack comes off the rails, time is of the essence. Reporting gaps grow larger. Hard-earned trust crumbles. Campaigns and comms are delayed.

In these high-stress moments, a deep understanding helps quickly identify where the issue has taken root and narrows the solutions to weed it out.

2) Identifying Opportunities

In a green-field project, it is quick to get going. After some time, initial implementation details begin to show their age. Refresh times slow down. Storage and compute costs come to a boil. Model sprawl undermines metric consistency.

At that time, the company will have two options:

Slash-and-burn to reestablish a green-field
Rework within the existing architecture constraints

The second option is cheaper and less disruptive. The second option also requires a knowledgeable guide to avoid repeating the same mistakes.

Bonus points if you proactively come to the team with these improvements.

3) Becoming a Knowledge Resource

When quickly clearing problems and offering new solutions, you grow a reputation. You become the go-to person for X. This opens the door to the most rewarding advantage - mentorship.

Mentorship is a win-win-win.

The mentee gets to level up.
The mentor reinforces their understanding.
The company improves its capabilities.

There is something special to seeing colleagues light up with personal aha moments. Your knowledge is valuable.

PERCOLATING PONDERINGS

🚧 Caution: opinions still settling, and I’d love to hear yours

How do you track significant events for analysts down the road?

Some events break the trend. These could be

a change in methodology

a product launch

a viral post

These changes are not self-evident to analysts coming in with fresh eyes. Keeping a record of notable events helps others quickly gain context.

Personally, I keep a markdown file with a running list of notable changes.

Adding that context to charts is even better, like how Google Trends notes their releases or how Evidence supports annotations.

These approaches are better than no context, but I'm genuinely curious - what works for you?

DATA DOODLES

Documenting analytics jargon, visually

This week, I’m adapting this section to share an interesting workflow using ChatGPT to enable me to make more doodles in less time.

I was given an iPad for my birthday in 2020. Since then, I’ve been trying to draw data diagrams and blog post artwork. I am no artist, so I call them doodles to avoid them being taken too seriously. Three years later, I have made over 300 doodles and settled into a personal style.

These doodles take more time than I care to admit, but they help my writing stand out. The desire to have a doodle to accompany each piece of content has constrained my publishing volume.

That constraint is coming to an end. Recently, I gained access to ChatGPT’s Vision and DALL·E 3 modules.

First, I used vision to have ChatGPT describe my drawing style.

Then I asked Chat to summarize the style as a prompt for DALL·E 3.

Next, I popped open a new chat to generate images. I passed in the suggested prompt from the first chat as the “Artist’s Style”, and provided a second input for the image I wanted.

They are not exactly my style. Much cleaner and professional than what I would be able to do. The images are usable - and that is a big speed unlock!

This approach was inspired by Nate Kadlac’s article “Discover your visual style using one word in Midjourney.”

I encourage you to try this out yourself. Whether you feed in your work or gather inspiration from other compelling visuals, you can quickly make new images in your style.