I blog about whatever is on my mind at the time. It’s typically about my career in software engineering, personal growth, personal hobbies, and occasional random thoughts.
I wrote this piece as a part of a weekly project update email at Bonusly. I wanted to give non-technical folks an understanding of why changes can take a long time to implement and why we sometimes need to do refactors. I iterated on it a bit to come to the correct metaphor, starting at the metaphor that our system is a “tangled mess of spaghetti” and ending at this version.
Today I learned about the Git --force-with-lease command.
I’ll restate a description of the command but it allows you to rebase and force push with less risks than using --force. If you had pushed your changes up to a remote branch and somebody else had pushed up new changes to that remote branch while you’re rebasing. After you finish rebasing, a normal git push will fail because the local refs don’t match the remote ones. Usually people will git push --force at this point because they were just rebasing their branch. However, this would wipe away commits on the remote branch. This is easy to do if you’re working on a branch and don’t realize that somebody else had commit to it.
Today I learned that DISTINCT statements can cause efficiency problems.
I was looking at some code that made a few joins across tables and used distinct for the column names. The query was causing high CPU load and it wasn’t immediately obvious from first glance why that was happening. You could imagine the query looked like this (with a few more joins and selects):
SELECT DISTINCT transactions.date, transactions.total_amount, transaction_items.amount, transaction_items.name
FROM transactions
INNER JOIN transaction_items ON transaction_items.transaction_id = transactions.id
WHERE transactions.date BETWEEN '2023-01-01' AND '2023-12-31';
From a high level, in order to execute this statement the database will filter for the rows that we need, grab the data we need from those rows, and return the unique ones. That seems simple enough for a database as long as the date column is indexed. However, the important part to consider is what the database needs to do to ensure all the data returned is unique. The database would need to keep all that data in memory someplace and use CPU time to sort these columns then remove the duplicates. Without the inclusion of a unique index, the sort will take time because there’s no guarantee that each column is unique.
Today I learned about MySQL transaction isolation levels.
I was trying to debug issues where concurrent processes didn’t seem to be locking rows in the database correctly. A simplified version of the offending code could look something like this (comments added for those unfamiliar with Rails):
# Reader: Assume the record we're processing is not processed
# Query for the row with ID 123 for table some_active_record_classes
some_active_record_class = SomeActiveRecordClass.where(id: 123).first
ActiveRecord::Base.transaction do # Open a database transaction
# Query for associated data from another table processed_items
processed_items = ProcessedItem.where(foreign_key: some_active_record_class.id)
# Lock the row with ID 123
some_active_record_class.lock!
# Requery the database for ID 123 to ensure we have the most up to date data after locking
some_active_record_class.reload!
# Early exit if the record is processed
return if some_active_record_class.processed == true
# Simulate long running logic by waiting for 5 seconds
sleep(5)
# Update the DB
some_active_record_class.update!(processed: true, processed_count: processed_items.size)
end # Release the lock, commit, and close the transaction
When running that code concurrently, you’d expect that only one process would do the updates and the other process(es) would exit immediately after grabbing the lock. However, we were seeing that all processes would update the database. Originally we thought the problem was that we weren’t opening a lock at all or we weren’t opening the lock correctly. We were correct on one point of the hypothesis: we were opening a lock but the logic to open it was incorrect.
Earlier this year, I got a new Asus RT-AX88U router for my home. We were having issues with the router randomly dropping internet connection and restarting itself. The range was also bad for our home.
We had some issues setting up PPPoE on the new router because it wasn’t obvious where to input the VLAN ID that we used for our old Netgear AC1750 router.
Luckily, I found this blog post about configuring an ASUS RT-N66U that worked well for our RT-AX88U.
Last year, we changed our internet service to a Gigabit line. The setup of Gigabit was pretty simple because our Internet Service Provider (CenturyLink) provided a free router. However, I wanted to use our router to ensure we had control over the hardware that performed our network and internet connections. In addition, the CenturyLink router underperformed our own router - the wifi range was too short to supply wifi to our entire home. So I did some investigating into how we could use our own router.
I was introduced to themes from this video.
After reflecting on my theme from last year and how I’m feeling now, I realize that I mostly want to continue what I established last year. I want to build on the foundation that I lay and expand it to areas that I feel like I’m missing.
I’ve decided my theme for 2022 is the year of foundation. I want to establish the foundation for who I am for the next few years.
2021 started as one of the most stressful periods of times for me as an adult. I typically keep my work life and personal life separated well but work stresses from the end of 2020 and beginning of 2021 affected my mood and seeped into my personal life. I was the lead for a project that was estimated to take 3 months and we needed to deliver in 2.5 months. To add, the project had an unmovable deadline due to hard-ish business constraints and hard government deadlines. There was immense pressure to deliver the project successfully and on time.
Warning: There will be major spoilers in this review, particularly about the beginning and end of the game.
I received The Last of Us Part 2 has a birthday present. I subsequently binge played the game in under 2 weeks, savoring every bit of it. The story of the game is very strong and it has good gameplay (mixed with some bad) to entertain in between the engrossing story bits.
Recently, I’ve been reading a lot to supplement my career development. I’ve slowly come to the realization that the greatest thing I can do to further my career is by improving my communication:
All of those are important but to really level up as an engineer you have to expand your impact on the organization. Instead of being singularly focused (how can I push out features faster?), a senior engineer has a broader organizational focus (how can we push out features faster?). Writing is one of the best ways to have impact. It has a multiplying factor that can touch a broad set of people. It gives people an opportunity to read and respond in their own time, at their own pace. This is especially true with COVID-19 and work from home mandates.
While at General Motors, I had a lot of opinionated discussions with friends. These conversations varied in topic and some of them involved ranking things in categories. After many of these discussions, I decided to create lists to rank the categories we discussed to compare with others. Unfortunately, I was the only one to create large and comrehensive tier lists.
This is a compilation of the tier lists I’ve created. I made 3 tier lists at GM (vegetables, fruits, desserts) and may add more tier lists in the future; I may also modify tier lists over time as I experience new items in a category or as my tastes change. I follow a scheme like fighting game tier lists: S > A > B > C > D. I also try to maintain ordering within a tier so the top item in the S tier is the best for the entire category. I’ll try to explain my reasoning for ratings and describe my criteria for rating each tier list. The criteria is an attempt to make my opinions at least partially objective. Tiers are definitely skewed by what I’ve been able to experience and have some natural inherent bias. In addition, I try to keep items general (like cake or pies) but if a specific item stands out of the general groupmfor positive or negative aspects, I’ll pull it out and rank it separately (like carrot cake).
The Creep movie series is a frightening exploration of a realistic situation with a mentally unstable person.
The realism of these movies is part of the reason they’re so scary. Instead of improbable situations or supernatural occurrences, everything is pretty much grounded in reality. The antagonist has mental health issues that drive him to be a serial killer and stalker. He acts just normal enough that you can believe his friendliness but he’s just weird enough to leave you unsettled. The antagonist’s weirdness is even explained in the first movie so his actions are all believable in the beginning. The protagonists give the serial killer multiple chances to explain himself but these chances don’t feel forced just to extend the movie. The endings of the movies just solidify the eeriness of the killer.