JohnI don’t get to code as often as I used to, but when I do and I solve a problem, I find it to be synonymous to the happy feeling you get when using cheat codes in games. Like, it can’t be this easy to win, right? It feels like when I need my minions to build things faster and I inconspicuously type “operation cwal” into the console. That was a StarCraft reference if you’ve never played it.
Anyway, let’s take this simplified scenario I had from earlier this week to show how the simple things in life make me happy. I had a 50 gig file where I needed to ignore the first 20 million records. I couldn’t quickly and easily open it in a text editor. So, I decided to write a python script to accomplish this:
row_counter = 0
ignore_count = 20000000
out_file = open('my_new_hefty_file', 'a')
with open('my_original_hefty_file.txt') as in_file:
for line in in_file:
row_counter += 1
if row_counter > ignore_count:
out_file.write(r)
out_file.close()Boom! Cheat code initiated and it only took about 10 seconds to execute. Awesome! Pandas DataFrame me 10 records to find that I forgot to grab my headers! Quick, add another cheat code:
out_file = open('my_new_hefty_file', 'a')
with open('my_original_hefty_file.txt') as in_file:
for line in in_file:
row_counter += 1
if row_counter > ignore_count or row_counter == 1:
out_file.write(r)
out_file.close()Hah! Awesome! Took another 20ish seconds to do and complete executing, but I won the game. Happiness ensues.
For the code-pedantic readers, I completely understand there are better ways to do this or that this isn’t a solution but a quick hot fix. First, my problem was resolved. Secondly, this isn’t enterprise production-level code and I’ll probably never look at this code again.
Anyway, that is why I love coding. The excitement and shots of dopamine you experience when you tell a computer to do something AND it does exactly what you want it to do – AND potentially faster. You gotta admit, that’s pretty cool.
Happy coding!
