Monday, May 6, 2019

Simple Fixes

I'm not sure if I'm a lazy developer, but I like my code like I like my computer's desktop - minimalist. I can appreciate engineering a solution for flexibility and robustness but either my skills are way below the people who write such code or their code is really difficult to decipher and update - or simply implement. If I'm looking for reusable code snippets I can use, as soon as my eyes start to glaze over I move on and look for a simpler piece of code. I'm pretty sure that says more about my skills than the snippets authors.

A great example of my minimalist tendencies is a fix I made this morning. I don't code in Python but I currently support an image data extraction script written in it (So I'm doing a lot of hilariously noobie web searches). I need to run large batches through it via input file of DB record IDs. The trouble is it isn't 'resumable'. So if it works all day and crashes before the batch is done, it has to reprocess everything from the top.

I'm not sure what other developers would do, here's what I did:

If it doesn't exist, I create a file:
<input file name>.bmark.txt

Where the script reads the input file into a tuple, I set bookmarkid the first row of the bmark file.

For each row of the input file... If bookmarkid is not "", is the current input id the bookmarkid? If so, clear the bookmarkid.
So the next row in the input, we have a bookmarkid of "" so we add the input id.

Later as we run the ids through SQL, as each one completes, I replace the bmark file contents with the successful id.

When everything is done, I delete the bmark file, just to be tidy.

So that was my minimalist solution to the problem. I added a little file to persist the last successful id that was processed. And when the script is started, I grab any 'bookmarked' id and check for it (The code was already reading the rows into a tuple, I just skipped adding them to the tuple until the bookmarkid == ""), resuming at the next row.

No comments:

Post a Comment

From Shotcut to Kdenlive

So I've been using Shotcut for a while now, for my YouTube videos... and music videos. I love the application. Slicing clips, doing fade...