I couldn’t think of a better title, but infinite second-guessing is what has kept this
blog update-less for two years. No more.
Making computers do stuff I want
I’ve recently been expanding my knowledge of Bash and the *NIX system in general1. It
has been extremely rewarding both as an intellectual hobby and in terms of increasing
my productivity and lower-level understanding of what exactly is going on under the hood
when I’m doing …well anything on a computer. In any creative or intellectual field
composed of an array of siloed information – a horizontal of verticals – the deeper down
you can get into the most number of verticals, the more useful your creative contribution
will be / the better you’ll understand the overall topic.
This excellent Wait But Why post
discusses this general concept quite a bit in the beginning and it really spoke to me.
I recommend you listen to it so it can speak to you also.
But I’m seriously digressing. This post is about how I set up some super simple and fun
Bash scripts in concert in order to solve an actual problem I often have:
I want to able to do certain minor custom command-line things on a number of different computers.
The tasks are simple: things like “open up vim inside a certain directory” or “transform
some string on my clipboard” – super easy stuff that doesn’t necessarily warrant an afternoon
of experimenting and blog-post-writing. But now I have a framework for myself to expand
upon, plus I learned some cool stuff along the way (I purposefully didn’t search Google
for a preexisting solution, which I am sure exists since I already use
something similar for dotfiles)
install.sh – this installs the scripts for you so you can use them right away.
directory and files example files – these serve as instructions. explained in readme.
the actual bash scripts.
The general idea is that the installer makes scripts executable and symlinks in a
place in your PATH, getting its relevant information from the directory and files
in order to be flexible and environment-semi-agnostic.
There are many things I should change2, so this blog post might quicky be reflective
of some past version of the repo rather than the present version you see but I’m sure
the general idea will hold true.
Serious things I learned:
The ~ character gets expanded by Bash. This can make weird things happen if you’re unsure
how you’re using the ~ in different parts of your script. I kept getting unknown
file or directory errors on ~/scripts when I was working on the chmod a+x part.
It was extremely confusing! which brings me to…
-xv is your best friend. Appending it like: #! /bin/bash -xv will activate debugging
and verbose output. Using that I was able to see the very important single quotes
in this output: chmod a+x '~/scripts/testing-executable'. Since the tilde there is
part of a literal string, Bash wasn’t expanding it to $HOME like I needed it to.
The NON--xv version of the output was simply: chmod: ~/scripts/testing-executable: No such file or directory
which I unfortunately could not make sense of for a little while :)
Comments, suggestions? Send me a tweet or email or pull request or postcard!
Recursion. What’s that? I dunno, should probably Google it….
Very funny, Google. (if you don’t understand that joke, you’ll understand it when you understand recursion jokes.) A friendlier web service starts off this way: “Recursion is the process of repeating items in a self-similar way.” The simplest example I can think of might be a spiral – it sort of goes around and in a little, and then goes around and in a little and goes around and in a little, etc.
Here’s another real-world example that is a little more procedural and will get our brains warmed-up to get fully into some real code:
You’re in front of a mailbox with these instructions:
if you are not holding an iPhone in your hand, open the next thing you can.
Since you have a thing (mailbox) that isn’t an iPhone, you open it and find a shipping box. It’s a thing that isn’t an iPhone so you open it and find an iPhone box. It’s not an iPhone and it’s a thing: you open it and have both an iPhone and a headphones box. You have an iPhone so you stop. Simple enough?
What if i left out the “if you’re not holding an iPhone” part? Your instructions would be:
open the next thing you can.
You’d have gotten to the iPhone and headphones box part and then opened the next thing you could and you’d have a pair of headphones. Since you’re a good instruction follower, you’d maybe tear the headphones open and have some sort of little speaker in your hand. You’d rip that open and have some other electronics wizardry and keep going – before you know it you’d be ripping apart the fabric of space-time* – and that’s what trying to imagine recursion can feel like if you’re approaching it the wrong way: Like the fabric of your brain is being shredded by a guy on a wild hunt for an iPhone. Well, not specifically like that, maybe.
Getting closer to the code…
If we were to write the above instructions in psuedocode it might look like this:
if not iPhone, open_thing(next_thing)
Let’s notice and remember a few important things about the above code.
The open_thing method is called inside itself
When it is called within itself, the argument it takes is similar but not identical to the argument it takes outside.
There’s an if somewhere
You could say it is repeated in a self-similar manner. It doesn’t open the same thing again and again – it opens a next_thing. So It’s recursive! But in a special way because there’s also that if. The if defines our Base Case. The thing that tells us to stop recursing. To avoid tearing open the headphones.
I forgot to tell you why I’m writing this. Recursion can be a bewildering topic for some. Before this post my understanding of it danced around the periphery of the concept. Yea, I knew what it meant and how it generally worked and about fractals and all that, but I couldn’t intuit the workings of it – and more importantly, relative to my current endeavor: I didn’t get how the heck a computer makes sense of it. I don’t want to take such an interesting, complex and pragmatically useful topic on faith. I want to get it. If you feel the same way then please read on. I promise your understanding will be different by the end.
Back to Learning!
As I see it, the typical confusion when thinking about recursion (in terms of programming) comes from two sources:
Poor understanding of how the base case works
Neglecting to visualize how the computer will interpret your code
The idea of repeated iteration is not foreign to a programmer. We each, loop all the time (heh). The key is to realize that recursion with a base case is similar to iteration – it’ll stop eventually – and that recursive functions get evaluated exactly the same as any other code.
Visuals help and as always:
Understanding is reached by moving methodically in very tiny steps while replacing large complex chunks with smaller, digestible ones
So let’s do that!
Visuals, Examples, Graphs, Yay!
Here is some Ruby code for a real recursive function that computes the Factorial of a number:
If you do factorial(5) you’ll receive 120.
Uh, ok I guess that makes sense. But how?
WELL I’M GLAD YOU ASKED I’M SO EXCITED TO SHOW YOU! (read on)
Let’s focus on this important line: number * factorial(number-1)
When we pass 5 into the function, the first thing it hits is the if block. 5 is not equal to 0, so we land in else – The important line I mentioned above.
let’s evaluate it from left to right, replacing number with its value, 5…
5 * (factorial(5-1))
and again evaluating further… 5 * (factorial(4))
Well hold on now. factorial(4) needs it’s own evaluation. Back to the important line above, when we pass 4 into factorial(number) we get: 4 * (factorial(4-1))
or 4 * factorial(3)
So now altogether we have something like:
5 * ( 4 * factorial(3))
Likewise, continuing to evaluate the Ruby expression factorial(3) we arrive at:
3 * factorial(2)
Replacing the expression with this value in our running chain we get:
5 * ( 4 * ( 3 * factorial(2)))
Following the trend of evaluation…
5 * ( 4 * ( 3 * ( 2 * factorial(1))))
…and one more time:
5 * ( 4 * ( 3 * ( 2 * ( 1 * factorial(0)))))
Hang on, something is different now. Look back up to the full method. Evaluating factorial(0) will land us in the if rather than the else part of the block. This part returns the value 1, not another call to the factorial method! Let’s replace that nice gentle number into our chain:
5 * ( 4 * ( 3 * ( 2 * ( 1 * ( 1 )))))
Every time we get a bundled-up expression rather than a nice, unpacked, math-able return value, we’re going to continue interpreting and replacing until we end up with something upon which we can do arithmetic.
This is a dense topic and it can never hurt to be more clear – a small confusion will recursively grow larger with no base case in sight! – so let’s make it a little more visually-appealing.
Here’s stepping through factorial(5) like an interpreter..
It’s beautiful. When the interpreter can finally stop recursing and start returning, the big stack of self-similar calls falls back into itself like an imploding building. Finite recursion, while dense, is not entirely elusive. It’s infinite recursion that is impossible to imagine (you think you’re imagining it right now but you’re merely imagining an infinitely small portion of it.)
Recursion is Fundamental to our Species
Maybe you’re sitting there thinking,
Duh… computers are stupid. This is boring.
That’s ok! Recursion is still a really cool concept and has connections to biology and other parts of the natural world. Do you think humans am stupid, too? There are theories that recursion is an integral part of what elevates human cognitive abilities above other non-human animals.
You can sort of see this yourself: (solipsists, look away now) I know that you’re thinking and I know that you know that I know it.
What is that if not a self-similar repetition?! Don’t take for granted the fact that you can imagine that very complex relationship with ease. I love my cat, but there’s no way that he knows that I know that he’s a cat. In his head it’s all just like, “cat cat cat cat cat”.
(I just tried to imagine his imagining without a base case to rescue me and almost got stuck in his head!)
It only takes a small stretch of imagination to entertain the idea that our ability to recursively model other minds is at the root of our capacity for empathy. That is cool.
* Ripping Spacetime, pretty awesome speed metal band name?
Earlier tonight I was flipping through Jason Arhart’s Speakerdeck presentation on the basics of Test-Driven Development and giving a little thought to the topic in general.
A little overview of TDD
TDD is a system in which you create a suite of tests to ensure your code stays in line during development. It takes advantage of a few truisms that surround programming:
Repetition is key to testing
Computers are good at repetition
Humans Programmers like automating repetitive tasks
“Why use TDD instead of whatever it is I normally do?”
– You, sounding a bit silly.
Well, see what you said there? The second part of the sentence? You most likely have a general plan for your code, write a little, play with it and see if it breaks, write a little more, etc. Before you know it you’ve got a whole bunch of code, testing it becomes cumbersome and your test coverage starts to get a little spotty. Worst of all, you don’t know when you’re missing something. – That is, until something breaks a little later.
“Ok, I guess that doesn’t sound so smart. What’s this thing you’re talking about?”
– You again, wising up.
This thing is test-first development. Before you write a line of production code, you’re going to write at least one (failing, for now) test that describes what your program should do when it’s working properly. By keeping the end goal in your sights you minimize the risk of diluting your focus throughout the coding process.
You start with a failing test and then write only enough code to make it pass
Then you write some more tests (a bunch, if you like) and write only enough code to make each of those tests pass, in order.
(check out the --fail-fast flag, when the time is right).
All of this careful attention to the intended end result has an added bonus, as if having working code isn’t pleasant enough: Refactoring is safer and more fun. The adage goes:
Red | Green | Refactor
And then [repeat]. Once you’ve got passing tests, refactor, look out for red (make it pass again) and continue. Never refactor unless all your tests pass.
It nicely echoes Kent Beck’s “Make it Work, Make it Right, Make it Fast”.
“That sounds great! Aren’t there any downsides, though?”
– You, bringing up a good point…
“Sure”. is the answer. “False Positives” is one elaboration of that answer. There is no clear way to test the tests, aside from being really careful and diligent when writing them – but even then there are bound to be mistakes sometimes. That’s the whole reason you write tests to begin with: to account for your mistakes or normal human inability to complete every complex task with 100% accuracy on the first try.
Another pitfall is the tendency to veer towards The Happy Path whenever possible. Your tests are only as good as your natural or tailored skepticism. Doubt yourself often and rigidly.
A final thought that intrigues me
…in the context of TDD. Looking at a full suite of tests with its such-and-such method should do this-thing
evokes a curious idea… Yes, that method should do that thing. Why should I have to tell it how? It seems like having a computer figure out how some not-yet-written code should behave based on some simple human-readable instruction is the next logical step in the process. As a programmer, you switch gears from “Test-Designer Overlord” to “Code Implementer Monkey” during the TDD process. Doesn’t the former sound more human and the latter more machinated? The same way an automatic transmission knows to shift when we want the car to drive at a certain speed, how long do we have to wait until an interpreter knows how to write a method, given some discrete instructions? Uh oh, sounds like some(many thousand)one would be out of a job!
Confirmed, the first post went live. Octopress up and
running. I spent some time today messing around in Octopress with the purpose of customizing a few cosmetic things and was generally successful. It helped me
realize I need to add SASS to my list of technologies to study harder.
Now that this blog is ready to go I’m going to start using it! I’ve got to finish
the static About page as well.
Some topics I am interested in and will likely blog about in the future:
Attempting to better intuit recursion (if it’s humanly possible)
The beauty of data modeling
Related: data and the relationship between computer data and the real stuff
My tendency towards using more methods (and ones with verbose names) rather than
less code/nifty loops to accomplish the same goal
Meditation on the interaction between the concepts of human-readable code and
computer-legible code (the two don’t necessarily share any attributes)
(I’m pretty excited about this one. there are a million ways to write code that
a computer is ok with, but none of them necessarily make sense to a person and
there are a million ways to write instructions for humans that a computer will
not understand – but the constraint on both sides is where the real interesting
The solubility of programming problems. What I mean is: if you spend enough time,
follow the stack trace carefully, observe the program flow and diligently replace
variables, you will eventually solve the problem. That’s a cool thing. Not all
endeavors can claim the same.