On thinking outside of the box
After I posted about optional values I have got all sorts of “counter examples”, like
Optiondoesn’t help if you have a bad block on the SSD, and when the OS swaps the memory page and reads it back, then the value might be corrupted.
> But now I have to pass
Option<T> everywhere and it makes my code unreadable!
> But now instead of checking for
null I have to check for
None everywhere, what’s the big difference?!
> But often I don’t need
Option<T>, I just need
T, so I always need to extract the value!
And I was like “no, and it shouldn’t”, “no, it’s not”, “no, you don’t”, “no, you can’t”…
These and similar concerns (except maybe the first one) were expressed by different people which indicates that some misunderstanding is going on.
I didn’t expect that, and it made me think about the difficulties people may experience in understanding such a simple (as it seemed) concept. This semi-philosophical post is about that.
I think the root of the problem is…
People think outside of the box
OK, probably it is not what you mean when you say “think outside of the box”, so let me explain.
What I mean is that often people miss the idea of a context.
Let’s take the following statement as an example:
One way of reading this code is: enumerate a list, apply the function to each element, and have a new list back. This is correct, but it is a very procedural way of thinking.
Alternatively we could look at it as at
DoSomething function applied inside of a context of a list. That’s it, list (or
IEnumerable) is a context in which values exist, and when we want to do something with these values we apply a function within that context.
A context is some kind of a “box”. We can go inside the box and do whatever we want with its value. An after we finished, we have the same box with a new (transformed) value in it.
Why this way of thinking is “better”?
Because it is a pattern, and a very powerful one. Once the idea of contexts is realised this pattern can be applied again and again and again.
Let me give you a couple of examples.
IEnumerable<T> represents a list of values (Non-determinism).
Task<T> represents an ongoing computation which, when finished, results in a value (Future).
Option<T> represents one value that may or may not exist.
All these three are very different, but they are also very common in one thing: they all provide a context for their values. In each case we can say that the value if type
T exists in a context of a promised future value (
Task<T>), or in a context of optionality (
Option<T>), or in a context of a list/non-determinism.
Then everything is simplified to: there is a value in some context, and I just want to do something with this value. I want to transform, or operate on the value that lives within a context. Which context? I don’t care much, but I know what to do with the value!
This is what
LINQ is about, not about traversing
IEnumerables! This is why
LINQ is designed to be a set of extension methods so any context can benefit from it in the same way.
All these examples above use different context, but apply the same pattern: they deal with the value within the box (context).
Here is a blog post from one of the MS employees about how to use
Task<T> if you are interested, but here is one example from it:
We see it an operation on two “contextful” values. Would it look much different the context was not
Task<T> but something else? It probably wouldn’t because the pattern is the same.
Just give me my value!
Why don’t we just get the value out of the context and just use it? Because it is generally not possible, and attempts to do so are usually dirty and lead to errors.
How can you extract a value from an
IEnumerable<T>? If there are many, which one? And what if it is empty? You have to run your computation within the context of a list.
How can you extract a value from a
Task<T>? The value is promised to you, true, but it may not be there yet, or even never in case if the task fails.
How can you extract a value from an
Option<T>? It is impossible because the value may not even exist.
We cannot “just get the value out of the box”, most of the contexts don’t have this luxury. Therefore we have to operate on values within their boxes.
So the pattern is: get a box with a value, deal with the value within the box as much as you want, and you have a box with a new value. But you always have a box. The context is preserved. And it is not a bad thing at all because each context has its meaning.
When we apply a function inside the box, we say that we map a function over a context.
I myself hate analogies, but if you like them then I have one for you.
Imagine having a box with that Schrödinger’s cat. You don’t know if the cat is alive or not. And you know how to play with cats, meaning that you have a
Play(cat) function. Then you can do
box.Select(cat => Play(cat)) to entertain the cat within the box. As a result you still have a box, and you still don’t know if the cat is alive, but if it is then you have a box with a happy cat.
Of course this analogy is terrible, all of them are.
What all this has to do with Option?
Option is just a context that represents optional values. It is just another “box”. As we said, there is no difference between
Option and other “boxes” when we look at them this way.
It is each box’s implementation details how exactly the function that is mapped over it gets applied. It is up to the
Select method implementation for that specific context.
List would handle non-determinism and would apply the function to each of its elements.
Option handles potentially non-existent values and would only apply the function if the value exists. Otherwise you have an empty box.
Other boxes do other things, but the concept stays the same.
Option helps to compensate for bad blocks on disk?
It doesn’t. It is a different concern. And to be honest, I’d like to see the code when you actually compensate for these things.
Option pollute my code because I have to pass it everywhere?
You don’t pass
Option everywhere to your functions, you simply don’t. It is possible for functions to declare their parameters as
Option, but does not happen often.
What you would typically do is you would map the function over the context of an option, using the
.Select method or a similar technique. Yes, you have an
Option of a new value back, which is only fair. But you don’t pass
Option to functions that supposed to get a “pure” value.
Do I need to check for
Just as above, usually you don’t need to check if the value exists or not. Map over an option.
There are rare cases when you do want to pass an option to a function or do want to check it, but they are rare. And in most of these cases you will be using a method like
.GetOrElse(defaultValue), which is a legit way to compensate for non-existent values according to the business logic.
How do you get
T out of
This is the funny one. You don’t. How do you get something if it does not exist? You don’t. That’s the power: there is no way to use the value that doesn’t exist as if you had one.
What if you want to use the value? So do it. Map your function over an option and let the function use the value.
.GetOrElse(defaultValue) help you to get the value out of
.GetOrElse(defaultValue) allows you to provide the
defaultValue if you don’t have one in the
Option. When you do it, the result value is not optional anymore. Now you always have a value, the whole concept of non-existence has gone, so the result of this function is just
But it is not really escaping the
Option. It is a safe way to compensate for non-existent values according to the business rules.
Thinking in terms of “boxes”, or “contexts” is often beneficial. It is a powerful pattern because there is a lot of commonality between contexts. Unfortunately C# developers don’t often think this way and C# doesn’t facilitate this way of thinking.
A bit of a hint: when there is some sort of a ceremony involved in handling a value it might be useful to think about which the context this value lives in. It could be all sorts of things: nullability/option, future, non-determinism, or something specific to your system. Embrace the context, don’t try to fight it with attempts to extract values by introducing
if s and checks, etc.
Just to blow your mind completely, a function (or
Func<T, R>) can also be seen as a value
R that lives in a context where it can only be produced after a
T is given :) Implementing a
Select method for this context is not hard, you can try it as a challenge ;)
There are some “scary” terms that can be said about this example, and the contexts in general, but I tried to avoid them and to save them for another time.
Have a nice day. Use contexts.