On thinking outside of the box – Alexey Raga

Alexey Raga

throw new InvalidIdeaException()

On thinking outside of the box

After I posted about optional values I have got all sorts of “counter examples”, like

But Option doesn’t help if you have a bad block on the SSD, and when the OS swaps the memory page and reads it back, then the value might be corrupted.

or > But now I have to pass Option<T> everywhere and it makes my code unreadable!

or > But now instead of checking for null I have to check for None everywhere, what’s the big difference?!

or > But often I don’t need Option<T>, I just need T, so I always need to extract the value!

And I was like “no, and it shouldn’t”, “no, it’s not”, “no, you don’t”, “no, you can’t”…

These and similar concerns (except maybe the first one) were expressed by different people which indicates that some misunderstanding is going on.

I didn’t expect that, and it made me think about the difficulties people may experience in understanding such a simple (as it seemed) concept. This semi-philosophical post is about that.

I think the root of the problem is…

People think outside of the box

OK, probably it is not what you mean when you say “think outside of the box”, so let me explain.

What I mean is that often people miss the idea of a context.

Let’s take the following statement as an example:

var newList = oldList.Select(x => DoSomething(x));

One way of reading this code is: enumerate a list, apply the function to each element, and have a new list back. This is correct, but it is a very procedural way of thinking.

Alternatively we could look at it as at DoSomething function applied inside of a context of a list. That’s it, list (or IEnumerable) is a context in which values exist, and when we want to do something with these values we apply a function within that context.

A context is some kind of a “box”. We can go inside the box and do whatever we want with its value. An after we finished, we have the same box with a new (transformed) value in it.

Why this way of thinking is “better”?
Because it is a pattern, and a very powerful one. Once the idea of contexts is realised this pattern can be applied again and again and again.

Let me give you a couple of examples.

IEnumerable<T> represents a list of values (Non-determinism).
Task<T> represents an ongoing computation which, when finished, results in a value (Future).
Option<T> represents one value that may or may not exist.

All these three are very different, but they are also very common in one thing: they all provide a context for their values. In each case we can say that the value if type T exists in a context of a promised future value (Task<T>), or in a context of optionality (Option<T>), or in a context of a list/non-determinism.

Then everything is simplified to: there is a value in some context, and I just want to do something with this value. I want to transform, or operate on the value that lives within a context. Which context? I don’t care much, but I know what to do with the value!

This is what LINQ is about, not about traversing IEnumerables! This is why LINQ is designed to be a set of extension methods so any context can benefit from it in the same way.

List<int> res1 = GetSomeNumbersList()
  .Where(x => x % 2 != 0)
  .Select(x => x * 2)

Task<Result> res2 = DownloadSomeDataTask()
  .Where(x => IsGoodData(x))
  .Select(x => ProcessData(x))

Option<string> res3 = LoadUserOption()
  .Where(u => u.Age > 21)
  .Select(u => u.FirstName + " " + u.LastName)

All these examples above use different context, but apply the same pattern: they deal with the value within the box (context).

Here is a blog post from one of the MS employees about how to use LINQ with Task<T> if you are interested, but here is one example from it:

var c = from first  in Task.Run(() => 1)
        from second in Task.Run(() => 2)
        select first + second;

We see it an operation on two “contextful” values. Would it look much different the context was not Task<T> but something else? It probably wouldn’t because the pattern is the same.

Just give me my value!

Why don’t we just get the value out of the context and just use it? Because it is generally not possible, and attempts to do so are usually dirty and lead to errors.

How can you extract a value from an IEnumerable<T>? If there are many, which one? And what if it is empty? You have to run your computation within the context of a list.
How can you extract a value from a Task<T>? The value is promised to you, true, but it may not be there yet, or even never in case if the task fails.
How can you extract a value from an Option<T>? It is impossible because the value may not even exist.

We cannot “just get the value out of the box”, most of the contexts don’t have this luxury. Therefore we have to operate on values within their boxes.

So the pattern is: get a box with a value, deal with the value within the box as much as you want, and you have a box with a new value. But you always have a box. The context is preserved. And it is not a bad thing at all because each context has its meaning.

When we apply a function inside the box, we say that we map a function over a context.

I myself hate analogies, but if you like them then I have one for you.
Imagine having a box with that Schrödinger’s cat. You don’t know if the cat is alive or not. And you know how to play with cats, meaning that you have a Play(cat) function. Then you can do box.Select(cat => Play(cat)) to entertain the cat within the box. As a result you still have a box, and you still don’t know if the cat is alive, but if it is then you have a box with a happy cat.
Of course this analogy is terrible, all of them are.

What all this has to do with Option?

Everything. Option is just a context that represents optional values. It is just another “box”. As we said, there is no difference between List and Option and other “boxes” when we look at them this way.

It is each box’s implementation details how exactly the function that is mapped over it gets applied. It is up to the Select method implementation for that specific context.
List would handle non-determinism and would apply the function to each of its elements. Option handles potentially non-existent values and would only apply the function if the value exists. Otherwise you have an empty box.
Other boxes do other things, but the concept stays the same.

So…

How does Option helps to compensate for bad blocks on disk?

It doesn’t. It is a different concern. And to be honest, I’d like to see the code when you actually compensate for these things.

Does Option pollute my code because I have to pass it everywhere?

You don’t pass Option everywhere to your functions, you simply don’t. It is possible for functions to declare their parameters as Option, but does not happen often.
What you would typically do is you would map the function over the context of an option, using the .Select method or a similar technique. Yes, you have an Option of a new value back, which is only fair. But you don’t pass Option to functions that supposed to get a “pure” value.

Do I need to check for None everytime?

Just as above, usually you don’t need to check if the value exists or not. Map over an option.
There are rare cases when you do want to pass an option to a function or do want to check it, but they are rare. And in most of these cases you will be using a method like .GetOrElse(defaultValue), which is a legit way to compensate for non-existent values according to the business logic.

How do you get T out of Option<T>?

This is the funny one. You don’t. How do you get something if it does not exist? You don’t. That’s the power: there is no way to use the value that doesn’t exist as if you had one.
What if you want to use the value? So do it. Map your function over an option and let the function use the value.

Doesn’t .GetOrElse(defaultValue) help you to get the value out of Option?

.GetOrElse(defaultValue) allows you to provide the defaultValue if you don’t have one in the Option. When you do it, the result value is not optional anymore. Now you always have a value, the whole concept of non-existence has gone, so the result of this function is just T, not Option<T>.
But it is not really escaping the Option. It is a safe way to compensate for non-existent values according to the business rules.

Conclusion

Thinking in terms of “boxes”, or “contexts” is often beneficial. It is a powerful pattern because there is a lot of commonality between contexts. Unfortunately C# developers don’t often think this way and C# doesn’t facilitate this way of thinking.

A bit of a hint: when there is some sort of a ceremony involved in handling a value it might be useful to think about which the context this value lives in. It could be all sorts of things: nullability/option, future, non-determinism, or something specific to your system. Embrace the context, don’t try to fight it with attempts to extract values by introducing if s and checks, etc.

Just to blow your mind completely, a function (or Func<T, R>) can also be seen as a value R that lives in a context where it can only be produced after a T is given :) Implementing a Select method for this context is not hard, you can try it as a challenge ;)
There are some “scary” terms that can be said about this example, and the contexts in general, but I tried to avoid them and to save them for another time.

Have a nice day. Use contexts.

Written on September 5, 2015