Rx Discussions Rss Feed

New Post: Rx++ bug?! Catching OnNext exceptions.

kirkshoop — Sat, 15 Jun 2013 18:02:18 GMT

Good discussion. I am convinced that the CreatedObserver should not be redirecting exceptions into OnError.

Tonko, please open an Issue for this. Thanks for the help!

Kirk

New Post: Rx++ bug?! Catching OnNext exceptions.

davedev — Tue, 11 Jun 2013 23:58:40 GMT

DELETED

New Post: Rx++ bug?! Catching OnNext exceptions.

davedev — Tue, 11 Jun 2013 14:51:02 GMT

Hi Tonko,

That's correct.

As you mentioned, if you catch an exception thrown by an observer and do nothing with it, then you may be suppressing valuable diagnostic information and may cause corruption to user data by letting the observable continue to run in an unpredictable state. Essentially, an operator can't possibly handle an observer's exceptions in any meaningful way since it doesn't know anything about the origin of the exception or the state of the observer.

Developers seem to agree with the former, but sometimes they draw an incorrect conclusion with the intention of avoiding crashes: If an operator can't handle exceptions thrown by observers in any meaningful way, then simply pass the exception to OnError. But that's incorrect too because the OnError handler has no way of distinguishing between exceptions thrown by the observable and exceptions thrown by the observer, thus the observer cannot handle exceptions in any meaningful way either.

For example, imagine that an observable may throw FileNotFoundException. It's expected as a possibility, so you add an onError handler for contingency. Your onError handler creates the missing file (if in fact it's still missing - you have to account for the implicit asynchrony in the file system) and re-subscribes to the observable. At runtime, onNext throws FileNotFoundException, which indicates a bug in your code because you weren't aware that this exception might be thrown from within the observer; i.e., it's truly exceptional. What's worse is that your program can't automatically recover from this exception, so the correct thing to do would be to crash; however, if the operator were to pass the observer's exception to OnError, then instead of crashing the application goes into an infinite fail-check_file_exists-subscribe loop because it can't distinguish between a FileNotFoundException thrown by the observable (expected) vs. one that is thrown by the observer (unexpected). If the operator were to ignore exceptions thrown by observers instead of passing them to OnError, then the program would crash, greatly simplifying debugging. Furthermore, you could imagine a scenario where instead of an infinite loop the program continues executing from the onError handler as if the caught exception wasn't thrown by the observer, which is now in a corrupt state. When it comes to user data, we should always assume the worst possible scenario to avoid corruption and loss.

Note that although the above situation may seem unlikely and we like to believe that we actually have more knowledge and control about which exceptions will be thrown by our code and the code that we call into, that excuse is actually the very reason why it's so important to catch as few exceptions as possible. "Unlikely" is "exceptional". The previous scenario is entirely possible, yet unexpected. By not catching exceptions that we possibly can't handle we avoid that scenario altogether.

Conclusion
Given that observable operators can't handle an observer's exceptions in any meaningful way, passing the exception back to the observer may create unresolvable ambiguity, and it's the unexpected from which we have to protect user data, I conclude that it's best practice to let exceptions from user code crash the application.

That goes not only for exceptions thrown by observers, but also exceptions thrown by selectors and other user code that's passed as arguments to operators. The difference is that user code passed as arguments to operators can in some situations be handled safely by an onError handler as if the exception originated from the observable itself, though it should be clear now that the possibility of catching ambiguous exceptions also exists in this scenario. Therefore, just because you can add an onError handler doesn't necessarily mean that you should, and in general you shouldn't.

The correct approach when defining an operator is to not catch exceptions thrown by observers at all. Observers can add try...catch blocks around code in which they expect vexing or exogenous exceptions to be thrown. Handling these kinds of exceptions locally is the safest approach. Truly exceptional exceptions should not be caught by observers at all, let alone caught by observables and passed back to observers!

Here's an example subscription based on correct expectations:

xs.Subscribe(
    value =>
    {
        try
        {
            parsed = Parse(value);
        }
        catch (ArgumentException ex)
        {
            Log("Parse failed: " + ex.ToString());

            parsed = new ParserError(ex);
        }

        OnParserStateChanged(ParserState.Updated);
    }, 
    ex =>
    {
        // The observable has faulted, but if the operators we've used have been 
        // implemented properly, then we can be sure that any unhandled exceptions
        // thrown by the Parse method above will crash the application instead of 
        // being passed here.
        Log("Parse observable failed: " + ex.ToString());

        OnParserStateChanged(ParserState.Faulted);
    });

Dave

New Post: Rx++ bug?! Catching OnNext exceptions.

Tonko — Tue, 11 Jun 2013 03:51:33 GMT

Thanks for this clarification Dave!

Let me see if I can come up with a correct, illustrative interpretation of this part:

Note: do not protect calls to Subscribe, Dispose, OnNext, OnError and OnCompleted methods. These calls are on the edge of the monad. Calling the OnError method from these places will lead to unexpected behavior

Imagine that OnNext call, after a, more or less, complex sequence of calls, ends up executing the code in some third-party, security-sensitive library.

The writer of that library throws a specific exception on some specific input.

The intention of the library writer is: if you read the documentation and know enough to go and catch that particular exception, it is your responsibility to handle it properly. However, if you do not know about it, your intent may be malicious and, in that case, the safest course of action is that program crashes, preventing more damage.

Reactive Framework, if doing catch(...) of all exceptions coming out of OnNext(), interferes with that intent. While gallant in preventing the outright crash, the catch all is not expected behavior.

New Post: Rx++ bug?! Catching OnNext exceptions.

davedev — Mon, 10 Jun 2013 20:53:15 GMT

Hi Tonko,

§6.4 addresses user code, except standard observer callbacks. It should probably be made clearer in the doc.

An example would be the selector function of the Select operator. The following is a bad implementation because it doesn't protect user code:

public static IObservable<R> Select<T, R>(this IObservable<T> source, Func<T, R> selector)
{
    return Observable.Create<R>(observer =>
    {
        return source.Subscribe(value =>
        {
            observer.OnNext(selector(value));
        }, 
        observer.OnError, 
        observer.OnCompleted);
    });
}

The naïve approach would be to wrap observer.OnNext(selector(value)) in a try...catch block, but that's incorrect as well because it catches exceptions thrown by the observer (the second paragraph that you cited).

Here's the correct implementation:

public static IObservable<R> Select<T, R>(this IObservable<T> source, Func<T, R> selector)
{
    return Observable.Create<R>(observer =>
    {
        return source.Subscribe(value =>
        {
            R result;
            try
            {
                result = selector(value);
            }
            catch (Exception ex)
            {
                observer.OnError(ex);
                return;
            }

            observer.OnNext(result);
        }, 
        observer.OnError, 
        observer.OnCompleted);
    });
}

Dave

New Post: Rx++ trampoline bug?!

Tonko — Mon, 10 Jun 2013 20:40:05 GMT

Yes, thanks!

New Post: Rx++ bug?! Catching OnNext exceptions.

Tonko — Mon, 10 Jun 2013 20:32:55 GMT

Well, according to Rx Design Guidelines pdf, protecting the user code from within an operator is ok:

6.4. Protect calls to user code from within an operator
When user code is called from within an operator, this is potentially happening outside of the execution context of the call to the operator (asynchronously). Any exception that happens here will cause the program to terminate unexpectedly. Instead it should be fed through to the subscribed observer instance so that the exception can be dealt with by the subscribers.

However, the following paragraph is what (now) I have hard time to understand. Does it only say that you should not call OnError from within OnNext. for example? I'll try find some related discussion in old Forum:

Note: do not protect calls to Subscribe, Dispose, OnNext, OnError and OnCompleted methods. These calls are on the edge of the monad. Calling the OnError method from these places will lead to unexpected behavior

New Post: No Range() function in Ix++?

kirkshoop — Mon, 10 Jun 2013 18:30:43 GMT

It did not disappear, it just hasn't been implemented yet (although I will point you to the int_range impl in the unittest for an example of how to fill in until it is available)

Kirk

New Post: lx++ groupby / aggregate

kirkshoop — Mon, 10 Jun 2013 18:27:52 GMT

The thing that is wanted here is template lambdas.

Without template lambdas the only thing a user can do is to build template functors by hand instead of using lambdas - yuck.

What Ix++ can do is to prettify the internal names like _List_iterator to make this a little less of an eyesore.

Kirk

New Post: Rx++ trampoline bug?!

kirkshoop — Mon, 10 Jun 2013 18:14:25 GMT

Thanks for the report.

I believe that this has been fixed.

Kirk

New Post: Rx++ bug?! Catching OnNext exceptions.

kirkshoop — Mon, 10 Jun 2013 18:12:28 GMT

Thanks for the report.

I think the code in CreatedObserver::OnNext is correct. When you introduce Schedulers an uncaught exception here would unwind the scheduler thread (not the callers thread) and the exception would either be swallowed or cause the process to exit.

Passing an OnError handler to subscribe will allow the caller to handle the exception.

Kirk

New Post: Ix-Main Nuget Package - Silverlight 5 support

davecz — Mon, 03 Jun 2013 16:12:58 GMT

Would it be possible to add Silverlight 5 libraries to the Ix-Main NuGet package? As of now, there is just Net35/4/45 and SL3-WP. And the SL3 libs conflict "native" SL5 libs.

New Post: lx++ groupby / aggregate

dlgd — Sat, 01 Jun 2013 21:53:26 GMT

Hi all,

I'm trying to use lx++ groupby in conjunction with aggregate but I couldn't come up with anything simpler than this:

  auto ints = {1,2,3,5,6};
  auto aggrInts = from(ints)
    .groupby([](int x)
             {
               return x % 2 == 0;
             })
    .select([](const cpplinq::group<std::_List_iterator<int>, bool> &a)
            {
              return from(a).aggregate(std::multiplies<int>());
            })

Can anybody think of another construction that would avoid the direct use of cpplinq::group type?

Thanks.

New Post: No Range() function in Ix++?

stugol — Thu, 09 May 2013 21:08:35 GMT

Hi. I'm trying out the Ix extensions for C++, and I notice there doesn't appear to be a Range function like in .NET LINQ. e.g.:

auto values = Range(0, 100);
auto n = from(values).where( ...etc... );

So....where's Range gone?

New Post: The trouble with Where()

adhemar82 — Wed, 10 Apr 2013 15:42:17 GMT

Dave, you are a gentleman and a scholar. Your suggestion is working like a charm. Not only that, my profiling shows the performance is just as good if not better than my homegrown solution where I was mapping subscribers myself.

As for the Replay semantics, fortunately in my use case I'm not concerned with that because I'm getting an infinite and stateless upstream feed, i.e., I don't care about previous values for one particular key and I already have to wait for a new feed to come for each key anyway. So I just put the GroupBy behind a Publish/Refcount pair.

Thanks so much!

New Post: The trouble with Where()

davedev — Wed, 10 Apr 2013 02:53:54 GMT

Hi,

How about using GroupBy or GroupByUntil with Publish?

For example: (Untested)

IConnectableObservable<IGroupedObservable<string, Foo>> foosByBar = 
    (from foo in foos
     group foo by foo.bar)
    .Publish();

foosByBar.Where(g => g.Key == "abc").Take(1).SelectMany(g => g).Subscribe(A);
foosByBar.Where(g => g.Key == "xyz").Take(1).SelectMany(g => g).Subscribe(B);
foosByBar.Where(g => g.Key == "bla").Take(1).SelectMany(g => g).Subscribe(C);

foosByBar.Connect();

GroupBy uses a dictionary lookup for every key to find the appropriate observable in which the value is pushed.

Publish broadcasts the group-by so that the dictionary lookup operation is shared by all observers.

Where / Take executes the predicate only once to locate the appropriate group, then it receives a broadcast of every value in that group along with any other observers interested in the same key.

Note that GroupBy doesn't replay IGroupedObservable so you must set up all of your subscriptions before connecting. If you'd rather use RefCount than Connect, then perhaps you should consider applying the Replay operator to the result of GroupBy.

- Dave

New Post: The trouble with Where()

ngm_codeplex — Wed, 10 Apr 2013 01:25:33 GMT

Do you really need separate subscribers?

If you know all keys at compile time, then you can use one subscriber with a switch statement for it's OnNext.

If you want to assemble a list of "handlers" at runtime, then yes, I think you may need a dictionary to make the lookup fast. But I think you can get by with one subscriber, and a concurrentdictionary. Use GetOrCreate (?) to register new handlers for each new key. You can even make a disposable which removes the handler from the dict.

If you want to register multiple handlers, then you may need locks. Unless you're ok with key->empty sticking around.

Hope that helps.

New Post: The trouble with Where()

adhemar82 — Wed, 10 Apr 2013 00:38:56 GMT

I love Rx, but I have a problem I keep running into.

Let's say we have a single upstream IObservable<Foo>, and N downsteam sequences attached to it, where each is only interested in those Foos that satisfy some simple predicate (say foo.bar == someKey).

Of course this is a simple job for the Where() operator:

IObservable<Foo> foos = ...;
foos.Where(foo => foo.bar == "abc").Subscribe(f => A(f));
foos.Where(foo => foo.bar == "xyz").Subscribe(f => B(f));
foos.Where(foo => foo.bar == "bla").Subscribe(f => C(f));
...
// Many more subscriptions for different bar values.

What will essentially happen here is that for each Foo produced upstream, the Where() predicate will be evaluated for that Foo N times. It acts like a linear search to find all subscribers that want this Foo. That's all well and good, and exactly what we (should) expect from using Where() here.

The problem I have is that in my case, N may be very large, but the subset of subscribers that want any particular Foo is very small. Typically, there will be only one for each Foo. This means I'm essentially doing a slow linear search when I could be doing a very efficient lookup to find the few downstream sequences that this Foo needs to be propagated too. My apps run in a very performance critical environment and I cannot afford this inefficiency.

I've racked my brain trying to find some elegant way of doing this more efficiently, but I can only come up with solutions that involve storing a lot of state (mapping subscribers, etc) and having to manage concurrency very carefully, which defeats a lot of the purpose of using Rx in the first place. I would prefer some way of dealing with this in terms of existing operators. Has anyone dealt with this issue before, or know of a good solution? I'm happy to provide more details.

New Post: IxJS on nuget

DGolubets — Mon, 08 Apr 2013 08:28:57 GMT

Hi,
do you plan to release ix.js and l2o.js to nuget? Would be nice to have them there.

New Post: Inconsistent API?

davedev — Sat, 30 Mar 2013 14:23:18 GMT

Hi,

Take a look at this video about hot and cold observables. There's also a section named Cold vs. Hot Observables in the official documentation.

.NET events are hot. Adding a handler does not cause any side-effects. The handler is not called until the next event is raised. Events that were raised before you added a handler are not seen by the handler.

Converting an event to an observable preserves this behavior. Subscribing to a hot observable does not cause any side-effects. The observer is not called until the next notification. Previous notifications that were pushed before you subscribed are not seen by your observer.

If you want to gather all notifications in memory and replay them to all subscribers, then you can apply the Replay operator.

-Dave