Rx (Reactive Extensions)

New Post: No Range() function in Ix++?

stugol — Thu, 09 May 2013 21:08:35 GMT

Hi. I'm trying out the Ix extensions for C++, and I notice there doesn't appear to be a Range function like in .NET LINQ. e.g.:

auto values = Range(0, 100);
auto n = from(values).where( ...etc... );

So....where's Range gone?

Commented Issue: Observable.Timer() and Observable.Interval() throw exception if period == 0 [13]

davedev — Tue, 30 Apr 2013 21:46:41 GMT

Even though the documentation says period == 0 is supported:

```
/// <param name="period">Period to produce subsequent values. If this value is equal to TimeSpan.Zero, the timer will recur as fast as possible.</param>
```

Here is an example exception stack trace:
```
at System.Reactive.Concurrency.ConcurrencyAbstractionLayerImpl.StartPeriodicTimer(Action action, TimeSpan period)
at System.Reactive.Concurrency.DefaultScheduler.SchedulePeriodic[TState](TState state, TimeSpan period, Func`2 action)
at System.Reactive.Concurrency.Scheduler.SchedulePeriodic_[TState](IScheduler scheduler, TState state, TimeSpan period, Func`2 action)
at System.Reactive.Concurrency.Scheduler.SchedulePeriodic[TState](IScheduler scheduler, TState state, TimeSpan period, Func`2 action)
at System.Reactive.Linq.Observαble.Timer.π.Run()
at System.Reactive.Linq.Observαble.Timer.Run(IObserver`1 observer, IDisposable cancel, Action`1 setSink)
(remainder elided)
```

The issue seems to be this code in __System.Reactive.Concurrency.ConcurrencyAbstractionLayerImpl.StartPeriodicTimer(Action action, TimeSpan period)__:
```
//
// MSDN documentation states the following:
//
// "If period is zero (0) or negative one (-1) milliseconds and dueTime is positive, callback is invoked once;
// the periodic behavior of the timer is disabled, but can be re-enabled using the Change method."
//
if (period <= TimeSpan.Zero)
throw new ArgumentOutOfRangeException("period");
```

Even the comment seems to indicate Period==0 is allowed.
Comments: Pull request: https://rx.codeplex.com/SourceControl/network/forks/davedev/RxPeriodZero/contribution/4608

Commented Issue: BehaviorSubject.Current Property [8]

davedev — Tue, 30 Apr 2013 14:18:44 GMT

Please add a Current or Value property to BehaviorSubject<T> returning the instantaneous value so that it can be used more elegantly as a backing field for observable properties where the containing class may need to extract the current value imperatively, without using it in an asynchronous query. This would avoid the more costly and verbose recommended alternative to use .FirstAsync().Wait() as of Rx 2.0.

Open discussion:
http://social.msdn.microsoft.com/Forums/en-US/rx/thread/7b5b6982-4d03-4d44-a450-efebd5b2102a/
Comments: Pull request: https://rx.codeplex.com/SourceControl/network/forks/davedev/RxBehaviorSubjectValue/contribution/4602

Commented Unassigned: Optimise ReplaySubject for specific cases [35]

LeeCampbell — Wed, 17 Apr 2013 21:46:42 GMT

Currently the ReplaySubject is a very versatile class that can perform 5 major use cases
1. Replay-One. Caching and replaying the last value.
2. Replay-Many. (i.e. a value > 1)
3. Replay-Period. Cache data for a period of time for late subscribers
4. Replay by count and time. A combination of caching by count and period.
5. Replay-All. Cache and replay all values.

It seems to me that the most common usage* of Replay, "Replay(1)" incurs the cost of all the Scheduling and Buffering of the other implementations (Stopwatches, Queues, ImmutableLists of ScheduledObservers and the relevant checks).

My proposal is this: as the _bufferSize_ and _window_ are readonly values of a ReplaySubject, a specialization of the inner workings of the ReplaySubject can be loaded at construction time. For example if the new Replay(1) constructor was called then an internal implementation could be resolved that was specific and optimized for the behavior required by a Replay-one subject :
* Use ImmutableList<IObserver<T>> instead of ImmutableList<SchduedledObserver<T>>,
* no StopWatch,
* no internal Queue of cached values
* no Time and queue depth checks on every OnNext

For Replay(n) where n>1, then an implementation with an intenal queue could be used. Still this implementation would have no need for the ImmutableList<SchduedledObserver<T>>, StopWatch and the time checks on the OnNext call.

The two constructors that take a Window and a bufferSize+Window, could share an internal implemenation as I doubt there would be much perf difference between the two.

Finally the Reply-all behaviour where the parameterless constructor is used, could use the Replay-Many implementation passing Int.Max as a param, but this would still needlessly be calling the Trim method for each OnNext, so again is a candidate for specialization.

*In my opinion Replay(1) would be the most used operation of the ReplaySubject or the associated Replay extension method.
Comments: The short answer is "Yes, we are seeing perf issues". We switched out a quick modification of the ReplaySubject and noticed a large change in the number of allocations being made. In ReplaySubject the root of the allocations seems to come from private ImmutableList<ScheduledObserver<T>> _observers; Which in turn ScheduledObserver<T> has private readonly ConcurrentQueue<T> _queue = new ConcurrentQueue<T>(); which is excessively allocating arrays. I say excessively, but really I mean redundantly as we have no scheduling requirements, so dont need to cost of the ScheduledObserver<T>. I will try to create a test pack to reproduce our findings. w.r.t your proposed solution, thanks for the thought. However, I see this as a work around/hack where perhaps a far better implementation can be provided. I imagine as Rx gains in popularity (which it certainly seems to be doing), there will be an expectation that it performs optimally. I have seen that Rx has gathered quite a ground swell in the Capital Markets/Investment Banking industry where often data is provided at a pretty quick rate (100s of values per second). With a high tick rate and unnecessary allocations happening Garbage Collection can have an impact on performance. Essentially, I don't think that new users should have to learn a hack to get assumed performance, when the library could be optimized. Having said this, I have taken a fork of the repo and am working on a proposed fix. I am not expecting someone else to do this for me, but thought I would raise an "Issue" for it so it can be tracked. Lee

Commented Unassigned: Optimise ReplaySubject for specific cases [35]

davedev — Wed, 17 Apr 2013 20:42:03 GMT

Currently the ReplaySubject is a very versatile class that can perform 5 major use cases
1. Replay-One. Caching and replaying the last value.
2. Replay-Many. (i.e. a value > 1)
3. Replay-Period. Cache data for a period of time for late subscribers
4. Replay by count and time. A combination of caching by count and period.
5. Replay-All. Cache and replay all values.

It seems to me that the most common usage* of Replay, "Replay(1)" incurs the cost of all the Scheduling and Buffering of the other implementations (Stopwatches, Queues, ImmutableLists of ScheduledObservers and the relevant checks).

My proposal is this: as the _bufferSize_ and _window_ are readonly values of a ReplaySubject, a specialization of the inner workings of the ReplaySubject can be loaded at construction time. For example if the new Replay(1) constructor was called then an internal implementation could be resolved that was specific and optimized for the behavior required by a Replay-one subject :
* Use ImmutableList<IObserver<T>> instead of ImmutableList<SchduedledObserver<T>>,
* no StopWatch,
* no internal Queue of cached values
* no Time and queue depth checks on every OnNext

For Replay(n) where n>1, then an implementation with an intenal queue could be used. Still this implementation would have no need for the ImmutableList<SchduedledObserver<T>>, StopWatch and the time checks on the OnNext call.

The two constructors that take a Window and a bufferSize+Window, could share an internal implemenation as I doubt there would be much perf difference between the two.

Finally the Reply-all behaviour where the parameterless constructor is used, could use the Replay-Many implementation passing Int.Max as a param, but this would still needlessly be calling the Trim method for each OnNext, so again is a candidate for specialization.

*In my opinion Replay(1) would be the most used operation of the ReplaySubject or the associated Replay extension method.
Comments: Hi Lee, Out of curiosity, are you experiencing real problems with the overhead incurred by the _count_ scenarios or did you simply notice that it's unnecessary in the implementation? Note that you can use `Publish(initial)` instead of `Replay(1)`. It uses __BehaviorSubject__ internally, which is essentially just a specialization of `ReplaySubject(1)`, perhaps due to its commonality as you've mentioned. If you don't need the initial value it can be excluded with __Where__, though in some cases you need to use __Select__ also if there is no sensible value to indicate exclusion in a particular domain; e.g., integers can be projected into nullable integers. For example: ```c# IObservable<int> xs = GetObservable(); IObservable<int?> ys = xs.Select(x => new int?(x)); IConnectableObservable<int?> p = ys.Publish(initialValue: null); IObservable<int> q = from x in p where x.HasValue select x.Value; q.Subscribe(...); ... p.Connect(); ``` \- Dave

Created Unassigned: Optimise ReplaySubject for specific cases [35]

LeeCampbell — Wed, 17 Apr 2013 16:38:17 GMT

Commented Unassigned: Rx.Net: Add Silverlight 5 support for portable library [34]

danneesset — Sat, 13 Apr 2013 19:48:15 GMT

Silverlight 5 should support all (or almost all) of the portable library API, please add it to the list of supported platforms for the portable library.

By not supporting Silverlight 5 you prevent everyone else from building portable libraries compatible with SL5 from using Rx. If you wan't async support just utilize the microsoft.bcl.async package since it add async support
Comments: I've looked a bit at the code and I hope t can be solved quite easily by following the following steps. But it would require a new version of all platforms. 1; We need 2 different portable libraries, the current one plus one which is "more portable" (target platforms where IObservable) does not exist. This second portable library would then include the implementation of the IObservable interfaces. BTW: I think it would be a good idea to utilize the Microsoft.Bcl.Async package in order to add task and async support to nearly all platforms. 2; Add TypeForwardedToAttribute's in order to forward the IObservable interfaces to all builds which does not define the NO_RXINTERFACES (like the current portable library). In this way allassemblies compiled against the "more portable" will work with the current portable assembly as well as all current platforms assemblies. You can then choose any of the libraries to compile against with the new portable library allowing us to build portable libraries which target more platforms.

Created Unassigned: Rx.Net: Add Silverlight 5 support for portable library [34]

Daniel_Svensson — Thu, 11 Apr 2013 10:11:42 GMT

New Post: The trouble with Where()

adhemar82 — Wed, 10 Apr 2013 15:42:17 GMT

Dave, you are a gentleman and a scholar. Your suggestion is working like a charm. Not only that, my profiling shows the performance is just as good if not better than my homegrown solution where I was mapping subscribers myself.

As for the Replay semantics, fortunately in my use case I'm not concerned with that because I'm getting an infinite and stateless upstream feed, i.e., I don't care about previous values for one particular key and I already have to wait for a new feed to come for each key anyway. So I just put the GroupBy behind a Publish/Refcount pair.

Thanks so much!

New Post: The trouble with Where()

davedev — Wed, 10 Apr 2013 02:53:54 GMT

Hi,

How about using GroupBy or GroupByUntil with Publish?

For example: (Untested)

IConnectableObservable<IGroupedObservable<string, Foo>> foosByBar = 
    (from foo in foos
     group foo by foo.bar)
    .Publish();

foosByBar.Where(g => g.Key == "abc").Take(1).SelectMany(g => g).Subscribe(A);
foosByBar.Where(g => g.Key == "xyz").Take(1).SelectMany(g => g).Subscribe(B);
foosByBar.Where(g => g.Key == "bla").Take(1).SelectMany(g => g).Subscribe(C);

foosByBar.Connect();

GroupBy uses a dictionary lookup for every key to find the appropriate observable in which the value is pushed.

Publish broadcasts the group-by so that the dictionary lookup operation is shared by all observers.

Where / Take executes the predicate only once to locate the appropriate group, then it receives a broadcast of every value in that group along with any other observers interested in the same key.

Note that GroupBy doesn't replay IGroupedObservable so you must set up all of your subscriptions before connecting. If you'd rather use RefCount than Connect, then perhaps you should consider applying the Replay operator to the result of GroupBy.

- Dave

New Post: The trouble with Where()

ngm_codeplex — Wed, 10 Apr 2013 01:25:33 GMT

Do you really need separate subscribers?

If you know all keys at compile time, then you can use one subscriber with a switch statement for it's OnNext.

If you want to assemble a list of "handlers" at runtime, then yes, I think you may need a dictionary to make the lookup fast. But I think you can get by with one subscriber, and a concurrentdictionary. Use GetOrCreate (?) to register new handlers for each new key. You can even make a disposable which removes the handler from the dict.

If you want to register multiple handlers, then you may need locks. Unless you're ok with key->empty sticking around.

Hope that helps.

New Post: The trouble with Where()

adhemar82 — Wed, 10 Apr 2013 00:38:56 GMT

I love Rx, but I have a problem I keep running into.

Let's say we have a single upstream IObservable<Foo>, and N downsteam sequences attached to it, where each is only interested in those Foos that satisfy some simple predicate (say foo.bar == someKey).

Of course this is a simple job for the Where() operator:

IObservable<Foo> foos = ...;
foos.Where(foo => foo.bar == "abc").Subscribe(f => A(f));
foos.Where(foo => foo.bar == "xyz").Subscribe(f => B(f));
foos.Where(foo => foo.bar == "bla").Subscribe(f => C(f));
...
// Many more subscriptions for different bar values.

What will essentially happen here is that for each Foo produced upstream, the Where() predicate will be evaluated for that Foo N times. It acts like a linear search to find all subscribers that want this Foo. That's all well and good, and exactly what we (should) expect from using Where() here.

The problem I have is that in my case, N may be very large, but the subset of subscribers that want any particular Foo is very small. Typically, there will be only one for each Foo. This means I'm essentially doing a slow linear search when I could be doing a very efficient lookup to find the few downstream sequences that this Foo needs to be propagated too. My apps run in a very performance critical environment and I cannot afford this inefficiency.

I've racked my brain trying to find some elegant way of doing this more efficiently, but I can only come up with solutions that involve storing a lot of state (mapping subscribers, etc) and having to manage concurrency very carefully, which defeats a lot of the purpose of using Rx in the first place. I would prefer some way of dealing with this in terms of existing operators. Has anyone dealt with this issue before, or know of a good solution? I'm happy to provide more details.

New Post: IxJS on nuget

DGolubets — Mon, 08 Apr 2013 08:28:57 GMT

Hi,
do you plan to release ix.js and l2o.js to nuget? Would be nice to have them there.

Source code checked in, #e24677887e1727fb3b5dd614d996aa6d113b3834

Kirk Shoop — Mon, 08 Apr 2013 06:21:12 GMT

add BehaviorSubject

Source code checked in, #615efd71bf8acb8f5adcad40ca86eda2d41c4dcd

Kirk Shoop — Mon, 01 Apr 2013 04:18:25 GMT

change from_iterable to from(Iterate())

Source code checked in, #05a8bb7e0d563360023fafa74d244ec1adea440f

Kirk Shoop — Sun, 31 Mar 2013 19:09:25 GMT

adds from_iterable from_iterable(r, opt - scheduler) adapts any r that satisfies std::begin(r) and std::end(r) into an Observable<decltype(*begin(r))>

Created Feature: RxC++ : Implement Timestamp [33]

kirkshoop — Sat, 30 Mar 2013 18:07:35 GMT

Implement Timestamp operator

Created Feature: RxC++ : Implement Timeout [32]

kirkshoop — Sat, 30 Mar 2013 18:06:56 GMT

Implement Timeout operator

Created Feature: RxC++ : Implement Timer [31]

kirkshoop — Sat, 30 Mar 2013 18:05:51 GMT

Implement Timer operator

Created Feature: RxC++ : Implement Window [30]

kirkshoop — Sat, 30 Mar 2013 18:05:11 GMT

Implement Window operator