… or how long is a piece of string?
From an IceStorm perspective, there are two types of clients:
- Publishers: Clients that publish messages to topics.
- Subscribers: Servers that receive messages from a topic.
The exact number of supported publishers and subscribers depends on the total load imposed by them. Load is composed of several factors.
The first is message throughput, expressed as events per second (EPS). Message throughput is a function of latency and the total number of raw messages sent. By way of example:
- You have 1 publisher publishing 1 EPS to 1 subscriber: throughput is 1 EPS.
- You have 1 publisher publishing 10 EPS to 1 subscriber: throughput is 10 EPS.
- You have 1 publisher publishing 10 EPS to 10 subscribers: throughput is 100 EPS.
- You have 10 publishers publishing 10 EPS to 10 subscribers: throughput is 1000 EPS.
The second factor that influences load is message size. By multiplying EPS with message size, you can roughly determine the amount of bandwidth IceStorm will consume.
Bigger events also mean that IceStorm’s memory consumption will increase. However, memory use is not a primarily a function of the number of subscribers, but a function of the number of publishers and the number of events that are queued for subscribers. A single event published to one subscriber will only consume marginally less memory than a single event that is published to one hundred subscribers. This is because the memory allocated to an event is shared among all subscribers. Each subscriber has a separate event queue that points at the event, so queuing an event for an additional subscriber only consumes memory to store a pointer to the event in the queue, not memory to store an extra copy of the event.
IceStorm provides four quality-of-service modes for event delivery: oneway, twoway, twoway-ordered, and batched. Of these, oneway is fastest, followed by twoway, and then twoway-ordered. In particular, oneway events (if you can live with the limitations imposed by them) provide a big increase in EPS. For high-latency networks, you can also set batch delivery, which increases overall throughput by combining several events into a single physical protocol message. The cost is that events arrive at subscribers in bursts, and the bursts are separated by longer gaps than for unbatched delivery. For low-latency networks, batching does not provide any benefits. (See Ice Manual for more information on batch delivery.)
The best way to determine IceStorm’s maximum raw throughput is to run a latency test for the delivery mode you intend to use. For example, if you intend to use twoway delivery, run the C++ version of the twoway latency demo on your machine (see cpp/demo/Ice/latency). This demo measures the latency of sending twoway messages on a single machine. On one of my dual-core machines, the latency demo measures 0.149ms per message, which works out to 6,716 twoway messages per second. If you have more than two cores, you will achieve higher throughput (but throughput will not increase linearly with the number of cores).
To get a feel for the maximum bandwidth, you can run the C++ throughput demo (cpp/demo/Ice/throughput). Of course, this demo does not take into account the physical bandwidth limitations imposed by your network because client and server communicate over the loopback interface. However, you can also run the throughput client and server on separate machines to see how your network affects bandwidth.
To get optimum throughput from IceStorm for your application, you will need to play with the IceStorm configuration settings. The main setting you will be concerned with is the number of threads in the publishing thread pool. This setting determines how many events can concurrently be received and published. (Note that, if you are publishing messages to IceStorm with a oneway proxy and also require strict event ordering, you cannot configure more than one thread in the IceStorm thread pool.)
A good starting point for the thread pool size is to set the size equivalent to the number of cores on your IceStorm host. For example, assuming the name of the service is IceStorm, and IceStorm runs on a four-core machine, define the following:
IceStorm.Publish.ThreadPool.Size=4
As you can see, the question of how many clients IceStorm can support cannot be answered in isolation. To obtain a realistic estimate, you have to understand the application structure, the load imposed on IceStorm, the network and bandwidth restrictions, as well as the hardware on which IceStorm executes. Running benchmarks in an environment that resembles the deployment configuration as closely as possible is the best way to determine the total possible load that can be supported. The general guidelines I’ve outlined above should point you in the right direction.