Stochastic Monotonicity of Markovian Multi-class Queueing Networks

Multi-class queueing networks (McQNs) extend the classical concept of Jackson network by allowing jobs of different classes to visit the same server. While such a generalization seems rather natural, from a structural perspective there is a significant gap between the two concepts. Nice analytical features of Jackson networks, such as stability conditions, product-form equilibrium distributions, and stochastic monotonicity do not immediately carry over to the multi-class framework. The aim of this paper is to shed some light on this structural gap, focusing on monotonicity properties. To this end, we introduce and study a class of Markov processes, which we call \emph{Q-processes}, modeling the time evolution of the network configuration of any open, work-conservative McQN having exponential service times and {Poisson input}. We define a new monotonicity notion tailored for this class of processes. Our main result is that we show monotonicity for a large class of McQN models, covering virtually all instances of practical interest. This leads to interesting properties which are commonly encountered for `traditional' queueing processes, such as (i)~monotonicity with respect to external arrival rates and (ii)~star-convexity of the stability region (with respect to the external arrival rates); such properties are well known for Jackson networks, but had not been established at this level of generality. This research was partly motivated by the recent development of a simulation-based method which allows one to numerically determine the stability region of a McQN parametrized in terms of the arrival rates vector.


Introduction.
Multi-class queueing networks (McQNs) arise as natural generalizations of conventional Jackson networks: in McQNs each network station (server) acts as a multi-class M/M/1 queue, whereas in Jackson networks each server is a single-class M/M/1. McQNs are particularly suitable for describing complex manufacturing systems (to be thought of as assembly lines) as they allow jobs (or, in queuing lingo, customers) visiting multiple times the same station to have different service requirements and/or a different routing scheme. One can think of situations in which a piece entering the system undergoes some physical transformation during the process, hence its processing time (as well as its next destination) at a given station might depend on the processing stage. Other possible applications include packet transmission models in telecommunication networks and distributed systems in computer science, where packets/tasks of different types can be routed to the same server in order to control resource utilization.
Motivation: Unfortunately, when generalizing the setup from Jackson networks to McQNs much of the nice mathematical structure is lost. For instance, the following complications appear: (I) Although the network-configuration process is expected to be of Markovian type (provided that all service and inter-arrival times are independent, exponentially distributed), the underlying state space is intimately related to the queue and service policies employed by each station. In other words, while for Jackson networks the vector recording the length of the queue at each server defines a Markov process, in the multi-class model it is not immediately clear what the Markovian statevariable is. Although in some special cases one could easily guess what the Markovian structure of a McQN is, a unified approach is lacking. The Markovian modeling of McQNs is equally important for both theoretical and practical reasons. On one hand it allows one to use the powerful Markov process machinery to derive analytical properties of the underlying network-configuration process, while on the other hand it facilitates the use of standard simulation methodology which could prove useful for numerical evaluation purposes.
(II) Stability is arguably a crucial property of a queueing network, as it entails that the network is able to cope with all incoming workload. While for Jackson networks stability is equivalent to subcriticality (a numerical condition on the parameters of the network which requires that the nominal workload is less than one at every station), in the multi-class framework this is not the case, as illustrated by numerous counter-examples; see, e.g., [9,6,4,3,2]. This is a major drawback of the McQN model and determining the set of input (arrival) rates for which an arbitrary network is stable is a challenging open problem.
(III) Stochastic monotonicity is in general a desirable property since it allows one to predict the behavior of a network when particular parameters are varied; monotonicity is widely used in e.g. control and optimization. Jackson networks are known to satisfy the following (stochastic) monotonicity condition: if one increases the flow of jobs entering the network then the resulting number of jobs in the network (at any given time) will (stochastically) increase; this corresponds to the intuition that, while the incoming work flow increases, the network processing capacity remains the same, resulting in a higher congestion. Surprisingly however, this fact does not extend in a straightforward way to the multi-class framework; in fact, it is not clear what the 'network processing capacity' could mean under the multi-class paradigm. To the best of our knowledge, no stochastic monotonicity results of this type have been established for networks of servers which are used by multiple classes of jobs.
Question (I) has been partially addressed in [6], where Markovian state-descriptions for networks having general service and interarrival time distributions are presented for a range of usual (network) service disciplines; nevertheless, the examples given in [6] suggest merely a list of heuristics, rather than instances of a more general structure. Regarding question (II), much research has been invested in the 1990's into studying the stability of McQNs. The most successful approach is based on fluid-limits, an asymptotic technique originally introduced in [9] and further expanded in [6]. It has been established in this way [5] that stability implies, in general, subcriticality but the converse is only true in specific cases; in particular, unlike Jackson networks, McQNs generally do not admit analytic conditions for stability.
Finally, question (III) has attracted considerable attention over the past decades; see e.g. [8] and the references therein. The monotonicity became relevant recently, when we investigated [7] simulationbased methods for detecting the stability region. More specifically, in [7] we argued that a weak form of stochastic monotonicity ensures that the stability region (w.r.t. arrival rates) of an McQN is a star-shaped domain which can be investigated (i.e., approximated) by usual root-finding schemes of Robbins-Monro type. As we did not have any formal proof of the monotonicity property at that point, we verified it empirically; extensive simulation experiments, including the ones reported in [7], indicated that a certain form of stochastic monotonicity could be expected to hold true under rather general conditions.

Contributions:
The present paper addresses questions (I) and (III) above, in a direct way, but also contributes to question (II) indirectly. More specifically, we identify the following contributions.
1. We introduce a class of Markovian processes, called Q-processes, modeling the dynamics of any work-conservative McQN with exponential service-time and interarrival-time distributions; this, in turn, facilitates a standardized simulation routine for such processes, where queue-disciplines and service allocations can be regarded as parameters.
2. We develop a stochastic monotonicity concept tailored to Q-processes, and identify a nontrivial subclass of McQNs (which includes the particularly relevant class of FIFO networks) satisfying such a monotonicity property. Furthermore, we prove that stochastic monotonicity of a Q-process implies monotonicity with respect to arrival rates and with respect to time (when started empty), extending in this way the well known results from the M/M/1 queue and Jackson networks [7].
3. Finally, we show that for Q-processes satisfying this type of monotonicity conditions, the stability region (w.r.t. arrival rates) defines an open, star-shaped domain. In particular, this proves that the numerical method introduced in [7] for determining the stability region associated with a McQN works for the networks specified at 2. In addition, the class of stochastically monotone networks covers the leading examples in [7], hence our results formally validate the numerical findings in [7].
Approach and Challenges: In order to define a concept general enough to cover a wide range of queuerelated processes, we need to introduce a rather intricate, abstract formalism. In principle, we need to construct a state-space which is wide enough to accommodate various types of queue configurations and an underlying structure which allows one to define the (Markovian) transitions performed by the network configuration processes associated with usual McQNs. Furthermore, we shall introduce a new concept of stochastic monotonicity suited to Q-processes, as the classical monotonicity results proved in [8], which have been successfully applied to Jackson networks, do not provide satisfactory results in the multi-class framework. More specifically, we define the concept of F -monotonicity w.r.t. a (sub)class of increasing performance measures; when F is the full space of non-decreasing functions we call it strong monotonicity and otherwise we call it weak monotonicity. We stress that, while our concept of strong monotonicity agrees with that in [8], our weak monotonicity concept is essentially different from that in [8] since it does not require that a certain subspace of (increasing) functions is left invariant by the Markovian transition operator of the embedded chain.
Organization of the paper : The paper is organized as follows. In Section 2 we present a compact account of the mathematical concept of multi-class queueing network. Then Section 3 introduces the concept of space of multi-class configurations, formalizing the queue dynamics at a single-server system. Section 4 defines the concept of Q-process, which appears as a natural aggregation of spaces of multi-class configurations. In Section 5 we introduce and elaborate on a stochastic monotonicity concept tailored to Q-processes, while in Section 6 we analyze the stability region (with respect to arrival rates) associated with a stochastically monotone Q-process.

Multi-class Queueing Networks: Description
Following the exposition in [6], we consider a multi-server network, comprising ℵ ≥ 1 single servers, labeled 1, . . . , ℵ. The network is used by d ≥ 1 classes of jobs in such a way that each class k job, at any time, requires service at a fixed server, denoted by S(k); once the service at S(k) is finished, it either becomes a job of class l, with probability R kl , independently of all routing history, or leaves the system with probability We assume w.l.o.g. that ℵ ≤ d and that the mapping k −→ S(k) is surjective, i.e., any server will be used by the network. The probability routing matrix R = {R kl } k,l=1,...,d is assumed transient (substochastic), i.e., the series I + R + R 2 + . . . converges, which guarantees that any job eventually leaves the system, almost surely; in standard queueing language, the network is open.
Each class k has its own exogenous (possibly null) arrival stream, regulated by a Poisson process with rate θ k ≥ 0 and requires i.i.d. service-times, exponentially distributed with rate β k > 0, independent of everything else; a null arrival process corresponds to a class with no external input, which models, for instance, intermediate processing stages of a certain class.
Each server is allowed to select its own service protocol. Informally, a service protocol consists of a queue policy, i.e., an ordering rule which dictates the order in which arriving jobs will be served, and of a service policy which specifies which jobs (in a given configuration) will be served simultaneously and what fractions from the total service capacity will be received by each one.
The above introduced McQN concept extends many known classes of queueing networks. For instance, when the mapping k −→ S(k) is bijective (in particular, ℵ = d) one obtains a Jackson network. Moreover, if the service-rate for any class k only depends on the server S(k) then we recover the concept of Kelly network ; see [?]. Finally, if there exists only one class with non-null exogenous arrival process and all jobs have the same (deterministic) routing, visiting all classes exactly once (in the same order), then the network is called a re-entrant line; finally, a re-entrant line which is also a Jackson network is simply a tandem of queues. Re-entrant lines provide the most popular instances of McQNs, as they can be used to model manufacturing lines; see

Spaces of multi-class configurations
In this section we introduce the space of multi-class configurations, the basic formalism required for a unified Markovian representation of McQNs; a Q-process will be obtained by aggregating such blocks in a standard way.
Let K denote a finite set (of classes) and consider the space Q[K] of all finite (ordered) sequences p = (k 1 , . . . , k n ), for n ≥ 1, with elements k m ∈ K, for any 1 ≤ m ≤ n. We augment this space with the empty sequence, denoted by ∅ and denote Q[K] := Q[K] ∪ {∅}. The space Q[K] covers all possible configurations of a multi-class queue with classes k ∈ K, where jobs belonging to the same class are (probabilistically) exchangeable. For each p ∈ Q[K], we denote by (p) ∈ N K the composition vector, with components indicating the number of each type of digits present in the ordered sequence p. For arbitrary x := (x k : k ∈ K) ∈ N K we define x := k x k (the norm) and σ[x] := {k ∈ K : x k ≥ 1} (the support). Furthermore, (p) k will denote the number of k-digits in the sequence p, p := (p) will denote the length (norm) of the sequence p and σ[p] := σ[(p)] will denote the support of p. Finally, for p = (k 1 , . . . , k n ) ∈ Q[K] we let κ(p) := k 1 denote the leading digit of the (nonempty) sequence p; κ(p) will typically denote the class being served in the queue configuration p.
A queue-policy on Q[K] is a family of insertion operators {I k : k ∈ K} such that for each k ∈ K, the mapping I k : Q[K] −→ Q[K] inserts a k-digit in a sequence, after the last (current) k-digit. More specifically, for any k ∈ K there exists some mapping j k : Q[K] −→ {2, 3, . . .} satisfying: • For any p ∈ Q[K] it holds that j k (p) ≤ (p) + 1.
• If j k (p) ≤ (p) then k m = k, for m ≥ j k (k 1 , . . . , k n ) and I k (k 1 , . . . , k n ) = (l 1 , . . . , l n , l n+1 ), where Alternatively, if for p, q ∈ Q[K] we denote by (p, q) the concatenation of (ordered) sequences p and q, it follows that for any p ∈ Q[K] there exists a decomposition p = (p ′ , p ′′ ) with p ′ ∈ Q[K] and p ′′ ∈ Q[K], such that I k (p) = (p ′ , k, p ′′ ). The interpretation is as follows: p ′′ denotes the part of the queue which is overtaken by k (and this does not contain any k-digit), whereas p ′ is the part of the queue not overtaken by k, which contains (at least) κ(p) and we have j k (p) = (p ′ ) + 1.
On Q[K] we also consider the family {D k : k ∈ K} of deletion operators, i.e., D k :

Remark 1. By convention, any insertion/deletion operator is extended to Q[K]
in a canonical way, as follows: A priority ranking establishes overtaking rules within the queue; the interpretation is that each C ı represents a caste (a subset of unranked classes) and representatives of higher castes will always overtake representatives of lower castes. When each C ı is a singleton, or equivalently if ν = #K, then the priority ranking is called total, as ≺ becomes a total ordering on K.
The following example illustrates that many important (and intensively studied) models correspond to the class of queue policies defined above. Example 1. The most common queue policies, which are used in applications are the following: • The first-come-first-served (FCFS) policy is defined by j k (p) = (p) + 1, for any k ∈ K; that is, we have I k (k 1 , . . . , k n ) = (k 1 , . . . , k n , k), for arbitrary (k 1 , . . . , k n ) ∈ Q[K] and k ∈ K. This insertion operator corresponds to the decomposition p ′ = p and p ′′ = ∅.
• A static buffer priority (SBP) policy assumes that a priority ranking {C 1 , . . . , C ν } exists on K. Then we define j k (p), for p = (k 1 , . . . , k n ) ∈ Q[K], as follows: by convention, j k (p) := 2 if n = 1, or k ≺ k m , for m = 1, . . . , n, and set j k (p) := n + 1 if k ⊀ k n . This insertion rule corresponds to the following decomposition: p ′′ is the maximal backwards subsequence of (k 2 , . . . , k n ) satisfying k ≺ l for any l ∈ σ[(p ′′ )]. Note that, in the trivial case ν = 1 the resulting SBP policy reduces to the above introduced FCFS policy. ⋄ Let P [K] denote the simplex of all probability vectors over K, i.e., ω := (ω k : k ∈ K) satisfying ω k ≥ 0 and k ω k = 1. A service allocation is a mapping W : . Intuitively, W k (p) indicates the fraction of service the server allocates to each class k; by assumption, classes which are not present in the configuration p receive a null fraction. The mapping W is called a head-of-the-queue (HQ) service allocation if W k (p) = 1{κ(p) = k} and is called an orderinsensitive (OI) service allocation if there exists some mappingW : in words, the service allocation is a symmetric function which only depends on the composition vector, but not on the order of digits. Typical examples of OI allocations are: • egalitarian allocation, specified by the mapping i.e., service capacity is equally divided among classes present in the buffer.
• proportional allocation, specified by the mapping i.e., service capacity is divided proportionally with the number of representatives of each class.
• preferential allocation, specified by the mapping where a total priority ranking is assumed on K and κ(x) denotes the highest-ranked class in σ[x].
By convention, we extend a service allocation mapping W to Q[K], by setting W k (∅) := 0, for any k ∈ K; similarly,W k (x) := 0. Note, however, that In what follows, a (server) protocol will refer to any combination of queue policy and service allocation; accordingly, a protocol will be called of HQ or OS type, depending on what type of service allocation it employs. Our next definition introduces the basic structure of the Q-process formalism, the space of multi-class queue configurations.
• the families {I k : k ∈ K} and {D k : k ∈ K} define insertion, resp. deletion, operators on Q[K]; will be called a space of multi-class configurations over the set K. In words, a space of multi-class configurations is a space of ordered sequences endowed with a protocol. ⋄ In some situations of interest, it is possible to reduce the complexity of the full space Q[K], by identifying equivalent configurations; more specifically, we have: Definition 3. A space of multi-class configurations (Q[K]; I k , D k , W k : k ∈ K) over the set K is reducible if there exists an equivalence relation ∼ on Q[K] such that p ∼ q entails (p) = (q) and W k (p) = W k (q), I k (p) ∼ I k (q) and D k (p) ∼ D k (q), for all k ∈ K. In particular, the mappings I k , D k , W k are well defined on the quotient space Q[K]/ ∼, which will be called a reduced space of multi-class configurations. ⋄ is reducible then the equivalence relation ∼ extends to Q[K] via p ∼ ∅ iff p = ∅; in words, the empty configuration is equivalent only to itself. In particular, it holds that this completes the construction of the reduced space of multi-class configurations. ⋄ Example 2. Instances of reducible spaces of multi-class configurations are given below: In particular, for single class stations all service/queue policies are equivalent.
2. Under a combination of an HQ service policy with a SBP queue-policy, having priorities regulated by ı denotes the ordered subsequence obtained by only keeping from p the digits in C ı . In this case, it holds that In particular, if the priority ranking is total, i.e., ν = #K, we have 3. Under an OI protocol, the resulting space of multi-class configurations is reducible w.r.t. the equiv- Note that there is a natural distinction (which can be viewed as a duality) between HQ and OI protocols. More specifically, for an HQ protocol the service allocation is fixed while there is flexibility in choosing the queue policy; on the other hand, for an OI protocol, the queue policy is irrelevant (in fact, there is no ordered queue and the space of multi-class configurations reduces to N K ) while one can freely choose the service allocation.
The above formalism covers essentially all server protocols used in applications. For instance, • the usual FCFS, resp. LCFS, protocol results from a combination of FCFS, resp. LCFS, queue policy and a HQ service allocation; • a non-preemptive SBP server protocol corresponds to a combination of SBP queue policy and a HQ service allocation; • a preemptive resume SBP protocol corresponds to an OI with preferential service allocation; • the usual processor sharing protocol corresponds to an OI with proportional service allocation.
Finally, we note that in this modeling paradigm all queue policies are of non-preemptive type, i.e., newly arriving jobs are not allowed to overtake the job being already in service, even if this has a lower priority ranking, whereas protocols of preemptive resume type belong to the OI class.

Q-processes
The objective of this section is to introduce the concept of Q-process corresponding to an McQN, as described in Section 2. Let 1 ≤ ℵ ≤ d and assume that On the class of real-valued functions φ : X −→ R we consider the following linear operators: • I denotes the identity operator.
Finally, we are now in a position to formally define the concept of Q-processes, combining the notions introduced above.
• In the above definition, (0, k)-transitions correspond to external arrivals to class k, (k, 0)-transitions correspond to external departures from class k, while (k, l)-transitions correspond to class changes from class k to class l; in fact, one can regard 0 as an external virtual class.
• When a certain component Q[K i ] is reducible then the state space X in (1) can be reduced accordingly by replacing Q[K i ] by Q[K i ]/ ∼; then the 'equivalence class' process (defined on the resulting quotient space) still defines a Markov process on the reduced space, having identical behavior as the original one (lumpability). By reducing the state-space in accordance with the rules put forward in Example 2 one recovers the Markovian models described in [6].
A Q-process, as introduced in Definition 4, allows modeling of any McQN with exponential interarrivaltime and service-time distributions. In this modeling paradigm, the order of the digits in a given multiclass configuration represents the order in which the current jobs in the buffer will be served, in the sense that a lower ranked job may not begin its service between a higher ranked one. Moreover, an FCFS discipline is assumed within each class, or subset of classes having the same priority ranking; in addition, it is assumed that only the first representative (with respect to ordering) of each class present in the buffer may receive service. Therefore, a Q-process models what is called (in standard queueing terminology) a network with all stations operating under a head-of-the-line policy. Nevertheless, it can been argued that, from a probabilistic standpoint, it makes no difference how many jobs in the same class are processed simultaneously, but it is only their cumulated service fraction which is relevant; [7]. Consequently, the Q-process formalism covers, in fact, any McQN with exponential interarrival times and service times.
For a Q-process X = {X t : t ≥ 0} on X we denote by P t the associated transition operator, defined as for φ ∈ R X . We set P t (ξ, Ω) := P t (ξ, 1 Ω ), for arbitrary Ω ⊂ X. Since the generator in (2) satisfies A < ∞, it follows that P t = exp(tA), i.e., P t (ξ, φ) = [exp(tA)φ](ξ), for any ξ ∈ X and φ ∈ R X . In addition, Q := I + (1/λ)A defines a Markov operator for λ ≥ A and we have (by uniformization) In the remainder of this paper, we let furthermore, {Ξ ξ m : m ≥ 0} shall denote the embedded Markov chain associated with the Q-process X , started in ξ, with transitions regulated by the Markov operator Q = I + (1/λ)A. It follows by the uniformization property that X t is distributed as Ξ Nt , where N t is the number of jumps of a Poisson process with rate λ until time t ≥ 0. • A k corresponds to an external arrival to class k; • D i corresponds to a (possible) departure from station i.
Assume further that {J n : n ≥ 1} is a sequence of i.i.d. variables on E, such that for k = 1, . . . , d and i = 1, . . . , ℵ; note that the distribution of the J n 's is independent of the state, queue policies or service allocations. For ξ ∈ X, a sample path of the chain Ξ ξ m can be constructed recursively as follows: for m ≥ 0, • sample J m+1 , independent of everything else; An alternative, more efficient way to sample the process X is by means of a state-dependent embedded chain, as illustrated in [7]. Nevertheless, the scheme in Remark 3 is better suited for our further analysis.

Stochastic Monotonicity of Q-processes
In this section we introduce a new stochastic monotonicity concept tailored to Q-processes, as it seems that standard stochastic montonicity concepts for Markov chains/processes are not suited to this framework. In addition, we identify a nontrivial subclass of McQNs (which covers the class of FCFS networks) satisfying such a monotonicity property.
On the state-space X -see (1) -we introduce a partial ordering, as follows. If K is an arbitrary set of classes, we define the partial ordering ⊆ on Q[K] via p ⊆ q iff p = (k 1 , . . . , k m ) and q = (l 1 , . . . , l n ), such that there exists some increasing sequence ν 1 < . . . < ν m such that k ı = l νı , for ı = 1, . . . , m; in words, the digits of p are to be found among the digits of q, in the same order. By convention, ∅ ⊆ p, for any p ∈ Q[K], hence ⊆ extends in a natural way to Q[K]. Furthermore, if ξ = [p 1 , . . . , p ℵ ] ∈ X and ζ = [q 1 , . . . , q ℵ ] ∈ X then ξ ⊆ ζ iff p i ⊆ q i , for any i = 1, . . . , ℵ. Note that, by definition p ⊆ I k (p) (for any queue policy), for any k ∈ K i and p ∈ Q[K], hence ξ ⊆ f (0,k) (ξ) for any ξ ∈ X and k = 1, . . . , d. Our next definition introduces the concept of stochastic monotonicity for Q-processes.
Definition 5. Let F ⊆ I[X] denote some class of increasing functions. The Q-process having generator A is called F -monotone if there exists some λ ≥ A such that the transition operator Q n = [I +(1/λ)A] n maps F onto I[X], for any n ≥ 0; more specifically, A is called F -monotone if ξ ⊆ ζ and φ ∈ F entails , then we call that the Q-process strongly monotone; otherwise we call it weakly monotone. Note that strong monotonicity amounts to Q, hence P t , leaving invariant the space I[X].
Remark 4. F -monotonicity of a Q-process implies that its transition operator P t maps F onto I[X], i.e., φ ∈ F entails P t (·, φ) ∈ I[X], for any t ≥ 0. The concept of stochastic monotonicity defined in Definition 5 is essentially different from the similar concepts introduced in [8], where stochastic monotonicity amounts to Q leaving invariant some subspace of I[X]. As such, proving stochastic monotonicity requires different techniques than those developed in [8].
⋄ Stochastic monotonicity of a Q-process is a rather powerful property, as it implies monotonicity of expressions of type P t θ (ξ, φ) also with respect to θ and t; the following result (proven in the appendix) establishes that.

II. For any
is non-decreasing. ⋄

Stochastic Monotonicity in the Single-class Case (Jackson networks)
In this case, the state-space X of network configurations can be reduced to N d and ⊆ corresponds to the usual componentwise ordering ≤ (see Example 2). In addition, all (server) protocols are equivalent (resulting in the same Q-process).
We claim that Q maps I[N d ] onto itself; in fact, it suffices to prove that this is the case for each for (k, l) ∈ T . The last claim now follows by Theorem 5.4 in [8] by noting that x ≤ z entails either therefore, one concludes that the Q-process corresponding to a Jackson network is strongly monotone, meaning that x ≤ z entails P t (x, φ) ≤ P t (z, φ), for any increasing φ : N d −→ R and t ≥ 0.

Stochastic Monotonicity in the Multi-class Case
In this section we aim to extend the monotonicity results from Jackson networks to the multi-class framework. However, strong monotonicity does not carry over to multi-class networks, the main reason being that transition rates from a given pair of ordered states are not ordered.
We shall establish instead a weak type of monotonicity, with respect to i.e., the class of real-valued functions which are non-decreasing in the (total) number of jobs in the network. It is worth noting that F -monotonicity can be expressed by means of the embedded chain Ξ, as follows: X is F -monotone iff for each ξ ⊆ ζ and n ≥ 0 it holds that Ξ ξ n ≤ st Ξ ζ n , where ≤ st denotes the usual stochastic ordering on R. Our next result establishes F -monotonicity for networks in which each station serves one class at any time (the service capacity is indivisible); in particular, this covers networks employing overall FCFS or SBP protocols, but also hybrid networks combining the two types of protocols.

Proposition 1.
Assume that X is a Q-process satisfying W k (ξ) = 1{k = κ i (ξ)}, for i = 1, . . . , ℵ and k ∈ K i , where κ i : X −→ K i denotes the head of the i-queue and is defined according to the protocol employed by station i (either FCFS or SBP). Then X is F -monotone.

Remark 5.
The key step in establishing Proposition 1 (which is done in the appendix) amounts to constructing a Markovian coupling {(Σ ξ m , Σ ζ m )} m≥0 such that the processes Σ ζ and Ξ ζ have the same distribution, while Σ ξ appears as a delayed version of the chain Ξ ξ , i.e., is obtained by freezing a random number of transitions. This coupling is possible since ξ ⊆ ζ means that the same type of transitions observed by the ξ-chain will occur with exactly the same probability in the ζ-chain at some random (later) moment. This reasoning does not extend, however, to networks featuring general OI protocols (except the ones with preferential allocation scheme) since in this case, although the transitions of the ξ-chain are still among those of the ζ-chain, the corresponding rates will be, in general, different due to the total service capacity of the server being divided in different ways. Therefore, Proposition 1 does not cover (nor can be extended to) networks in which some station(s) employ an OI protocol with equalitarian or proportional allocation. ⋄

Stability of Stochastically Monotone Q-processes
In this section we study stability of Q-processes, where stability is understood in a Markovian sense, i.e., positive Harris recurrence, which in this context is equivalent to ergodicity. To start with, note that a Q-process is irreducible on X if and only if (I − R ′ ) −1 θ > 0; for instance, for reentrant lines this amounts to the input rate θ being strictly positive. In fact, if the condition is violated then there exist some 'dead' classes which do not receive any new jobs over time, so that the process will be eventually absorbed by the subspace of configurations obtained by removing the dead classes from the network; all other configurations are transient states. In what follows, we restrict the parameter space to The standard theory states that an irreducible continuous-time Markov chain is either positive (Harris) recurrent or leaves eventually any compact (finite) subset of X. Moreover, positive recurrence is equivalent to boundedness in probability and (asymptotic) stability, hence one can define the stability region associated with the Q-process X as follows: where P θ denotes the probability law of the Q-process with parameter (θ, β, R); throughout this section, β and R are fixed while θ is regarded as a variable vector. An alternative way to characterize stability is the following: let T ξ := inf{t > 0 : X t = ξ, X t− = ξ}, for ξ ∈ X 0 denote the (first) hitting time of the state ξ; then the process X is stable if and only if E ξ θ [T ξ ] < ∞, for any ξ ∈ X. Moreover, it suffices that the expected return time is finite (only) for some particular ξ; see e.g.
[?] for a comprehensive treatment of stability of Markov processes.
We say that φ : X −→ R is vanishing at infinity if there exists a sequence of exhausting compacts {Ω n } n∈N , i.e., finite sets satisfying Ω n ⊂ Ω n+1 and ∪ n Ω n = X, such that sup{|φ(ξ)| : ξ / ∈ Ω n } −→ 0, for n → ∞. Our next result, also proven in the appendix, shows that provided that the Q-process is F -monotone, with F including the negative of some function vanishing at infinity, the stability region Θ s is a star shaped domain in Θ, having the origin as a vantage point. In addition, the stability region can be characterized as the support of some continuous, decreasing functional.
In particular, the stability region Θ s is an open, star shaped domain in the parameter space Θ.
By Theorem 2 (combined with Proposition 1), for any FCFS or SBP network the stability region along any positive direction (one-dimensional manifold is an open interval of form (0, θ * ), where θ * = α * · v is some (stability) threshold, as it follows by taking φ(ξ) = exp(−α ξ ), for some given α > 0, in Theorem 2; in particular, the stability region associated with a reentrant line is an interval (0, θ * ) ⊂ R. This justifies the findings in [7], where such thresholds are numerically approximated under monotonicity assumptions.
Finally, our next result further elaborates on stochastic monotonicity of Q-processes and shows that, provided that the process is stable in some neighborhood of θ, then strict monotonicity holds for the equilibrium distribution; such a property is important in the context of [7].

Concluding Remarks and Discussion
This paper has addressed challenges that arise when generalizing Jackson networks to multi-class queueing networks (McQNs). In the first place, we have introduced the class of Q-processes, which provide an appropriate framework to model any work-conservative McQN with exponential service-time and interarrival-time distributions. Secondly, we have developed a stochastic monotonicity concept for such processes, which we proved to hold for a nontrivial subclass, covering FCFS and SBP networks (of both preemptive and non-preemptive type), but also hybrid networks in which each station is allowed to choose one of the aforementioned protocols. In addition, we generalized existing results for Jackson networks to McQNs by showing that stochastic monotonicity of a Q-process implies monotonicity with respect to arrival rates and with respect to time (when started empty). The third contribution concerns the property that, provided that stochastic monotonicity holds, the stability region (with respect to arrival rates) defines an open, star-shaped domain, thus rigorously justifying the approach proposed in [7]. Proposition 1 states that F -monotonicity holds for a wide category of networks, where at most one class is served at each station; this category covers e.g. networks employing overall FCFS, SBP protocols, but on the other hand restricts the range of models which can be analyzed. It is not clear whether or not monotonicity holds for networks outside this category, and therefore this issue constitutes an important subject for further study. While the technique developed in the proof of Proposition 1 does not carry over to general McQNs (including OI stations), numerical simulations seem to indicate that stochastic monotonicity (in the sense of Theorem 2 I.) goes beyond the setup of Proposition 1; see the Appendix, for a range of numerical results corresponding to a simple OI reentrant line.
In this auxiliary section we provide some complementary material supporting the facts presented in this paper. In the first part we present the proofs of the main results stated in the paper, while the second part consists of an extensive set of numerical results suggesting that stochastic monotonicity goes beyond the framework of Proposition 1.

Proofs of the Results
Proof of Theorem 1: I. Define for any 0 ≤ θ ≤ ϑ and 0 ≤ s ≤ t the expression where A θ denotes the generator corresponding to the arrival rate vector θ; since E(0, t) = exp(tA θ ) and and the claim follows from the fact that with the r.h.s. being nonnegative for any φ ∈ F , by the monotonicity assumption.
Finally, we note that both expressions in the r.h.s. of (6) define continuous functions, hence we only need to verify that ϕ is continuous at boundary points of Θ s . To this end, let θ * ∈ ∂Θ s and note that, since Θ s is open, we have ϕ(θ * ) = 0. On the other hand, ϕ = inf t ϕ t appears as the infimum of a family of continuous functions, hence ϕ is upper semi-continuous and we have 0 ≤ lim sup θ→θ * ϕ(θ) ≤ ϕ(θ * ) = 0; that is, lim θ→θ * ϕ(θ) = 0, which shows that ϕ is continuous at θ * . This concludes the proof.
Furthermore, by the previous step, using the identity π θ A θ = 0 (for all θ) one obtains Finally, we note that the r.h.s. above equals which is strictly positive for θ < ϑ, by assumption and the fact that π θ is fully supported on X.

Numerical Tables
In this section we consider a variation of the classical example of Lu and Kumar [1] (a preemptive SBP reentrant line) in which we replace the preferential allocation at each station by a proportional allocation. The network consists of two stations, four classes, s.t. K 1 = {1, 4} and K 2 = {2, 3}, having deterministic routing, as indicated in Figure 2. For our numerical experiments, we fix two sets of service rates β 1 , β 2 , β 3 , β 4 and evaluate by simulation