DES: STRENGTHS & WEAKNESSES

THIS ARTICLE PRESENTS THE FINDINGS OF AN EXTENSIVE THREE-STAGE SCREENING PROCESS TO EVALUATE AND SELECT THE MOST SUITABLE COMMERCIAL DISCRETE-EVENT SIMULATOR (DES) WITH GRAPHICAL INTERFACE FOR SIMULATION OPTIMIZATION. The process was based on a set of desired features (including a subset that was considered critical) and was applied to a collection of 52 commercial packages. The list of DES software was built from information provided by publications (Swain, 2005; Tewoldeberhan et al, 2002), the Internet and simulation practitioners. During the compilation of the list, though the focus was on DES, packages that had this modeling technique as part of a broader set of capabilities were also considered. The packages were screened and evaluated through a three-stage evaluation strategy along the lines of Tewoldeberhan et al (2002).

In the first stage, a set of criteria was selected as critical – namely, graphical model creation, communication with lower languages (C/C++, C# or Java), dynamic model updating, extensibility, model reusability, modularity and animation availability. Following a Web search, all the simulators that clearly did not have some or any of the critical factors were disqualified. In the second stage, vendors from the 23 remaining firms were contacted with a questionnaire about the specific implementation of the critical issues. Sixteen vendors provided thorough answers to our questions, while the rest did not satisfy the required level of information or did not answer at all. Based on the responses, nine firms were invited to participate in the final stage of study, of which seven (eM Plant, Extend, Flexsim, Micro Saint, Quest, SimCad and Workbench) accepted the invitation.

Vendor participation at this stage was conditioned on providing us with a fully functional version of the software and unlimited interaction with their technical support. The study was developed to reflect the needs of simulation developers and final users in terms of implementing a simulation optimization framework, and limited to built-in functionalities (no source code customization). Therefore, the evaluation was framed with four aspects in mind:

• capability of the DES to be integrated with user written optimizers and optimization libraries;
  
  
• calability of the combined DES-optimizer structure;
  
  
• flexibility in modeling any system within the enterprise; and
  
  
• ability to use historical data.

These aspects were broken down in various criteria (23 in total) to provide a clear picture of the capabilities of each package. Each of the criteria fell into the categories of:

1. model building structure,
   
   
2. user-defined elements,
   
   
3. interaction with external applications,
   
   
4. dynamic model updating, and
   
   
5. miscellaneous topics such as preemption and animation capabilities.

The generic aspects in the evaluation of DES such as simulation speed, model robustness, etc. were ignored to keep the size of the study manageable.

 To avoid confusion that may arise if the specific terminology used within each package was quoted, the following terms were used to standardize the reports:

• Flow items: Entities that move through the model.
  
  
• Blocks: Entities that delay (process) flow items.
  
  
• Elements: Any entity used to build the model (flow items, blocks and connections).
  
  
• Extensions: Features that allow the user to customize the functionalities of the blocks
  
  
• Parameters: Inputs required to define the behavior of an element.
  
  
• Dynamic update: Change implemented at run time by the user or an external application.

Evaluation Results

THIS SECTION SUMMARIZES the strengths and weaknesses of the seven software packages included in the third phase of the evaluation. While each of the software exhibited different levels of performance in most of the criteria evaluated, we found that for some of the criteria all the software performed at a superior level. These criteria were communication with lower languages (C/C++, C# or Java) and user-defined routing of flow items. In addition, all but one software package exhibited superior performance in accessibility and extensibility of built-in elements, model reusability and ability to run multiple simulations. For a detailed discussion of the strengths and weaknesses of all software, see the full version of this article online (http://lionhrtpub.com/orms/orms-10-07/frdes.html).

eM Plant v 7.6. This software has a very flexible set of tools for model building and manipulation due to strong adherence to object-oriented paradigms such as “composition”(hierarchical model building), inheritance and modularity. Elements are completely self-contained (modular) with respect to variables, arrays and functionality extensions. They can be accessed using pathnames reflecting levels of modeling hierarchy. eM Plant also has excellent tools for array creation (at any scope) and manipulation, as well as a unique tool that allows manipulating any parameter of any element in the model from a single window.

An important strength of this software is that user-defined elements are completely indistinguishable from built-in elements in terms of features and functionality. Other strengths include efficient communication with spreadsheets as well as databases, and the fact that the model structure and queueing policies can be updated dynamically. While built-in features are sufficient to accomplish most tasks, the software requires user-written code to handle the consequences of implementing preemption and to model non-empty starts. Some other shortcomings are the absence of an “Undo/Redo” functionality as well as the lack of clarity in various segments of the documentation.

Extend v 7.0. This software provides very flexible sets of tools to handle logic driven preemption, non empty starts and create (at any scope) and manipulate arrays. User-defined elements can be completely indistinguishable from built-in elements in terms of features and functionality.A custom environment is provided for their creation, but the development is completely code-based. Other strengths include efficient communication with spreadsheets as well as databases, and the fact that the model structure and queueing policies can be updated dynamically. In terms of object-oriented paradigms, hierarchical model building and modularity are completely implemented. However, from an implementation perspective, multiple blocks are needed to accomplish a single task, which significantly adds to modeling effort required to maintain modularity. Though the strategy of using blocks with basic events provides great flexibility, it results in an overwhelming number of blocks in large models, making them difficult to maintain. This situation is alleviated to an extent by the ability to centrally access any parameter of any block through a database tool. Other weaknesses include very limited implementation of inheritance concepts and the inability to define multiple types of flow items.

FlexSim v 3.5. This software follows the objected-oriented paradigms of modularity and hierarchical model building, but it does not implement inheritance concepts. Elements are completely self-contained with respect to variables, arrays and functionality extensions. They can be accessed using pathnames reflecting levels of modeling hierarchy. User-defined elements are found to be a big strength and are completely indistinguishable from built-in elements in terms of features and functionalities. Other strengths include efficient communication with spreadsheets, as well as databases and the ability of modeling user defined queuing policies. On the negative side, significant weaknesses are the absence of features to dynamically update the model structure and the constraint that the built-in functionality for resource management allows the consideration of only one type of resource. Other shortcomings include the need for user-written code to model non-empty starts and a cumbersome process to manipulate arrays.

Micro Saint v 2.2. Micro Saint doesn’t belong to the same class of software as the others evaluated. Its main objective is to provide a framework to create elements while avoiding the specifics of discrete-event simulation and animation from a programming perspective. However, it requires an entire logic coding exercise to mould the only type of block available according to the desired functionality (right from the smallest details). Due to the fact that our criteria were designed for simulators with built-in elements, the performance of this software may be misleading. Nevertheless, the software was evaluated, whenever possible, for completeness purposes.

From an object-oriented perspective, Micro Saint allows hierarchical model building. Modularity is affected by the need to declare variables and arrays outside the blocks, and no inheritance concepts are implemented. User-defined elements are very rudimentary to the extent that no GUI can be associated with them and they cannot be made part of the built-in library. Other weaknesses include the inability to implement preemption, the need for user-written code to handle non-empty starts, very basic animation capabilities (coding is required to tap the advanced features), and the lack of dynamic update capabilities for model structure and queueing policies.

Quest v 5R17. This software has a unique set of tools to realistically model the behavior of resources and their interaction with the modeling environment. Not only can different types of resources be allocated to a single task in a user-specified sequence, they can also be governed by “resource controllers” according to specific policies. Quest also provides a comprehensive set of tools to dynamically update the model structure and queueing policies. Hierarchical model building is the only object-oriented paradigm completely implemented in the software. Modularity is affected by the partition of the elements into three different areas (logics, processes and blocks), while inheritance is completely absent. Quest’s most significant weaknesses are the inability to customize the simulation environment through user defined elements, to communicate with databases and spreadsheets, and to handle preemption.

SimCad v 7.2. SimCad provides very flexible sets of tools to handle non-empty starts and to model resource behavior. Different combinations of resources can be associated with different types of flow items. Hierarchical model building is the only object-oriented paradigm completely implemented in the software. Inheritance is completely absent, and modularity is limited by the need to declare extensions (with some degree of complexity), variables and arrays outside blocks. Though variables can be created at different scopes, arrays are limited to the model scope. SimCad’s most significant weaknesses include lack of user-defined elements which handicaps users from extending functionalities of built-in elements, limited interaction with spreadsheets and databases, and lack of dynamic update functionalities for model structure and queueing policies.

Workbench v 5.2. The modularity of this software is evident in the fact that elements are completely self-contained with respect to variables, arrays and functionality extensions. Hierarchical model building and inheritance are supported to some extent. Hierarchical model building is limited up to four levels, which correspond to the programming levels underneath (directories, files, functions and commands). Limited inheritance is possible at the function and file level. Workbench provides a very flexible set of tools to handle logic-driven preemption. User-defined elements can be created and made part of the built-in element libraries, but they are code-based and there is no built-in functionality to associate GUIs with them. The most significant weaknesses of the software are the inability to update the model structure, the limitations in defining user-defined queueing policies and lack of interaction with spreadsheets and databases. Workbench’s built-in animation is sequential in nature, which can be used as a good troubleshooting tool, but limits the use of animation as a way to understand the behavior of the model, especially in the case of large models.

Conclusions

THE STUDY SHOWS that though the market currently offers DES that are close to satisfying most of the criteria required to implement simulation optimization strategies,there is no product that completely satisfies all of them. The final selection would depend on the specific system being modeled and the particular priorities given by users and developers to the trade-offs between the different unmet criteria. For a full version of this paper, see http://lionhrtpub.com/orms/orms-10-07/frdes.html.

REFERENCES

1. Swain, J. J., 2005, “Seventh Biennial Survey of Discrete-Event Simulation Software,” OR/MS Today, Vol. 30, No. 4.

2. Tewoldeberhan, T. W., Verbraeck, A., Valentin, E., and Bardonnet, G., “An evaluation and selection methodology for discrete-event simulation software,” Proceedings of the 2002 Winter Simulation Conference, San Diego, Calif.