An Empirical Performance Study on PSIM

It is well known that errors caught in the early stages of system development life cycle greatly helped reduce the expense for later error fixing and mitigate risks and complications. Performance analysis is one of the methods to ensure satisfactory system performance that can avoid redesign and patch in the later stage of system development when performance problems emerge. Much work has been done to address such concerns collectively known as the 'performance engineering' problem. However, a comprehensive method is still needed to provide an end-to-end support for performance evaluation on software architecture design. Automated tools to support the method must be developed, and its validation must be carefully planned and conducted. In response to this need, we propose our approach, PSIM (performance simulation and modeling). PSIM is a performance simulation and modeling tool that integrated performance properties into software architecture specifications expressed in several major UML diagrams. PSIM models can be transformed into Colored GSPN (colored generalized stochastic Petri nets) via an automated tool. As a result, the Colored GSPN models can be simulated to perform model-based performance evaluation. In this paper PSIM is first briefly reviewed and illustrated through modeling a web-based electronic conferencing system, called M-Net, to derive performance metrics. We then conduct runtime performance testing to the implementation of M-Net and compare the simulation results against the runtime testing data. PSIM is shown to be effective in predicting system performance and in identifying system performance bottlenecks through this experimentation.