Tuesday, April 24, 2007

an extract from a paper I wrote a while ago on BPM

[Note: some of this material comes from presentations I attended given by various Gartner analysts]

Background Material on BPM Technology

A Business Process Management System (BPMS) is:
• A development and runtime environment that enables process modeling and design, development and execution, and ongoing management and optimization;
• Automates and manages the end-to-end flow of work as it progresses across system and human boundaries;
• Unifies previously independent software infrastructure categories (such as workflow, EAI, document/content mgmt., mgmt., portals, Web servers and application servers);
• Supports SOA principles and XML Web services standards;
• Among many emerging technologies that can be used for creating SOA composite applications.

In general, as a management discipline, BPM is the ability to continuously optimize operational processes that most directly affects the achievement of corporate performance goals. When implemented in technology as an integrated business process management suite, business process architects and users can:
• Model the interactions among workers, systems and information that are necessary for accomplishing work;
• Consistently execute the optimal process;
• Coordinate and manage the handoff of work across boundaries;
• Adjust organizational structure and incentives to foster new behaviors;
• Monitor process outcomes to performance targets and seek continuous improvement.

There is no one technology that does all of the functions needed to support BPM. Enterprises will require some degree of integration across these technologies to enable consistency and reuse. Some will seek an integrated business process management suite from a single vendor (or its strategic partners that provide tight levels of integration). However, most will need to integrate tools from multiple vendors themselves.

The origin of BPM is in the Quality and BPR domains. In the 80s, the focus was on Quality. In the 1990s, focus shifted to BPR. In the 2000s, it has evolved to BPM. Thus, a focus on processes is not new. We have seen it before in the era of quality programs, followed by, and even more explicit, in business process re-engineering. The current phase is built on a stronger synergy of IT capabilities
and business understanding, but can equally be expected to follow the typical hype cycle. Gartner feels that we are still at an early stage in the current [BPM] wave. Management writers have been espousing a new found belief in process (Michael Hammer's book "TheAgenda" was published in 2001 and demonstrated a rebirth of process thinking from the ashes of BPR). Start-up companies have been pursuing BPM as a technology initiative (Gartner 2004 Magic Quadrant for BPM was constructed from consideration of 100 vendors — see "Magic Quadrant for Pure-Play BPM, 2Q04," June 2003, M-22-9774). Standardization initiatives have started and proliferated — Business Process Execution Language (BPEL), Business Process Modeling Language (BPML) and Web Service Choreography Interface (WSCI) — and according to Gartner are beginning to consolidate around BPEL. Gartner believes that the hype around BPM will continue to build before the industry will stabilize. Furthermore, Gartner believes that early adopters will gain their greatest benefit during the trough of disillusionment, which they feel we will see shortly.

Looking at figure 1, we are currently in the Service-Oriented era. In the future, industry will continue to evolve towards a more event-driven architecture and finally towards even a more adaptive and dynamic environment than will be available via the SOA. We will only briefly touch on the Event Driven Architecture in this report. The adaptive/dynamic era is beyond the scope of this paper.

The kinds of products that are included within the BPM market are shown in figure 2. BPMS contains core BPM-enabling tools: orchestration engines, business intelligence and analysis tools, rules engines; repositories (for process definitions, process components, process models, business rules, process data); simulation and optimization tools; integration tools.

A core component of BPM is modeling -- its importance cannot be understated. Good process models communicate how work is accomplished, reflecting the concerns of all of the stakeholders and participating functions. Process models are needed to help business and IT managers understand actual processes and enable them, by visualization and simulation, to propose improvements. Explicit process models are easily changed because non-technical managers understand them easily and they are independent of the underlying resources. Models provide a basis for cross-organizational collaboration between managers responsible for the separate tasks within a process, as well as with IT professionals on the implementation of the resulting design. The key elements to be identified in a process model are the business events that trigger actions, the sequence of steps, and the business rules used in and between those steps to support decision making and execution flow. Once this is done, IT professionals (architects and systems analysts) can begin to map the work tasks and information dependencies to existing logic, data and user interfaces. This kind of multilevel modeling effort identifies valuable existing IT assets to be leveraged in new process designs, and highlights those areas where business users want more control over process change. As in manufacturing (where a broken finished product can easily be fixed when its design is based on a component-assembly approach), so too can re-engineering to SOA turn existing IT assets into reusable services to achieve the desired flexibility. However, modeling is still just one of ten major components of a BPM platform!

Action Item: Modeling must become a business discipline — not a creative pastime.

With a BPMS, the full business process is made explicit in a graphical process flow model. This process model is completely autonomous from the resources performing the work steps, whether they are human or machine resources. By making it explicit and autonomous, changes to the process model can be made independently from changes in the resources.

This is the "loose coupling" principle familiar to many from earlier middleware forms and from SOA design guidelines.. Making process flow control explicit and decoupling it from the underlying technology allows processes to be changed more quickly. For example: Processes are easily changed since other system elements are not affected by flow control changes and need not, therefore, be retested. Some process changes are made by business professionals who need only limited knowledge of IT systems. In a growing number of cases, changes such as work item routing, business rule overrides, and parametric changes to approval levels, are made in real time to executing process. Near-real-time reporting of process steps deliver never-before-seen analysis of current operating conditions customized as appropriate to the organization. Tightly coupling business dashboards/cockpits/business activity monitoring (BAM) to the underlying runtime allows rapid and precise process change. Orchestration engines are the runtime environments. There will be different kinds of runtime environments — some for Web-service-based components, others for human workflow components, others for rules. The process model is simply an XML metadata description of how all the activities and events should be coordinated. And because it is XML, it can be made executable at runtime and the resources that perform the steps can be dynamically late-bound into the execution.

Figure 3 shows the traditional software development lifecycle along with that for BPM development (and integration). The traditional lifecycle model is shown simply t contrast it with today’s BPM-focused, highly agile one. We will focus on the BPM lifecycle (on the right) as most are quite familiar with that model shown on the left. (see next post for image)

Specifically, BPM consists of the following iterative phases:
• Definition (and/or discovery) identifies intricacies of how a process executes
• Modeling is valuable because it shows easy improvement opportunities, or at least the scale of the problem. Helps process owners collaborate on potential process improvements that will help achieve corporate goals.
• Simulation reveals bottlenecks not obvious during static modeling. Helps fine-tune adjustments in process model.
• Deployment creates detailed process execution scripts and makes required changes in systems. Training and facility changes must be coordinated; deployment usually involves integration with external systems and may include converting application code segments into sets of reusable web service components.
• Execution is where the main value of BPM is realized because its where the actual improvements are first seen.
• Monitoring collects information from executing processes in real time and helps facilitate immediate corrections.
• Analysis creates further value when key performance indicators based on process execution are linked directly to business objectives.
• Optimization is a fact-based approach to process scenario optimization, greatly reducing risk and eliminating guesswork.

Action Item: The separation between process design and process implementation and execution (a separation not found in conventional applications) will permit the emergence of a market for "process components" — a new kind of intellectual property. These components will include templates for specific horizontal and vertical processes, business content, and rule sets.

Up to now, this report has focused on the business side of BPM. Briefly addressing the information technology aspect of business management, SOA is the current best practice for defining service modularity and interoperability. SOA is a style of software system architecture in which certain discrete functions are packaged into modular, encapsulated, shareable elements ("services") which can be invoked in a loosely coupled manner by local or remote "consumer" parts of the system. Thus, SOA is an architectural style that is modular, distributable and loosely coupled. A service (or component) is a software process that acts in response to a request. Business services comprise a type of service; those that have semantic meaning in a business context. An SOA service refers to the combination of a provider service and its exposed interface to consumer services. The interface is the contract between consumer implementations and provider implementations. It is the consumer's view of a provider's capability and the provider's view of the consumer's responsibilities. Lastly, web services are software components that employ one or more of the following technologies — SOAP, WSDL and UDDI — to perform distributed computing.

Action Item: IT must decide on tools and approaches (wrapping or rewriting) to be used for migrating existing business capabilities into reusable services, and make them available to business analysts, process modelers and architects.

Service-oriented systems do not require the Web services suite of protocols, but the Web services protocol family provides a standard, widely supported foundation for intercommunication between SOA systems. Portal technology predates Web services, but has evolved to incorporate support for this distributed computing technology, along with other technology for application integration (for example, message queues and connectors). Another benefit of an SOA is that two subsystems can each pursue their own destinies or life cycles. As long as there is a contract that governs the interaction between the two, they can evolve separately. Thus, SOAs offer greater flexibility and agility. Organizations can recombine, reassemble and orchestrate their systems with other systems to create composites more quickly than they would by developing a monolithic system from scratch.

Figure 4 depicts how a business process platform that supported SOA might be visualized. SOA enables software to be defined as independent services that can be "composed" into operational systems. The composition process is driven largely by mapping the use of services within a business process (i.e., process orchestration). The services generally are assumed to already exist either as components delivered within new Service Oriented Business Applications (SOBAs) or accessed from an external source. In addition, established applications may be segmented and components wrapped to create service interfaces, so these legacy systems can be exploited within the composition platform, or new services may be developed from scratch to meet unique needs. Services will be managed and stored in a repository along with rules for maintaining their integrity (the repository may also point to external services, acting, in that case, as a registry). The composition process also looks out to the user experience and the way in which the functionality is delivered (typically via multiple channels and alternate device types). Management and security mechanisms must still span the repository, composition platform and user experience. The creation of this new framework for delivering applications is generating new families of integrated products and technologies from middleware, platform and application companies. In turn, this is creating new areas of competition and collaboration between technology suppliers. The result for businesses will be the reconstituting of their application software as a BPP delivering greater flexibility in running the business.

They equate Web services with SOA and distributed computing with SOA, or they combine events and services with SOA. This debate seems academic, but the position you take determines what your company gets out of your SOA initiative. Plain use of Simple Object Access Protocol (SOAP) enables client/server exchanges across the Internet (and through firewalls). This is a benefit for many applications, but users who expect additional SOA benefits from deploying a SOAP-based interchange will be disappointed. As users make advanced investment in their architectures, the benefits will include incremental engineering of software, technical or business reuse of software, business-to-business (B2B) opportunities, business activity monitoring (BAM), managed inter-application scalability and availability of software. Ultimately, the movement toward SOA and beyond promises to bring business-IT alignment closer — a long-time pursuit of the software industry. However, greater benefits require a greater investment. Declaring a distributed computing deployment an SOA will not deliver the full benefits of SOA. Adopting the vision of business semantics for services and adding integrated support of business events requires greater systematic effort, but it is essential to achieving the potential of the fully expressed architecture of business components.

Action Item: Seek an integrated platform, comprised of a business services repository and composition technologies, to enable the Business Process Platform.

SOA initiatives should be linked to BPM for mutual benefit! Although SOA is complimentary to BPM, BPM can be accomplished with or without SOA. The more the organization desires flexible processes with greater flexibility in the systems automated steps, the more important SOA becomes. Nevertheless, while the organization builds up a portfolio of reusable services, BPMS technology can be used to take early baby steps toward adaptive processes. The shift in control over processes from IT to line-of-business managers is best accomplished gradually, allowing everyone to gain confidence in the required skills and technologies. In this presentation, we've tried to present a number of pragmatic approaches to accomplishing high-value business initiatives leveraging existing IT assets. For those strategic solution areas, meant to differentiate the enterprise, and where the rate of change is expected to be high, re-engineering of automated business capabilities into SOA Web services is highly justified.

Action Item: To get the most from your SOA investment, think of SOA as long-term software architecture and engineering practice, not just as Web services or another tool.

Bottom Line: Migration to SOA is not an "all or nothing" proposition.

There are many reasons to use event-driven applications, rather than monolithic application architectures or even SOAs. Events can scale to higher volumes of business transactions and more users; they can simplify application development (AD) by reducing the complexity and amount of code in application programs; they can decrease the latency (response time) for a function and reduce the elapsed time for an entire end-to-end business process; they can facilitate data quality by being closer to real time; they can enable better auditability (track and trace); they can provide earlier warning of threats and earlier notice of emerging opportunities; and, perhaps most importantly, they provide the maximum application software "plugability" to facilitate agility through continuous adjustments to business processes.

Ultimately, the SOA is not as flexible or efficient as an event-driven design, but it is more flexible and agile than traditional tightly coupled or monolithic application architectures. SOA incurs the minimum possible lock-in among components in situations that involve collaboration. Companies that seek to maximize the effectiveness of their IT usage must invest in event-oriented design (as well as SOA). Their architects must have a thorough understanding of business events. They must identify and document business events in the earliest stages of business process design and follow through by implementing event-driven business components in the later stages of development and maintenance.
In the very long run, the Model Driven and Event Driven Architectures (closely related) will replace today’s SOAs.

Background Material on BPM Market

The BPM market is in tremendous flux and is rapidly growing and developing. Illustrating the rapid growth in this product arena, Gartner’s first BPM conference, held in 2005, had around 600 attendees. A year later, there was a 50% increase in the number of attendees at their next BPM conference. Along similar lines, Gartner conducted a survey among a representative population of CIOs in thirty-plus countries. Their survey results, summarized in a news story posted on www.bpm.com, contained a list of the absolute top business priorities in 2006. “Business process improvement” was the phrase that conveyed these representative CIO’s number one single priority. Similarly, among technology priorities, at least five of the top ten are those that concern areas which are directly addressed by BPM. These five in relative order of priority are: business intelligence applications, mobile workforce enablement, collaboration technologies, service oriented architecture, and workflow management.

The growth and excitement in the BPM space is important to consider when evaluating individual products. One of many reasons why it is important to consider this explosion of growth is that we should remember that it is common in the information technology field for a new technology to be initially pursued by numerous vendors before the actual leaders of the new technology area surface and show sustainability. Accordingly, in March of 2006, Gartner stated the following about the BPM market:

“The "BPM market" is really multiple markets or segments with over 160 software vendors vying for leadership. Most vendors are small, privately held companies with revenues under $50 million. With so many vendors, few have really enjoyed significant growth rates and few, if any, can legitimately claim leadership. As is typical of early markets, there will be multiple waves of consolidation. This will continue through 2009, as buyers become more sophisticated, and the technology matures into more-predictable sets of features and functionalities (0.7 probability)…Of the contenders in the BPMS market in 2005, we anticipate that only 25 will continue to compete beyond this time frame, with the others moving into potentially adjacent market spaces. There has already been some market consolidation through acquisition, although we expect the number of acquisitions to steadily rise through mid-2006, once IBM, Microsoft and Oracle start delivering more of their complete BPM architecture.”

Some time earlier, Gartner had also stated: “because many BPM pure-play vendors are small companies with limited resources, no more than 25 of the 140+ competitors of 2005 will transition to the emerging BPMS market even by 2008. The rest will transition to providing more specialized tools, become suppliers of packaged processes for specific industries, geographies or horizontal processes, migrate to alternative adjacent tool markets, be acquired or cease trading (0.8 probability).”

When one looks at the range of products that comprise the BPM product space, one sees a broad range of products. These include such diverse software products as those that provide workflow, enterprise application integration, business process modeling, business rules management, and business intelligence analysis, to name just a few. Ultimately, it is possible the future will consist of industry specific markets, rather than any specific and generic market, since it may ultimately prove to be easier to integrate all the elements for an industry rather than arrive at a generic, universal BPM suite solution.

According to Gartner, a comprehensive BPM suite needs to deliver 10 major areas of functionality:

• Human task support facilitating the execution of human-focused process steps (cross referenced (ref.) #1)
• Business process/policy modeling and simulation environment (ref. #2)
• Pre-built frameworks, models, flows, rules and services (ref. #3)
• Human interface support and content management (ref. #4)
• Collaboration anywhere support (ref. #5)
• System task and integration support (ref. #6)
• Business activity monitoring (BAM) (ref. #7)
• Runtime simulation, optimization and predictive modeling (ref. #8)
• Business policy/rule management support (heuristics and inference) (ref. #9)
• Real-time agility infrastructure supports (ref. #10)

In the 2005 Business Process Trends Report (also referred to as the 2005 BP Suites Report and available at www.bptrends.com), Derek Miers and Paul Harmon list ten BPM product areas. There is some similarity in this list to the preceding one. To highlight the similarities and differences, I have attempted to map these two lists. Thus, in the Gartner list, I have arbitrarily assigned reference numbers to which the BPM Suites Report subsequently cross references. Some of this mapping was subjective because product categories easily overlap and the difference between tools ultimately comes down to how practitioners use specific products. We will follow up these two lists with a report provided by Forrester. The reason I have chosen to include data by three different industry analyst organizations is to ensure completeness. The ten product areas listed in the BPM Trends report were:

• Process modeling tools (see ref. #1)
• Simulation tools (see ref. #8)
• Business rule management tools (see ref. #9)
• BPM applications (meaning complete turn-key enterprise resource planning (ERP) system with BPM elements which are found within a BPM suite)
• Business process monitoring tools (see ref. #7)
• Software development tools (not included by Gartner, and ref. #11)
• Enterprise Application Integration (EAI) tools (see ref. #6)
• Workflow tools (see ref. #1 and #4)
• Business process languages (not included by Gartner, and ref. #12)
• Organization and enterprise modeling tools (not included by Gartner, and ref. #13)

dated trip report following Gartner's 2006 BPM conference

This post is dated but still relevant. I found this while looking for some other stuff...

Following are my thoughts and perceptions from the 2006 Gartner BPM conference held in Nashville Tennessee March 26-29. The official days were March 27-29 but there were some meetings on the 26th and so I am including the 26th as a conference day.

My first observation was that Gartner’s first BPM conference was one year ago and there were around 600 people at that and within a year the same conference had grown 50%. This just highlights the growth in the BPM space.

In general, my expectations were a bit higher than they should have been. I had believed that Gartner might be publishing an updated Pure Play BPM report since their last report was published in 2004. Unfortunately, this was not to be. There are rumors that some time within the next 45-60 days their next report (or set of reports) on BPM will be published. We will have to wait and see. In all, however, I feel the conference was worthwhile and that I gained some general as well as specific insight into the BPM product space. I would have preferred more analysis on specific vendor offerings; I would have preferred more of a technical focus and less of a management or business perspective. But again, BPM does refer to Business Process Management so it is not surprising that a good deal of focus would be on business, management, and specifically on business management.

Gartner consistently says that the 1980s were the years of Quality (such as TQM, CMM, etc.); the 1990s were the years of BPR; and the 2000s the years of BPM.

A key difference between BPR and BPM is that in the BPR era, we (as in the software industry) thought in terms of “if we could just get it right”. Thus, BPR followed more of a waterfall life cycle. The focus on BPM is on agility, iteration, and adaptability.

Concepts that Gartner was particularly trying to sell include that of Programmatic Integration Server. Programmatic Integration Servers is a software infrastructure category defined by Gartner, and Gartner produces an annual Magic Quadrant for this software category. According to Gartner, "Programmatic integration servers enable a lightweight approach to providing a service-oriented architecture on top of legacy applications.”Within a single software technology, Programmatic Integration Servers enable newly developed composite applications, generally running inside application servers, to combine new business logic, with the existing business logic, screen logic, components and data present in legacy systems. Most legacy applications were not designed with integration in mind, therefore the "smarts" in programmatic integration servers center around mapping proprietary and often human interfaces to more standardized interfaces. A key consideration when selecting programmatic integration servers is the level of granularity they expose. Most legacy systems have usage profiles that define granularity from a navigation and transactional perspective which do not map well to new business logic within composite applications. The more sophisticated Programmatic Integration Servers products allow the granularity of reused legacy functions to match the needs of the composite application rather than being hampered by the original legacy application's design. The more sophisticated Programmatic Integration Servers support Service-Oriented Architectures (SOA), Event-Driven Architectures, direct data and program level access, and presentation integration support for screen-based applications. I am including two Gartner graphics which will quickly say more than I have time to explain.

One of the best sessions I attended was titled “When will the Power Vendors Offer Credible BPM Solutions?” This session was one that I really wanted other team members (i.e. Dave, Steve) to hear. Gartner evaluated the power vendors based on ten criterion: human task support; pre-built frameworks, models, flows, rules, and services; business process/policy modeling and simulation environment; human interface support and content management; collaboration anywhere support; system task and integration support; business activity monitoring; runtime simulation, optimization, and predictive analysis; business policy/rule management support (heuristics and inference); and real-time agility infrastructure supports. The power vendors named were: Fujitsu, IBM, Microsoft, Oracle, and SAP (now partnered with IDS Scheer). Gartner says that power vendors are 12-18 months away from offering a full solution but in three years could own 50% of the market. To date, the power vendor that shows the greatest capability and maturity in the BPM space is Fujitsu. Second to Fujitsu and considered a power vendor with BPM strength is Oracle. Another strong vendor that is not considered as a “power vendor” but has considerable corporate strength and viability as well as outstanding BPM capabilities is Tibco. The strongest BPM based vendors are Pega, FileNet, and Global 360. The next group of BPM based vendors with strength but less strength than the three previously named are comprised of Appian, Lombardi, Savvion, Metastorm, and Ultimus. Lastly, Gartner states that EMC, CA, and Autonomy may be interesting to watch.

Let’s digress a bit and hear Gartner describe the BPM market:

“The "BPM market" is really multiple markets or segments with over 160 software vendors vying for leadership. Most vendors are small, privately held companies with revenues under $50 million. With so many vendors, few have really enjoyed significant growth rates and few, if any, can legitimately claim leadership. As is typical of early markets, there will be multiple waves of consolidation. This will continue through 2009, as buyers become more sophisticated, and the technology matures into more-predictable sets of features and functionalities (0.7 probability)…Of the contenders in the BPMS market in 2005, we anticipate that only 25 will continue to compete beyond this time frame, with the others moving into potentially adjacent market spaces. There has already been some market consolidation through acquisition, although we expect the number of acquisitions to steadily rise through mid-2006, once IBM, Microsoft and Oracle start delivering more of their complete BPM architecture. Another indication of the early nature of this market is the tremendous diversity still in the licensing and pricing models seen and the average selling price. Through 2004, we saw many "conference room pilots" for $50,000. In 2005, we saw many more project deployments, ranging from $150,000 to $800,000. In 2006, we expect momentum to be increasing and deal sizes to get even larger…By 2007, the major infrastructure and development tool vendors (for example, IBM, Microsoft, SAP, Oracle and BEA) will deliver model-driven development frameworks and begin to challenge then-leading BPMS tool vendors for becoming the preferred platform for process modeling and design, SOA Web services development and composite solutions deployment.”

Again, I will emphasize that Gartner states that “because many BPM pure-play vendors are small companies with limited resources, no more than 25 of the 140+ competitors of 2005 will transition to the emerging BPMS market even by 2008. The rest will transition to providing more specialized tools, become suppliers of packaged processes for specific industries, geographies or horizontal processes, migrate to alternative adjacent tool markets, be acquired or cease trading (0.8 probability).

As could be expected, there was much discussion on Service Oriented Architectures. Gartner states: “SOAs and the resulting service-oriented business applications (SOBAs) have begun to give IT a method of composing new business processes from services. However, without structure, security,and integrated composition technologies, this method of composing new processes can lead to issues such as greater inflexibility, the need to maintain these new processes as custom applications, and lack of data/process integrity. In addition, this method is very IT-centric, and ultimately it inhibits super-users in building/composing their own business processes...It is also assumed that SOA enables users to buy processes from multple sources and somehow mold those into a new process. This is a false assumption, because vendors will define beginning and end points of services in different ways. Users should seek to define new processes in a controlled and manageable environment both for technology and service content.“

Gartner also states there are many reasons to use event-driven applications, rather than monolithic application architectures or even SOAs. Events can scale to higher volumes of business transactions and more users; they can simplify application development (AD) by reducing the complexity and amount of code in application programs; they can decrease the latency (response time) for a function and reduce the elapsed time for an entire end-to-end business process; they can facilitate data quality; they can enable better auditability (track and trace); they can provide earlier warning of threats and earlier notice of emerging opportunities; and, perhaps most importantly, they provide the maximum application software "plugability" to facilitate agility through continuous adjustments to business processes.

Furthermore, SOA is not as flexible or efficient as an event-driven design, but it is more flexible and agile than traditional tightly coupled or monolithic application architectures. SOA incurs the minimum possible lock-in among components in situations that involve collaboration. Companies that seek to maximize the effectiveness of their IT usage must invest in event-oriented design (as well as SOA). Their architects must have a thorough understanding of business events. They must identify and document business events in the earliest stages of business process design and follow through by implementing event-driven business components in the later stages of development and maintenance.

In the interest of time, I am going to throw out a few other observations and close with a few of the many graphics contained in Gartner’s documentation. Fujitsu made a strong presentation on standards including BPEL, BPMN, and BPDL, focusing on an explanation of the differences of purpose. In short, BPMN is about 18 months old and is simply a graphical standard whereby shapes and other graphical features used by BPM modeling tools are given standard meaning so that one can quickly glance at a model and immediately know its interpretation. BPMN does NOT imply any type of file format and it does not facilitation machine interoperability. BPEL as we all probably know is a specific machine readable programming language that facilitates machine to machine interoperability. XPDL is strong in modeling of the human elements and is used so that a model can be shared among different modeling tools. As for BPEL For People, it might be / probably will be another 3 or more years before any specification is established. Until then, it will remain a high-level concept described in a white paper. Finally, BPDM is Business Process Definition Metamodel and identifies the constrained subset of UML 2.0 that is appropriate for business process modeling. We spent the least time looking at BPDM, which is not really developed as of yet.

Monday, April 16, 2007

Friday, April 13, 2007

I wonder what happened to OO databases?

I remember in the 1990s a plethora of databases were springing up that claimed to be object-oriented. The truth is however other than attend some vendor presentations and demos, I have hardly ever encountered actual use of any of these products.

From wikipedia, object database management systems grew out of research during the early to mid-1970s into having intrinsic database management support for graph-structured objects. The term "object-oriented database system" first appeared around 1985. Notable research projects included Encore-Ob/Server (Brown University), EXODUS (University of Wisconsin), IRIS (Hewlett-Packard), ODE (Bell Labs), ORION (Microelectronics and Computer Technology Corporation or MCC), Vodak (GMD-IPSI), and Zeitgeist (Texas Instruments). The ORION project had more published papers than any of the other efforts. Won Kim of MCC compiled the best of those papers in a book published by The MIT Press. Early commercial products included GemStone (Servio Logic, name changed to GemStone Systems), Gbase (Graphael), and Vbase (Ontologic). The early to mid-1990s saw additional commercial products enter the market. These included ITASCA (Itasca Systems), Jasmine (Fujitsu, marketed by Computer Associates), Matisse (Matisse Software), Objectivity/DB (Objectivity, Inc.), ObjectStore (Progress Software, acquired from eXcelon which was originally Object Design), ONTOS (Ontos, Inc., name changed from Ontologic), O2[2] (O2 Technology, merged with several companies, acquired by Informix, which was in turn acquired by IBM), POET (now FastObjects from Versant which acquired Poet Systems), and Versant Object Database (Versant Corporation). Some of these products remain on the market and have been joined by new products (see the product listings below). Object database management systems added the concept of persistence to object programming languages. The early commercial products were integrated with various languages, such as GemStone with Smalltalk. For much of the 1990s, C++ dominated the commercial object database management market. Vendors added Java in the late 1990s and more recently, C#.

Also according to wikipedia, as of 2004, object databases have seen a second growth period when open source object databases emerged that were widely affordable and easy to use, because they are entirely written in OOP languages like Java or C#, such as db4o (db4objects) and Perst (McObject). Recently another open source object database Magma has been in development. Magma is written in Squeak. (I haven't ran across any of these, however...and until reading this from wikipedia had never even heard of last three products...)

So my question today, is if this is a market that will FINALLY take off...

Thursday, April 12, 2007

really showing my age... (LOL)

Okay, this is not a complete list (and its not in any specific order)...I could actually never re-create a full list of all of the languages, hardware platforms, and operating systems I have worked on (there has just been too many) but its a near complete list other than a few assorted academic-purpose languages and a couple of completely obscure languages used just in bar-coding etc.

Programming languages that I have either been paid to write in or studied in school or elsewhere: Ada, Active Server Pages (ASP), Assembler (both for IBM mainframe and IBM pc), BASIC, C, C++, C#, Clipper, CLIPS, COBOL, Cold Fusion, DBase III+/IV, Delphi, DHTML, EXEC/EXEC II, Fortran, HTML, Java, JavaScript, JCL, LISP, Motorola's Four-Phase minicomputer language Vision, Paradox, Pascal, Perl, PL1, PL/SQL, Prolog, R:Base, Rexx, RPG II/III, Transact-SQL, VBScript, Visual Basic, XSLT

Of these, the languages with which I have the most experience (in order) are COBOL, 370 Assembler, all variants of SQL, Pascal, ASP, HTML, JavaScript, VBScript, C++, Java, and Cold Fusion

Most of the operating systems I have played with in no particular order are: VAX/VMS, Unix, Linux, Primos, IBM 360/65, IBM 370, Windows 2x, Windows 3x, Windows for Workgroups, Windows 95, Windows NT, Windows 98, Windows 2000, Windows XP, Tandy Deskmate, DOS, VM/CMS, MVS, Solaris, MS-DOS, CICS, DOS/VSE, AS/400, Novell Netware, and OSF/1

Today it appears that the winners of the language competition has been won by Java, C++/C#, and variants of XML with some scripting languages used as needed (i.e. JavaScript--now evolved somewhat into AJAX) and of course all supported by SQL. The three principal database platforms are Oracle, DB2, and SQL Server with lagging support for MySQL. And of course Microsoft rules the OS world (at least in my world it does).

What's interesting is not so much to see what was popular when--but why. For instance, Pascal was a superb language. It could easily have evolved into market dominance as it continued to evolve into more and more of a robust OO language. But Borland made some fatal marketing errors and Microsoft was a little less stupid even if it was strongly hated by the majority of those who called themselves "developers". True geeks still love all variants of Unix and those most true geeks are most enamored by anything that is labeled "open source". But business people are tired of bowing down to IT types and so superior products are frequently abandoned in favor of those products who were simply better marketed for business users.

I remember probably 15 years ago being told that software vendors were going to revolutionize the world so that software products would handle more of the detailed and repetitive "coding" aspects. As a "programmer" at the time, I found this appalling and insulting, of course. But that is exactly what is evolving in this area called BPM.

Of course, at least for the very foreseeable future, there will remain a need for folks who can program. Just like buyers ERPs learned they had to customize (ergo, I mean, code), companies that have jumped on the BPM bandwagon are finding they still need custom code (particularly integration code). This is actually a good thing for programmers. Instead of spending their time maintaining things such as user interfaces, they can attempt to resolve more complex problems. This is why my job is that of systems integrator. Although I will admit its a lot of fun to design user interfaces. ;-)

my dad's home computer...

I guess because I am reading this book called "In Search of Stupidity: 20 Years of High-Tech Marketing Disasters", I have been experiencing a lot of memories from Computer Yesteryear. I found the story below that describes my Dad's first (and only--since he passed away at such a young age) home computer. My son came home today and asked me if I knew Moore's Law. He was surprised when I described it to him. He is reading a book called "The Singularity Is Near: When Humans Transcend Biology" by Ray Kurzweil. In it, he discusses Moore's Law among an assortment of many other ideas including nanotechnology. By the way, I got distributed a new computer today at work. Boy, is it nice! Sleek flat screen monitor for my new HP notebook. I signed up today for yet another class (at my boss' urging), this one is on Oracle's products for SOA and BPM (such as their BPEL Manager). I am already registered for a 3 day class this month at IBM on the Websphere product. (This will not the first time I have attended a class on Websphere.) On Sunday I return to Cambridge for a third week of Pegasystems training. In addition I am studying for certification in XML-related technologies and will be participating in several courses on rather "old technologies" such as XML, XSLT, and Web Services within the .Net or Java frameworks. Boy, between all this reading and sitting in class I think I better get some exercise (LOL).


Sperry Introduces Personal Computer
By THE ASSOCIATED PRESS
Published: November 30, 1983
The Sperry Corporation became the latest major entrant in the personal-computer market with an I.B.M.- compatible machine that Sperry says is 50 percent faster than I.B.M.'s machine.

Sperry, a leading maker of mainframe computers, said its personal computer, with 128,000 characters of basic memory, is a Sperry design that uses several components made by the Mitsubishi Electric Industrial Company of Japan.

Sperry said its machine can run on the same software and peripheral equipment designed for the International Business Machines Corporation Personal Computer, but is faster and less costly than I.B.M.'s machine. Sperry's machine, to be available in January, will cost between $2,643 and $5,753, depending on options ordered.

Wednesday, April 11, 2007

memories




This is definitely NOT one of my first computers to ever touch but I do remember one of these computers. I took my first Pascal course on this computer. You had to boot the operating system from a floppy and store your data on another floppy. Life has improved! :-) If you get really bored or need to take a short break, check out: http://www.old-computers.com

my latest book assignment titled In Search of Stupidity

I just finished reading Why Software Sucks and am now reading In Search of Excellence (pun on the classic book In Search of Excellence). The subtitle is Over 20 Years of High-Tech Marketing Disasters. So far I have enjoyed the book. (I just started it 2 days ago). The basic argument presented in the book is: "Remember: The race goes not to the strong, nor swift, nor more intelligent but to the less stupid"

Here are some pages from the author's website:

http://www.insearchofstupidity.com (home page...duh)

http://www.insearchofstupidity.com/m_collateral.htm (examples of some really stupid corporate marketing pieces)

http://www.insearchofstupidity.com/m_pandw.htm (couple of examples of stupid writing...the best example is about the Mercator Integration Broker)

and http://www.insearchofstupidity.com/m_products.htm (some examples of stupid hardware and software products)

I also stumbled across PC World's article from June of 2006 listing the 25 worst tech products of all time: http://www.pcworld.com/printable/article/id,125772/printable.html

Kind of amusing if you are like me and don't have much of a life (LOL--just kidding--or NOT!)

Wednesday, April 4, 2007

what is Pegasystems?

Pegasystems is in Gartner's magic quadrant for both rules management and BPMS. I found this article that does okay describing the 5.1 version of their SmartBPM product. (They have just released their 5.2 product.) And they have been in the "rules" business for 25 years. This is an eternity in the IT world. I have read stuff by this guy before, as well as hear him speak at BPM conferences. Anyway, I am including this because people keep asking me why I am spending so many weeks in a training class in Cambridge. I haven't been very successful at explaining myself to these folks. Of course, most (not all) of these folks do not work in IT.

You can find this article at:
http://www.intelligententerprise.com/showArticle.jhtml?articleID=191902421

Put to the Test: Pegasystems' SmartBPM Suite 5.1

With this latest version, Pegasystems has made its powerful product easier to use for process participants, business analysts and developers.

By Derek Miers

With the 5.1 release of its SmartBPM Suite, Pegasystems has made its powerful product easier to use for process participants, business analysts and developers. Although the suite is one of the more daunting BPM systems to deploy, this new release is significant because of the kind of business process management suite it offers. Pega's package manages processes in a fundamentally different way than any other BPMS I've examined.

SmartBPM is based on a unified rules-and-process engine in which processes are first-class citizens alongside different types of business rules. Pega effectively binds all the elements required to deliver an application at run time, including process fragments, business rules, presentation elements, integration calls and security controls. Everything is dynamically selected and bound based on the work context, as defined by the events and attributes of the case (process instance).

Competitive approaches tend to limit the use of business rules to decision points calling a standalone rules engine. With Pega, the inferencing capability of the core rules engine detects changes in the state of the related information and then works out what to do based on the goals of the process. That could be just about anything, from forward chaining (moving to the next step in the process), invoking a separate process thread in parallel, raising an alert to a manager or even backward chaining through the rule set to retrieve some piece of missing information automatically.

Keep Data In The Domain

Another key differentiator for Pega is that rather than regarding the information domain as out of the scope of the BPMS, the entire SmartBPM environment is based on a componentized, service-oriented run-time environment in which data classes are specialized alongside the business processes and declarative rules. This approach lets Pega resolve the right rules and processes to bind to the case based on the context of the work. Although specialization delivers tremendous downstream flexibility, enabling better market segmentation, it also presents challenges in the early stages of deployment.

Overall, the approach facilitates app customization to meet particular needs. Say you have a standard way of processing orders, for example, but when an order comes in for a key customer and the product is out of stock, you want to offer a special alternative. Or perhaps it is a first-time customer and, as a result of a directive from on high, you want to use a special set of customer-satisfaction checks. Pega handles these situations by layering on specializations from the rule base, adding alternatives without your having to go back and manually weave these revisions into the baseline process.

Other BPM suites require a cut-and-paste approach, where each scenario requires a copy of the process that is then adapted. Over time, this can lead to fragmented process architectures and a higher cost of ownership. However, creating the right class structures to meet the downstream goals of the organization demands long-range planning and expertise. For a major enterprisewide project, it's best to involve a Pega-certified system architect from the outset.

Deliver Pop-ups In The Portal

The new Ajax-based portal environment serves both process development and run-time delivery. The portal uses business rules to build the Ajax environment dynamically. As with the previous version of Process Commander, the user experience is composed (and driven) at run time based on the rules and process definitions. With the Ajax upgrade, the user interface is more intelligent, as it responds to the intent that was defined in the process and rules base. New rule-driven pop-ups help the user identify related information quickly.

The Process Commander portal is role-based and augmented by a rich case- and content-management model that ensures people see only the information and processes they're supposed to see. What's more, the buttons and choices made available are constructed based on the context of the case. This functionality supports a "delegated development model" that ensures the appropriate managers, analysts and admins maintain the rule sets that directly affect their part of the business.

The look and feel of the development environment has been updated. The developer's desktop is split across six areas representing the different development roles: process, decisions, user interface, integration, reports and security. All six areas are under the control of the security model to ensure the right user interface is delivered to the right class of user.

Within the development tooling, users can see snapshots of an object just by hovering over its name. Control over the content of a pop-up in the user portal is defined declaratively using rules, and the corresponding Ajax code is generated and managed automatically. Moreover, these intent-driven interfaces can be updated instantly as higher-level policy changes ripple down through the rule sets.

For modeling, Process Commander relies on Microsoft Visio, which must be available on the designer's desktop. It's launched through an OCX control into the browser window (along with the relevant stencil). A check-in/out mechanism ensures that the process model is only worked on by one person at a time.

The system stops short of providing process support for the development activity (using the power of the process model and rules to guide the developer). However, a context-sensitive help system calls the relevant references at the field level. In the next version, this will extend right into facilitated online discussion groups and product support.


Few Rough Spots

Developers face some concerns I would not have expected. Since the entire execution environment relies on Java, there are no built-in dialogs for role assignment, for example. Work is assigned based on a property set of a Java class. Creating a new role for a given process model is cumbersome and convoluted.

Although a business user may be unnerved to see the underlying Java code exposed in the configuration dialog (role-based security can protect the faint of heart), the IT community will feel relatively comfortable inspecting the dynamically generated code. Depending on the needs of the app, this code can be embedded or, if need be, developers can embed their own Java code.

For the business analyst, a new testing feature steps through the model, automatically navigating across roles. This also includes the ability to inspect the rules, HTML and other properties of the case. A Rule Referential Integrity feature (shown on page 18) is included to identify any conflicts in the rules sets or between rules and processes.

Version 5.1 facilitates introspection of third-party apps, letting developers represent such apps in the development environment without connecting to live production systems. An LDAP wizard guides a system admin through the process of creating single-sign-on access to third-party applications.

In the Process Analyzer component, wizards are used to build differing "scenario definitions" that become automated tests for the application. These simulations use existing performance data, with measurement carried out automatically.

Bottleneck Management

Version 5.1 also delivers new functionality to support deployment, including an Autonomic Event Services module, which monitors the health of third-party apps, measuring fine-grained SLA (service-level agreement) information. This helps in the overall management of the environment, tracking response times and helping eliminate bottlenecks associated with third-party apps.

Pegasystems has set itself a challenge: to unify and simplify the worlds of business process and rules as they interact with line-of-business data. Competitive products tend to treat these as distinct disciplines, which implies multiple skill sets, but perhaps more importantly, a less-extensible application. All BPM products require you to jump through a few hoops to get started; it's just that Pega's hoops are higher than most. However, given Pega's focus on supporting BPM apps across global organizations (rather than just a process or two at a time), the rewards--like some of the challenges--are bigger.

Pegasystems SmartBPM Suite 5.1, starts at $100,000; usage-model pricing based on the number of rule invocations

in response to post about PowerPoint

Here is a good piece on instructional design:

http://educationnew.arts.unsw.edu.au/staff/sweller/clt

criticism of PowerPoint

This comes from wikipedia:

One major source of criticism of PowerPoint comes from Yale professor of statistics and graphic design Edward Tufte, who criticizes many emergent properties of the software:

Its use to guide and reassure a presenter, rather than to enlighten the audience;

Unhelpfully simplistic tables and charts, resulting from the low resolution of computer displays;

The outliner causing ideas to be arranged in an unnecessarily deep hierarchy, itself subverted by the need to restate the hierarchy on each slide;

Enforcement of the audience's linear progression through that hierarchy (whereas with handouts, readers could browse and relate items at their leisure);

Poor typography and chart layout, from presenters who are poor designers and who use poorly designed templates and default settings;

Simplistic thinking, from ideas being squashed into bulleted lists, and stories with beginning, middle, and end being turned into a collection of disparate, loosely disguised points. This may present a kind of image of objectivity and neutrality that people associate with science, technology, and "bullet points".

Tufte's criticism of the use of PowerPoint has extended to its use by NASA engineers in the events leading to the Columbia disaster. Tufte's analysis of a representative NASA PowerPoint slide is included in a full page sidebar entitled "Engineering by Viewgraphs" in Volume 1 of the Columbia Accident Investigation Board's report.

Critics of the prewar planning by the U.S. Department of Defense prior to the invasion of Iraq in 2003 also point to a reliance on PowerPoint presentations as a poor alternative to traditional military briefing methods. A report issued by The National Security Archive at George Washington University in February 2007 released declassified PowerPoint slides relating to prewar assumptions and also criticized the reliance on PowerPoint itself as a presentation medium. The report included quotes from two senior officers regarding PowerPoint:

Lt. Gen. McKiernan later told Washington Post reporter Thomas E. Ricks (Fiasco, p. 75):

"It's quite frustrating the way this works, but the way we do things nowadays is combatant commanders brief their products in PowerPoint up in Washington to OSD and Secretary of Defense... In lieu of an order, or a frag [fragmentary] order, or plan, you get a set of PowerPoint slides... [T]hat is frustrating, because nobody wants to plan against PowerPoint slides."

Retired Army Col. Andrew Bacevich told Ricks (Fiasco, pp. 75-76) that PowerPoint war planning was the ultimate insult:

"Here may be the clearest manifestation of OSD's [Office of Secretary of Defense] contempt for the accumulated wisdom of the military profession and of the assumption among forward thinkers that technology -- above all information technology -- has rendered obsolete the conventions traditionally governing the preparation and conduct of war. To imagine that PowerPoint slides can substitute for such means is really the height of recklessness."

observation made during Pegasystems training

In a couple of my last posts I discussed the IT jobs that are moving to India. I am close to wrapping up two very intense weeks of training in this software produced by Pegasystems. Hence, I have spent the past 2 weeks in the area across the street from MIT in Cambridge MA, 5 miles from Boston. The observation is just that several (half) of the class is Indian, with some Asian mix. When I graduated from George Mason University (in 2000), out of the hundreds of people graduating with a technical masters of science degree, very few had last names I could pronounce (I mean something like 10 out of 1,000). I am just saying this because it suports some of what the author says in the book "My Job Moved to India and All I Got Was This Lousy Book". We Americans really cannot afford to be complacent.