Rulebase Benchmarks 2009 Part 3

Greetings:

Once again we return to the world of Rulebase Benchmarks. Benchmarks for rulebased systems seem to have three camps: Those who love them and usually have good numbers to report. Those whose benchmarks are not remarkable one way or the other and are not worried about them; yet. Those who benchmarks are not very good and keep telling their users and/or customers, "Yes. That's OK for academics. But those benchmarks don't represent the 'real world' in which we have to live and work every day."

At October Rules Fest 2009 I asked the T3 (Thursday Think Tank) if they thought benchmarks were important. Dr. Forgy (who has the fastest rulebase engine on the planet Earth - called TECH) and myself (who has spent the past decade exploring benchmarks, their cause and effect) were ardently in favor of such things. FICO (Blaze Advisor) folks were kind of OK with the idea, especially since they use Dr. Forgy's engine in their rulebase. The rest of the group, whose performance was anywhere from just barely OK to abysmal, were not really against using benchmarks but it just wasn't something that they wanted to discuss in public.

So, here's my proposal: If anyone can produce a "real world" benchmark that can be run on any rulebase engine on any platform, PLEASE DO SO!! Otherwise, those of us who are in this for the long haul will continue to examine a rulebase from many different viewpoints but we shall NOT throw out tried and true tests that continually disclose fatal faults in fancy products that will not perform with large data sets combined with large rule sets. Such applications are large banks, large insurance companies, Homeland Security along with "real world" AI problems in rulebased forecasting, petro-chemical processing problems, electrical power grid production problems, shift scheduling for large plants and/or hospitals and anywhere else that will have the many objects - many patterns complex cross products.

Speaking of which, I guess it's about time to publish the 2009 Benchmarks. Our goal is to finish up in December some time so that at least we can say that we finished this year with a completed product. :-)

SDG

jco

World Conference on Artificial Intelligence

Dallas, Texas; Oct 18, 2009, KBSC:

I'm not sure if I learned this poem in Junior High School or High School, probably the latter:

Listen my children and you shall hear

Of the midnight ride of Paul Revere.

Hardly a man is now alive

Who remembers that famous day and year.

Of course you recognize "The Midnight Ride of Paul Revere" by Henry Wadsworth Longfellow. I thought of that poem last night when I was thinking of the upcoming October Rules Fest 2009 to be held in Dallas, Texas, at the Adolphus Hotel from October 26th - 30th. This conference reminded me of the 1956 Dartmouth College conference where the term Artificial Intelligence, AI, was first used. Those were the giants in the computer age then and ORF 2009 has their pride and joy students presenting the future of rulebased computing (a part of AI) in Dallas.

For example, Dr. Charles Forgy, considered one of the godfathers of that technology, will be speaking on parallel rulebased systems, something that no one is doing today. Well, not yet, anyway... But he and a few others at Carnegie Mellon University, CMU, in Pittsburgh, PA, pioneered that concept back in the 1980's. Thomas Cooper, another CMU grad, worked on the early implementations for XCON, a computer configuration program done by CMU under a DoD grant for Digital Equipment Corporation, DEC, that saved DEC over $1M per year in manpower. Tom is the keynote speaker on where the industry started, where they are now, and where they are going. Tom went on to co-author the first book on the language, OPS5, that was used to make life easier for all concerned.

The other pioneers who will be presenting will be Dr. Richard Hicks, inventor of EZ-Xpert rulebased system and considered to be THE expert on Validation and Verification of rulebased systems; Gary Riley, co-inventor of CLIPS (C-Language Interface to Production Systems) which was the first C/C++ implementation of rulebased system and it was specially developed for NASA; Carlos Seranno-Morales, the co-inventor of Neuron Data Advisor, one of the first Java (a computing language like BASIC or COBOL) versions of a rulebased system; Mark Proctor, co-inventor of Drools (Dynamic Object Oriented Language System) that was one of the very first implementations of an XML/Java version of a rulebase and is totally open source; Dr. Hafedh Mili, who wrote all of the early ILOG JRules rulebase tutorials; Dr. Jacob Feldman, inventor of Open Rules, a rulebase that comes with source code; Jason Morris, an early adopter of Jess (Java Expert System Shell), which was the first of the Java implementations of a rulebase a close copy of the CLIPS system, who is a recognized authority, lecturer and teacher on that system; Dr. Daniel Levine, a local professor at University of Texas Arlington, who is not only an early pioneer in Artificial Neural Networks (another division of AI) but also works with rulebased systems, psychology of the inner workings of the mind and is a widely published author; Dr. Gopal Gupta a local professor at University of Texas Dallas who works with constraint-based programming, another part of rulebased systems; Dr. Leon Kappelman (North Texas University) and John Zachman, the leaders in the world of Enterprise Architecture, will be presenting a two hour talk on how all of this can be road-mapped for executives, managers, programmers and users.

All in all, this is one of the most technical conferences in the world. It bridges the gap between academia, industry and software vendors so that there will be greater cohesion and understanding between all concerned. The T3 (Thursday Think Tank) is a highlight of the conference wherein all of the thought leaders get together in a four-hour, open panel discussion (open to all ORF attendees) and discuss what is needed by rulebased systems that will probably set the directions for the rulebase industry for the next decade. Oh, what does all of this have to do with Paul Revere? Nobody seems to remember the Dartmouth College Conference that started all of this.

SDG

jco

(James Owen is the founder of ORF and a globe-trotting consultant on rulebase systems architecture and implementation.)

[Note: The above is being made available to The Dallas Morning News, Fort Worth Star Telegram, London Financial Times, New York Times, and other major media outlets for publication during the week of Oct 19th - 26th. Should you wish to forward to your local newspaper, please feel free to do so.]

Small Corrections

Greetings:

Small corrections to the previous post: Neither Gary nor Leon will actually have their books on sale AT the conference itself. However, should you already have the book(s) mentioned I sure that they would agree to sign them. :-)

SDG

jco