Monkeys, Queens and Cats

Greetings:

[Short rant]

I have been doing AI type work since the late 80's (seems like only yesterday) and, like most who start in AI work or any kind of programming, we start with very simple problems. The first is always the ubiquitous "Hello World" program. The second, if you are learning inheritance or something of that nature, is checking to see if a child is a grandchild of another person or object or checking to see if this object or person is human or a toad and therefore has five fingers on each hand. Something like that.

For most who use rulebase tools we learn (or were supposed to have learned) the problem of how to help direct a monkey to find bananas to satisfy its hunger OR we learn how to solve the constraint problem of four, eight or sixteen queens on a chess board so that they none can capture the other one. The other one (thanks to the Monday night CBS TV series of "The Big Bang") is Schrodinger's cats. I really do think that these are excellent problems for initial learning experience in our little world of AI or its subset of rulebased systems.

Also, I consider MAB the best teaching tools for AI that we have - but it is, after all, a teaching tool for AI geeks, not something that the normal person outside of AI would ever consider. My Daddy used to say that people who do things like that are Educated Idiots or just Book Smart - meaning that they could solve complex problems in class but couldn't hammer a nail into a board nor saw a straight line and, therefore, useless in the real world.

The objection with continuing to solve and/or teach these problems is that once we have done them in class we tend to use them over and over and over in more advanced discussions as though we could not think beyond the beginner level of thought. Whenever we have to explain something outside of our close circle of friends and workmates, we always run back to our "comfort zone" and show "Hello World" or MAB. Also, to those outside of AI, who have never had to solve anything more complex than balancing their checkbooks, these problems seem rather silly and beyond the ken of the ordinary mortal and therefore not something for every day life.

PLEASE! Let's try and bring something to the table that (1) the ordinary mortal can understand and (2) that possibly has some real-world application(s). For example, what if we (a) had a problem in forecasting, or (b) a problem of finding a terrorist in L.A. or (c) how to process fluid flow in a petro-chemical plant so that we ensure that it will not blow up [and what to do if there is that possibility] or (d) complex shift scheduling or maintenance crew assignment or (e) complex resource assignment in a large manufacturing plant or (f) how to configure specialty helicopters?

OK, these are just thoughts. But if we, the AI community, can begin to establish some better examples then maybe we can be accepted and assimilated into the mainstream river of human thought. Maybe...

SDG

jco

Sunday Ramblings

Greetings:

I just went to the following link that was sent to me by a friend:

http://java.sun.com/developer/media/deepdivejavawarehouse.jsp

It is, indeed, refreshing to see the older guys teaching the younger guys in a Sun-sponsored video. Having children (anyone less than 30) teaching a class should be an anomaly, not a normal thing. While we older troops should be enjoying our advancing years, at the same time we should recognize that we have something to contribute back to those same "children" who are still struggling with undocumented code, insufficient requirements and poorly constructed architecture. When they become frustrated with these daily problems, we can help them to understand those problems and show them how to deal with each one in the appropriate manner.

I think that as we deal with old age in our every day life, so then we learn to deal with death when it finally arrives. Some of us feel that we are still young, 20-year-old studs that have become trapped in in old man's body that is heir to all of the aches and pains of advancing years. I have come to think of that as God's way of preparing us for the end of life so that we welcome Death as a friend who has come to take us home and not as a protagonist against whom we have to fight.

But, so long as we stay alert and up to date on changes in the industry, do our own research, analyze our own data and prepare our own reports, then we will continue to amass knowledge and we can filter that new knowledge through the years of the pain of working with those who always "take the easy way out." This would be those programmers and scientists who don't try to understand the underlying principles of the art and science of their industry that is sometimes so elusive and seemingly unavailable to those who have had only a few years trying to make things work the right way when using apparently insufficient material. Nevertheless, having worked with younger programmers most of my life, I have found that if I can show them something new or a better way to do things, then I am usually accepted by my cohorts as just another geek trying to make sense out of senseless requirements.

So, to the older, more experienced technical personnel I say this; keep on learning. Never give up on a problem and turn it over to someone else just to make life easier. It is that very struggle, this exercise of the brain, to solve complex problems that makes us who we are. This struggle that gives us strength is much like a butterfly coming out of a cocoon that gives it the strength to fly away. The daily solving of those complex problems gives us the strength and the courage to manage smaller problems of everyday life.

SDG

jco

Rulebase Benchmarks 2009 Part 3

Greetings:

Once again we return to the world of Rulebase Benchmarks. Benchmarks for rulebased systems seem to have three camps: Those who love them and usually have good numbers to report. Those whose benchmarks are not remarkable one way or the other and are not worried about them; yet. Those who benchmarks are not very good and keep telling their users and/or customers, "Yes. That's OK for academics. But those benchmarks don't represent the 'real world' in which we have to live and work every day."

At October Rules Fest 2009 I asked the T3 (Thursday Think Tank) if they thought benchmarks were important. Dr. Forgy (who has the fastest rulebase engine on the planet Earth - called TECH) and myself (who has spent the past decade exploring benchmarks, their cause and effect) were ardently in favor of such things. FICO (Blaze Advisor) folks were kind of OK with the idea, especially since they use Dr. Forgy's engine in their rulebase. The rest of the group, whose performance was anywhere from just barely OK to abysmal, were not really against using benchmarks but it just wasn't something that they wanted to discuss in public.

So, here's my proposal: If anyone can produce a "real world" benchmark that can be run on any rulebase engine on any platform, PLEASE DO SO!! Otherwise, those of us who are in this for the long haul will continue to examine a rulebase from many different viewpoints but we shall NOT throw out tried and true tests that continually disclose fatal faults in fancy products that will not perform with large data sets combined with large rule sets. Such applications are large banks, large insurance companies, Homeland Security along with "real world" AI problems in rulebased forecasting, petro-chemical processing problems, electrical power grid production problems, shift scheduling for large plants and/or hospitals and anywhere else that will have the many objects - many patterns complex cross products.

Speaking of which, I guess it's about time to publish the 2009 Benchmarks. Our goal is to finish up in December some time so that at least we can say that we finished this year with a completed product. :-)

SDG

jco

World Conference on Artificial Intelligence

Dallas, Texas; Oct 18, 2009, KBSC:

I'm not sure if I learned this poem in Junior High School or High School, probably the latter:

Listen my children and you shall hear

Of the midnight ride of Paul Revere.

Hardly a man is now alive

Who remembers that famous day and year.

Of course you recognize "The Midnight Ride of Paul Revere" by Henry Wadsworth Longfellow. I thought of that poem last night when I was thinking of the upcoming October Rules Fest 2009 to be held in Dallas, Texas, at the Adolphus Hotel from October 26th - 30th. This conference reminded me of the 1956 Dartmouth College conference where the term Artificial Intelligence, AI, was first used. Those were the giants in the computer age then and ORF 2009 has their pride and joy students presenting the future of rulebased computing (a part of AI) in Dallas.

For example, Dr. Charles Forgy, considered one of the godfathers of that technology, will be speaking on parallel rulebased systems, something that no one is doing today. Well, not yet, anyway... But he and a few others at Carnegie Mellon University, CMU, in Pittsburgh, PA, pioneered that concept back in the 1980's. Thomas Cooper, another CMU grad, worked on the early implementations for XCON, a computer configuration program done by CMU under a DoD grant for Digital Equipment Corporation, DEC, that saved DEC over $1M per year in manpower. Tom is the keynote speaker on where the industry started, where they are now, and where they are going. Tom went on to co-author the first book on the language, OPS5, that was used to make life easier for all concerned.

The other pioneers who will be presenting will be Dr. Richard Hicks, inventor of EZ-Xpert rulebased system and considered to be THE expert on Validation and Verification of rulebased systems; Gary Riley, co-inventor of CLIPS (C-Language Interface to Production Systems) which was the first C/C++ implementation of rulebased system and it was specially developed for NASA; Carlos Seranno-Morales, the co-inventor of Neuron Data Advisor, one of the first Java (a computing language like BASIC or COBOL) versions of a rulebased system; Mark Proctor, co-inventor of Drools (Dynamic Object Oriented Language System) that was one of the very first implementations of an XML/Java version of a rulebase and is totally open source; Dr. Hafedh Mili, who wrote all of the early ILOG JRules rulebase tutorials; Dr. Jacob Feldman, inventor of Open Rules, a rulebase that comes with source code; Jason Morris, an early adopter of Jess (Java Expert System Shell), which was the first of the Java implementations of a rulebase a close copy of the CLIPS system, who is a recognized authority, lecturer and teacher on that system; Dr. Daniel Levine, a local professor at University of Texas Arlington, who is not only an early pioneer in Artificial Neural Networks (another division of AI) but also works with rulebased systems, psychology of the inner workings of the mind and is a widely published author; Dr. Gopal Gupta a local professor at University of Texas Dallas who works with constraint-based programming, another part of rulebased systems; Dr. Leon Kappelman (North Texas University) and John Zachman, the leaders in the world of Enterprise Architecture, will be presenting a two hour talk on how all of this can be road-mapped for executives, managers, programmers and users.

All in all, this is one of the most technical conferences in the world. It bridges the gap between academia, industry and software vendors so that there will be greater cohesion and understanding between all concerned. The T3 (Thursday Think Tank) is a highlight of the conference wherein all of the thought leaders get together in a four-hour, open panel discussion (open to all ORF attendees) and discuss what is needed by rulebased systems that will probably set the directions for the rulebase industry for the next decade. Oh, what does all of this have to do with Paul Revere? Nobody seems to remember the Dartmouth College Conference that started all of this.

SDG

jco

(James Owen is the founder of ORF and a globe-trotting consultant on rulebase systems architecture and implementation.)

[Note: The above is being made available to The Dallas Morning News, Fort Worth Star Telegram, London Financial Times, New York Times, and other major media outlets for publication during the week of Oct 19th - 26th. Should you wish to forward to your local newspaper, please feel free to do so.]

Small Corrections

Greetings:

Small corrections to the previous post: Neither Gary nor Leon will actually have their books on sale AT the conference itself. However, should you already have the book(s) mentioned I sure that they would agree to sign them. :-)

SDG

jco

ORF 2009 and the Dartmouth Conference

Greetings:

As you may not know, the term AI (Artificial Intelligence) was first coined by those attending the Dartmouth College Conference way back in 1956. You can read the invitation list at http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html if you like. The absolute best attended that conference. And that is what started our industry.

So, what does that have to do with ORF 2009? Because we have the best of the best of the best speaking at ORF 2009. And some of them are students of those who started all of this way back in 1956. ORF 2009 will be world-shattering and I, for one, have no idea of how to tell the world what we have here. Think of Woodstock - who knew? Well, I know from the white papers that are coming in and from the presentations that are coming in AND from the credentials of those who are speaking that we might never have such an event again.

If you know of anyone in the media who can see the dream, who can grasp what is happening here later this month, PLEASE tell them about it. This is not a Dallas thing. This is not a Texas thing. This is a world-shaking event. And, to reiterate, it might never happen again. The responses that I have heard so far is, "OK, it's a conference on geek stuff. So what? Those happen all the time." Not true. Not like this. Not with THESE kinds of speakers and topics.

I'm not a PR guy. I'm not a marketeer. I'm a geek and I don't know how to get the word out to the rest of the world that this could very well be a once-in-a-lifetime event. So, if you do know someone who can help, please ask them to do that very thing. Thanks,

SDG

jco

October Rules Fest 2009 Update

Greetings:

Lots and lots of things happening at ORF 2009 so let's just get right to it:

Drools Boot Camp, all day Sunday and Monday morning. Free for ORF attendees. Small charge for DBC only. Mark Proctor and Edson Tirelli teaching. See link for details.

Jess Boot Camp, all day Sunday and Monday morning. Free for ORF attendees. Small charge for JBC only. Jason Morris (Jess guru and certified trainer) teaching. See link for details.

Tom Cooper, co-author of "Rule-based Programming With OPS5", will be our Keynote Speaker. Tom goes WAY back and is one of the pioneers of rulebased systems. The book is still available but has been out of print for quite some time.

Discounted Books. We don't have the complete list yet but there will be lots of rulebase specific books deeply discounted (more than Amazon) for ORF attendees. Dr. Kappelman and Gary Riley will have theirs there for signing (if you're into the groupie thing - I am) and I'm sure that we will have some more before the conference starts.

T3 - Thursday Think Tank - is really gearing up to be one of the highlights of the conference. Our take on this is that the outcome might set the direction(s) for rulebased systems (BRMS) for the next decade.

TECH, Dr. Forgy's new algorithm, will be discussed.

Several talks on Complex Event Processing (CEP) by the leaders in the field. Dr. Forgy has stated that this could well be one of the most important advances for rulebased systems for this century.

The Adolphus has dropped their room rates to $129 / night in order to encourage you to stay where the action is. BUT, we have only a limited number of rooms at this rate so get your reservation in early.

STUDENT FEES will be $150 which will also allow them to attend either of the boot camps. We just announced this and already students are signing up. Meaning that they might jolly well take up the conference slots so you need to hurry and get your conference registration NOW !!

Ping Pong / Table Tennis and/or Foosball has been cancelled due to lack of interest. Sorry about that. I was looking forward to a tournament kind of evening.

Speakers: New speakers and assistants - we have 30 of them now and it is a single track conference so you don't have to make those mind-wrenching decisions as to which one to attend.

White Papers and Presentations will be available during the conference for you to load on your computer to follow along.

Brochures will be available at the conference and electronically later this week. (Thanks, Chelanie). This will be a real keep-sake for your Golden Years.

So, if you haven't registered already, do so NOW. He who hesitates is lost. :-)

SDG

jco

Day Two - FICO Decision Management Tools User Group

Greetings:

Wow. Yesterday was good but today was even better. One interesting this is that Oracle was presenting on an idea and their presenter did not know that Oracle had bought a rulebase (two of them, in fact: Rule Burst & Haley Expert Systems) of their own that is a direct competitor to FICO's Blaze Advisor. Weird, really weird.

OK, performance issues - I ran through a brief (55 minute) presentation on how to improve performance. More of how NOT to get into trouble although I did point out some time-proven tips on how to make even Advisor run faster.

Then came the Illinois Death March: We took a bus to the lake for a river cruise. (whater...) The bus dropped us off and we walked past ship after ship after ship. Now, understand, walking a mile is not such a big deal UNLESS you have a bad back, two knees that need to be replaced and pinched nerves in your neck. Then (after a rather nice meal onboard the ship) we had to walk BACK! So, thank the Lord for Hydrocodone and Soma. Between the two I made it through the night.

So, wrapping up today. Tonight begins Rosh Hoshanna so a lot of the guys have to get home in time for services. People are here from all over the world; Russia, Lithuania, Israel, France and even one or two from Texas. :-) So, this will be the last report on this conference. Nothing technical. You'll have to get that from FICO since everything is copyrighted into their vault of knowledge.

SDG

jco

Day One - FICO Decision Management Tools User Group

Greetings:

First day started at 13:00 but I couldn't get through traffic and arrived at 15:00. I went to the Decision Support clinic headed by Don Grice and Maarten Van Lier that started at 13:00 or so and I was so far behind that I'll have to catch up later.

The hotel is nice but the 17:30 cheese-and-wine was awesome. Lots of brisket, pasta bar, chips-and-dips, beer/wine/etc. bars made it nice. But with my bad back I could only stand up for so long. Lots of old friends there: Tom Travis, Vlad Silverman, Carole Ann Berlioz-Matignon, Carlos Seranno-Morales (Europeans are really big on double last names). Willie Hall and Mark Eastwood are supposed to arrive tomorrow.

The conference itself seems focused on business users rather than geeks. (So what am I doing here?) Several good talks lined up for today including some on Blaze Advisor and business optimization tools. My talk will be on optimizing Blaze Advisor at 16:00 today and then FICO has a river-boat tour and supper planned. I'm not sure if the PPTs or talks will be available to the general public but if they are I'll let you know. Maarten's talk last night was pretty good so I hope that they do release them to the great, unwashed masses. :-)

SDG

jco

MIND Over Matter: If You Don't Mind Then It Doesn't Matter

Greetings:

If you don't already subscribe to the Mind magazine, may I suggest that you consider it. Drop by your local magazine store or book store and take a look at what is on the shelf and then save a few coins by subscribing to the magazine on an annual basis. Why? OK, here's a couple of thoughts that I got from one of the more recent issues:

Do you remember the last (or next to last) session of M.A.S.H. on TV? It seems that there were some villagers trying to hide from some enemy soldiers and, if caught, all would be killed. One of the young mothers had a baby with her and the baby, as they will do from time to time, was scared and began to cry. The rest of the villagers kept telling her to be quiet or they all would die. The mother ended up suffocating the baby in her attempt to keep it quiet. Questions: Was she right to kill her child to keep the other villagers from being found out and not only killed but probably tortured as well? What did this have to do with MIND magazine?

It seems that whenever the mind is presented with a conundrum such as this that certain portions of the brain are activated. In younger minds, they are, basically, stored in one location. In older minds, the job is distributed out to more than one part of the brain. What I think is happening is that older minds have to consider more than one answer and that the outcomes are different depending on which solution is implemented and when it is done and how it is done. This comes all the way back around to rulebased systems: It's the Senior KE, the Senior Knowledge Engineer, the Rule Architect that will make a huge difference in the overall outcome.

Why? Because that Senior KE has seen the pitfalls of what happens when certain logical paths are followed and has experienced the outcomes of failures in the past. What was it that George Satayana said? "Those who will not learn from history are forced to repeat it." (Often attributed to Ben Franklin but supposedly he got it from George...) We learn from our mistakes. The more intelligent ones learn from the mistakes of others. It seems that some never learn but have keep making the same mistake over and over and over. My favorite is the company that hears about off-shoring work at really, really cheap rates. What they don't hear about are all of the problems connected with different cultures, different languages, different work habits, the advantage of face-to-fact meetings, etc.

The other one is when a company refuses to pay the higher rates of a senior consultant and hires a programmer who is really, really good at Java or C/C++ and assigns them to do the work that truly needs a Senior KE to direct the effort. the project goes down the porcelain receptacle reserved for such sludge and management brazenly decides that a rulebased solution was not needed for a project such as this. No one really loses their jobs because the smart ones bailed out after the first few months and went on the bigger and better things. Those stuck with the smelly thing can then blame those who started it. Those who started it blame those who followed it with not carrying out their particular vision and, of course, it went bad.

Now, wasn't that fun? And I'm sure that most of the senior guys have been on the Project From Hell, Death March, etc. By whatever name, it wasn't much fun if you were on the final march and, as consultants called in to save such a fiasco, it's usually impossible with what is left of the budget. But, once in a while, it is possible and what remains of management is so desperate that they actually let you do the right thing. And you get to be the hero. One or two of those go a long, long way toward making up for all of the fiascos foisted upon you in the past.

So, what is the lesson to be learned here? Take care and be wary of what you accept when management is willing to pay a higher rate plus expenses. There HAS to be a catch somewhere and I'll bet it's because some ninny has already messed up the playground.

More and more good news on ORF 2009

Greetings:

First, some GREAT news! We have re-arranged the Agenda so that we could accommodate a "Thursday Afternoon Think Tank on Needful Things." This would be a round table discussion (with some attendee participation) on various subjects that concern the more technical aspects of rulebased systems rather than their direct application to problems per se. Those who have agreed to be on the panel are Dr. Charles Forgy, Mark Proctor, Gary Riley, Dr. Jacob Feldman, Dr. Richard Hicks, Carole Ann Berlioz-Matignon, Carlos Seranno-Morales, Jason Morris and Paul Vincent. For example, a few of the topics will (might) be Complex Event Processing, Benchmarks in This Century, Patterns in Rulebased Systems, Finite State Machines as a Special Type of Rules, etc. This is focused more on the "hard core" developers and "thought leaders" in the rulebase field. However, ALL attendees and/or speakers are invited to attend and be part of the direction(s) that rulebased systems might take for the next decade.

Another change is that there will be another Drools Boot Camp (like last year) headed up by Mark Proctor. This will be co-located with October Rules Fest in The Adolphus hotel and will be all day SUNDAY and MONDY morning. Since Mark and Edson will be there anyway this was a great opportunity for another in a great series of DBCs that Mark has done in the past. The DBC will be absolutely free for ORF attendees but there will be a charge if you are attending DBC only so that we can pay for the room, coffee, etc.

Finally, the conference proper will start on MONDAY AFTERNOON rather than Tuesday morning. The lead-off speaker will be Lawrence Terrill on a really good explanation of the Rete Algorithm - one of the best that I have ever seen.

I've been reading through some of the early submissions of the White Papers for October Rules Fest and they are really outstanding! Dr. Charles Forgy's paper on parallel rulebased systems (need and solutions) is fantastic. As is that of Gary Riley on optimizing performance in CLIPS. Andrew Waterman submitted one on games and is looking for folks to submit more add-ons. Dr. Richard Hicks has submitted one on Validation and Verification that everyone should read. (He is giving each attendee a free copy of the game.) Dr. Daniel Levine's paper explores the inner-dimension of the brain itself, its own self-imposed rules and problems with "rational thought process." Remember, all of these will be available at no charge only to attendees at ORF 2009.

Anyway, you need to sign up (if you haven't already done so) and be sure to reserve a hotel room. We still have a few places left.

SDG

jco

http://www.OctoberRulesFest.org

He Who Knows...

Greetings:

Nothing much to do with rules specifically but more with things in general. I ran across an old by-line that I used to use a long time ago:

"He who knows not and knows not that he knows not is a fool.

Shun him.

He who knows not and knows that he knows not is a child.

Teach him.

He who knows and knows not that he knows is asleep.

Wake him.

He who knows and knows that he knows is a genius.

Follow him."

This was supposedly an old Hindu proverb but who knows? Maybe Ben Franklin made it up but it sounds too psychological to be Ben. On large projects I've met all kinds distributed in a natural distribution pattern. Maybe one or two at the top and one or two at the top. Maybe. But lots in between which is pretty much a compliment to programmers everywhere.

Here's the problem: What to do with the first one? Management picked this guy and he/she is a "valuable" member of the development team. I had to point this out to a company once after a two-week gig to help them get started with rules. I told ONLY the Senior Director for the project after he pointedly asked about each person. But his pride-and-joy manger had picked that individual to move from being a contractor to leading the rule development team. The guy had been doing C/C++ stuff for two years and had made very little progress toward a real rulebased system. But, in his defense, his GUI stuff looked really cool.

Truly, he didn't know rulebase development and he didn't know that he didn't know. Their regular employee on the team (who would have to work for this ninny) was more of the second kind; eager to learn anything that you could teach him. I'm not sure where the manager fit on the chart but she certainly was convinced that she had made her decision and she was going to back it up 100%. Also, unfortunately for me, she was the one who hired me and I had to tell her the same thing. They were ecstatic about the analysis that I had done until 2:00 the afternoon that I was leaving. By 2:15 all bets were off. So, I didn't get called back for the traditional "sanity check" because the bare, naked truth was out of the bag and on the table. Now they had to justify keeping these plans and that meant that they did NOT want anything to do with facts.

As one of my old managers used to tell me, "James, you have great technical skills. It's your 'soft skills" that need work. You don't know how to tell anyone diplomatically that what they are doing is trash and that their development personnel need five years training just to get up to the zero level. Sometimes you can't tell someone the truth straight out - you have to sugar coat it so that it tastes better." He was right, of course. Maybe one day I'll find a home where I can just DO things and not have to worry about feelings. So, that's my tale of adventures with the upper class of management. I don't belong there. Hopefully you can learn from that and be nice to nit-wits and dweebs. After all, if God didn't love them he wouldn't have made so many of them. :-)

SDG

jco

Benchmarks 2009 to follow ORF 2009

Greetings:

Just a quick follow-up on all of the earlier posts about Benchmarks. It looks like I'll be doing them AFTER the October Rules Fest rather than before. Too much stuff has piled up on my plate. I do plan on doing CLIPS and Jess on WaltzDB-200 since the code is already written for those. And I've run most of the OPSJ and TECH stuff already. But the multi-platform and optimization as well as the code for Advisor, JRules, Drools and others will have to wait until after ORF 2009 is over. If anyone has a suggestion, or wants to write the code for the last three, I would appreciate it.

It's beginning to look as though the Annual Benchmark Cook-Off will have to be done the last two months of each year since that seems to be when most companies begin to shut down and then crank up again in January.

SDG

jco

FICO Forum: Decision Management Tools User Group

Greetings:

The FICO Forum: Decision Management Tool User Group will meet on September 16 - September 18 at the Hotel Allegro in Chicago, IL. FICO has kindly asked me to chat with the FICO Blaze Advisor users about optimizing rules, benchmarks and things of that nature. My talk will be on September 17th so if you happen to be in Chicago at the time be sure to come by the conference. There is a lot more than my talk, of course, and this can be found at

http://www.online-reg.com/FICO/2009/dmtools/

http://www.online-reg.com/FICO/2009/dmtools/speakers.lasso

if you would like more information about the conference itself or the other speakers. Like most other conferences, I would like to be able to attend all four tracks but I can only attend one session at a time - so in addition to having to attend my own presentation (one that I already know quite well) I'll miss out on 3/4 of the great stuff, including the half-day tutorials on Wednesday. All-in-all, it should be a really great conference.

SDG

jco

October Rules Fest 2009 Books available

Greetings:

More conference news:

More good news for attendees: Dr. Leon Kappleman, a widely recognized authority on Enterprise Architcture, has made it possible for attendees to order his new book on "The SIM Guide to Enterprise Architecture" that normally retails for $59.95 and get a 40% discount. You can get better preliminary details at http://www.routledge.com/books/The-SIM-Guide-to-Enterprise-Architecture-isbn9781439811139 where you can some more details about the book itself. [Added note 10 Aug] Here is a list of contributors to the book: http://courses.unt.edu/kappelman/aboutwork/Book%20Slide%201.pdf

So: Register for the conference THEN register at the hotel for the conference rate. If you have ANY problems (registering OR getting the special ORF rate at the hotel) PLEASE let either myself or Chelanie know about it. See you in October!

Finally: If anyone else has a book that they would like to offer discounts to ORF conference attendees, please let us know and we will start a listing of all of them. But, right now, this is the only one. Enjoy...

SDG

jco

Benchmarking at ORF 2009, Part 7

Greetings:

Conference News:

Dr. Forgy has just about finished up his new algorithm - there's always some last minute thing that needs to be done. While testing his new TECH algorithm on my variousl platforms I have found it to be about an order of magnitude faster then Rete 2 or Rete III on both my MacIntosh, Core2Duo Unix environment and the Windows 64-bit Dell i7. On the i7 (that is effectively 8 CPUs) it scales nicely and provides even faster performance. I can't wait to get my 8-core Mac Pro (the Nehalem 2x4 CPU) and see how it runs there. Dr. Forgy has agreed to discuss the new algorithm after hours during Pub Nights in the Walt Garrison restaurant and bar that is located in The Adolphus Hotel.

OH! You wanted numbers? OK, the only comparison that I have right now is OPSJ versus TECH but here is what I have so far - lots more to come later and I'll publish it all at ORF. We had to use a new benchmark because TECH runs so blooming fast that it's hard to compare the difference in 1.2 seconds and 0.57 seconds as anything meaningful when running WaltzDB-16. OPSJ runs that one 2.7 seconds on the Core2Duo Mac and in 0.433 seconds on the Dell i7. TECH is not really measurable.

WaltzDB-200 (New Benchmark for 2009) - in seconds

Core2Duo Mac, Leopard, 3GB RAM:

OPSJ: 46.467

TECH: 4.549

Dell i7 64-bit Vista, 6GB RAM

OPSJ: 17.466

TECH: 2.362

So, that's pretty much a 10:1 (or thereabouts) speed improvement firing 246,233 rules. Remember, OPSJ is the underlying engine in Java that incorporates Rete 2 / Rete III, the fastest Java rulebase on the planet Earth. Until now, that is. Also, remember that I have NOT finished the full comparisons checkout and double-checked everything to be sure that they are the same and firing the same rules and doing everything. But, so far, it looks pretty fantastic. As soon as I get a chance I'll do the other BRMS and rulebased systems including Drools, Blaze Advisor, Jess, CLIPS and (possibly) JRules.

Depending on when his patents are approved and finalized, Dr. Forgy will be happy to talk with you at ORF about this ground-breaking technology.

SDG

jco

Manny and James

Greetings:

I could start this blog with a cute saying, but the essence of the blog is more serious so I shan't. Manny Gandarillas posted an article http://www.BRCommunity.com/a2009/b487.html on BR Forum that was a tongue-in-cheek view of the different views of a rulebase as experienced by different personnel on any rulebased program. Subsequently, James Taylor posted a response on BR Forum at http://www.BRCommunity.com/a2009/b487.html where he took Manny to task for some of the things that he said. So, just to add another view, permit me just a quick view of both articles and an even briefer rejoinder.

First, neither Haley Expert Rules nor Mind Box are considered "leaders" in the world of BRMS. Way back in 2004, with some agreement from both ILOG and FICO, I wrote a brief definition for a BRMS at http://www.infoworld.com/t/business/business-rules-management-systems-548 that was pretty much as brief as one could be and still say anything about it in a 400-word side bar. In that description I did say that, for the most part, the business analysts should be writing the code BUT with the guidance of a full-blown knowledge engineer. Since that time I have seen situations where the BAs did not want any help whatsoever from the IT guys except to construct the architecture for the Java classes, database, web screens, whatever. In other situations, the BAs wanted no part of writing the rules except for writing them into the rules document and letting the IT department implement them in what ever fashion they deemed fit so long as it passed the final testing for both validation, verification and speed.

Let's face it: The business guys have the money but have not the technical expertise to use it properly if left alone. The IT guys know how to implement things but the real rules of business are usually beyond their job skills. The rules help implement the "what" and the "why" of what happens in the coropration, not the "how". The "how" is normally left up to IT. BUT, at what point do we draw the line between the BAs and the IT guys? This last bit is what can cause a BRMS to fail if one party or the other cannot resolve their power struggles and work in harmony with the other. Business is, after all, the customer of the IT department. And if business wants things to work correctly, they need to be friends with the geeks.

I do think that James is spot-on about the business personnel not needing to care about forward or backward chaining, sequential or Rete or Dete or whatever - it's the geeks who will have to make this blooming thing perform properly and scale to enterprise proportions who will be, should be, concerned about those things. And the BAs will HAVE to work with the IT guys to accept that some things will work and some things won't work.

However, let's get one thing perfectly clear: When the BAs take over the logic of the rules then THEY are the ones responsible for the testing and verification of the results, not the IT guys. To me, this has and should be one of the great things about a real BRMS; the BAs take responsibility for what actually happens. Normally, this kind of arrangement just provides more and more work for the IT guys who have to design more enterprise architecture to handle all of the various rulebased system (BRMS) that will begin to crop up once the word gets out that the rules CAN be changed, tested, verified and put into productions in days (three to five) rather than months. BUT, and this is equally important, the design of a proper BRMS is not an easy task and it DOES require the expertise of someone who has done this several times for many different companies and knows where the land mines are located.

Bottom line: Even with the shameless plug for BR Forum and his book, James is right on several of the points. However, Manny did bring up some bullet points that gave us a focus on our discussion. Personally, while I might not agree with all of the contained thoughts in Manny's chart, it was unique and should be shared for further consideration by most vendors. (OK, it's a twist on the old view of an elephant joke.) Now, for my shameless plug: If you guys really, really ant to get into the technical semantics of BRMS, come to October Rules Fest in Dallas, http://www.OctoberRulesFest.org and see what the CIOs and technical architects think about the BRMS hype.

BTW, what is the best way to eat that infamous elephant? (A: One bite at a time.) The same applies to a BRMS or rulebase or any other unfamiliar territory. Take it slow, make something small work first with the ultimate goal to incorporate all of the corporate knowledge into the rules. Not today, not this year. But maybe in five years. :-)

SDG

jco

Only Three More Days !!

Greetings:

If you want to attend the October Rules Fest AND you would like to avail yourself of the Early Bird Discount of $50, you should do so by midnight (PDT, GMT -8) THIS Friday night, July 31st. That's when it expires.

Also, remember that the first 100 attendees to sign up will get the best seating in the house. All attendees and speakers will have power outlets and WiFi available in the meeting room itself. But the first 100 or so will be two persons at a 6' (just under two meters) table.

I have said this so many times that I'm beginning to sound like a cracked record - if you're old enough to remember records made of vinyl that had a tendency to get caught in a groove and just repeat itself over and over and over... Anyway, ORF 2009 has absolutely the BEST Technical Speakers in the world. AND, these guys will be hanging around during the conference to chat with the attendees - especially during Pub Nights.

Who else can say that they bought a beer for Mark "Drools" Proctor, Charles "Dr. Rete" Forgy, Jason "The Jess Answer Man" Morris, Gary "CLIPS" Riley, John "EA" Zachman, Carole Ann Berlioz-Matignon, Carlos "Advisor" Seranno-Morales, Paul "Tibco" Vincent, Edson "CEP" Terilli, Daniel "TurboGeek" Brookshier, Rick "V&V" Hicks, Daniel "ANN" Levine, Hafedh "I wrote the books on JRules" Mili or Jacob "Open Rules" Feldman?? But, if you are at the conference and hang out on Pub Night at the Walt Garrison Room, YOU might !! :-)

The main theme of this year is just like last year: Have Fun! We are doing the one thing that we enjoy the most (outside of one or two indoor activities) and we should be having lots and lots of fun doing it. Last year was simply great. This year is even better. Better Hotel. Better location. Better presentations. I would say better speakers but some are the same as last year BUT we have many new speakers who will knock your socks off!! And the vendors who are the sponsors will have the very best technical personnel to answer your most perplexing problems.

This year, we will have redundancy on the video / audio recording so that we don't have gaps like last year. We learned our lesson about depending on a single source of technology. Greg Barton has promised the best recording and Chelanie Israel has promised to deliver a truly professional, well-polished product for those who attended the meeting. And, no, we do not have plans to sell the video to anyone. It's our free gift to those who attend the conference - and ONLY to them.

Sign up NOW and avoid the rush AND get an extra 10% along the way. Also, when you sign up, be SURE to sign up at the hotel for the special rates. If you have any trouble whatsoever, send me an email and let me know. [ jco@kbsc.com ]

SDG

jco

Agile Programming

Greetings:

Recently I've been thinking that I need another blog - one that I would use just to vent and blow off steam at the stupidity of some of the things that we do as programmers. One of those is something called "Agile Programming" where two programmers work on one computer, both at the same time; one writing, one watching and commenting. This was all hyped up a couple of years ago that, in my own opinion, was to help the "less developed" programmers ramp up quicker by being teamed with a more experienced programmer. (Surely you wouldn't put two of the newbies together, would you?)

Anyway, the sheer though of having to explain everything that I do to a newbie who hasn't taken the time to read the first book on rulebase theory, has forgotten everything that he/she might ever have known about statistics, has never had an accounting course in their life and reads only blogs like this for information (OK, they don't even read blogs any more; they just listen to "tweets" and such) sends me into tremors of sheer terror. Don't get me wrong; there is a place for training. But that place is NOT when you are behind schedule (aren't you always?) and you need to get something "done" by the end of the day/week/month.

So, when do the newbies get training from the old timers? When those things are scheduled for (maybe) an hour or two each day or each week. When the trainer can have time to properly present the complete thought process that goes behind each precept of a programming paradigm. When the logic of a though process is presented in a proper methodology rather than a series of "Why" statements followed by an answer that is intended to just shut the other person up rather than teach. This is because you are trying actually to THINK through a laborious process. When done in this manner, sometimes learning takes place. Hopefully.

At our last Java MUG meeting the speaker was pointing out the time that it took to build an idea or a process in your head and that, if interrupted by various things, how long it would take to get back on track. It was actually very revealing. Without intending to do so, he proved my point. Once you are interrupted you can not just turn around and pick up where you left off. Depending on how long was the interruption time and for what reason, it could easily take 15 or 20 minutes to get back to that level of concentration and/or that step in the workflow process.

Which is why I take the phone off the hook, shut the door to my office and put my headphones on when I am trying to concentrate and work on something that requires that level of focus and thought. BTW, during the meeting I did ask if anyone had ever done Agile Programming. Only two guys raised their hands and only one thought that it was a good idea to do programming in that manner. BUT, he did qualify that it was only effective for about four hours a day. According to him, productivity and quality of programming went (way?) up when Agile programming was used. That tells me that the problem "might" have been way too many newbies who needed to improve their programming skills. But, that's probably just my prejudices talking.

When did I personally use Agile Programming? Only two clients have ever tried to force me to do that. One was for a major insurance company in the USA and the other was in the UK. At the USA company it came down to either I would have to leave or they would have lower their expectations of what I was doing. After my rebellion all of the other senior consultants did the same thing and the whole idea was thrown out.

The client in the UK kept sending me really low-paid technicians hired from 4th-world countries who didn't have the foggiest idea of how to use a rulebase nor what was the thought process. I was working as an emloyee at that time for a major BRMS vendor and, at first, they resisted. But when the 4th-world vendor sent two or three of their techies to school and became a partner, then I was asked to reconsider. I told them the same thing; I could do that but production would suffer if I had to stop and explain every little thing to a newbie. Besides, I was not being paid to train personnel outside of my company. If they needed more training then they could go to the vendor-provided school for more advanced courses. They didn't and neither did I.

So, did I ever try it and have it fail? Nope. My clients wanted me working on rulebase problems 100% of the time and were not willing to pay me my normal rate to train a newbie; not even one of their own employees.

Comments to this blog are always welcome. Even when you believe in something as silly as Agile Programming and how it can improve the quality and quantity of programming code. I would ask only that if you do think Agile Programming is a good thing that you tell us of your personal experience and how it helped the quality and quantity of the programming code. And be specific; not that the programming was "better" but how this "better" was measured with and without Agile Programming. What benchmarks did you or your company use that would help guide the rest of us in measuring the "quality and quantity" of the programming code. And if you believe, as I do, that Agile Programming is for really strange people then tell us why in very specific term and in reference to personal experience. Thanks,

SDG

jco

More on Benchmarks for 2009

Greetings:

Just another post to TRY and get some discussion started on benchmarks. All I seem to get from the vendors is, "Those old things? Those are not 'real-world' benchmarks!" OK, no argument here... So, does anyone have a "real-world" benchmark? No? Fine, then we will use what we have and hope for the best since they seem to work fine on checking the rule processing performance (using almost "real-world" rules) on most engines.

Checking performance on a modeling tool, such as Corticon, Visual Rules, Rule Burst, VisiRules, any Decision Table or Decision Tree, is absolutely NOT what checking rule processing power is all about. [I'll probably catch some flak on that remark but those just my personal feelings.] If you already have the rules hard-coded using either straight-up Java or some kind of modeling tool that produces Java code (such as sequential rules or something else along those lines) then that SHOULD run faster than a real rulebase engine that is designed from the ground up to be an inferencing engine based on whatever algorithm you like.

However, I would like this year to allow a couple of things that I have not done in the past: (1) Allow a "warm-up" time of maybe three or four code passes through all of the rules using different date each time and then running the rules for 10 consecutive passes using 10 different sets of data and taking the average time for the benchmark time. In years past, rules did not run under EJB/J2EE or similar environments (we had Java for several years before we had J2EE/EJB) and we did not allow such things. However, with the increased overhead of having to have that in the core part of the engine I think that it should be allowed. (2) I'm going to drop the old version of Miss Manners 8, 16, 32, 64, 128 and 256 and substitute Miss Manners 2009 - which is the ORF example for this year. (3) The other two benchmarks from the old days are still good, Waltz-50 and WaltzDB-16. (4) However, we are introducing a new WaltzDB-200 this year just to really get some long lead times. (5) We will run these all on the following systems

a1. Mac Core2Duo, 3GB RAM, OS X Leopard, 64-bit (which is Free BSD Unix with a pretty face)

a2. Mac Dual-Quad Core, 8GB RAM, OS X White Leopard, 64-bit [maybe...]

b. HP Intel, 3GB RAM, Dual Threaded, Windows XP, 32-bit

c. Dell Intel i7, 4-core, 8-threads, 6GB RAM, Windows Vitria 64-bit

I might try to work in some Linux if there seems to be any significant speed differentiation on an Intel running Linux or Windows - but experience teaches that usually the Windows version runs faster. But I will check it anyway just to be sure. (6) The systems that I am hoping to check will be (alphabetical order)

a. Blaze Advisor 6.7

b. Drools Version 5.x

c. CLIPS Version 3.0b

d. Jess 7.0-p2

e. JRules Version 6.7.3 or 7.x (depends...)

f. OPSJ Version 6.0

g. OPSJ-NT Version 1.0

Probably, I will publish the results here, along with the previous years of Performance benchmarks, as well as on the KBSC home page. The comparisons of 32-bit and 64-bit should tell us something about scalability. The comparisons of different OS should tell us something about scalability and transportability.

One more thing: If any of the other vendors can demonstrate a suitable version of the benchmarks I will include them - but NOT the same thing that I did a few years ago when I allowed a "similar" version of the benchmarks to be used by a vendor that could not code straight-up IF - THEN - ELSE rules using a NOT statement in there somewhere.

I do expect cheating on the part of the vendors. Somehow, I must find a benchmark somewhere that will not allow that so I'll probably throw in a one that has lots of NOT statements in it or something really rude like that. I know that the vendors don't really pay attention to benchmarks any more so I'm hoping that the customers of these and other vendors will stress performance benchmarks to their suppliers as another check of good engineering. Layering GUI after GUI after Model after Model is cool EXCEPT when you forget how to perform under the pressure of millions of transactions per day that need complex, real rulebase-type analysis.

SDG

jco

2009 Benchmarks - Again

Greetings:

Sometimes I wonder how many folks find this subject of intense interest and how many are totally bored with the idea of comparing benchmarks?  Why even have benchmarks?  After all, most vendors seem to think that companies are not interested in them so long as the product will "do the job in the real world."  OK, let's pursue that line of thought for just a moment.  

Let's say that you are buying a car for your company.  Some of the constraints are the mileage, longevity, repair frequency (time and costs), ROI of the auto, etc.  All of these are benchmarks.  Let's say that you are not buying a car for yourself nor for your company but you buying for the military and you want an armored vehicle.  Then the constraints would be how effective is the bullet proofing, what firepower is mounted standard, what is the performance in sand and mud, what is the carrying capacity, etc.  Again, all of this is benchmarking.

Now, let's say that you pick up any car magazine in your local bookstore.  Check the charts and you will find under performance the times for 0 - 60 mph, standing 1/4 mile performance and top speed.  Again, benchmarks.  And, agreed, none of these are real-world benchmarks since very few people take their Corvette or Camero from 0 to 60 more than once or twice and almost none of them would ever take their auto over 150 mph.  BUT, these are benchmarks for automobiles.  This is why we have such things as the Le Mans 24 Hour race - another benchmark to determine the best of breed.

Benchmarks for databases have been here since their advent and no one has ever complained about them.  OK, not very much.  Spec-Int and Spec-fp tell the potential customer how well the database that is being considered for purchase will perform under certain standard, non-real-world conditions and from this they will extrapolate whether a particular product has improved its performance as compared to previous versions or against other products OR how well the product has improved from one date to another, and the client will try to determine whether the product will or will not perform as expected in their particular application(s).  

So why, Oh Why, can't the rulebase / BRMS vendors agree on at least one benchmark for performance of their products.  Mostly it is fear that this will become the only basis for comparison of products.  

This belief is based on past experience when a potential customer would use the published benchmarks of KBSC or other independent consultants to beat them across the head and shoulders so that the vendor would have to lower their price to get an order.  (Actually, the vendor didn't have to lower the price but salesmen are notoriously weak-willed creatures who find it easier to sell on the basis of price alone without any consideration to customer needs and their ability to fulfill those needs.)  

After all, one does not purchase a Corvette on the need for transportation, they purchase the Corvette because they like its looks and its performance.  The same thing can be said for a Mercedes, Cadillac or pickup truck - they are purchased NOT on performance alone but also on a myriad of little factors that make up the overall buying experience.

All of that to say this:  I think I'll have the new benchmarks ready for most rulebase / BRMS systems ready before September 15th this year and ready for publication.  Right now we're going to use Waltz-50, WaltzDB-200 and the Manners128-2009 when loaded on Unix (Mac OS X), Windows XP (32 bit) and Windows 64 bit.  That's a lot of work between now and then but with the help of some friends maybe I'll make it.

Watch this space for more news on benchmarks.   BTW, the BEST benchmark (after reviewing the standard benchmarks) is to put the product on your system, use your data, use your process, use your rules and run your tests.  Only then will you know for sure whether the potential product is suitable for your application(s).

Looking forward to seeing all of you at the October Rules Fest.  Check out http://www.OctoberRulesFest.org for more details on the world class lineup of speakers and activities.

SDG

jco

Open Source Leeches

Greetings:

InfoWorld ran an article this week on "Open Source Leeches"  and it got some responses, one from Daniel Selman of IBM/ILOG.  While the article is spot-on about some companies using Open Source software and keeping their own solutions secret in order to "get the jump" on their competition, it is decidedly UNFAIR to broadly paint all companies with the same swath using a really wide brush.   

For example, IBM/ILOG contributes back to Eclipse.  And the mentioned Southwest Airlines continually contributes back to the Drools project.  Mark Proctor (Drools) is always commenting that "There is no such thing as a free lunch."  His way of handling "leeches" is to just quit responding to their questions after a while.  

My own thoughts here are that many companies either (1) don't NEED to make changes to the product and can use it right out of the box or (2) don't feel comfortable making changes to a vendor product.  In the second case, that is  normally the result is decades of using vendor products "as is" and only making suggestions.

So, maybe Dave Rosenberg (and Bill Snyder) need to research the "problem" more thoroughly and give us some facts and statistics.  For example, how many companies are using Drools, Eclipse, JBoss, etc. for free.  And, out of those companies, how many have made NO contributions to improve the product.  I have a feeling that the "NO contributor" companies will be a really small percentage.  

After all, the "spirit of community software" is NOT that you have everyone contributing to the product, but that you have a few really solid professionals leading the way for the newbies.  Personally, I use lots of "free software" (Mark hates that expression) and rarely contribute to the effort - not because I don't want to but I would have to learn a whole new product each time.  And I only have time for one or two "projects" in my life at one time.   So, I use a lot of them and contribute to only a couple of them.  

Again, folks need to get the facts straight before they leap to unsupportable conclusions.

SDG

jco

ORF 2009 Super Early Bird Discount Extension

Greetings:

October Rules Fest 2009 has extended the Super Early Bird discount of 20% off of normal registration fees to midnight, June 6th, 2009. Regular registration is $500 so that represents $100 savings. If you have any questions, check out http://www.OctoberRulesFest.org for more info on speakers, agenda, etc. The hotel should be determined not later than Friday of this week, June 5th. Anyway, be SURE to register THIS WEEK to get the best savings. :-)

With parking at ANY hotel in downtown Dallas being astronomical (cheaper than New York, San Francisco, London or Paris though) it might be more economical to park-and-ride. There will be directions on the ORF web page at a later date about how to do that.

SDG
jco

October Rules Fest 2009

Greetings:

Well, ORF 2009 is ready to rock and roll in downtown Dallas again, only this year the conference will be in the center of the downtown Dallas restaurant district rather than on the outskirts like last year. Just point your Google Earth to 1400 Commerce Street in downtown Dallas and you'll see what I mean. Everything from MacDonald's to The French Room to the "ultra cool" Fuse restaurant. After we check them out, the ORF web site have an ever-expanding list of top (not necessarily expensive) restaurants within a two- or three-block walking radius.

We won't have the same space as last year so, counting the 30+ speakers that we will have, there will be room for only about 170 attendees - first come first served. So, register early and you will still get your 20% discount. There is still no extra charge for tutorials if you are attending the conference and this year's tutorial have a great line up. As a matter of fact, the entire agenda is great. As you can see from the speaker bios those who are returning and the new speakers for this year are strictly "top drawer" - the best of the best.

If your company is interested in becoming a sponsor, contact info@OctoberRulesFest.org and let them know. Again, remember that there is room this year for only 170 attendees. Maybe 180, but that's about it. More on the hotel as it develops but it is down to a choice of only two really great hotels for the same price as last year - but more goodies.

So, register early and ensure that you have a great spot at the Second Annual October Rules Fest. Most of all, like last year, we're going to have FUN and we're going to enjoy ourselves.

SDG

jco

MacWorld is now iWorld

Greetings:

Apple is finally showing its true colors:  Green, as in the almighty dollar.  Yes, you have to make money as a corporation otherwise you become a philantrophy and not a commercial organization.  

So, what is the problem?  The latest MacWorld came today.  And there is NOTHING in there about the Mac computers EXCEPT in how to do movies or photos in the back.  The rest is devoted to iPhone, iPod, iShuffle, and all of the other iCrap toys that Apple has foisted on the world of mindless teeny-boppers and drugged-up skateboarders.  

Even the products page failed to mention the latest advance of the Nehalem processors for the Mac Pro - all that they mentioned was the Xeon Quad core that is several months out of date now.  (I guess that the single article from last month on the Nehalem processor failed to catch the attention of the deely-boppers in the game room.)  Ergo, my reasoning for calling it iWorld rather than MacWorld.

OK.  I'm an old dude who just doesn't get it, right?  Probably.  But Apple's foundation is (was) computers, not toys.  They seem to have forgotten who brought them to the party.  If they are going to degenerate into a world of i-R-Us then I'll have to find another Unix hardware provider.  And, no, Linux is not a good substitute.  Linux is for Windows-haters who don't have the bollocks to do real, hard-core Unix.  (You know, I think all of this started when Steve Jobs decided to use Intel consumer junk for the CPU rather than the more advanced, more stable Power PC of the former Mac Pro.)

SDG

jco

Rulebase Benchmarks 2009

Greetings:

It is beginning to look like Miss Manners 2009 might (just might, not will) be a good benchmark for checking rules.  The other will be the WaltzDB-200.  I will be depending on vendors to double check all of my rules for them on that one since the database is generated from WaltzDB.java file that puts it into the "expected" format for the rules.  (Thanks to Dr. Forgy for re-writing the WaltzyDB.c file into Java for OPSJ but it can be extended to any rulebase needed.)  

Georrrey De Smet has made some changes to the original (hastily done) data file and has blogged about it http://blog.athico.com/2009/05/miss-manners-2009-yet-another-drools.html  in case you want to read his blog as well.  

SDG

jco

In Memory of Bubba the Rules Cat

Greetings:

For those who knew him, Bubba the Rules Cat (aka, Richard Halsey) was one of the more interesting folks in the rulebase space.  I worked on a couple of projects with him; Ericsson for 18 months and GMAC Insurance for another five or six.  Whatever the subject of conversation during the day, Bubba rarely watched TV and would spend the evening hours researching arkane and little-know facts and technology.  The next morning he had several sites for our consideration.  He was one of the very best researchers that I have had the privilege of knowing.  And he had a low tolerance for fools and bad products.

Bubba was 68 years old.  He was born June 14, 1942 and died "suddenly" on January 17, 2009.  He loved his mother and his wife, Wanda with a deep affection.  He spent six years in the Air Force with tours in Germany and South Africa and attended the University of Central Florida.

In his local community he was called "The Mayor of Vizcaya" - maybe due to his efforts in organizing a home owners association and he was later elected the president of that group.  The HOA put a plaque in his honor at the base of a pear tree that he planted there.

I remember when we were working in St. Louis and staying at the same motel he got up early every morning, ate breakfast, went outside to smoke and feed the birds.  He told me that his mother always wanted to come back as a bird so if she made it he was going to be sure and bring her breakfast every morning.  

Bubba wanted to be cremated and buried next to his mother so when he died Wanda did just that.  They had a memorial service in Pensacola and then his brother, Jim, took his ashes to New Jersey for another service and to be buried there.

Quite a guy.  He lived a full life.  He and I used to chat at least once a month about various and sundry things and I will sorely miss those calls.  I do know that the last few years (he retired for all practical purposes a few years ago) were probably the happiest that I have known him to be.  He quit smoking about the time he retired but he still had his two (or sometimes three) Cuba Libras every day - one at lunch, a nap, a long walk on the beach, and then a couple of more.  He had a specific method for making them using only the best rum and only smooth-skinned limes.  They really were quite good.

So - Bon Voyage Richard.  We'll miss you.  But I'm looking forward to seeing you again one day.  :-)

SDG

jco

IBM/ILOG in Health Care - NY Times

Greetings:

Well, it had to happen. The Doctor Will B.R.M.S. you now is an article that appeared in the NY Times. It does give a good shot-in-the-arm to the industry as a whole and ILOG in particular.  There's also an article on Google in the health industry that is interesting reading.  I only hope that someone there actually read the MYCIN book before starting out on this project.  What's new is old and what's old is new.

MYCIN was done in the late 70's and early 80's by the uber-geeks at Stanford in CA and CMU in PA.  And it did a good job.  Back then nobody trusted computers so they didn't use it in diagnosis even though it did a better job than did the doctors.  Today, maybe it will have a better chance.  

SDG

jco

Rulebase Benchmarks 2009

Greetings:

In concert with certain other "technical" persons in this field (Dr. Forgy, Gary Riley, et al) I would like to propose a new-and-improved benchmark for rulebased systems. Waltz and WaltzDB are still valid benchmarks but, unfortunately, most folks (read: programmers) have trouble visualizing the lines of logic that would help move lines from a 2D drawing to a 3D drawing. This is elementary for a "normal" engineer because they have had to endure Mechanical Engineering 101, aka Drafting Class.

I was "privileged" to see some recent emails from a "Benchmark" consortium and the vendors (surprise of surprises) did not like the Waltz nor the WaltzDB benchmarks because they were not "real world." First, the vendors have not defined what is a "real world" and second, since when was a benchmark "real world"?? The "real world" of most vendors these days is composed of financial problems that, while they are sometimes quite large, are never complex. (Think Abstract Algebra or Partial Differential Equation level of "complexity.")

WHY would we need a rulebase that can handle both massive numbers of rules and objects as well as very complex rules? Think about Homeland Security where there are thousands of Ports of Entry (POE) as well as perhaps a million travelers every day and millions of cargo boxes being shipped into the USA. The database query alone could take days when you have only a few minutes to determine (1) does a threat exist and (2) what is the level of that threat? An automobile of a specific nature parked too long in one place (think OKC) could be a "clue" on the threat level. A new object of a certain type on a roadway (think IED) is a potential threat in some areas but not in others.

One person from Saudi might not be (most likely is not) a threat. But knowing that he/she is related to another person of the same village entering at another port combined with another person of the same village either already here or entering from another port might quite possibly raise the threat level. (Meaning, what are the odds of three persons from the same, small village in Saudi Arabia entering the USA over a one, two or three day period?) These decisions sometimes have to be done within seconds while a person is standing at the counter - not hours later when that same person has been passed through and disappears into the local population.

Think of what happens in health underwriting when the things that must be considered are many and related. For example, a bad back (or knee or foot or whatever) could lead to declining health and possible heart attack depending on the severity of the injury. A heart attack could lead to even more declining health and death. Family history can and does play a huge part in underwriting. For example, being overweight means a potential increased risk in diabetes. If most everyone in the family has had diabetes (of either type) then the risks escalate. Having a family history of heart problems as well makes the problem even riskier. This is a large and complex problem that needs fast resolution. Assuming, of course, that the data are available in the first place. The reasoning process here is (can be) extremely complex and most times the human underwriter is the only person who can make that kind of determination when a rulebase would be a much better approach.

Fraud detection is a complex issue that is normally addressed from a superficial viewpoint rather than something "in depth" that might be reasonably accurate. Some of the issues of fraud detection (or homeland security or underwriting) could be handled with Rule-Based Forecasting (RBF) system as well as possibly linking the rulebase with Neural Networks to help predict what will happen. It has been shown (back in 1989) that neural net was much better at forecasting a time-dependent series than even the far more popular Box-Jenkins method of analysis and forecasting. It isn't much good at non-temporal situations but that's another story.

Let us return to our primary discussion of what should compose a rulebase benchmark. A rulebase benchmark should be composed of several tests:

Forward Chaining
Backward Chaining
Non-Monotonicity
Complex Rules
Rules with a high level of Specificity
Lots of (maybe 100 or more) "simple" rules that chain between themselves
Stress the conflict resolution strategy
Stress pattern matching

Just having an overwhelming amount of data is not sufficient for a rulebase benchmark - that would be more in line with a test of the database efficiency and/or the available memory. Further, it has been "proven" over time that compiling rules into Java code or into C++ code (something that vendors call "sequential rules") is much faster than using the inference engine. True, and it should be. After all, most inference engines are based in Java or C++ code and the rules are merely an extension, another layer of abstraction, if you will. But sequential rules do not have the flexibility of the engine and, in most cases, have to be "manually" arranged so that they fire in the correct order. An inference engine, being non-monotonic, does not have that restriction.

Simply put, most rulebased systems cannot pass muster on the simple WaltzDB-16 benchmark. We now have a WaltzDB-200 test should they want to try something more massive.

New Benchmarks: Perhaps we should try some of the NP-hard problems - that would eliminate most of the "also ran" tools. Also, perhaps we should be checking on the "flexibility" of a rulebase by processing on multiple platforms (not just Windows) as well as checking performance and scalability on multiple processors; perhaps 4, 8 or 16 (or more) CPU machines. An 8/16 CPU Mac is now available at a reasonable price as is the i7 Intel (basically 4/8 cores) CPU. But these are 64-bit CPUs and some rule engines are not supported for 64-bit platforms. Sad, but true. Some won't even run on Unix but only on LInux. Again, sad, but true.

So, any ideas? I'm thinking that someone, somewhere has a better suggestion than 64-Queens or Sudoku. Hopefully...


SDG
jco

Benchmarks 2009

Greetings:

In concert with certain other "technical" persons in this field (Dr. Forgy, Gary Riley, et al) I would like to propose a new-and-improved benchmark for rulebased systems. Waltz and WaltzDB are still valid benchmarks but, unfortunately, most folks (read: programmers) have trouble visualizing the lines that would help move from a 2D drawing to a 3D drawing. This is elementary for a real engineer because they have had to endure Mechanical Engineering 101, aka Drafting Class.

I was "privileged" to see some recent emails from a "Benchmark" consortium and the vendors (surprise of surprises) did not like the Waltz nor the WaltzDB benchmarks because they were not "real world." First, they have not defined what is a "real world" and second, since when was a benchmark "real world"?? The "real world" of most vendors these days is composed of financial problems that, while they are sometimes quite large, are never complex. (Think Abstract Algebra or Partial Differential Equation level of "complex.")

WHY would we need a rulebase that can handle both massive and complex? Think about Homeland Security where there are thousands of Ports of Entry (POE) as well as perhaps a million travelers every day and millions of cargo boxes being shipped into the USA. The database query alone could take days when you have only a few minutes to determine (1) does a threat exist and (2) what is the level of that threat? An automobile of a specific nature parked too long in one place could be a "clue" on the threat level. A new object of a certain type on a roadway is a potential threat in some areas but not in others.

One person from Saudi might not be (most likely is not) a threat. But knowing that he/she is related to another person of the same village entering at another port combined with another person of the same village either already here or entering from another port definitely raises the threat level. (Meaning, what are the odds of three persons from the same village entering the USA over a one, two or three day period?) These decisions sometimes have to be done within seconds while a person is standing at the counter - not hours later when that same person has been passed through and disappears into the local population.

Think of what happens in health underwriting when the things that must be considered are many and related. For example, a bad back (or knee or foot or whatever) could lead to declining health and possible heart attack depending on the severity of the injury. A heart attack could lead to even more declining health and death. Family history can and does play a huge part in underwriting. For example, being overweight means a potential increased risk in diabetes. If everyone in the family has had diabetes (of either type) then the risks escalate. Having a family history of heart problems as well makes the problem even riskier. This is a large and complex problem that needs fast resolution. Assuming, of course, that the data are available in the first place. The reasoning process here is (can be) extremely complex and most times the human underwriter is the only person who can make that kind of determination when a rulebase would be a much better approach.

Fraud detection is a complex issue that is normally addressed from a superficial viewpoint rather than something "in depth" that might be reasonably accurate. Some of the issues of fraud detection (or homeland security or underwriting) could be handled with Rule-Based Forecasting (RBF) system as well as possibly linking the rulebase with Neural Networks to help predict what will happen. It has been shown (back in 1989) that neural net was much better at forecasting a time-dependent series than even the far more popular Box-Jenkins method of analysis and forecasting.

But, I digress. Let us return to our primary discussion of what should compose a rulebase benchmark. A rulebase benchmark should be composed of several tests:

Forward Chaining
Backward Chaining
Non-Monotonicity
Complex Rules
Rules with a high level of Specificity
Lots of (maybe 100 or more) "simple" rules that chain between themselves

Just having an overwhelming amount of data is not sufficient for a rulebase benchmark - that would be more in line with a test of the database efficiency and/or the available memory. Further, it has been "proven" over time that compiling rules into Java code or into C++ code (something that vendors call "sequential rules") is much faster than using the inference engine. True, and it should be. After all, most inference engines are based in Java or C++ code and the rules are merely an extension. But sequential rules do not have the flexibility of the engine and, in most cases, have to be "manually" arranged so that they fire in the correct order. An inference engine, being non-monotonic, does not have that restriction.

Simply put, most rulebased systems cannot pass muster on the simple WaltzDB-16 benchmark. We now have a WaltzDB-200 test should they want to try something more massive.

New Benchmarks: Perhaps we should try some of the NP-hard problems - that would eliminate most of the "also ran" tools. Also, perhaps we should be checking on the "flexibility" of a rulebase by processing on multiple platforms (not just Windows) as well as checking performance and scalability on multiple processors; perhaps 4, 8 or 16 (or more) CPU machines. An 8/16 CPU Mac is now available at a reasonable price as is the i7 Intel (basically 4/8 cores) CPU. But these are 64-bit CPUs and some rule engines are not supported for 64-bit platforms. Sad, but true. Some won't even run on Unix but only on Linux or only on Windows.  Again, sad, but true.

Anyway, the next blog on ORF 2009 will be about Ms. Manners - the new version where we don't tell you HOW to solve the problem (we don't give you the rules) but you have to get the right answer.  Probably, in order to get the "right" answer we will have to provide the data.  Unless, of course, some kind vendor would find a college intern to do that.  :-)  


SDG
jco

A Rare Retraction

Greetings:

OK. I was wrong. Visual Rules is NOT a spreadsheet-looking development GUI but rather more of a modeling environment. My upcoming article in InfoWorld (hopefully, next week or the week after) will more than demonstrate that. Their GUI is one of the coolest in the industry eclipsed only by (maybe) Haley Expert Rules (not Haley Office Rules.) There! I reviewed the product in July of 2008 and again in December 2008 but somehow (old age? Nahhh - I've always been like this.) I lumped it in with Corticon and other spreadsheet-looking GUI environments.

OK, I blogged the error. Now I've blogged the retraction. Sorry, David K.

SDG
jco

Ladder Logic Relays, Computers and MYCIN

Greetings:

If you have a sufficient mass of grey hair (or should that be "gray hair") you might remember "Ladder Logic" that was used for relays (those electro-mechanical monstrosities long, long ago) that were used in order to control machines in the dark ages before solid-state, ladder-logic that came to be called a PLC (Programmable Logic Controller) as popularized by both Square D and Allen-Bradley companies.  A good view of this kind of logic is available on http://en.wikipedia.org/wiki/File:Ladder_diagram.png if you would like to see it.

Today's spreadsheets and decision tables are about as archaic as that particular concept and serve basically the same function.  

IF this is true AND this is true AND this is true 

THEN do that

How simple.  Just like Ladder Logic.  Yet, when many of these IF-THEN rules are strung together, how complex they swiftly seem to become.  (Not really - but the human brain cannot usually do more than four or five things at once.)  So, we bunch them up and put them in groups to make it simpler for the simple minds of humans.   And we are left with simple solutions for simple minds.

But, think about it...  What is happening is not very complex.  To move just ONE STEP toward complexity consider what the group of MYCIN did; they implemented the concept of "probability of belief and disbelief" into the rule as well as some other statistical concepts and nobody has EVER gone back to that concept since it is too complicated for the simple minds of most "computer scientists" - those minions to whom we trust our programming today.  If you can obtain the book by Buchannan and Shorliffe on the MYCIN project, GET IT, READ IT, UNDERSTAND IT !!  

Then, after you read it and understand it, let's move beyond MYCIN.  What if we integrate rulebased systems and neural nets?  What if we integrate digital computers and analog computers?  (Typical response to that one is:  "Tried it.  Failed.  Not possible.")  Cyc (Doug Lenat's huge, monstrous, immensely complicated 'real' KnowledgeBased System to be completed in 2025) may or may not be the answer, but surely we can move toward something in between the brain-dead, incredibly simplistic BRMS that are out there and immensely complex Cyc Project.  Any ideas?  

SDG

jco

Why Can't The English Teach Their Children How to Speak?

Greetings:

English is the language spoken (usually very poorly) around the world.  But there are some countries where English should be spoken and written properly.  (England, Scotland, Wales, Ireland, USA, Australia, New Zealand and Eastern Canada.)  It's the national language of India and Pakistan but you could not tell this from the language spoken or written there.   While in London and Southampton for long periods I was determined to develop a true British accent.  The problem was that not one person there spoke like the other one.  Finally, I gave up and decided that speaking properly with a decent mid-western-USA accent was far better than ANY of those terrible dialects.  Even if it does have slight Texas twang.  :-)

What set me off this time was the phrase from a fairly popular blog, "Is the data flat?"  Data are plural, Datum is singular.  So the sentence should read, "Are the data flat?"  It could read, "Is the data structure flat?" or "Is the structure of the data flat?"  Agenda (plural) is a list of agendum (singular).  

The other one drives me nuts is when someone says, "Me and Bob are going to lunch.  Want to go?"  It should be, "Bob and I are going to lunch."  What about where someone uses "were" for "we're" going to do something.  Or ending a sentence in a preposition as in, "Where are you going to?"  when  "Where are you going?" is sufficient.  Or the infamous, "What did you do that for?"   The most egregious of all, "You need to get aholt of the data."  OUCH!  Get a hold on the data?  Even that is terrible.  

What about the spoken variety?  Such as using "git" rather than "get."  Or using "caint" (rhymes with paint) for "can't", which rhymes with pant.  Or saying, "O'tel" rather than "Hotel".   I had the ignomious displeasure of listening to a linguist preach today - but she used the most terrible English when I KNOW that she fully knows how to speak properly - she was either heavily influenced by her congregation or her surrounding growing up and just never got out of improper usage of the English language.  (She speaks and writes about 10 or 12 languages fairly  fluently.)

I'm reminded every day of the musical "My Fair Lady" in which Professor Higgins asks the eternal question, "What can't the English teach their children how to speak?"  If you haven't seen it, check it out of your local DVD store and at least watch the opening song by Rex Harrison.  "The way an Englishman speaks absolutely classifies him  From the moment an Englishman speaks he makes some other Englishman despise him."  (Also from the movie and that song.) 

So brush up on your English by reading something of quality.  Shakespeare comes to mind.  Or the bible.  Get out of the code books for a while.  STOP reading all of the infernal, terrible documentation produced by someone in another country - or even in our own English-speaking countries when it was written by some dim-witted, High School Dropouts!

SDG

jco

Corticon and Tibco

Greetings:

There is a link called "The Orange Mile" [takeoff on "The Green Mile" ?] that published a "review" of Corticon as used with Tibco. They were even less impressed than I was when I reviewed Corticon several years ago - about 2006 or so. The link to the Orange Mile article is at http://orangemile.blogspot.com/2008/06/corticon-rule-engine-review.html so you can read it for yourself. I could not find an author's name or homebase link so it must be anonymous or something.  The link to my article is at http://www.infoworld.com/article/06/07/07/28TCcorticon_1.html if you would like to read the original broohaha.

Corticon calls what they do DETI, or Design Time Inferencing. The math that they claim to be part of the optimization process is doubtful since they don't show you what they do. ("Pay no attention to the man behind the curtain." - Wizard of Oz) That being said, what they do is basically static and, to my way of thinking, not a flexible as a true rulebased system.

On the other hand, you must consider that ILOG, FICO, and others now have sequential rules that are nothing more than what Corticon is doing. Even the "Decision Tables" by FICO, ILOG and Drools are extremely similar to Corticon in that each row is a rule that is static as well.  True, the row (rule) is processed by the rule engine BUT it's still pretty much a static process.  FICO has just recently introduced a "gap analysis" tool that Corticon has had for years.

Visual Rules and VisiRules are code generators, Visual Rules from a spreadsheet and VisiRules from a model driven process. They, too, are static processes.  Visual Rules generates plain-jane Java code while VisiRules generates a high-level processor for Prolog. 

Being a purist, I prefer a "real" inference engine that can deal with anomalies and incomplete rules and does have some kind of Conflict Resolution Strategy - as has been mentioned and discussed in other blogs at this link.  This is the BEST way to handle complex logic, incomplete rules and anything that might require thinking.

On the other hand, if ALL that you are doing is processing straight "out of the procedure book" business logic, why would you NEED and inference engine????  The answer is obvious:  You don't.  They may as well use a spreadsheet and get it over with as easily as possible.  And, so long as you don't have over a few hundred rules in each set, it should work quite well.  

All of that being said, why be critical of a company who is doing the same thing to the rulebase industry as the others except they don't have the fall-back position of a real inference engine should they need it? They (ILOG, FICO et al) gave their "stamp of approval" to the bastardization of the rulebase industry when they started down the Decision Table, Decision Tree, Compiled Sequential route a long time ago. Now that somebody is giving them the "come-uppance" that they deserve, they begin to whine like a mule. It makes you want to gag at the gall and hypocrisy of it the "Big Four" vendors.

SDG

jco