This is in reference to Daniel Selman's posting on Meta-Policies where he quietly snuck in some references to benchmarks that were totally out of whack.
I turned all of my benchmarking for InfoWorld over to Steve Nunez back in late December 2006. BUT, as I recall, I did run a BUNCH of tests - over and over and over again - before I released all of it. The last published results still can be found at http://www.kbsc.com/Performance2006.xls - and for JRules "optimized" versus JBoss. Daniel shows Waltz (I can only guess that it's Waltz 50 or something like that) for JRules and Drools as
JRules: 2.65 seconds
Drools: 170.578 seconds
WRONG!!! I did these on earlier versions and the speed hasn't improved dramatically since that time. What I got way back then on a WinXP at 2.0 GHz was
Blaze Advisor: 2.266
Drools 4.x is much improved but I have NOT re-run the benchmarks. Mark Proctor has and (I actually trust his results) reports that they have just about halved the previous times.
BTW, it should also be noted that, to my knowledge and not a fact stated from ILOG, that the "optimized" version of JRules is nothing more than compiled sequential rules - which should give great results. But compiled sequential loses the versatility of straight Rete rules where we can add rules wherever we like. If you look at my old spreadsheets, the straight Rete version of JRules puked (ran out of memory) on the 6.1.1 version. I would hope that the later versions are better but I seriously doubt it.
Blaze Advisor has several methods of running as well: Rete, Rete 2 (or Rete III) or Compiled Sequential. Compiled Sequential will "usually" run just a bit faster than Rete III in most commercial applications. BUT, compiled sequential, in my own humble opinion, is a violation of most rulebased principles. I mean, after all, why not do it in Java or C++ or even VB??? Sorry, I'm from the old school where a rulebase is part of the AI environment. Assembly (or plain-jane C) will usually run faster than most anything else.
So, if speed were the ONLY consideration, throw out the rulebase and use Assembly. And hard code it on a chip - bypass the OS completely. BUT, if you want intelligence, versatility Yaakov Kohen and not as any kind of representative of any company.
Who stole my spring?? - After a nice 20C degrees day yesterday, I woke up this morning to this:
3 years ago