PAW … What is it good for?...
… Well quite a lot actually.
The inaugural Predictive Analytics World (PAW) ran in San Francisco from Feb 18th -19th. Originated, chaired and orchestrated by Eric Siegel of Prediction Impact and supported by the ever-efficient Rising Media.
To my mind PAW distinguishes itself among the various conferences on similar topics primarily because of its business-focus and software vendor-neutrality. The vendors were there (more on that below) and there was certainly some theory in the mix but plenty of visionary thought, conceptual thinking and real-life applications.
Over the coming blogs I hope to expand on some of the themes which emerged. But let me start with a twitter-style front-of-mind download of some of the quotes/points I remember. As usual I didn’t see all the presentations because of the streaming. So I am sure I missed a lot and I’m looking forward to seeing the other stuff. In the meantime, and in no particular order …
· Eric Siegel hopped in (he's had a skiing accident)
· Ann Milley of SAS followed him and said “at the end of the day it is about the people”. I couldn’t agree more!
· David McMichael of MetLife, in his presentation with Kathy Konkel of SPSS on Insurance Claim Fraud Detection, emphasised how important it is that any Predictive Analytics effort is aligned with the priorities and agendas of senior management. That there needs to be a commitment to a project team which covers all the disciplines required to make a PA project successful. In particular he identified the need for an IT advocate in the mix.
· John Elder (Elder Research) gets to ply his trade across an interesting variety of industries and business/research areas and he presented them with great humour. He was the first to make the point – of what became a recurrent theme - that you have to be cynical about any model and try to “break it in the lab”. Linked to this he said he looks to recruit analysts with humility. A most under-rated characteristic in my opinion.
· Related to Evaluation and Quality Assurance was the importance of “out-of-time” testing.
· Various definitions of the differences/similarities between Data Mining (DM) and Predictive Analytics (PA). The correct consensus (I believe) is that PA is a subset of DM.
· The bright young Austrian guys from Commendo (project team "BigChaos") explained how they lead the Netflix Prize.
· Decision Tree models seemed to be in the majority.
· There weren’t too many new algorithms/tools in evidence but there were plenty of examples of how familiar algorithms are used collaboratively. Ensemble modelling in particular was a frequent topic. That said the number of applications with Support Vector Machines in the mix also seems to be growing. Likewise for Survival Analysis.
· The cluster visualisation work (some of which was also based on the Netflix data) presented by Todd Holloway of Ingenuity was probably the most interesting to me from a technology perspective. It is a minor observation but i felt we could have done with a bit more visualisation throughout.
· One of the ensemble modelling fans is Dean Abbott of Abbott Analytics who presented his National Rifle Association case study. I found Dean to be the most infectiously enthusiastic speaker of the 2 days (and I had to follow him!). Like many of the US based practitioners in attendance he is remarkably understated in many ways. You’d trust him with your data.
· The almost subversive (in a good way) presence of R and many discussions about what it really means for PA and Data Mining in the future – particularly from the perspective of the established software vendors.
The REvolution badges drove the point home.
· The keynotes were from Usama Fayyad, formerly of Yahoo and now with Open Insights and Andreas S. Weigend, Ph.D. who used to be the Chief Scientist at Amazon. Both took an appropriately higher level view of where PA could be heading in the near future. On-line behavioural applications seem to dominate their visions. Andreas speculated about the largely untapped potential of mobile devices and the implicit location data they carry. It made me wonder if I really need to work so hard on Google to find the best Dim Sum in Chinatown.
· As we’ve been doing a lot of on-line work in recent years I found Vincent Granville’s (Click Forensics) presentation on Pay-Per-Click (PPC) optimisation particularly interesting.
The two points that stick in my mind are his scepticism about real-time bid management – because it is moving the goal posts before you have had time to assess them (if I understood correctly) and the way in which he decomposed the textual constructs within ads to build potential predictors of response.
· Richard G. Vlasimsky, (Valen Technologies) presentation was interesting from a number of angles.
Foremost among them is that it is a true Predictive Analytics application as it focuses on PA for the Insurance industry.
Moreover Valant use a “consortia” approach to model data across a number of their client’s portfolios. This helps to smooth out some bias arising from the idiosyncrasies of individual customer business models.
Richard was another speaker who emphasised the importance of validating models and data. It makes me think that we are really into an era where the Quality Assurance around models is becoming paramount – for the more valuable ones at least.
Last but not least Valen typically deliver the model and results to their clients using Software as a Service (SaaS) platform.
If you take all of these points together I feel that they are very much at the front of the PA delivery curve.
· Conferences of like-minded people are always somewhat reassuring to the attendees but I left PAW thinking that there is much more yet to be done with Predictive Analytics. I think I might have referred to the current situation in my presentation as “the tip of the iceberg”.
· Eric Siegel hopped off into the distance.